Upload folder using huggingface_hub

.gitattributes

@@ -36,3 +36,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 qwen3-4b-instruct/base/tokenizer/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 qwen3-4b-instruct/dp8/tokenizer/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 tokenizer/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+qwen3-4b-instruct/dp3/tokenizer/tokenizer.json filter=lfs diff=lfs merge=lfs -text

qwen3-4b-instruct/dp3/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: Qwen/Qwen3-4B-Instruct-2507
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3-4B-Instruct-2507
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

qwen3-4b-instruct/dp3/adapter/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-4B-Instruct-2507",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": true,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "lm_head",
+    "embed_tokens"
+  ],
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

qwen3-4b-instruct/dp3/adapter/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d0844d1d1b84e1c37902c9865b63a922a40f22a74e52fc9fe297a73324652cde
+size 4721857072

qwen3-4b-instruct/dp3/audit_results.json

@@ -0,0 +1,137 @@
+{
+  "delta": 1e-05,
+  "num_canaries": 500,
+  "num_members": 250,
+  "paper_guess_fraction": 0.2,
+  "paper_guess_steps": 20,
+  "loss": {
+    "auc": 0.504792,
+    "empirical_epsilon": {
+      "0.05": 0.0,
+      "0.01": 0.0
+    },
+    "empirical_epsilon_details": {
+      "0.05": {
+        "epsilon": 0.0,
+        "num_guesses": 0,
+        "correct_guesses": 0,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      },
+      "0.01": {
+        "epsilon": 0.0,
+        "num_guesses": 0,
+        "correct_guesses": 0,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      }
+    }
+  },
+  "embedding": {
+    "auc": 0.51504,
+    "empirical_epsilon": {
+      "0.05": 0.0,
+      "0.01": 0.0
+    },
+    "empirical_epsilon_details": {
+      "0.05": {
+        "epsilon": 0.0,
+        "num_guesses": 0,
+        "correct_guesses": 0,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      },
+      "0.01": {
+        "epsilon": 0.0,
+        "num_guesses": 0,
+        "correct_guesses": 0,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      }
+    }
+  }
+}

qwen3-4b-instruct/dp3/audit_scores.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3cdc062093b88f95ceb1289319631cbeab3c23e571ca5bbfb90fda14e85498d3
+size 12784

qwen3-4b-instruct/dp3/codecarbon.csv

@@ -0,0 +1,2 @@
+timestamp,project_name,run_id,experiment_id,duration,emissions,emissions_rate,cpu_power,gpu_power,ram_power,cpu_energy,gpu_energy,ram_energy,energy_consumed,water_consumed,country_name,country_iso_code,region,cloud_provider,cloud_region,os,python_version,codecarbon_version,cpu_count,cpu_model,gpu_count,gpu_model,longitude,latitude,ram_total_size,tracking_mode,cpu_utilization_percent,gpu_utilization_percent,ram_utilization_percent,ram_used_gb,on_cloud,pue,wue
+2026-03-18T06:02:23,codedp-qwen3-4b-instruct-cpt-dp3,4b4c965b-253a-402e-94f1-cd15878f8181,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,2457.6702942838892,0.09496692637153915,3.8641036021965845e-05,72.03190690693239,3303.243429373142,54.0,0.04736684855609284,2.25075239893377,0.03550715308856684,2.3336264005784284,0.0,Sweden,SWE,östergötland county,,,Linux-6.8.0-94-generic-x86_64-with-glibc2.39,3.11.0,3.2.3,256,AMD EPYC 9554 64-Core Processor,8,8 x NVIDIA H200,16.1885,58.594,1511.49019241333,machine,3.8756856324191915,94.07749795249795,5.391281211624831,81.57497969388083,N,1.0,0.0

qwen3-4b-instruct/dp3/epochs/epoch_001/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: Qwen/Qwen3-4B-Instruct-2507
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3-4B-Instruct-2507
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

qwen3-4b-instruct/dp3/epochs/epoch_001/adapter/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-4B-Instruct-2507",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": true,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "lm_head",
+    "embed_tokens"
+  ],
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

qwen3-4b-instruct/dp3/epochs/epoch_001/adapter/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f74fff041f7595364c070bb9a1766fbaf33b5ef486cfd83c339ae5646e8b8931
+size 4721857072

qwen3-4b-instruct/dp3/epochs/epoch_001/audit_results.json

@@ -0,0 +1,137 @@
+{
+  "delta": 1e-05,
+  "num_canaries": 500,
+  "num_members": 250,
+  "paper_guess_fraction": 0.2,
+  "paper_guess_steps": 20,
+  "loss": {
+    "auc": 0.505664,
+    "empirical_epsilon": {
+      "0.05": 0.0,
+      "0.01": 0.0
+    },
+    "empirical_epsilon_details": {
+      "0.05": {
+        "epsilon": 0.0,
+        "num_guesses": 0,
+        "correct_guesses": 0,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      },
+      "0.01": {
+        "epsilon": 0.0,
+        "num_guesses": 0,
+        "correct_guesses": 0,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      }
+    }
+  },
+  "embedding": {
+    "auc": 0.51372,
+    "empirical_epsilon": {
+      "0.05": 0.0,
+      "0.01": 0.0
+    },
+    "empirical_epsilon_details": {
+      "0.05": {
+        "epsilon": 0.0,
+        "num_guesses": 0,
+        "correct_guesses": 0,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      },
+      "0.01": {
+        "epsilon": 0.0,
+        "num_guesses": 0,
+        "correct_guesses": 0,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      }
+    }
+  }
+}

qwen3-4b-instruct/dp3/epochs/epoch_001/audit_scores.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d5805d6b95eeea5aa8c4e47bf2e70ccba9c96374348e289a8b67bb9f60dced7c
+size 12784

qwen3-4b-instruct/dp3/epochs/epoch_002/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: Qwen/Qwen3-4B-Instruct-2507
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3-4B-Instruct-2507
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

qwen3-4b-instruct/dp3/epochs/epoch_002/adapter/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-4B-Instruct-2507",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": true,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "lm_head",
+    "embed_tokens"
+  ],
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

qwen3-4b-instruct/dp3/epochs/epoch_002/adapter/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d0844d1d1b84e1c37902c9865b63a922a40f22a74e52fc9fe297a73324652cde
+size 4721857072

qwen3-4b-instruct/dp3/epochs/epoch_002/audit_results.json

@@ -0,0 +1,137 @@
+{
+  "delta": 1e-05,
+  "num_canaries": 500,
+  "num_members": 250,
+  "paper_guess_fraction": 0.2,
+  "paper_guess_steps": 20,
+  "loss": {
+    "auc": 0.504792,
+    "empirical_epsilon": {
+      "0.05": 0.0,
+      "0.01": 0.0
+    },
+    "empirical_epsilon_details": {
+      "0.05": {
+        "epsilon": 0.0,
+        "num_guesses": 0,
+        "correct_guesses": 0,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      },
+      "0.01": {
+        "epsilon": 0.0,
+        "num_guesses": 0,
+        "correct_guesses": 0,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      }
+    }
+  },
+  "embedding": {
+    "auc": 0.51504,
+    "empirical_epsilon": {
+      "0.05": 0.0,
+      "0.01": 0.0
+    },
+    "empirical_epsilon_details": {
+      "0.05": {
+        "epsilon": 0.0,
+        "num_guesses": 0,
+        "correct_guesses": 0,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      },
+      "0.01": {
+        "epsilon": 0.0,
+        "num_guesses": 0,
+        "correct_guesses": 0,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      }
+    }
+  }
+}

qwen3-4b-instruct/dp3/epochs/epoch_002/audit_scores.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3cdc062093b88f95ceb1289319631cbeab3c23e571ca5bbfb90fda14e85498d3
+size 12784

qwen3-4b-instruct/dp3/metrics.jsonl

@@ -0,0 +1,27 @@
+{"timestamp": 1773811506.4695234, "event": "train_step", "step": 10, "epoch": 1, "metrics": {"train/step_loss": 1.616579128034187, "train/step_real_loss": 1.2012769132852554, "train/lr": 0.0002, "train/step_canary_loss": 14.90625, "perf/step_duration_sec": 12.063218676950783, "perf/samples_per_sec": 5.471176621054408, "perf/tokens_per_sec": 4265.196659189263, "perf/logical_batch_size": 66.0, "perf/logical_token_count": 51452.0, "perf/physical_batches": 9.0, "privacy/epsilon": 0.7638484077719963, "system/cuda_memory_allocated_gb": 12.68057107925415, "system/cuda_max_memory_allocated_gb": 86.2213044166565}}
+{"timestamp": 1773811628.0656273, "event": "train_step", "step": 20, "epoch": 1, "metrics": {"train/step_loss": 1.7544667159809786, "train/step_real_loss": 1.0926365107297897, "train/lr": 0.0001984111204336116, "train/step_canary_loss": 12.34375, "perf/step_duration_sec": 11.990010250825435, "perf/samples_per_sec": 5.671387978614834, "perf/tokens_per_sec": 4520.429830013588, "perf/logical_batch_size": 68.0, "perf/logical_token_count": 54200.0, "perf/physical_batches": 9.0, "privacy/epsilon": 1.016371889021752, "system/cuda_memory_allocated_gb": 13.261098384857178, "system/cuda_max_memory_allocated_gb": 86.2213044166565}}
+{"timestamp": 1773811750.6200142, "event": "train_step", "step": 30, "epoch": 1, "metrics": {"train/step_loss": 1.0279401504632197, "train/step_real_loss": 1.0600632801651955, "train/lr": 0.0001936949724999762, "train/step_canary_loss": 0.0, "perf/step_duration_sec": 12.954613069072366, "perf/samples_per_sec": 4.94032509182328, "perf/tokens_per_sec": 3957.7407466073655, "perf/logical_batch_size": 64.0, "perf/logical_token_count": 51271.0, "perf/physical_batches": 10.0, "privacy/epsilon": 1.2175571913066654, "system/cuda_memory_allocated_gb": 14.42089033126831, "system/cuda_max_memory_allocated_gb": 86.2213044166565}}
+{"timestamp": 1773811872.468319, "event": "train_step", "step": 40, "epoch": 1, "metrics": {"train/step_loss": 1.4674463272094727, "train/step_real_loss": 1.0572493374347687, "train/lr": 0.00018600142402077006, "train/step_canary_loss": 14.59375, "perf/step_duration_sec": 11.865780014079064, "perf/samples_per_sec": 5.562213349791522, "perf/tokens_per_sec": 4472.188085152074, "perf/logical_batch_size": 66.0, "perf/logical_token_count": 53066.0, "perf/physical_batches": 9.0, "privacy/epsilon": 1.3913063211429064, "system/cuda_memory_allocated_gb": 12.680593967437744, "system/cuda_max_memory_allocated_gb": 86.22135162353516}}
+{"timestamp": 1773811994.1354816, "event": "train_step", "step": 50, "epoch": 1, "metrics": {"train/step_loss": 1.257238564124474, "train/step_real_loss": 1.081570416688919, "train/lr": 0.00017557495743542585, "train/step_canary_loss": 12.5, "perf/step_duration_sec": 12.282207927200943, "perf/samples_per_sec": 5.292208077347962, "perf/tokens_per_sec": 4310.462769707012, "perf/logical_batch_size": 65.0, "perf/logical_token_count": 52942.0, "perf/physical_batches": 9.0, "privacy/epsilon": 1.5474611608458564, "system/cuda_memory_allocated_gb": 12.39033031463623, "system/cuda_max_memory_allocated_gb": 86.22135162353516}}
+{"timestamp": 1773812006.9120464, "event": "eval_step", "step": 50, "epoch": 1, "metrics": {"eval/loss": 0.9748443443423662, "eval/duration_sec": 12.77427957393229}}
+{"timestamp": 1773812128.636093, "event": "train_step", "step": 60, "epoch": 1, "metrics": {"train/step_loss": 2.1636021409715926, "train/step_real_loss": 1.041244499385357, "train/lr": 0.0001627469007380852, "train/step_canary_loss": 14.135416984558105, "perf/step_duration_sec": 12.324101509992033, "perf/samples_per_sec": 5.67992725013227, "perf/tokens_per_sec": 4058.7137293088017, "perf/logical_batch_size": 70.0, "perf/logical_token_count": 50020.0, "perf/physical_batches": 9.0, "privacy/epsilon": 1.6892902398315597, "system/cuda_memory_allocated_gb": 13.841647148132324, "system/cuda_max_memory_allocated_gb": 86.22135162353516}}
+{"timestamp": 1773812251.0683832, "event": "train_step", "step": 70, "epoch": 1, "metrics": {"train/step_loss": 1.8121652182410746, "train/step_real_loss": 1.0885114818811417, "train/lr": 0.0001479248986720057, "train/step_canary_loss": 13.390625, "perf/step_duration_sec": 12.67668361403048, "perf/samples_per_sec": 5.364178997473597, "perf/tokens_per_sec": 3911.117569040308, "perf/logical_batch_size": 68.0, "perf/logical_token_count": 49580.0, "perf/physical_batches": 9.0, "privacy/epsilon": 1.8217089373472166, "system/cuda_memory_allocated_gb": 13.261121273040771, "system/cuda_max_memory_allocated_gb": 86.22135162353516}}
+{"timestamp": 1773812373.025167, "event": "train_step", "step": 80, "epoch": 1, "metrics": {"train/step_loss": 1.8635356987223906, "train/step_real_loss": 1.08547542989254, "train/lr": 0.0001315799587615025, "train/step_canary_loss": 14.3125, "perf/step_duration_sec": 12.623636846896261, "perf/samples_per_sec": 5.386720231635859, "perf/tokens_per_sec": 3980.3109523349326, "perf/logical_batch_size": 68.0, "perf/logical_token_count": 50246.0, "perf/physical_batches": 9.0, "privacy/epsilon": 1.946082329590544, "system/cuda_memory_allocated_gb": 13.261121273040771, "system/cuda_max_memory_allocated_gb": 86.22135162353516}}
+{"timestamp": 1773812495.4775374, "event": "train_step", "step": 90, "epoch": 1, "metrics": {"train/step_loss": 1.5933425817916642, "train/step_real_loss": 1.0586555153131485, "train/lr": 0.00011423148382732853, "train/step_canary_loss": 13.0, "perf/step_duration_sec": 11.945233571808785, "perf/samples_per_sec": 5.608931763219984, "perf/tokens_per_sec": 4434.907001318536, "perf/logical_batch_size": 67.0, "perf/logical_token_count": 52976.0, "perf/physical_batches": 9.0, "privacy/epsilon": 2.06391520460725, "system/cuda_memory_allocated_gb": 12.970857620239258, "system/cuda_max_memory_allocated_gb": 86.22135162353516}}
+{"timestamp": 1773812537.9189491, "event": "train_epoch", "step": 93, "epoch": 1, "metrics": {"train/epoch_loss": 1.6919119858601415, "train/epoch_real_loss": 1.0723268173838774, "train/epoch_canary_loss": 13.472450833857557, "perf/epoch_duration_sec": 1142.3914510426112, "perf/epoch_samples_per_sec": 43.508728951566745, "perf/epoch_tokens_per_sec": 33067.53649593877, "perf/epoch_samples": 49704.0, "perf/epoch_tokens": 37776071.0, "system/cuda_epoch_peak_memory_gb": 86.22135162353516, "eval/loss": 0.9517239639774346, "eval/duration_sec": 12.793823145795614, "privacy/epsilon": 2.0981420051109834}}
+{"timestamp": 1773812551.2877524, "event": "audit_epoch", "step": 93, "epoch": 1, "metrics": {"audit/delta": 1e-05, "audit/num_canaries": 500.0, "audit/num_members": 250.0, "audit/paper_guess_fraction": 0.2, "audit/paper_guess_steps": 20.0, "audit/loss/auc": 0.505664, "audit/loss/empirical_epsilon/0.05": 0.0, "audit/loss/empirical_epsilon/0.01": 0.0, "audit/loss/empirical_epsilon_details/0.05/epsilon": 0.0, "audit/loss/empirical_epsilon_details/0.05/num_guesses": 0.0, "audit/loss/empirical_epsilon_details/0.05/correct_guesses": 0.0, "audit/loss/empirical_epsilon_details/0.01/epsilon": 0.0, "audit/loss/empirical_epsilon_details/0.01/num_guesses": 0.0, "audit/loss/empirical_epsilon_details/0.01/correct_guesses": 0.0, "audit/embedding/auc": 0.51372, "audit/embedding/empirical_epsilon/0.05": 0.0, "audit/embedding/empirical_epsilon/0.01": 0.0, "audit/embedding/empirical_epsilon_details/0.05/epsilon": 0.0, "audit/embedding/empirical_epsilon_details/0.05/num_guesses": 0.0, "audit/embedding/empirical_epsilon_details/0.05/correct_guesses": 0.0, "audit/embedding/empirical_epsilon_details/0.01/epsilon": 0.0, "audit/embedding/empirical_epsilon_details/0.01/num_guesses": 0.0, "audit/embedding/empirical_epsilon_details/0.01/correct_guesses": 0.0, "perf/audit_duration_sec": 6.93540578102693}}
+{"timestamp": 1773812637.181298, "event": "train_step", "step": 100, "epoch": 2, "metrics": {"train/step_loss": 2.226203179695237, "train/step_real_loss": 0.9423502832651138, "train/lr": 9.643076661610196e-05, "train/step_canary_loss": 13.964286804199219, "perf/step_duration_sec": 12.210506019182503, "perf/samples_per_sec": 5.814664837678322, "perf/tokens_per_sec": 4380.080556528868, "perf/logical_batch_size": 71.0, "perf/logical_token_count": 53483.0, "perf/physical_batches": 9.0, "privacy/epsilon": 2.176204855129592, "system/cuda_memory_allocated_gb": 14.131910800933838, "system/cuda_max_memory_allocated_gb": 86.22135162353516}}
+{"timestamp": 1773812649.9187398, "event": "eval_step", "step": 100, "epoch": 2, "metrics": {"eval/loss": 0.9495426886356794, "eval/duration_sec": 12.734526461921632}}
+{"timestamp": 1773812771.789377, "event": "train_step", "step": 110, "epoch": 2, "metrics": {"train/step_loss": 1.4468027822899097, "train/step_real_loss": 1.0408434942364693, "train/lr": 7.874347104470234e-05, "train/step_canary_loss": 14.4375, "perf/step_duration_sec": 12.185235306154937, "perf/samples_per_sec": 5.416391094775368, "perf/tokens_per_sec": 4356.419770834176, "perf/logical_batch_size": 66.0, "perf/logical_token_count": 53084.0, "perf/physical_batches": 9.0, "privacy/epsilon": 2.2837597517001855, "system/cuda_memory_allocated_gb": 12.680593967437744, "system/cuda_max_memory_allocated_gb": 86.22137594223022}}
+{"timestamp": 1773812893.198121, "event": "train_step", "step": 120, "epoch": 2, "metrics": {"train/step_loss": 1.616717566305132, "train/step_real_loss": 1.0352746099233627, "train/lr": 6.173165676349103e-05, "train/step_canary_loss": 14.020833969116211, "perf/step_duration_sec": 12.073515899013728, "perf/samples_per_sec": 5.549336296105193, "perf/tokens_per_sec": 4502.996513587329, "perf/logical_batch_size": 67.0, "perf/logical_token_count": 54367.0, "perf/physical_batches": 9.0, "privacy/epsilon": 2.387262501265619, "system/cuda_memory_allocated_gb": 12.970857620239258, "system/cuda_max_memory_allocated_gb": 86.22137594223022}}
+{"timestamp": 1773813014.0904934, "event": "train_step", "step": 130, "epoch": 2, "metrics": {"train/step_loss": 1.9267044274703315, "train/step_real_loss": 0.9727360382676125, "train/lr": 4.593591825444028e-05, "train/step_canary_loss": 14.137499809265137, "perf/step_duration_sec": 11.957372829318047, "perf/samples_per_sec": 5.770498334786406, "perf/tokens_per_sec": 4662.479024096767, "perf/logical_batch_size": 69.0, "perf/logical_token_count": 55751.0, "perf/physical_batches": 9.0, "privacy/epsilon": 2.4870228609511393, "system/cuda_memory_allocated_gb": 13.55138349533081, "system/cuda_max_memory_allocated_gb": 86.22137594223022}}
+{"timestamp": 1773813135.2851143, "event": "train_step", "step": 140, "epoch": 2, "metrics": {"train/step_loss": 1.6469186313116728, "train/step_real_loss": 1.0473601296544075, "train/lr": 3.185820604061088e-05, "train/step_canary_loss": 14.4375, "perf/step_duration_sec": 12.27732455311343, "perf/samples_per_sec": 5.4572150235296455, "perf/tokens_per_sec": 4390.940368708353, "perf/logical_batch_size": 67.0, "perf/logical_token_count": 53909.0, "perf/physical_batches": 9.0, "privacy/epsilon": 2.5836117621736387, "system/cuda_memory_allocated_gb": 12.970857620239258, "system/cuda_max_memory_allocated_gb": 86.22137594223022}}
+{"timestamp": 1773813257.0352345, "event": "train_step", "step": 150, "epoch": 2, "metrics": {"train/step_loss": 2.0192771579908286, "train/step_real_loss": 1.04129096865654, "train/lr": 1.994587590756397e-05, "train/step_canary_loss": 14.537500381469727, "perf/step_duration_sec": 11.960256013087928, "perf/samples_per_sec": 5.769107277009316, "perf/tokens_per_sec": 4232.434485065052, "perf/logical_batch_size": 69.0, "perf/logical_token_count": 50621.0, "perf/physical_batches": 9.0, "privacy/epsilon": 2.6773301521119843, "system/cuda_memory_allocated_gb": 13.55138349533081, "system/cuda_max_memory_allocated_gb": 86.22137594223022}}
+{"timestamp": 1773813269.8009667, "event": "eval_step", "step": 150, "epoch": 2, "metrics": {"eval/loss": 0.9415916662949781, "eval/duration_sec": 12.763515894301236}}
+{"timestamp": 1773813391.1203394, "event": "train_step", "step": 160, "epoch": 2, "metrics": {"train/step_loss": 1.4993842009342078, "train/step_real_loss": 1.1243649572134018, "train/lr": 1.057747301402887e-05, "train/step_canary_loss": 13.5, "perf/step_duration_sec": 12.41904565365985, "perf/samples_per_sec": 5.314418018952207, "perf/tokens_per_sec": 3556.311912500654, "perf/logical_batch_size": 66.0, "perf/logical_token_count": 44166.0, "perf/physical_batches": 9.0, "privacy/epsilon": 2.768504028383432, "system/cuda_memory_allocated_gb": 12.680593967437744, "system/cuda_max_memory_allocated_gb": 86.22137594223022}}
+{"timestamp": 1773813513.5407782, "event": "train_step", "step": 170, "epoch": 2, "metrics": {"train/step_loss": 1.6632740604343699, "train/step_real_loss": 1.0752243772149086, "train/lr": 4.050702638550275e-06, "train/step_canary_loss": 14.208333969116211, "perf/step_duration_sec": 12.597235643770546, "perf/samples_per_sec": 5.318627188904904, "perf/tokens_per_sec": 3936.101649771056, "perf/logical_batch_size": 67.0, "perf/logical_token_count": 49584.0, "perf/physical_batches": 9.0, "privacy/epsilon": 2.8571453531213207, "system/cuda_memory_allocated_gb": 12.970857620239258, "system/cuda_max_memory_allocated_gb": 86.22137594223022}}
+{"timestamp": 1773813635.9173324, "event": "train_step", "step": 180, "epoch": 2, "metrics": {"train/step_loss": 1.7563851160161636, "train/step_real_loss": 1.0712373107671738, "train/lr": 5.729698228102653e-07, "train/step_canary_loss": 12.71875, "perf/step_duration_sec": 12.466372530907393, "perf/samples_per_sec": 5.454674150913608, "perf/tokens_per_sec": 4245.501236930199, "perf/logical_batch_size": 68.0, "perf/logical_token_count": 52926.0, "perf/physical_batches": 9.0, "privacy/epsilon": 2.9437684274367397, "system/cuda_memory_allocated_gb": 13.261121273040771, "system/cuda_max_memory_allocated_gb": 86.22137594223022}}
+{"timestamp": 1773813716.1885753, "event": "train_epoch", "step": 186, "epoch": 2, "metrics": {"train/epoch_loss": 1.6068291228622247, "train/epoch_real_loss": 1.0283400972633716, "train/epoch_canary_loss": 13.124244338677878, "perf/epoch_duration_sec": 1152.0965769039467, "perf/epoch_samples_per_sec": 42.999867366267054, "perf/epoch_tokens_per_sec": 32782.04514893617, "perf/epoch_samples": 49540.0, "perf/epoch_tokens": 37768082.0, "system/cuda_epoch_peak_memory_gb": 86.22137594223022, "eval/loss": 0.9410304912389854, "eval/duration_sec": 12.763936698902398, "privacy/epsilon": 2.9947529815620726}}
+{"timestamp": 1773813729.4597642, "event": "audit_epoch", "step": 186, "epoch": 2, "metrics": {"audit/delta": 1e-05, "audit/num_canaries": 500.0, "audit/num_members": 250.0, "audit/paper_guess_fraction": 0.2, "audit/paper_guess_steps": 20.0, "audit/loss/auc": 0.504792, "audit/loss/empirical_epsilon/0.05": 0.0, "audit/loss/empirical_epsilon/0.01": 0.0, "audit/loss/empirical_epsilon_details/0.05/epsilon": 0.0, "audit/loss/empirical_epsilon_details/0.05/num_guesses": 0.0, "audit/loss/empirical_epsilon_details/0.05/correct_guesses": 0.0, "audit/loss/empirical_epsilon_details/0.01/epsilon": 0.0, "audit/loss/empirical_epsilon_details/0.01/num_guesses": 0.0, "audit/loss/empirical_epsilon_details/0.01/correct_guesses": 0.0, "audit/embedding/auc": 0.51504, "audit/embedding/empirical_epsilon/0.05": 0.0, "audit/embedding/empirical_epsilon/0.01": 0.0, "audit/embedding/empirical_epsilon_details/0.05/epsilon": 0.0, "audit/embedding/empirical_epsilon_details/0.05/num_guesses": 0.0, "audit/embedding/empirical_epsilon_details/0.05/correct_guesses": 0.0, "audit/embedding/empirical_epsilon_details/0.01/epsilon": 0.0, "audit/embedding/empirical_epsilon_details/0.01/num_guesses": 0.0, "audit/embedding/empirical_epsilon_details/0.01/correct_guesses": 0.0, "perf/audit_duration_sec": 7.0606719348579645}}
+{"timestamp": 1773813742.732494, "event": "audit_final", "step": 186, "epoch": 2, "metrics": {"audit/delta": 1e-05, "audit/num_canaries": 500.0, "audit/num_members": 250.0, "audit/paper_guess_fraction": 0.2, "audit/paper_guess_steps": 20.0, "audit/loss/auc": 0.504792, "audit/loss/empirical_epsilon/0.05": 0.0, "audit/loss/empirical_epsilon/0.01": 0.0, "audit/loss/empirical_epsilon_details/0.05/epsilon": 0.0, "audit/loss/empirical_epsilon_details/0.05/num_guesses": 0.0, "audit/loss/empirical_epsilon_details/0.05/correct_guesses": 0.0, "audit/loss/empirical_epsilon_details/0.01/epsilon": 0.0, "audit/loss/empirical_epsilon_details/0.01/num_guesses": 0.0, "audit/loss/empirical_epsilon_details/0.01/correct_guesses": 0.0, "audit/embedding/auc": 0.51504, "audit/embedding/empirical_epsilon/0.05": 0.0, "audit/embedding/empirical_epsilon/0.01": 0.0, "audit/embedding/empirical_epsilon_details/0.05/epsilon": 0.0, "audit/embedding/empirical_epsilon_details/0.05/num_guesses": 0.0, "audit/embedding/empirical_epsilon_details/0.05/correct_guesses": 0.0, "audit/embedding/empirical_epsilon_details/0.01/epsilon": 0.0, "audit/embedding/empirical_epsilon_details/0.01/num_guesses": 0.0, "audit/embedding/empirical_epsilon_details/0.01/correct_guesses": 0.0}}
+{"timestamp": 1773813743.2978687, "event": "energy_final", "step": 186, "epoch": null, "metrics": {"energy/codecarbon/duration": 2457.6702942838892, "energy/codecarbon/emissions": 0.09496692637153915, "energy/codecarbon/emissions_rate": 3.8641036021965845e-05, "energy/codecarbon/cpu_power": 72.03190690693239, "energy/codecarbon/gpu_power": 3303.243429373142, "energy/codecarbon/ram_power": 54.0, "energy/codecarbon/cpu_energy": 0.04736684855609284, "energy/codecarbon/gpu_energy": 2.25075239893377, "energy/codecarbon/ram_energy": 0.03550715308856684, "energy/codecarbon/energy_consumed": 2.3336264005784284, "energy/codecarbon/water_consumed": 0.0, "energy/codecarbon/cpu_count": 256.0, "energy/codecarbon/gpu_count": 8.0, "energy/codecarbon/longitude": 16.1885, "energy/codecarbon/latitude": 58.594, "energy/codecarbon/ram_total_size": 1511.49019241333, "energy/codecarbon/cpu_utilization_percent": 3.8756856324191915, "energy/codecarbon/gpu_utilization_percent": 94.07749795249795, "energy/codecarbon/ram_utilization_percent": 5.391281211624831, "energy/codecarbon/ram_used_gb": 81.57497969388083, "energy/codecarbon/pue": 1.0, "energy/codecarbon/wue": 0.0}}

qwen3-4b-instruct/dp3/pretrain_lm_head.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc44b7d60b8e2cf912e4233ff02bc57bb7e91f7a3ba6aa8ea10b7767ca29954a
+size 779106920

qwen3-4b-instruct/dp3/resolved_config.yaml

@@ -0,0 +1,101 @@
+model:
+  name: Qwen/Qwen3-4B-Instruct-2507
+  tokenizer_name: Qwen/Qwen3-4B-Instruct-2507
+  max_length: 1024
+  dtype: bfloat16
+  trust_remote_code: true
+  use_fast_tokenizer: true
+  cache_dir: null
+  local_files_only: false
+  low_cpu_mem_usage: true
+  tie_word_embeddings: true
+  gradient_checkpointing: false
+  use_chat_template: false
+dataset:
+  name: melihcatal/codedp-cpt
+  split: train
+  mode: cpt
+  text_column: text
+  validation_ratio: 0.05
+  max_samples: -1
+lora:
+  enabled: true
+  r: 16
+  alpha: 32
+  dropout: 0.05
+  target_modules:
+  - q_proj
+  - k_proj
+  - v_proj
+  - o_proj
+  modules_to_save:
+  - lm_head
+  bias: none
+training:
+  seed: 42
+  epochs: 2
+  warmup_steps: null
+  warmup_ratio: 0.05
+  mixed_precision: false
+  mixed_precision_dtype: bfloat16
+  batch_size: 8
+  eval_batch_size: 8
+  eval_every_steps: 50
+  eval_every_epochs: 1
+  learning_rate: 0.0002
+  optimizer: adamw
+  lr_scheduler: cosine
+  adam_beta1: 0.9
+  adam_beta2: 0.999
+  adam_epsilon: 1.0e-08
+  sgd_momentum: 0.9
+  weight_decay: 0.01
+  max_grad_norm: 1.0
+  log_every: 10
+  gradient_accumulation_steps: 8
+  num_workers: 4
+  output_dir: runs/cpt/qwen3-4b-instruct/dp3
+distributed:
+  strategy: dpddp
+  backend: nccl
+  devices: null
+dp:
+  module_validator: auto
+  target_delta: 1.0e-05
+  noise_multiplier: null
+  max_grad_norm: 1.0
+  grad_sample_mode: hooks
+  secure_mode: false
+  enabled: true
+  target_epsilon: 3.0
+  clipping: flat
+audit:
+  enabled: true
+  run_every_epoch: true
+  epoch_device: cuda
+  q_canary: auto
+  num_canaries: 500
+  prefix_length: 49
+  num_digits: 12
+  batch_size: 32
+  delta: 1.0e-05
+  p_values:
+  - 0.05
+  - 0.01
+  paper_guess_fraction: 0.2
+  paper_guess_steps: 20
+  enable_holdout_empirical_epsilon: false
+  holdout_seed: 42
+  tie_seed: 42
+tracking:
+  enabled: true
+  tensorboard: true
+  wandb: false
+  wandb_project: codedp-finetune-h200-audit
+  wandb_run_name: qwen3-4b-instruct-cpt-dp3
+  wandb_mode: online
+  codecarbon: true
+  codecarbon_output_file: codecarbon.csv
+  codecarbon_measure_power_secs: 15
+  codecarbon_country_iso_code: null
+  codecarbon_project_name: codedp-qwen3-4b-instruct-cpt-dp3

qwen3-4b-instruct/dp3/scalars.csv

@@ -0,0 +1,358 @@
+timestamp,event,step,epoch,key,value
+1773811506.4695234,train_step,10,1,train/step_loss,1.616579128034187
+1773811506.4695234,train_step,10,1,train/step_real_loss,1.2012769132852554
+1773811506.4695234,train_step,10,1,train/lr,0.0002
+1773811506.4695234,train_step,10,1,train/step_canary_loss,14.90625
+1773811506.4695234,train_step,10,1,perf/step_duration_sec,12.063218676950783
+1773811506.4695234,train_step,10,1,perf/samples_per_sec,5.471176621054408
+1773811506.4695234,train_step,10,1,perf/tokens_per_sec,4265.196659189263
+1773811506.4695234,train_step,10,1,perf/logical_batch_size,66.0
+1773811506.4695234,train_step,10,1,perf/logical_token_count,51452.0
+1773811506.4695234,train_step,10,1,perf/physical_batches,9.0
+1773811506.4695234,train_step,10,1,privacy/epsilon,0.7638484077719963
+1773811506.4695234,train_step,10,1,system/cuda_memory_allocated_gb,12.68057107925415
+1773811506.4695234,train_step,10,1,system/cuda_max_memory_allocated_gb,86.2213044166565
+1773811628.0656273,train_step,20,1,train/step_loss,1.7544667159809786
+1773811628.0656273,train_step,20,1,train/step_real_loss,1.0926365107297897
+1773811628.0656273,train_step,20,1,train/lr,0.0001984111204336116
+1773811628.0656273,train_step,20,1,train/step_canary_loss,12.34375
+1773811628.0656273,train_step,20,1,perf/step_duration_sec,11.990010250825435
+1773811628.0656273,train_step,20,1,perf/samples_per_sec,5.671387978614834
+1773811628.0656273,train_step,20,1,perf/tokens_per_sec,4520.429830013588
+1773811628.0656273,train_step,20,1,perf/logical_batch_size,68.0
+1773811628.0656273,train_step,20,1,perf/logical_token_count,54200.0
+1773811628.0656273,train_step,20,1,perf/physical_batches,9.0
+1773811628.0656273,train_step,20,1,privacy/epsilon,1.016371889021752
+1773811628.0656273,train_step,20,1,system/cuda_memory_allocated_gb,13.261098384857178
+1773811628.0656273,train_step,20,1,system/cuda_max_memory_allocated_gb,86.2213044166565
+1773811750.6200142,train_step,30,1,train/step_loss,1.0279401504632197
+1773811750.6200142,train_step,30,1,train/step_real_loss,1.0600632801651955
+1773811750.6200142,train_step,30,1,train/lr,0.0001936949724999762
+1773811750.6200142,train_step,30,1,train/step_canary_loss,0.0
+1773811750.6200142,train_step,30,1,perf/step_duration_sec,12.954613069072366
+1773811750.6200142,train_step,30,1,perf/samples_per_sec,4.94032509182328
+1773811750.6200142,train_step,30,1,perf/tokens_per_sec,3957.7407466073655
+1773811750.6200142,train_step,30,1,perf/logical_batch_size,64.0
+1773811750.6200142,train_step,30,1,perf/logical_token_count,51271.0
+1773811750.6200142,train_step,30,1,perf/physical_batches,10.0
+1773811750.6200142,train_step,30,1,privacy/epsilon,1.2175571913066654
+1773811750.6200142,train_step,30,1,system/cuda_memory_allocated_gb,14.42089033126831
+1773811750.6200142,train_step,30,1,system/cuda_max_memory_allocated_gb,86.2213044166565
+1773811872.468319,train_step,40,1,train/step_loss,1.4674463272094727
+1773811872.468319,train_step,40,1,train/step_real_loss,1.0572493374347687
+1773811872.468319,train_step,40,1,train/lr,0.00018600142402077006
+1773811872.468319,train_step,40,1,train/step_canary_loss,14.59375
+1773811872.468319,train_step,40,1,perf/step_duration_sec,11.865780014079064
+1773811872.468319,train_step,40,1,perf/samples_per_sec,5.562213349791522
+1773811872.468319,train_step,40,1,perf/tokens_per_sec,4472.188085152074
+1773811872.468319,train_step,40,1,perf/logical_batch_size,66.0
+1773811872.468319,train_step,40,1,perf/logical_token_count,53066.0
+1773811872.468319,train_step,40,1,perf/physical_batches,9.0
+1773811872.468319,train_step,40,1,privacy/epsilon,1.3913063211429064
+1773811872.468319,train_step,40,1,system/cuda_memory_allocated_gb,12.680593967437744
+1773811872.468319,train_step,40,1,system/cuda_max_memory_allocated_gb,86.22135162353516
+1773811994.1354816,train_step,50,1,train/step_loss,1.257238564124474
+1773811994.1354816,train_step,50,1,train/step_real_loss,1.081570416688919
+1773811994.1354816,train_step,50,1,train/lr,0.00017557495743542585
+1773811994.1354816,train_step,50,1,train/step_canary_loss,12.5
+1773811994.1354816,train_step,50,1,perf/step_duration_sec,12.282207927200943
+1773811994.1354816,train_step,50,1,perf/samples_per_sec,5.292208077347962
+1773811994.1354816,train_step,50,1,perf/tokens_per_sec,4310.462769707012
+1773811994.1354816,train_step,50,1,perf/logical_batch_size,65.0
+1773811994.1354816,train_step,50,1,perf/logical_token_count,52942.0
+1773811994.1354816,train_step,50,1,perf/physical_batches,9.0
+1773811994.1354816,train_step,50,1,privacy/epsilon,1.5474611608458564
+1773811994.1354816,train_step,50,1,system/cuda_memory_allocated_gb,12.39033031463623
+1773811994.1354816,train_step,50,1,system/cuda_max_memory_allocated_gb,86.22135162353516
+1773812006.9120464,eval_step,50,1,eval/loss,0.9748443443423662
+1773812006.9120464,eval_step,50,1,eval/duration_sec,12.77427957393229
+1773812128.636093,train_step,60,1,train/step_loss,2.1636021409715926
+1773812128.636093,train_step,60,1,train/step_real_loss,1.041244499385357
+1773812128.636093,train_step,60,1,train/lr,0.0001627469007380852
+1773812128.636093,train_step,60,1,train/step_canary_loss,14.135416984558105
+1773812128.636093,train_step,60,1,perf/step_duration_sec,12.324101509992033
+1773812128.636093,train_step,60,1,perf/samples_per_sec,5.67992725013227
+1773812128.636093,train_step,60,1,perf/tokens_per_sec,4058.7137293088017
+1773812128.636093,train_step,60,1,perf/logical_batch_size,70.0
+1773812128.636093,train_step,60,1,perf/logical_token_count,50020.0
+1773812128.636093,train_step,60,1,perf/physical_batches,9.0
+1773812128.636093,train_step,60,1,privacy/epsilon,1.6892902398315597
+1773812128.636093,train_step,60,1,system/cuda_memory_allocated_gb,13.841647148132324
+1773812128.636093,train_step,60,1,system/cuda_max_memory_allocated_gb,86.22135162353516
+1773812251.0683832,train_step,70,1,train/step_loss,1.8121652182410746
+1773812251.0683832,train_step,70,1,train/step_real_loss,1.0885114818811417
+1773812251.0683832,train_step,70,1,train/lr,0.0001479248986720057
+1773812251.0683832,train_step,70,1,train/step_canary_loss,13.390625
+1773812251.0683832,train_step,70,1,perf/step_duration_sec,12.67668361403048
+1773812251.0683832,train_step,70,1,perf/samples_per_sec,5.364178997473597
+1773812251.0683832,train_step,70,1,perf/tokens_per_sec,3911.117569040308
+1773812251.0683832,train_step,70,1,perf/logical_batch_size,68.0
+1773812251.0683832,train_step,70,1,perf/logical_token_count,49580.0
+1773812251.0683832,train_step,70,1,perf/physical_batches,9.0
+1773812251.0683832,train_step,70,1,privacy/epsilon,1.8217089373472166
+1773812251.0683832,train_step,70,1,system/cuda_memory_allocated_gb,13.261121273040771
+1773812251.0683832,train_step,70,1,system/cuda_max_memory_allocated_gb,86.22135162353516
+1773812373.025167,train_step,80,1,train/step_loss,1.8635356987223906
+1773812373.025167,train_step,80,1,train/step_real_loss,1.08547542989254
+1773812373.025167,train_step,80,1,train/lr,0.0001315799587615025
+1773812373.025167,train_step,80,1,train/step_canary_loss,14.3125
+1773812373.025167,train_step,80,1,perf/step_duration_sec,12.623636846896261
+1773812373.025167,train_step,80,1,perf/samples_per_sec,5.386720231635859
+1773812373.025167,train_step,80,1,perf/tokens_per_sec,3980.3109523349326
+1773812373.025167,train_step,80,1,perf/logical_batch_size,68.0
+1773812373.025167,train_step,80,1,perf/logical_token_count,50246.0
+1773812373.025167,train_step,80,1,perf/physical_batches,9.0
+1773812373.025167,train_step,80,1,privacy/epsilon,1.946082329590544
+1773812373.025167,train_step,80,1,system/cuda_memory_allocated_gb,13.261121273040771
+1773812373.025167,train_step,80,1,system/cuda_max_memory_allocated_gb,86.22135162353516
+1773812495.4775374,train_step,90,1,train/step_loss,1.5933425817916642
+1773812495.4775374,train_step,90,1,train/step_real_loss,1.0586555153131485
+1773812495.4775374,train_step,90,1,train/lr,0.00011423148382732853
+1773812495.4775374,train_step,90,1,train/step_canary_loss,13.0
+1773812495.4775374,train_step,90,1,perf/step_duration_sec,11.945233571808785
+1773812495.4775374,train_step,90,1,perf/samples_per_sec,5.608931763219984
+1773812495.4775374,train_step,90,1,perf/tokens_per_sec,4434.907001318536
+1773812495.4775374,train_step,90,1,perf/logical_batch_size,67.0
+1773812495.4775374,train_step,90,1,perf/logical_token_count,52976.0
+1773812495.4775374,train_step,90,1,perf/physical_batches,9.0
+1773812495.4775374,train_step,90,1,privacy/epsilon,2.06391520460725
+1773812495.4775374,train_step,90,1,system/cuda_memory_allocated_gb,12.970857620239258
+1773812495.4775374,train_step,90,1,system/cuda_max_memory_allocated_gb,86.22135162353516
+1773812537.9189491,train_epoch,93,1,train/epoch_loss,1.6919119858601415
+1773812537.9189491,train_epoch,93,1,train/epoch_real_loss,1.0723268173838774
+1773812537.9189491,train_epoch,93,1,train/epoch_canary_loss,13.472450833857557
+1773812537.9189491,train_epoch,93,1,perf/epoch_duration_sec,1142.3914510426112
+1773812537.9189491,train_epoch,93,1,perf/epoch_samples_per_sec,43.508728951566745
+1773812537.9189491,train_epoch,93,1,perf/epoch_tokens_per_sec,33067.53649593877
+1773812537.9189491,train_epoch,93,1,perf/epoch_samples,49704.0
+1773812537.9189491,train_epoch,93,1,perf/epoch_tokens,37776071.0
+1773812537.9189491,train_epoch,93,1,system/cuda_epoch_peak_memory_gb,86.22135162353516
+1773812537.9189491,train_epoch,93,1,eval/loss,0.9517239639774346
+1773812537.9189491,train_epoch,93,1,eval/duration_sec,12.793823145795614
+1773812537.9189491,train_epoch,93,1,privacy/epsilon,2.0981420051109834
+1773812551.2877524,audit_epoch,93,1,audit/delta,1e-05
+1773812551.2877524,audit_epoch,93,1,audit/num_canaries,500.0
+1773812551.2877524,audit_epoch,93,1,audit/num_members,250.0
+1773812551.2877524,audit_epoch,93,1,audit/paper_guess_fraction,0.2
+1773812551.2877524,audit_epoch,93,1,audit/paper_guess_steps,20.0
+1773812551.2877524,audit_epoch,93,1,audit/loss/auc,0.505664
+1773812551.2877524,audit_epoch,93,1,audit/loss/empirical_epsilon/0.05,0.0
+1773812551.2877524,audit_epoch,93,1,audit/loss/empirical_epsilon/0.01,0.0
+1773812551.2877524,audit_epoch,93,1,audit/loss/empirical_epsilon_details/0.05/epsilon,0.0
+1773812551.2877524,audit_epoch,93,1,audit/loss/empirical_epsilon_details/0.05/num_guesses,0.0
+1773812551.2877524,audit_epoch,93,1,audit/loss/empirical_epsilon_details/0.05/correct_guesses,0.0
+1773812551.2877524,audit_epoch,93,1,audit/loss/empirical_epsilon_details/0.01/epsilon,0.0
+1773812551.2877524,audit_epoch,93,1,audit/loss/empirical_epsilon_details/0.01/num_guesses,0.0
+1773812551.2877524,audit_epoch,93,1,audit/loss/empirical_epsilon_details/0.01/correct_guesses,0.0
+1773812551.2877524,audit_epoch,93,1,audit/embedding/auc,0.51372
+1773812551.2877524,audit_epoch,93,1,audit/embedding/empirical_epsilon/0.05,0.0
+1773812551.2877524,audit_epoch,93,1,audit/embedding/empirical_epsilon/0.01,0.0
+1773812551.2877524,audit_epoch,93,1,audit/embedding/empirical_epsilon_details/0.05/epsilon,0.0
+1773812551.2877524,audit_epoch,93,1,audit/embedding/empirical_epsilon_details/0.05/num_guesses,0.0
+1773812551.2877524,audit_epoch,93,1,audit/embedding/empirical_epsilon_details/0.05/correct_guesses,0.0
+1773812551.2877524,audit_epoch,93,1,audit/embedding/empirical_epsilon_details/0.01/epsilon,0.0
+1773812551.2877524,audit_epoch,93,1,audit/embedding/empirical_epsilon_details/0.01/num_guesses,0.0
+1773812551.2877524,audit_epoch,93,1,audit/embedding/empirical_epsilon_details/0.01/correct_guesses,0.0
+1773812551.2877524,audit_epoch,93,1,perf/audit_duration_sec,6.93540578102693
+1773812637.181298,train_step,100,2,train/step_loss,2.226203179695237
+1773812637.181298,train_step,100,2,train/step_real_loss,0.9423502832651138
+1773812637.181298,train_step,100,2,train/lr,9.643076661610196e-05
+1773812637.181298,train_step,100,2,train/step_canary_loss,13.964286804199219
+1773812637.181298,train_step,100,2,perf/step_duration_sec,12.210506019182503
+1773812637.181298,train_step,100,2,perf/samples_per_sec,5.814664837678322
+1773812637.181298,train_step,100,2,perf/tokens_per_sec,4380.080556528868
+1773812637.181298,train_step,100,2,perf/logical_batch_size,71.0
+1773812637.181298,train_step,100,2,perf/logical_token_count,53483.0
+1773812637.181298,train_step,100,2,perf/physical_batches,9.0
+1773812637.181298,train_step,100,2,privacy/epsilon,2.176204855129592
+1773812637.181298,train_step,100,2,system/cuda_memory_allocated_gb,14.131910800933838
+1773812637.181298,train_step,100,2,system/cuda_max_memory_allocated_gb,86.22135162353516
+1773812649.9187398,eval_step,100,2,eval/loss,0.9495426886356794
+1773812649.9187398,eval_step,100,2,eval/duration_sec,12.734526461921632
+1773812771.789377,train_step,110,2,train/step_loss,1.4468027822899097
+1773812771.789377,train_step,110,2,train/step_real_loss,1.0408434942364693
+1773812771.789377,train_step,110,2,train/lr,7.874347104470234e-05
+1773812771.789377,train_step,110,2,train/step_canary_loss,14.4375
+1773812771.789377,train_step,110,2,perf/step_duration_sec,12.185235306154937
+1773812771.789377,train_step,110,2,perf/samples_per_sec,5.416391094775368
+1773812771.789377,train_step,110,2,perf/tokens_per_sec,4356.419770834176
+1773812771.789377,train_step,110,2,perf/logical_batch_size,66.0
+1773812771.789377,train_step,110,2,perf/logical_token_count,53084.0
+1773812771.789377,train_step,110,2,perf/physical_batches,9.0
+1773812771.789377,train_step,110,2,privacy/epsilon,2.2837597517001855
+1773812771.789377,train_step,110,2,system/cuda_memory_allocated_gb,12.680593967437744
+1773812771.789377,train_step,110,2,system/cuda_max_memory_allocated_gb,86.22137594223022
+1773812893.198121,train_step,120,2,train/step_loss,1.616717566305132
+1773812893.198121,train_step,120,2,train/step_real_loss,1.0352746099233627
+1773812893.198121,train_step,120,2,train/lr,6.173165676349103e-05
+1773812893.198121,train_step,120,2,train/step_canary_loss,14.020833969116211
+1773812893.198121,train_step,120,2,perf/step_duration_sec,12.073515899013728
+1773812893.198121,train_step,120,2,perf/samples_per_sec,5.549336296105193
+1773812893.198121,train_step,120,2,perf/tokens_per_sec,4502.996513587329
+1773812893.198121,train_step,120,2,perf/logical_batch_size,67.0
+1773812893.198121,train_step,120,2,perf/logical_token_count,54367.0
+1773812893.198121,train_step,120,2,perf/physical_batches,9.0
+1773812893.198121,train_step,120,2,privacy/epsilon,2.387262501265619
+1773812893.198121,train_step,120,2,system/cuda_memory_allocated_gb,12.970857620239258
+1773812893.198121,train_step,120,2,system/cuda_max_memory_allocated_gb,86.22137594223022
+1773813014.0904934,train_step,130,2,train/step_loss,1.9267044274703315
+1773813014.0904934,train_step,130,2,train/step_real_loss,0.9727360382676125
+1773813014.0904934,train_step,130,2,train/lr,4.593591825444028e-05
+1773813014.0904934,train_step,130,2,train/step_canary_loss,14.137499809265137
+1773813014.0904934,train_step,130,2,perf/step_duration_sec,11.957372829318047
+1773813014.0904934,train_step,130,2,perf/samples_per_sec,5.770498334786406
+1773813014.0904934,train_step,130,2,perf/tokens_per_sec,4662.479024096767
+1773813014.0904934,train_step,130,2,perf/logical_batch_size,69.0
+1773813014.0904934,train_step,130,2,perf/logical_token_count,55751.0
+1773813014.0904934,train_step,130,2,perf/physical_batches,9.0
+1773813014.0904934,train_step,130,2,privacy/epsilon,2.4870228609511393
+1773813014.0904934,train_step,130,2,system/cuda_memory_allocated_gb,13.55138349533081
+1773813014.0904934,train_step,130,2,system/cuda_max_memory_allocated_gb,86.22137594223022
+1773813135.2851143,train_step,140,2,train/step_loss,1.6469186313116728
+1773813135.2851143,train_step,140,2,train/step_real_loss,1.0473601296544075
+1773813135.2851143,train_step,140,2,train/lr,3.185820604061088e-05
+1773813135.2851143,train_step,140,2,train/step_canary_loss,14.4375
+1773813135.2851143,train_step,140,2,perf/step_duration_sec,12.27732455311343
+1773813135.2851143,train_step,140,2,perf/samples_per_sec,5.4572150235296455
+1773813135.2851143,train_step,140,2,perf/tokens_per_sec,4390.940368708353
+1773813135.2851143,train_step,140,2,perf/logical_batch_size,67.0
+1773813135.2851143,train_step,140,2,perf/logical_token_count,53909.0
+1773813135.2851143,train_step,140,2,perf/physical_batches,9.0
+1773813135.2851143,train_step,140,2,privacy/epsilon,2.5836117621736387
+1773813135.2851143,train_step,140,2,system/cuda_memory_allocated_gb,12.970857620239258
+1773813135.2851143,train_step,140,2,system/cuda_max_memory_allocated_gb,86.22137594223022
+1773813257.0352345,train_step,150,2,train/step_loss,2.0192771579908286
+1773813257.0352345,train_step,150,2,train/step_real_loss,1.04129096865654
+1773813257.0352345,train_step,150,2,train/lr,1.994587590756397e-05
+1773813257.0352345,train_step,150,2,train/step_canary_loss,14.537500381469727
+1773813257.0352345,train_step,150,2,perf/step_duration_sec,11.960256013087928
+1773813257.0352345,train_step,150,2,perf/samples_per_sec,5.769107277009316
+1773813257.0352345,train_step,150,2,perf/tokens_per_sec,4232.434485065052
+1773813257.0352345,train_step,150,2,perf/logical_batch_size,69.0
+1773813257.0352345,train_step,150,2,perf/logical_token_count,50621.0
+1773813257.0352345,train_step,150,2,perf/physical_batches,9.0
+1773813257.0352345,train_step,150,2,privacy/epsilon,2.6773301521119843
+1773813257.0352345,train_step,150,2,system/cuda_memory_allocated_gb,13.55138349533081
+1773813257.0352345,train_step,150,2,system/cuda_max_memory_allocated_gb,86.22137594223022
+1773813269.8009667,eval_step,150,2,eval/loss,0.9415916662949781
+1773813269.8009667,eval_step,150,2,eval/duration_sec,12.763515894301236
+1773813391.1203394,train_step,160,2,train/step_loss,1.4993842009342078
+1773813391.1203394,train_step,160,2,train/step_real_loss,1.1243649572134018
+1773813391.1203394,train_step,160,2,train/lr,1.057747301402887e-05
+1773813391.1203394,train_step,160,2,train/step_canary_loss,13.5
+1773813391.1203394,train_step,160,2,perf/step_duration_sec,12.41904565365985
+1773813391.1203394,train_step,160,2,perf/samples_per_sec,5.314418018952207
+1773813391.1203394,train_step,160,2,perf/tokens_per_sec,3556.311912500654
+1773813391.1203394,train_step,160,2,perf/logical_batch_size,66.0
+1773813391.1203394,train_step,160,2,perf/logical_token_count,44166.0
+1773813391.1203394,train_step,160,2,perf/physical_batches,9.0
+1773813391.1203394,train_step,160,2,privacy/epsilon,2.768504028383432
+1773813391.1203394,train_step,160,2,system/cuda_memory_allocated_gb,12.680593967437744
+1773813391.1203394,train_step,160,2,system/cuda_max_memory_allocated_gb,86.22137594223022
+1773813513.5407782,train_step,170,2,train/step_loss,1.6632740604343699
+1773813513.5407782,train_step,170,2,train/step_real_loss,1.0752243772149086
+1773813513.5407782,train_step,170,2,train/lr,4.050702638550275e-06
+1773813513.5407782,train_step,170,2,train/step_canary_loss,14.208333969116211
+1773813513.5407782,train_step,170,2,perf/step_duration_sec,12.597235643770546
+1773813513.5407782,train_step,170,2,perf/samples_per_sec,5.318627188904904
+1773813513.5407782,train_step,170,2,perf/tokens_per_sec,3936.101649771056
+1773813513.5407782,train_step,170,2,perf/logical_batch_size,67.0
+1773813513.5407782,train_step,170,2,perf/logical_token_count,49584.0
+1773813513.5407782,train_step,170,2,perf/physical_batches,9.0
+1773813513.5407782,train_step,170,2,privacy/epsilon,2.8571453531213207
+1773813513.5407782,train_step,170,2,system/cuda_memory_allocated_gb,12.970857620239258
+1773813513.5407782,train_step,170,2,system/cuda_max_memory_allocated_gb,86.22137594223022
+1773813635.9173324,train_step,180,2,train/step_loss,1.7563851160161636
+1773813635.9173324,train_step,180,2,train/step_real_loss,1.0712373107671738
+1773813635.9173324,train_step,180,2,train/lr,5.729698228102653e-07
+1773813635.9173324,train_step,180,2,train/step_canary_loss,12.71875
+1773813635.9173324,train_step,180,2,perf/step_duration_sec,12.466372530907393
+1773813635.9173324,train_step,180,2,perf/samples_per_sec,5.454674150913608
+1773813635.9173324,train_step,180,2,perf/tokens_per_sec,4245.501236930199
+1773813635.9173324,train_step,180,2,perf/logical_batch_size,68.0
+1773813635.9173324,train_step,180,2,perf/logical_token_count,52926.0
+1773813635.9173324,train_step,180,2,perf/physical_batches,9.0
+1773813635.9173324,train_step,180,2,privacy/epsilon,2.9437684274367397
+1773813635.9173324,train_step,180,2,system/cuda_memory_allocated_gb,13.261121273040771
+1773813635.9173324,train_step,180,2,system/cuda_max_memory_allocated_gb,86.22137594223022
+1773813716.1885753,train_epoch,186,2,train/epoch_loss,1.6068291228622247
+1773813716.1885753,train_epoch,186,2,train/epoch_real_loss,1.0283400972633716
+1773813716.1885753,train_epoch,186,2,train/epoch_canary_loss,13.124244338677878
+1773813716.1885753,train_epoch,186,2,perf/epoch_duration_sec,1152.0965769039467
+1773813716.1885753,train_epoch,186,2,perf/epoch_samples_per_sec,42.999867366267054
+1773813716.1885753,train_epoch,186,2,perf/epoch_tokens_per_sec,32782.04514893617
+1773813716.1885753,train_epoch,186,2,perf/epoch_samples,49540.0
+1773813716.1885753,train_epoch,186,2,perf/epoch_tokens,37768082.0
+1773813716.1885753,train_epoch,186,2,system/cuda_epoch_peak_memory_gb,86.22137594223022
+1773813716.1885753,train_epoch,186,2,eval/loss,0.9410304912389854
+1773813716.1885753,train_epoch,186,2,eval/duration_sec,12.763936698902398
+1773813716.1885753,train_epoch,186,2,privacy/epsilon,2.9947529815620726
+1773813729.4597642,audit_epoch,186,2,audit/delta,1e-05
+1773813729.4597642,audit_epoch,186,2,audit/num_canaries,500.0
+1773813729.4597642,audit_epoch,186,2,audit/num_members,250.0
+1773813729.4597642,audit_epoch,186,2,audit/paper_guess_fraction,0.2
+1773813729.4597642,audit_epoch,186,2,audit/paper_guess_steps,20.0
+1773813729.4597642,audit_epoch,186,2,audit/loss/auc,0.504792
+1773813729.4597642,audit_epoch,186,2,audit/loss/empirical_epsilon/0.05,0.0
+1773813729.4597642,audit_epoch,186,2,audit/loss/empirical_epsilon/0.01,0.0
+1773813729.4597642,audit_epoch,186,2,audit/loss/empirical_epsilon_details/0.05/epsilon,0.0
+1773813729.4597642,audit_epoch,186,2,audit/loss/empirical_epsilon_details/0.05/num_guesses,0.0
+1773813729.4597642,audit_epoch,186,2,audit/loss/empirical_epsilon_details/0.05/correct_guesses,0.0
+1773813729.4597642,audit_epoch,186,2,audit/loss/empirical_epsilon_details/0.01/epsilon,0.0
+1773813729.4597642,audit_epoch,186,2,audit/loss/empirical_epsilon_details/0.01/num_guesses,0.0
+1773813729.4597642,audit_epoch,186,2,audit/loss/empirical_epsilon_details/0.01/correct_guesses,0.0
+1773813729.4597642,audit_epoch,186,2,audit/embedding/auc,0.51504
+1773813729.4597642,audit_epoch,186,2,audit/embedding/empirical_epsilon/0.05,0.0
+1773813729.4597642,audit_epoch,186,2,audit/embedding/empirical_epsilon/0.01,0.0
+1773813729.4597642,audit_epoch,186,2,audit/embedding/empirical_epsilon_details/0.05/epsilon,0.0
+1773813729.4597642,audit_epoch,186,2,audit/embedding/empirical_epsilon_details/0.05/num_guesses,0.0
+1773813729.4597642,audit_epoch,186,2,audit/embedding/empirical_epsilon_details/0.05/correct_guesses,0.0
+1773813729.4597642,audit_epoch,186,2,audit/embedding/empirical_epsilon_details/0.01/epsilon,0.0
+1773813729.4597642,audit_epoch,186,2,audit/embedding/empirical_epsilon_details/0.01/num_guesses,0.0
+1773813729.4597642,audit_epoch,186,2,audit/embedding/empirical_epsilon_details/0.01/correct_guesses,0.0
+1773813729.4597642,audit_epoch,186,2,perf/audit_duration_sec,7.0606719348579645
+1773813742.732494,audit_final,186,2,audit/delta,1e-05
+1773813742.732494,audit_final,186,2,audit/num_canaries,500.0
+1773813742.732494,audit_final,186,2,audit/num_members,250.0
+1773813742.732494,audit_final,186,2,audit/paper_guess_fraction,0.2
+1773813742.732494,audit_final,186,2,audit/paper_guess_steps,20.0
+1773813742.732494,audit_final,186,2,audit/loss/auc,0.504792
+1773813742.732494,audit_final,186,2,audit/loss/empirical_epsilon/0.05,0.0
+1773813742.732494,audit_final,186,2,audit/loss/empirical_epsilon/0.01,0.0
+1773813742.732494,audit_final,186,2,audit/loss/empirical_epsilon_details/0.05/epsilon,0.0
+1773813742.732494,audit_final,186,2,audit/loss/empirical_epsilon_details/0.05/num_guesses,0.0
+1773813742.732494,audit_final,186,2,audit/loss/empirical_epsilon_details/0.05/correct_guesses,0.0
+1773813742.732494,audit_final,186,2,audit/loss/empirical_epsilon_details/0.01/epsilon,0.0
+1773813742.732494,audit_final,186,2,audit/loss/empirical_epsilon_details/0.01/num_guesses,0.0
+1773813742.732494,audit_final,186,2,audit/loss/empirical_epsilon_details/0.01/correct_guesses,0.0
+1773813742.732494,audit_final,186,2,audit/embedding/auc,0.51504
+1773813742.732494,audit_final,186,2,audit/embedding/empirical_epsilon/0.05,0.0
+1773813742.732494,audit_final,186,2,audit/embedding/empirical_epsilon/0.01,0.0
+1773813742.732494,audit_final,186,2,audit/embedding/empirical_epsilon_details/0.05/epsilon,0.0
+1773813742.732494,audit_final,186,2,audit/embedding/empirical_epsilon_details/0.05/num_guesses,0.0
+1773813742.732494,audit_final,186,2,audit/embedding/empirical_epsilon_details/0.05/correct_guesses,0.0
+1773813742.732494,audit_final,186,2,audit/embedding/empirical_epsilon_details/0.01/epsilon,0.0
+1773813742.732494,audit_final,186,2,audit/embedding/empirical_epsilon_details/0.01/num_guesses,0.0
+1773813742.732494,audit_final,186,2,audit/embedding/empirical_epsilon_details/0.01/correct_guesses,0.0
+1773813743.2978687,energy_final,186,,energy/codecarbon/duration,2457.6702942838892
+1773813743.2978687,energy_final,186,,energy/codecarbon/emissions,0.09496692637153915
+1773813743.2978687,energy_final,186,,energy/codecarbon/emissions_rate,3.8641036021965845e-05
+1773813743.2978687,energy_final,186,,energy/codecarbon/cpu_power,72.03190690693239
+1773813743.2978687,energy_final,186,,energy/codecarbon/gpu_power,3303.243429373142
+1773813743.2978687,energy_final,186,,energy/codecarbon/ram_power,54.0
+1773813743.2978687,energy_final,186,,energy/codecarbon/cpu_energy,0.04736684855609284
+1773813743.2978687,energy_final,186,,energy/codecarbon/gpu_energy,2.25075239893377
+1773813743.2978687,energy_final,186,,energy/codecarbon/ram_energy,0.03550715308856684
+1773813743.2978687,energy_final,186,,energy/codecarbon/energy_consumed,2.3336264005784284
+1773813743.2978687,energy_final,186,,energy/codecarbon/water_consumed,0.0
+1773813743.2978687,energy_final,186,,energy/codecarbon/cpu_count,256.0
+1773813743.2978687,energy_final,186,,energy/codecarbon/gpu_count,8.0
+1773813743.2978687,energy_final,186,,energy/codecarbon/longitude,16.1885
+1773813743.2978687,energy_final,186,,energy/codecarbon/latitude,58.594
+1773813743.2978687,energy_final,186,,energy/codecarbon/ram_total_size,1511.49019241333
+1773813743.2978687,energy_final,186,,energy/codecarbon/cpu_utilization_percent,3.8756856324191915
+1773813743.2978687,energy_final,186,,energy/codecarbon/gpu_utilization_percent,94.07749795249795
+1773813743.2978687,energy_final,186,,energy/codecarbon/ram_utilization_percent,5.391281211624831
+1773813743.2978687,energy_final,186,,energy/codecarbon/ram_used_gb,81.57497969388083
+1773813743.2978687,energy_final,186,,energy/codecarbon/pue,1.0
+1773813743.2978687,energy_final,186,,energy/codecarbon/wue,0.0

qwen3-4b-instruct/dp3/summary.json

@@ -0,0 +1,72 @@
+{
+  "audit/delta": 1e-05,
+  "audit/embedding/auc": 0.51504,
+  "audit/embedding/empirical_epsilon/0.01": 0.0,
+  "audit/embedding/empirical_epsilon/0.05": 0.0,
+  "audit/embedding/empirical_epsilon_details/0.01/correct_guesses": 0.0,
+  "audit/embedding/empirical_epsilon_details/0.01/epsilon": 0.0,
+  "audit/embedding/empirical_epsilon_details/0.01/num_guesses": 0.0,
+  "audit/embedding/empirical_epsilon_details/0.05/correct_guesses": 0.0,
+  "audit/embedding/empirical_epsilon_details/0.05/epsilon": 0.0,
+  "audit/embedding/empirical_epsilon_details/0.05/num_guesses": 0.0,
+  "audit/loss/auc": 0.504792,
+  "audit/loss/empirical_epsilon/0.01": 0.0,
+  "audit/loss/empirical_epsilon/0.05": 0.0,
+  "audit/loss/empirical_epsilon_details/0.01/correct_guesses": 0.0,
+  "audit/loss/empirical_epsilon_details/0.01/epsilon": 0.0,
+  "audit/loss/empirical_epsilon_details/0.01/num_guesses": 0.0,
+  "audit/loss/empirical_epsilon_details/0.05/correct_guesses": 0.0,
+  "audit/loss/empirical_epsilon_details/0.05/epsilon": 0.0,
+  "audit/loss/empirical_epsilon_details/0.05/num_guesses": 0.0,
+  "audit/num_canaries": 500.0,
+  "audit/num_members": 250.0,
+  "audit/paper_guess_fraction": 0.2,
+  "audit/paper_guess_steps": 20.0,
+  "energy/codecarbon/cpu_count": 256.0,
+  "energy/codecarbon/cpu_energy": 0.04736684855609284,
+  "energy/codecarbon/cpu_power": 72.03190690693239,
+  "energy/codecarbon/cpu_utilization_percent": 3.8756856324191915,
+  "energy/codecarbon/duration": 2457.6702942838892,
+  "energy/codecarbon/emissions": 0.09496692637153915,
+  "energy/codecarbon/emissions_rate": 3.8641036021965845e-05,
+  "energy/codecarbon/energy_consumed": 2.3336264005784284,
+  "energy/codecarbon/gpu_count": 8.0,
+  "energy/codecarbon/gpu_energy": 2.25075239893377,
+  "energy/codecarbon/gpu_power": 3303.243429373142,
+  "energy/codecarbon/gpu_utilization_percent": 94.07749795249795,
+  "energy/codecarbon/latitude": 58.594,
+  "energy/codecarbon/longitude": 16.1885,
+  "energy/codecarbon/pue": 1.0,
+  "energy/codecarbon/ram_energy": 0.03550715308856684,
+  "energy/codecarbon/ram_power": 54.0,
+  "energy/codecarbon/ram_total_size": 1511.49019241333,
+  "energy/codecarbon/ram_used_gb": 81.57497969388083,
+  "energy/codecarbon/ram_utilization_percent": 5.391281211624831,
+  "energy/codecarbon/water_consumed": 0.0,
+  "energy/codecarbon/wue": 0.0,
+  "eval/duration_sec": 12.763936698902398,
+  "eval/loss": 0.9410304912389854,
+  "perf/audit_duration_sec": 7.0606719348579645,
+  "perf/epoch_duration_sec": 1152.0965769039467,
+  "perf/epoch_samples": 49540.0,
+  "perf/epoch_samples_per_sec": 42.999867366267054,
+  "perf/epoch_tokens": 37768082.0,
+  "perf/epoch_tokens_per_sec": 32782.04514893617,
+  "perf/logical_batch_size": 68.0,
+  "perf/logical_token_count": 52926.0,
+  "perf/physical_batches": 9.0,
+  "perf/samples_per_sec": 5.454674150913608,
+  "perf/step_duration_sec": 12.466372530907393,
+  "perf/tokens_per_sec": 4245.501236930199,
+  "privacy/epsilon": 2.9947529815620726,
+  "system/cuda_epoch_peak_memory_gb": 86.22137594223022,
+  "system/cuda_max_memory_allocated_gb": 86.22137594223022,
+  "system/cuda_memory_allocated_gb": 13.261121273040771,
+  "train/epoch_canary_loss": 13.124244338677878,
+  "train/epoch_loss": 1.6068291228622247,
+  "train/epoch_real_loss": 1.0283400972633716,
+  "train/lr": 5.729698228102653e-07,
+  "train/step_canary_loss": 12.71875,
+  "train/step_loss": 1.7563851160161636,
+  "train/step_real_loss": 1.0712373107671738
+}

qwen3-4b-instruct/dp3/tensorboard/events.out.tfevents.1773811283.7b654b6988b0.3379.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf62293c09b26e3b14cebd16fd5677f0089fba642744c9d5e6c8b124fd776594
+size 25026

qwen3-4b-instruct/dp3/tokenizer/chat_template.jinja

@@ -0,0 +1,61 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if message.content is string %}
+        {%- set content = message.content %}
+    {%- else %}
+        {%- set content = '' %}
+    {%- endif %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

qwen3-4b-instruct/dp3/tokenizer/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0e9c8aef460c70c1e1c32afe895f455856c0075e5706f06e6d80b2f581137715
+size 11517150

qwen3-4b-instruct/dp3/tokenizer/tokenizer_config.json

@@ -0,0 +1,516 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "865331112869",
+    "569765693871",
+    "485177821815",
+    "135441121756",
+    "367459894796",
+    "877482678543",
+    "457919547633",
+    "765474393376",
+    "114848338811",
+    "746285987371",
+    "649291669397",
+    "927914615679",
+    "445925149649",
+    "691587454538",
+    "143777992227",
+    "997981281989",
+    "425949483533",
+    "982993456429",
+    "718726519731",
+    "172599315861",
+    "643489267333",
+    "282322838685",
+    "781653545886",
+    "796415361892",
+    "841991688488",
+    "211411365397",
+    "698218415444",
+    "355977139358",
+    "682564697312",
+    "383837596997",
+    "689362171782",
+    "749966767285",
+    "753159165157",
+    "795693824762",
+    "669689115557",
+    "327491773134",
+    "983569279932",
+    "612128769512",
+    "374327157578",
+    "311632789559",
+    "523918658846",
+    "765981581453",
+    "794825141891",
+    "873898736873",
+    "447445629421",
+    "473822473819",
+    "181439694557",
+    "592538279337",
+    "668134915514",
+    "643692393748",
+    "696651276628",
+    "853859348234",
+    "778466723723",
+    "929826356991",
+    "272362973463",
+    "694235616268",
+    "281673864127",
+    "479676316326",
+    "646979124677",
+    "922327493433",
+    "883685933161",
+    "264259917554",
+    "836746273134",
+    "658481324922",
+    "481884157827",
+    "587787496812",
+    "579184949249",
+    "912193598348",
+    "529679678956",
+    "795838284624",
+    "159337222655",
+    "173781362446",
+    "773687856563",
+    "535787224917",
+    "351885857332",
+    "578827344666",
+    "198462689911",
+    "722618266242",
+    "952872416512",
+    "517778845323",
+    "749665846687",
+    "661436365453",
+    "259666844669",
+    "242851284913",
+    "514532995959",
+    "161588262349",
+    "742765629356",
+    "225164373623",
+    "676539973863",
+    "826214551218",
+    "182345464792",
+    "232776999554",
+    "337326533813",
+    "676676697292",
+    "929185622831",
+    "545512344383",
+    "499444466686",
+    "314697386682",
+    "517379856925",
+    "379557332953",
+    "614797267726",
+    "429781429464",
+    "922466849763",
+    "721737645236",
+    "479227349997",
+    "136931728327",
+    "259533577263",
+    "488538864842",
+    "937495658852",
+    "489991411364",
+    "499148455254",
+    "441373944925",
+    "899151413682",
+    "467893531755",
+    "527117488925",
+    "928335588653",
+    "374439448821",
+    "879425227932",
+    "867678158885",
+    "399749397872",
+    "129693547287",
+    "689285841825",
+    "771619544974",
+    "724883568652",
+    "516968424863",
+    "733737988257",
+    "852347289392",
+    "296953381169",
+    "377273562477",
+    "262296912232",
+    "547149832394",
+    "298464134954",
+    "216667245274",
+    "843998562287",
+    "572154333646",
+    "124589118494",
+    "841824384614",
+    "232896526252",
+    "295448593321",
+    "123741461297",
+    "653573457168",
+    "196735786156",
+    "377338713663",
+    "964342468552",
+    "586855179568",
+    "484773717614",
+    "894885246797",
+    "677896358599",
+    "848845611563",
+    "851852651677",
+    "398549545767",
+    "454244839926",
+    "799364566435",
+    "967114116556",
+    "817378986438",
+    "233795848681",
+    "824387273757",
+    "916198946615",
+    "563117729724",
+    "951794811935",
+    "374598961236",
+    "922867396683",
+    "765737843639",
+    "175469284871",
+    "231853711778",
+    "662426712668",
+    "711412347158",
+    "753466987363",
+    "513361312532",
+    "712992815957",
+    "971621888444",
+    "829235161526",
+    "585544633356",
+    "582471228164",
+    "678666359123",
+    "557533689478",
+    "632962475133",
+    "484489193824",
+    "489562189822",
+    "589547936288",
+    "363214487524",
+    "244885399387",
+    "431751228368",
+    "433581868192",
+    "486391569221",
+    "185438575221",
+    "126574388585",
+    "741757479784",
+    "529854679937",
+    "996116119839",
+    "616248973917",
+    "763531783491",
+    "955456118295",
+    "364196983365",
+    "195792996468",
+    "151859598873",
+    "399223169721",
+    "938488813964",
+    "961981959227",
+    "183368827562",
+    "533417736566",
+    "786391632558",
+    "665661658354",
+    "693281533643",
+    "475794684356",
+    "652154162978",
+    "753233719644",
+    "668514843129",
+    "819162623892",
+    "941169431859",
+    "877385381798",
+    "752644929761",
+    "881136466196",
+    "275597777299",
+    "731681792655",
+    "961133895172",
+    "864718285734",
+    "963852916563",
+    "319584985416",
+    "563365646341",
+    "811371928234",
+    "837131396371",
+    "267514771964",
+    "944513428457",
+    "117298239631",
+    "158142752582",
+    "252867443568",
+    "839269684865",
+    "612788593128",
+    "145669731981",
+    "121557291859",
+    "245416776926",
+    "799417897197",
+    "997958836435",
+    "892336777248",
+    "158929292238",
+    "581976444672",
+    "897784492783",
+    "492373714791",
+    "512659818733",
+    "881112998642",
+    "619454958782",
+    "431149748713",
+    "624221476921",
+    "125866399464",
+    "339882449689",
+    "186198784585",
+    "943193294691",
+    "955668961269",
+    "232787996724",
+    "215671314196",
+    "286173241916",
+    "745977673725",
+    "556976448182",
+    "599961512792",
+    "766294538337",
+    "934912591213",
+    "295118729589",
+    "529455466433",
+    "196119929397",
+    "379571934299",
+    "251789649997",
+    "564544131355",
+    "244371196654",
+    "384598329253",
+    "887753195844",
+    "364947325679",
+    "655517954651",
+    "673948786567",
+    "857231548835",
+    "816115936673",
+    "644234165531",
+    "182782912224",
+    "234316622259",
+    "421369185549",
+    "434632855397",
+    "921889371893",
+    "415956914763",
+    "598916996413",
+    "773671349113",
+    "952465217972",
+    "117657531962",
+    "729825168745",
+    "691315125346",
+    "768461952319",
+    "664847713559",
+    "953267689786",
+    "886464195129",
+    "824488329416",
+    "837873762491",
+    "532833541879",
+    "669183782449",
+    "941976537588",
+    "739394546916",
+    "267954879268",
+    "637551427887",
+    "217756494954",
+    "524444658383",
+    "117783274348",
+    "138218735276",
+    "814611949491",
+    "711641973413",
+    "499156317423",
+    "515856611931",
+    "454164859837",
+    "345271433112",
+    "462294118988",
+    "511785788222",
+    "497294727353",
+    "866519986723",
+    "334513529294",
+    "549946382131",
+    "284445431422",
+    "396521188476",
+    "421435255895",
+    "133373659361",
+    "322683334381",
+    "228358422847",
+    "291762694874",
+    "143182978129",
+    "511923256573",
+    "327158398268",
+    "879764613759",
+    "564395222747",
+    "451161679736",
+    "538631466654",
+    "221762325616",
+    "218391991184",
+    "322589379462",
+    "876537814263",
+    "152676556624",
+    "332522971941",
+    "884354318946",
+    "513349618943",
+    "116639746413",
+    "635185846287",
+    "993832498489",
+    "813981174797",
+    "438745114173",
+    "983493951323",
+    "724492262421",
+    "622553389126",
+    "889965243135",
+    "364492359246",
+    "154962668224",
+    "179564995814",
+    "418412875665",
+    "718951851413",
+    "699446724178",
+    "624266421831",
+    "815458725125",
+    "455423278865",
+    "393741199486",
+    "328552864359",
+    "211662639865",
+    "218784516525",
+    "762486672996",
+    "142799718159",
+    "858146415154",
+    "767858144912",
+    "571317457151",
+    "635127952696",
+    "116427191984",
+    "268921994538",
+    "523937669294",
+    "165429152138",
+    "739246183345",
+    "591464355756",
+    "212985874612",
+    "191887635211",
+    "967214577653",
+    "119342152414",
+    "946444632795",
+    "618423867817",
+    "228565148417",
+    "729116422489",
+    "527874729936",
+    "739784153482",
+    "387763951128",
+    "331369926711",
+    "562716493614",
+    "739667844957",
+    "562389434565",
+    "256497188281",
+    "859927364588",
+    "417668946583",
+    "357621613582",
+    "438435178228",
+    "485692541169",
+    "825815739116",
+    "342221452223",
+    "697747991249",
+    "716763689965",
+    "141499982867",
+    "818479319499",
+    "336813343298",
+    "594688742928",
+    "472129283475",
+    "514354144759",
+    "349249721685",
+    "546276298359",
+    "353755529131",
+    "315534574435",
+    "523723475786",
+    "215826764872",
+    "367968398551",
+    "569853653352",
+    "389715484387",
+    "293847485454",
+    "714738141818",
+    "178478368922",
+    "581493616981",
+    "589439538674",
+    "846657726193",
+    "722339992679",
+    "138154781148",
+    "757785319772",
+    "492516914298",
+    "919181521716",
+    "985781138935",
+    "476969195485",
+    "313145133463",
+    "758963111966",
+    "147541537162",
+    "557163366873",
+    "144373897488",
+    "522515164754",
+    "724964923582",
+    "284776712475",
+    "375429755114",
+    "181233596124",
+    "948585673431",
+    "243165586174",
+    "396847976144",
+    "997724962668",
+    "558837194455",
+    "163165456396",
+    "378749551722",
+    "161238482259",
+    "754978243758",
+    "195388849133",
+    "229775525672",
+    "262437452884",
+    "441377892146",
+    "451885565366",
+    "981277526855",
+    "762495822823",
+    "368763327262",
+    "757422791351",
+    "636324136426",
+    "214193645583",
+    "412843856172",
+    "179386156569",
+    "756916173536",
+    "892697125149",
+    "625334487352",
+    "941861857715",
+    "887417525236",
+    "649516938598",
+    "717628619782",
+    "438124184139",
+    "547563892268",
+    "856317483891",
+    "313313831273",
+    "371496153876",
+    "587541149322",
+    "265847332563",
+    "449549215429",
+    "163497196769",
+    "861342291298",
+    "268433315926",
+    "774679513717",
+    "851254219729",
+    "583527834464",
+    "488496781997",
+    "556814553861",
+    "482829231639",
+    "618878266619",
+    "147444452794",
+    "949235426629",
+    "357299947518",
+    "175528632226",
+    "645527857972",
+    "186872457894",
+    "552738847828",
+    "626748382482",
+    "921894985642",
+    "943878645871",
+    "859289776479",
+    "614583493135",
+    "933775286797",
+    "332234613346",
+    "325196781219",
+    "142526557681",
+    "356722692178",
+    "449318681694",
+    "687284547244",
+    "947262995132",
+    "893974619684",
+    "797238311233"
+  ],
+  "is_local": false,
+  "model_max_length": 1010000,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

qwen3-4b-instruct/dp3/train.log

@@ -0,0 +1,21 @@
+2026-03-18 05:25:06,469 [INFO] new_opacus_codex.train_steps: epoch=1 step=10 loss=1.6915
+2026-03-18 05:27:08,065 [INFO] new_opacus_codex.train_steps: epoch=1 step=20 loss=1.6692
+2026-03-18 05:29:10,619 [INFO] new_opacus_codex.train_steps: epoch=1 step=30 loss=1.7137
+2026-03-18 05:31:12,467 [INFO] new_opacus_codex.train_steps: epoch=1 step=40 loss=1.6041
+2026-03-18 05:33:14,135 [INFO] new_opacus_codex.train_steps: epoch=1 step=50 loss=1.6079
+2026-03-18 05:33:26,911 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=50 eval_loss=0.9748 duration_sec=12.77
+2026-03-18 05:35:28,635 [INFO] new_opacus_codex.train_steps: epoch=1 step=60 loss=1.5953
+2026-03-18 05:37:31,067 [INFO] new_opacus_codex.train_steps: epoch=1 step=70 loss=1.7617
+2026-03-18 05:39:33,024 [INFO] new_opacus_codex.train_steps: epoch=1 step=80 loss=1.7465
+2026-03-18 05:41:35,477 [INFO] new_opacus_codex.train_steps: epoch=1 step=90 loss=1.6112
+2026-03-18 05:43:57,180 [INFO] new_opacus_codex.train_steps: epoch=2 step=100 loss=1.7602
+2026-03-18 05:44:09,918 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=100 eval_loss=0.9495 duration_sec=12.73
+2026-03-18 05:46:11,788 [INFO] new_opacus_codex.train_steps: epoch=2 step=110 loss=1.7710
+2026-03-18 05:48:13,197 [INFO] new_opacus_codex.train_steps: epoch=2 step=120 loss=1.5877
+2026-03-18 05:50:14,090 [INFO] new_opacus_codex.train_steps: epoch=2 step=130 loss=1.5471
+2026-03-18 05:52:15,284 [INFO] new_opacus_codex.train_steps: epoch=2 step=140 loss=1.7275
+2026-03-18 05:54:17,034 [INFO] new_opacus_codex.train_steps: epoch=2 step=150 loss=1.5942
+2026-03-18 05:54:29,800 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=150 eval_loss=0.9416 duration_sec=12.76
+2026-03-18 05:56:31,119 [INFO] new_opacus_codex.train_steps: epoch=2 step=160 loss=1.3974
+2026-03-18 05:58:33,540 [INFO] new_opacus_codex.train_steps: epoch=2 step=170 loss=1.6737
+2026-03-18 06:00:35,917 [INFO] new_opacus_codex.train_steps: epoch=2 step=180 loss=1.6957