Add files using upload-large-folder tool

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+qwen3-4b-instruct/base/tokenizer/tokenizer.json filter=lfs diff=lfs merge=lfs -text

qwen3-4b-instruct/base/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: Qwen/Qwen3-4B-Instruct-2507
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3-4B-Instruct-2507
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

qwen3-4b-instruct/base/adapter/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-4B-Instruct-2507",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": true,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "lm_head",
+    "embed_tokens"
+  ],
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "k_proj",
+    "v_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

qwen3-4b-instruct/base/adapter/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:951990d4c34c593ed5c2777e6ebe986acc6b01f8038c050c69b3cd13c8dbc3af
+size 4721857072

qwen3-4b-instruct/base/audit_results.json

@@ -0,0 +1,137 @@
+{
+  "delta": 1e-05,
+  "num_canaries": 500,
+  "num_members": 250,
+  "paper_guess_fraction": 0.2,
+  "paper_guess_steps": 20,
+  "loss": {
+    "auc": 0.968584,
+    "empirical_epsilon": {
+      "0.05": 3.4791953936219215,
+      "0.01": 3.023197554051876
+    },
+    "empirical_epsilon_details": {
+      "0.05": {
+        "epsilon": 3.4791953936219215,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      },
+      "0.01": {
+        "epsilon": 3.023197554051876,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      }
+    }
+  },
+  "embedding": {
+    "auc": 0.883776,
+    "empirical_epsilon": {
+      "0.05": 3.4791953936219215,
+      "0.01": 3.023197554051876
+    },
+    "empirical_epsilon_details": {
+      "0.05": {
+        "epsilon": 3.4791953936219215,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      },
+      "0.01": {
+        "epsilon": 3.023197554051876,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      }
+    }
+  }
+}

qwen3-4b-instruct/base/audit_scores.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cd3dd9fd37e8fbc94a3c62416c334e11928ed1d4d26de64dd88997b4246b9fcb
+size 12784

qwen3-4b-instruct/base/codecarbon.csv

@@ -0,0 +1,2 @@
+timestamp,project_name,run_id,experiment_id,duration,emissions,emissions_rate,cpu_power,gpu_power,ram_power,cpu_energy,gpu_energy,ram_energy,energy_consumed,water_consumed,country_name,country_iso_code,region,cloud_provider,cloud_region,os,python_version,codecarbon_version,cpu_count,cpu_model,gpu_count,gpu_model,longitude,latitude,ram_total_size,tracking_mode,cpu_utilization_percent,gpu_utilization_percent,ram_utilization_percent,ram_used_gb,on_cloud,pue,wue
+2026-03-17T16:14:46,codedp-qwen3-4b-instruct-cpt-base,c4455506-be9b-4ebf-9411-44a1ca74c256,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,2345.9966679112986,0.09022432714096462,3.8458847096868924e-05,72.02285277932866,3280.290622412428,54.0,0.045218986505725985,2.137964725370466,0.03390259211605879,2.2170863039922497,0.0,Sweden,SWE,östergötland county,,,Linux-6.8.0-94-generic-x86_64-with-glibc2.39,3.11.0,3.2.3,256,AMD EPYC 9554 64-Core Processor,8,8 x NVIDIA H200,16.1885,58.594,1511.49019241333,machine,3.3142796066695253,88.58721675929884,5.287772552372644,79.7947571596665,N,1.0,0.0

qwen3-4b-instruct/base/epochs/epoch_001/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: Qwen/Qwen3-4B-Instruct-2507
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3-4B-Instruct-2507
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

qwen3-4b-instruct/base/epochs/epoch_001/adapter/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-4B-Instruct-2507",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": true,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "lm_head",
+    "embed_tokens"
+  ],
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "k_proj",
+    "v_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

qwen3-4b-instruct/base/epochs/epoch_001/adapter/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6d15fef335597efc6c9627b295490c7398f384f70071dc4ff04661f4776a9e5f
+size 4721857072

qwen3-4b-instruct/base/epochs/epoch_001/audit_results.json

@@ -0,0 +1,137 @@
+{
+  "delta": 1e-05,
+  "num_canaries": 500,
+  "num_members": 250,
+  "paper_guess_fraction": 0.2,
+  "paper_guess_steps": 20,
+  "loss": {
+    "auc": 0.907944,
+    "empirical_epsilon": {
+      "0.05": 3.4791953936219215,
+      "0.01": 3.023197554051876
+    },
+    "empirical_epsilon_details": {
+      "0.05": {
+        "epsilon": 3.4791953936219215,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      },
+      "0.01": {
+        "epsilon": 3.023197554051876,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      }
+    }
+  },
+  "embedding": {
+    "auc": 0.876048,
+    "empirical_epsilon": {
+      "0.05": 3.4791953936219215,
+      "0.01": 3.023197554051876
+    },
+    "empirical_epsilon_details": {
+      "0.05": {
+        "epsilon": 3.4791953936219215,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      },
+      "0.01": {
+        "epsilon": 3.023197554051876,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      }
+    }
+  }
+}

qwen3-4b-instruct/base/epochs/epoch_001/audit_scores.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b450e4b28f68344d4b013d7fd537cafa20bac640bcb9c531ee461b22e316ca0
+size 12784

qwen3-4b-instruct/base/epochs/epoch_002/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: Qwen/Qwen3-4B-Instruct-2507
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3-4B-Instruct-2507
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

qwen3-4b-instruct/base/epochs/epoch_002/adapter/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-4B-Instruct-2507",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": true,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "lm_head",
+    "embed_tokens"
+  ],
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "k_proj",
+    "v_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

qwen3-4b-instruct/base/epochs/epoch_002/adapter/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:951990d4c34c593ed5c2777e6ebe986acc6b01f8038c050c69b3cd13c8dbc3af
+size 4721857072

qwen3-4b-instruct/base/epochs/epoch_002/audit_results.json

@@ -0,0 +1,137 @@
+{
+  "delta": 1e-05,
+  "num_canaries": 500,
+  "num_members": 250,
+  "paper_guess_fraction": 0.2,
+  "paper_guess_steps": 20,
+  "loss": {
+    "auc": 0.968584,
+    "empirical_epsilon": {
+      "0.05": 3.4791953936219215,
+      "0.01": 3.023197554051876
+    },
+    "empirical_epsilon_details": {
+      "0.05": {
+        "epsilon": 3.4791953936219215,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      },
+      "0.01": {
+        "epsilon": 3.023197554051876,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      }
+    }
+  },
+  "embedding": {
+    "auc": 0.883776,
+    "empirical_epsilon": {
+      "0.05": 3.4791953936219215,
+      "0.01": 3.023197554051876
+    },
+    "empirical_epsilon_details": {
+      "0.05": {
+        "epsilon": 3.4791953936219215,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      },
+      "0.01": {
+        "epsilon": 3.023197554051876,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      }
+    }
+  }
+}

qwen3-4b-instruct/base/epochs/epoch_002/audit_scores.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cd3dd9fd37e8fbc94a3c62416c334e11928ed1d4d26de64dd88997b4246b9fcb
+size 12784

qwen3-4b-instruct/base/metrics.jsonl

@@ -0,0 +1,49 @@
+{"timestamp": 1773761894.8111923, "event": "train_step", "step": 10, "epoch": 1, "metrics": {"train/step_loss": 1.8352766107110416, "train/step_real_loss": 1.028106451034546, "train/lr": 5.2631578947368424e-05, "train/step_canary_loss": 14.75, "perf/step_duration_sec": 6.234770041890442, "perf/samples_per_sec": 5.453288536956348, "perf/tokens_per_sec": 3980.098677781523, "perf/logical_batch_size": 34.0, "perf/logical_token_count": 24815.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 94.4762544631958}}
+{"timestamp": 1773761950.7987902, "event": "train_step", "step": 20, "epoch": 1, "metrics": {"train/step_loss": 1.0323970019817352, "train/step_real_loss": 1.0323970019817352, "train/lr": 9.999797424944042e-05, "perf/step_duration_sec": 5.150427320972085, "perf/samples_per_sec": 6.213076703305535, "perf/tokens_per_sec": 4927.9406189561805, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 25381.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 94.4762544631958}}
+{"timestamp": 1773762007.3004794, "event": "train_step", "step": 30, "epoch": 1, "metrics": {"train/step_loss": 0.8551503717899323, "train/step_real_loss": 0.8551503717899323, "train/lr": 9.975508273693644e-05, "perf/step_duration_sec": 5.69609066285193, "perf/samples_per_sec": 5.617888108539722, "perf/tokens_per_sec": 4432.689276641233, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 25249.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 94.4762544631958}}
+{"timestamp": 1773762065.1568909, "event": "train_step", "step": 40, "epoch": 1, "metrics": {"train/step_loss": 0.8950656801462173, "train/step_real_loss": 0.8950656801462173, "train/lr": 9.910929512300672e-05, "perf/step_duration_sec": 6.2338299779221416, "perf/samples_per_sec": 5.133280842328368, "perf/tokens_per_sec": 4016.4714290693023, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 25038.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 94.4762544631958}}
+{"timestamp": 1773762121.2732794, "event": "train_step", "step": 50, "epoch": 1, "metrics": {"train/step_loss": 0.8450518101453781, "train/step_real_loss": 0.8450518101453781, "train/lr": 9.806584072891234e-05, "perf/step_duration_sec": 5.423216213937849, "perf/samples_per_sec": 5.900557664980961, "perf/tokens_per_sec": 5482.724425329497, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 29734.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 94.4762544631958}}
+{"timestamp": 1773762134.073646, "event": "eval_step", "step": 50, "epoch": 1, "metrics": {"eval/loss": 0.8533683589253671, "eval/duration_sec": 12.798461285419762}}
+{"timestamp": 1773762190.2760499, "event": "train_step", "step": 60, "epoch": 1, "metrics": {"train/step_loss": 1.114606170943289, "train/step_real_loss": 0.8427969664335251, "train/lr": 9.663316901718597e-05, "train/step_canary_loss": 9.8125, "perf/step_duration_sec": 5.968041606713086, "perf/samples_per_sec": 5.5294520673046765, "perf/tokens_per_sec": 4259.353683360148, "perf/logical_batch_size": 33.0, "perf/logical_token_count": 25420.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 94.4762544631958}}
+{"timestamp": 1773762249.207255, "event": "train_step", "step": 70, "epoch": 1, "metrics": {"train/step_loss": 1.1624658794114084, "train/step_real_loss": 0.8745741844177246, "train/lr": 9.48228811713756e-05, "train/step_canary_loss": 10.375, "perf/step_duration_sec": 6.334186799824238, "perf/samples_per_sec": 5.209824250986045, "perf/tokens_per_sec": 3829.378697936894, "perf/logical_batch_size": 33.0, "perf/logical_token_count": 24256.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 16.205660820007324, "system/cuda_max_memory_allocated_gb": 94.4762544631958}}
+{"timestamp": 1773762305.719841, "event": "train_step", "step": 80, "epoch": 1, "metrics": {"train/step_loss": 1.157600255573497, "train/step_real_loss": 0.8803409039974213, "train/lr": 9.26496361544538e-05, "train/step_canary_loss": 5.59375, "perf/step_duration_sec": 5.699764240998775, "perf/samples_per_sec": 5.9651590070052, "perf/tokens_per_sec": 4690.720329743854, "perf/logical_batch_size": 34.0, "perf/logical_token_count": 26736.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 16.205660820007324, "system/cuda_max_memory_allocated_gb": 94.4762544631958}}
+{"timestamp": 1773762363.2025864, "event": "train_step", "step": 90, "epoch": 1, "metrics": {"train/step_loss": 0.8746908158063889, "train/step_real_loss": 0.8746908158063889, "train/lr": 9.013103200659241e-05, "perf/step_duration_sec": 5.422750173136592, "perf/samples_per_sec": 5.901064769408466, "perf/tokens_per_sec": 4211.147346069116, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 22836.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 94.4762544631958}}
+{"timestamp": 1773762419.340239, "event": "train_step", "step": 100, "epoch": 1, "metrics": {"train/step_loss": 1.227062124194521, "train/step_real_loss": 0.9060328304767609, "train/lr": 8.728746334350483e-05, "train/step_canary_loss": 11.5, "perf/step_duration_sec": 5.688522285781801, "perf/samples_per_sec": 5.801155087760134, "perf/tokens_per_sec": 4329.068036096396, "perf/logical_batch_size": 33.0, "perf/logical_token_count": 24626.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 16.205660820007324, "system/cuda_max_memory_allocated_gb": 94.4762544631958}}
+{"timestamp": 1773762432.1276581, "event": "eval_step", "step": 100, "epoch": 1, "metrics": {"eval/loss": 0.829470864473245, "eval/duration_sec": 12.785310188308358}}
+{"timestamp": 1773762489.3303852, "event": "train_step", "step": 110, "epoch": 1, "metrics": {"train/step_loss": 1.011215921604272, "train/step_real_loss": 0.9119570553302765, "train/lr": 8.414195620927492e-05, "train/step_canary_loss": 4.1875, "perf/step_duration_sec": 5.6974792359396815, "perf/samples_per_sec": 5.792035149831895, "perf/tokens_per_sec": 4259.603062159705, "perf/logical_batch_size": 33.0, "perf/logical_token_count": 24269.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 94.4762544631958}}
+{"timestamp": 1773762544.9202547, "event": "train_step", "step": 120, "epoch": 1, "metrics": {"train/step_loss": 0.7180200964212418, "train/step_real_loss": 0.7180200964212418, "train/lr": 8.071998162096612e-05, "perf/step_duration_sec": 5.694211829919368, "perf/samples_per_sec": 5.619741758088605, "perf/tokens_per_sec": 4598.00245969612, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 26182.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 94.4762544631958}}
+{"timestamp": 1773762601.86658, "event": "train_step", "step": 130, "epoch": 1, "metrics": {"train/step_loss": 0.8463245183229446, "train/step_real_loss": 0.8463245183229446, "train/lr": 7.704924931484997e-05, "perf/step_duration_sec": 5.1536158989183605, "perf/samples_per_sec": 6.209232629602092, "perf/tokens_per_sec": 4267.683201733388, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 21994.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 101.70386934280396}}
+{"timestamp": 1773762659.5638738, "event": "train_step", "step": 140, "epoch": 1, "metrics": {"train/step_loss": 0.9742496162652969, "train/step_real_loss": 0.9742496162652969, "train/lr": 7.315948336441117e-05, "perf/step_duration_sec": 5.969049285165966, "perf/samples_per_sec": 5.360987733762741, "perf/tokens_per_sec": 3927.091045847888, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 23441.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 101.70386934280396}}
+{"timestamp": 1773762717.6212678, "event": "train_step", "step": 150, "epoch": 1, "metrics": {"train/step_loss": 0.914526179432869, "train/step_real_loss": 0.914526179432869, "train/lr": 6.908218148708247e-05, "perf/step_duration_sec": 5.703813333064318, "perf/samples_per_sec": 5.610281776666824, "perf/tokens_per_sec": 4645.31331108013, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 26496.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 101.70386934280396}}
+{"timestamp": 1773762730.4167268, "event": "eval_step", "step": 150, "epoch": 1, "metrics": {"eval/loss": 0.8174186625923866, "eval/duration_sec": 12.793565314263105}}
+{"timestamp": 1773762785.9700294, "event": "train_step", "step": 160, "epoch": 1, "metrics": {"train/step_loss": 0.9113822728395462, "train/step_real_loss": 0.9113822728395462, "train/lr": 6.485035998874356e-05, "perf/step_duration_sec": 5.425368802621961, "perf/samples_per_sec": 5.898216538668322, "perf/tokens_per_sec": 4679.128907832313, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 25386.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 101.70386934280396}}
+{"timestamp": 1773762841.0786932, "event": "train_step", "step": 170, "epoch": 1, "metrics": {"train/step_loss": 0.8910564256436897, "train/step_real_loss": 0.8441949039697647, "train/lr": 6.049828641131825e-05, "train/step_canary_loss": 2.390625, "perf/step_duration_sec": 5.690398690290749, "perf/samples_per_sec": 5.799242161415912, "perf/tokens_per_sec": 4355.582332445254, "perf/logical_batch_size": 33.0, "perf/logical_token_count": 24785.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 16.205660820007324, "system/cuda_max_memory_allocated_gb": 101.70386934280396}}
+{"timestamp": 1773762898.8441253, "event": "train_step", "step": 180, "epoch": 1, "metrics": {"train/step_loss": 1.0836328773787527, "train/step_real_loss": 0.8538245260715485, "train/lr": 5.6061202048379124e-05, "train/step_canary_loss": 8.4375, "perf/step_duration_sec": 5.692985306028277, "perf/samples_per_sec": 5.7966072677293665, "perf/tokens_per_sec": 4476.7373583439585, "perf/logical_batch_size": 33.0, "perf/logical_token_count": 25486.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 101.70386934280396}}
+{"timestamp": 1773762934.6675656, "event": "train_epoch", "step": 184, "epoch": 1, "metrics": {"train/epoch_loss": 0.9805822407983543, "train/epoch_real_loss": 0.9071368900552877, "train/epoch_canary_loss": 8.42140839386602, "perf/epoch_duration_sec": 1085.6805565529503, "perf/epoch_samples_per_sec": 43.826887856473604, "perf/epoch_tokens_per_sec": 34574.90951038253, "perf/epoch_samples": 47582.0, "perf/epoch_tokens": 37537307.0, "system/cuda_epoch_peak_memory_gb": 101.70386934280396, "eval/loss": 0.8121764394335258, "eval/duration_sec": 12.826699289958924}}
+{"timestamp": 1773762949.1512606, "event": "audit_epoch", "step": 184, "epoch": 1, "metrics": {"audit/delta": 1e-05, "audit/num_canaries": 500.0, "audit/num_members": 250.0, "audit/paper_guess_fraction": 0.2, "audit/paper_guess_steps": 20.0, "audit/loss/auc": 0.907944, "audit/loss/empirical_epsilon/0.05": 3.4791953936219215, "audit/loss/empirical_epsilon/0.01": 3.023197554051876, "audit/loss/empirical_epsilon_details/0.05/epsilon": 3.4791953936219215, "audit/loss/empirical_epsilon_details/0.05/num_guesses": 100.0, "audit/loss/empirical_epsilon_details/0.05/correct_guesses": 100.0, "audit/loss/empirical_epsilon_details/0.01/epsilon": 3.023197554051876, "audit/loss/empirical_epsilon_details/0.01/num_guesses": 100.0, "audit/loss/empirical_epsilon_details/0.01/correct_guesses": 100.0, "audit/embedding/auc": 0.876048, "audit/embedding/empirical_epsilon/0.05": 3.4791953936219215, "audit/embedding/empirical_epsilon/0.01": 3.023197554051876, "audit/embedding/empirical_epsilon_details/0.05/epsilon": 3.4791953936219215, "audit/embedding/empirical_epsilon_details/0.05/num_guesses": 100.0, "audit/embedding/empirical_epsilon_details/0.05/correct_guesses": 100.0, "audit/embedding/empirical_epsilon_details/0.01/epsilon": 3.023197554051876, "audit/embedding/empirical_epsilon_details/0.01/num_guesses": 100.0, "audit/embedding/empirical_epsilon_details/0.01/correct_guesses": 100.0, "perf/audit_duration_sec": 8.130579099990427}}
+{"timestamp": 1773762984.332577, "event": "train_step", "step": 190, "epoch": 2, "metrics": {"train/step_loss": 0.8655764758586884, "train/step_real_loss": 0.8655764758586884, "train/lr": 5.157503657571385e-05, "perf/step_duration_sec": 5.690848938189447, "perf/samples_per_sec": 5.623062630472995, "perf/tokens_per_sec": 4602.301042334944, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 26191.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 87.30217599868774}}
+{"timestamp": 1773763040.7157884, "event": "train_step", "step": 200, "epoch": 2, "metrics": {"train/step_loss": 0.8589679941986547, "train/step_real_loss": 0.8308791071176529, "train/lr": 4.7076117107656534e-05, "train/step_canary_loss": 1.7578125, "perf/step_duration_sec": 5.69332688068971, "perf/samples_per_sec": 5.796259496697344, "perf/tokens_per_sec": 5059.0806752537455, "perf/logical_batch_size": 33.0, "perf/logical_token_count": 28803.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 87.30217599868774}}
+{"timestamp": 1773763053.5208263, "event": "eval_step", "step": 200, "epoch": 2, "metrics": {"eval/loss": 0.8110936123591204, "eval/duration_sec": 12.803085402119905}}
+{"timestamp": 1773763110.1544352, "event": "train_step", "step": 210, "epoch": 2, "metrics": {"train/step_loss": 0.7684839069843292, "train/step_real_loss": 0.7684839069843292, "train/lr": 4.2600874035126046e-05, "perf/step_duration_sec": 5.4159107422456145, "perf/samples_per_sec": 5.908516872405425, "perf/tokens_per_sec": 4630.061534138701, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 25076.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 87.30229806900024}}
+{"timestamp": 1773763168.1105735, "event": "train_step", "step": 220, "epoch": 2, "metrics": {"train/step_loss": 0.8040641099214554, "train/step_real_loss": 0.8040641099214554, "train/lr": 3.818554602737332e-05, "perf/step_duration_sec": 5.967140641994774, "perf/samples_per_sec": 5.36270249351834, "perf/tokens_per_sec": 4430.932935269528, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 26440.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 87.30229806900024}}
+{"timestamp": 1773763224.221138, "event": "train_step", "step": 230, "epoch": 2, "metrics": {"train/step_loss": 0.806927278637886, "train/step_real_loss": 0.806927278637886, "train/lr": 3.386588658621128e-05, "perf/step_duration_sec": 5.424597659613937, "perf/samples_per_sec": 5.899055009782496, "perf/tokens_per_sec": 4482.175734620363, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 24314.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 87.30229806900024}}
+{"timestamp": 1773763282.06351, "event": "train_step", "step": 240, "epoch": 2, "metrics": {"train/step_loss": 0.9124463796615601, "train/step_real_loss": 0.9124463796615601, "train/lr": 2.967687452893051e-05, "perf/step_duration_sec": 5.692487298045307, "perf/samples_per_sec": 5.621444251792743, "perf/tokens_per_sec": 4766.282044988772, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 27132.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 94.47624206542969}}
+{"timestamp": 1773763337.7170568, "event": "train_step", "step": 250, "epoch": 2, "metrics": {"train/step_loss": 0.7962393760681152, "train/step_real_loss": 0.7962393760681152, "train/lr": 2.5652430744289756e-05, "perf/step_duration_sec": 5.419141778722405, "perf/samples_per_sec": 5.904994057480479, "perf/tokens_per_sec": 4963.885629569528, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 26900.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 94.47624206542969}}
+{"timestamp": 1773763350.5270941, "event": "eval_step", "step": 250, "epoch": 2, "metrics": {"eval/loss": 0.8086002505360506, "eval/duration_sec": 12.808165564201772}}
+{"timestamp": 1773763406.908792, "event": "train_step", "step": 260, "epoch": 2, "metrics": {"train/step_loss": 0.7549401223659515, "train/step_real_loss": 0.7549401223659515, "train/lr": 2.1825143515174878e-05, "perf/step_duration_sec": 5.96535021904856, "perf/samples_per_sec": 5.364312039520762, "perf/tokens_per_sec": 4828.551374573625, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 28804.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 94.47624206542969}}
+{"timestamp": 1773763462.8593152, "event": "train_step", "step": 270, "epoch": 2, "metrics": {"train/step_loss": 0.8835187554359436, "train/step_real_loss": 0.8835187554359436, "train/lr": 1.822600463214922e-05, "perf/step_duration_sec": 6.238990655634552, "perf/samples_per_sec": 5.129034769606553, "perf/tokens_per_sec": 4065.8820312690445, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 25367.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 94.47624206542969}}
+{"timestamp": 1773763519.993293, "event": "train_step", "step": 280, "epoch": 2, "metrics": {"train/step_loss": 0.9217604398727417, "train/step_real_loss": 0.9217604398727417, "train/lr": 1.488415843473942e-05, "perf/step_duration_sec": 5.145696292165667, "perf/samples_per_sec": 6.2187890973511335, "perf/tokens_per_sec": 4732.109828765628, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 24350.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 94.47624206542969}}
+{"timestamp": 1773763577.139687, "event": "train_step", "step": 290, "epoch": 2, "metrics": {"train/step_loss": 0.9289029836654663, "train/step_real_loss": 0.9481655806303024, "train/lr": 1.1826665812616183e-05, "train/step_canary_loss": 0.3125, "perf/step_duration_sec": 5.694677841849625, "perf/samples_per_sec": 5.794884437094274, "perf/tokens_per_sec": 4061.3359776095867, "perf/logical_batch_size": 33.0, "perf/logical_token_count": 23128.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 94.47624206542969}}
+{"timestamp": 1773763634.8883243, "event": "train_step", "step": 300, "epoch": 2, "metrics": {"train/step_loss": 0.9466440713766849, "train/step_real_loss": 0.8473204374313354, "train/lr": 9.078285077691178e-06, "train/step_canary_loss": 4.125, "perf/step_duration_sec": 6.2339927861467, "perf/samples_per_sec": 5.293557617412269, "perf/tokens_per_sec": 4104.752905210987, "perf/logical_batch_size": 33.0, "perf/logical_token_count": 25589.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 94.47624206542969}}
+{"timestamp": 1773763647.6901762, "event": "eval_step", "step": 300, "epoch": 2, "metrics": {"eval/loss": 0.8076710475560945, "eval/duration_sec": 12.799929299857467}}
+{"timestamp": 1773763704.81611, "event": "train_step", "step": 310, "epoch": 2, "metrics": {"train/step_loss": 0.9043312668800354, "train/step_real_loss": 0.9043312668800354, "train/lr": 6.661271481537157e-06, "perf/step_duration_sec": 5.423097257036716, "perf/samples_per_sec": 5.900687095087322, "perf/tokens_per_sec": 4037.360748341778, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 21895.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 94.47624206542969}}
+{"timestamp": 1773763760.6698298, "event": "train_step", "step": 320, "epoch": 2, "metrics": {"train/step_loss": 0.8735850304365158, "train/step_real_loss": 0.8735850304365158, "train/lr": 4.595197001556562e-06, "perf/step_duration_sec": 5.15081740077585, "perf/samples_per_sec": 6.212606176872034, "perf/tokens_per_sec": 4842.532370928723, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 24943.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 94.47624206542969}}
+{"timestamp": 1773763818.6010072, "event": "train_step", "step": 330, "epoch": 2, "metrics": {"train/step_loss": 0.7620985209941864, "train/step_real_loss": 0.7620985209941864, "train/lr": 2.8967918551955297e-06, "perf/step_duration_sec": 6.058840225916356, "perf/samples_per_sec": 5.281538843543317, "perf/tokens_per_sec": 4943.025213289962, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 29949.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 94.47624206542969}}
+{"timestamp": 1773763875.2762098, "event": "train_step", "step": 340, "epoch": 2, "metrics": {"train/step_loss": 0.9000806212425232, "train/step_real_loss": 0.9000806212425232, "train/lr": 1.5798090255558617e-06, "perf/step_duration_sec": 5.154371100012213, "perf/samples_per_sec": 6.2083228737496565, "perf/tokens_per_sec": 4376.285595724094, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 22557.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 94.47624206542969}}
+{"timestamp": 1773763932.3082285, "event": "train_step", "step": 350, "epoch": 2, "metrics": {"train/step_loss": 0.7404757142066956, "train/step_real_loss": 0.7404757142066956, "train/lr": 6.54912895420573e-07, "perf/step_duration_sec": 5.694617530796677, "perf/samples_per_sec": 5.619341391575985, "perf/tokens_per_sec": 4990.501968974935, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 28419.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 94.47624206542969}}
+{"timestamp": 1773763945.1132307, "event": "eval_step", "step": 350, "epoch": 2, "metrics": {"eval/loss": 0.807514699605795, "eval/duration_sec": 12.803151289001107}}
+{"timestamp": 1773764000.5064592, "event": "train_step", "step": 360, "epoch": 2, "metrics": {"train/step_loss": 0.8157700151205063, "train/step_real_loss": 0.8157700151205063, "train/lr": 1.295928914885336e-07, "perf/step_duration_sec": 5.695594378747046, "perf/samples_per_sec": 5.618377621729371, "perf/tokens_per_sec": 4667.116060650316, "perf/logical_batch_size": 32.0, "perf/logical_token_count": 26582.0, "perf/gradient_accumulation_steps": 4.0, "system/cuda_memory_allocated_gb": 15.915565013885498, "system/cuda_max_memory_allocated_gb": 94.47624206542969}}
+{"timestamp": 1773764058.7973218, "event": "train_epoch", "step": 368, "epoch": 2, "metrics": {"train/epoch_loss": 0.856036927981092, "train/epoch_real_loss": 0.8280306565727147, "train/epoch_canary_loss": 3.6806401156922846, "perf/epoch_duration_sec": 1096.7485609338619, "perf/epoch_samples_per_sec": 43.38460217306761, "perf/epoch_tokens_per_sec": 34225.847506959835, "perf/epoch_samples": 47582.0, "perf/epoch_tokens": 37537149.0, "system/cuda_epoch_peak_memory_gb": 94.47624206542969, "eval/loss": 0.8075161480750794, "eval/duration_sec": 12.857198356185108}}
+{"timestamp": 1773764072.6744637, "event": "audit_epoch", "step": 368, "epoch": 2, "metrics": {"audit/delta": 1e-05, "audit/num_canaries": 500.0, "audit/num_members": 250.0, "audit/paper_guess_fraction": 0.2, "audit/paper_guess_steps": 20.0, "audit/loss/auc": 0.968584, "audit/loss/empirical_epsilon/0.05": 3.4791953936219215, "audit/loss/empirical_epsilon/0.01": 3.023197554051876, "audit/loss/empirical_epsilon_details/0.05/epsilon": 3.4791953936219215, "audit/loss/empirical_epsilon_details/0.05/num_guesses": 100.0, "audit/loss/empirical_epsilon_details/0.05/correct_guesses": 100.0, "audit/loss/empirical_epsilon_details/0.01/epsilon": 3.023197554051876, "audit/loss/empirical_epsilon_details/0.01/num_guesses": 100.0, "audit/loss/empirical_epsilon_details/0.01/correct_guesses": 100.0, "audit/embedding/auc": 0.883776, "audit/embedding/empirical_epsilon/0.05": 3.4791953936219215, "audit/embedding/empirical_epsilon/0.01": 3.023197554051876, "audit/embedding/empirical_epsilon_details/0.05/epsilon": 3.4791953936219215, "audit/embedding/empirical_epsilon_details/0.05/num_guesses": 100.0, "audit/embedding/empirical_epsilon_details/0.05/correct_guesses": 100.0, "audit/embedding/empirical_epsilon_details/0.01/epsilon": 3.023197554051876, "audit/embedding/empirical_epsilon_details/0.01/num_guesses": 100.0, "audit/embedding/empirical_epsilon_details/0.01/correct_guesses": 100.0, "perf/audit_duration_sec": 7.556974642910063}}
+{"timestamp": 1773764086.367738, "event": "audit_final", "step": 368, "epoch": 2, "metrics": {"audit/delta": 1e-05, "audit/num_canaries": 500.0, "audit/num_members": 250.0, "audit/paper_guess_fraction": 0.2, "audit/paper_guess_steps": 20.0, "audit/loss/auc": 0.968584, "audit/loss/empirical_epsilon/0.05": 3.4791953936219215, "audit/loss/empirical_epsilon/0.01": 3.023197554051876, "audit/loss/empirical_epsilon_details/0.05/epsilon": 3.4791953936219215, "audit/loss/empirical_epsilon_details/0.05/num_guesses": 100.0, "audit/loss/empirical_epsilon_details/0.05/correct_guesses": 100.0, "audit/loss/empirical_epsilon_details/0.01/epsilon": 3.023197554051876, "audit/loss/empirical_epsilon_details/0.01/num_guesses": 100.0, "audit/loss/empirical_epsilon_details/0.01/correct_guesses": 100.0, "audit/embedding/auc": 0.883776, "audit/embedding/empirical_epsilon/0.05": 3.4791953936219215, "audit/embedding/empirical_epsilon/0.01": 3.023197554051876, "audit/embedding/empirical_epsilon_details/0.05/epsilon": 3.4791953936219215, "audit/embedding/empirical_epsilon_details/0.05/num_guesses": 100.0, "audit/embedding/empirical_epsilon_details/0.05/correct_guesses": 100.0, "audit/embedding/empirical_epsilon_details/0.01/epsilon": 3.023197554051876, "audit/embedding/empirical_epsilon_details/0.01/num_guesses": 100.0, "audit/embedding/empirical_epsilon_details/0.01/correct_guesses": 100.0}}
+{"timestamp": 1773764086.9161372, "event": "energy_final", "step": 368, "epoch": null, "metrics": {"energy/codecarbon/duration": 2345.9966679112986, "energy/codecarbon/emissions": 0.09022432714096462, "energy/codecarbon/emissions_rate": 3.8458847096868924e-05, "energy/codecarbon/cpu_power": 72.02285277932866, "energy/codecarbon/gpu_power": 3280.290622412428, "energy/codecarbon/ram_power": 54.0, "energy/codecarbon/cpu_energy": 0.045218986505725985, "energy/codecarbon/gpu_energy": 2.137964725370466, "energy/codecarbon/ram_energy": 0.03390259211605879, "energy/codecarbon/energy_consumed": 2.2170863039922497, "energy/codecarbon/water_consumed": 0.0, "energy/codecarbon/cpu_count": 256.0, "energy/codecarbon/gpu_count": 8.0, "energy/codecarbon/longitude": 16.1885, "energy/codecarbon/latitude": 58.594, "energy/codecarbon/ram_total_size": 1511.49019241333, "energy/codecarbon/cpu_utilization_percent": 3.3142796066695253, "energy/codecarbon/gpu_utilization_percent": 88.58721675929884, "energy/codecarbon/ram_utilization_percent": 5.287772552372644, "energy/codecarbon/ram_used_gb": 79.7947571596665, "energy/codecarbon/pue": 1.0, "energy/codecarbon/wue": 0.0}}

qwen3-4b-instruct/base/pretrain_lm_head.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc44b7d60b8e2cf912e4233ff02bc57bb7e91f7a3ba6aa8ea10b7767ca29954a
+size 779106920

qwen3-4b-instruct/base/resolved_config.yaml

@@ -0,0 +1,100 @@
+model:
+  name: Qwen/Qwen3-4B-Instruct-2507
+  tokenizer_name: Qwen/Qwen3-4B-Instruct-2507
+  max_length: 1024
+  dtype: bfloat16
+  trust_remote_code: true
+  use_fast_tokenizer: true
+  cache_dir: null
+  local_files_only: false
+  low_cpu_mem_usage: true
+  tie_word_embeddings: true
+  gradient_checkpointing: false
+  use_chat_template: false
+dataset:
+  name: melihcatal/codedp-cpt
+  split: train
+  mode: cpt
+  text_column: text
+  validation_ratio: 0.05
+  max_samples: -1
+lora:
+  enabled: true
+  r: 16
+  alpha: 32
+  dropout: 0.05
+  target_modules:
+  - q_proj
+  - k_proj
+  - v_proj
+  - o_proj
+  modules_to_save:
+  - lm_head
+  bias: none
+training:
+  seed: 42
+  epochs: 2
+  warmup_steps: null
+  warmup_ratio: 0.05
+  mixed_precision: false
+  mixed_precision_dtype: bfloat16
+  batch_size: 8
+  eval_batch_size: 8
+  eval_every_steps: 50
+  eval_every_epochs: 1
+  learning_rate: 0.0001
+  optimizer: adamw
+  lr_scheduler: cosine
+  adam_beta1: 0.9
+  adam_beta2: 0.999
+  adam_epsilon: 1.0e-08
+  sgd_momentum: 0.9
+  weight_decay: 0.01
+  max_grad_norm: 1.0
+  log_every: 10
+  gradient_accumulation_steps: 4
+  num_workers: 4
+  output_dir: runs/cpt/qwen3-4b-instruct/base
+distributed:
+  strategy: dpddp
+  backend: nccl
+  devices: null
+dp:
+  module_validator: auto
+  target_delta: 1.0e-05
+  noise_multiplier: null
+  max_grad_norm: 1.0
+  grad_sample_mode: ghost
+  secure_mode: false
+  enabled: false
+  target_epsilon: 8.0
+audit:
+  enabled: true
+  run_every_epoch: true
+  epoch_device: cuda
+  q_canary: auto
+  num_canaries: 500
+  prefix_length: 49
+  num_digits: 12
+  batch_size: 32
+  delta: 1.0e-05
+  p_values:
+  - 0.05
+  - 0.01
+  paper_guess_fraction: 0.2
+  paper_guess_steps: 20
+  enable_holdout_empirical_epsilon: false
+  holdout_seed: 42
+  tie_seed: 42
+tracking:
+  enabled: true
+  tensorboard: true
+  wandb: false
+  wandb_project: codedp-finetune-h200-audit
+  wandb_run_name: qwen3-4b-instruct-cpt-base
+  wandb_mode: online
+  codecarbon: true
+  codecarbon_output_file: codecarbon.csv
+  codecarbon_measure_power_secs: 15
+  codecarbon_country_iso_code: null
+  codecarbon_project_name: codedp-qwen3-4b-instruct-cpt-base

qwen3-4b-instruct/base/scalars.csv

@@ -0,0 +1,537 @@
+timestamp,event,step,epoch,key,value
+1773761894.8111923,train_step,10,1,train/step_loss,1.8352766107110416
+1773761894.8111923,train_step,10,1,train/step_real_loss,1.028106451034546
+1773761894.8111923,train_step,10,1,train/lr,5.2631578947368424e-05
+1773761894.8111923,train_step,10,1,train/step_canary_loss,14.75
+1773761894.8111923,train_step,10,1,perf/step_duration_sec,6.234770041890442
+1773761894.8111923,train_step,10,1,perf/samples_per_sec,5.453288536956348
+1773761894.8111923,train_step,10,1,perf/tokens_per_sec,3980.098677781523
+1773761894.8111923,train_step,10,1,perf/logical_batch_size,34.0
+1773761894.8111923,train_step,10,1,perf/logical_token_count,24815.0
+1773761894.8111923,train_step,10,1,perf/gradient_accumulation_steps,4.0
+1773761894.8111923,train_step,10,1,system/cuda_memory_allocated_gb,15.915565013885498
+1773761894.8111923,train_step,10,1,system/cuda_max_memory_allocated_gb,94.4762544631958
+1773761950.7987902,train_step,20,1,train/step_loss,1.0323970019817352
+1773761950.7987902,train_step,20,1,train/step_real_loss,1.0323970019817352
+1773761950.7987902,train_step,20,1,train/lr,9.999797424944042e-05
+1773761950.7987902,train_step,20,1,perf/step_duration_sec,5.150427320972085
+1773761950.7987902,train_step,20,1,perf/samples_per_sec,6.213076703305535
+1773761950.7987902,train_step,20,1,perf/tokens_per_sec,4927.9406189561805
+1773761950.7987902,train_step,20,1,perf/logical_batch_size,32.0
+1773761950.7987902,train_step,20,1,perf/logical_token_count,25381.0
+1773761950.7987902,train_step,20,1,perf/gradient_accumulation_steps,4.0
+1773761950.7987902,train_step,20,1,system/cuda_memory_allocated_gb,15.915565013885498
+1773761950.7987902,train_step,20,1,system/cuda_max_memory_allocated_gb,94.4762544631958
+1773762007.3004794,train_step,30,1,train/step_loss,0.8551503717899323
+1773762007.3004794,train_step,30,1,train/step_real_loss,0.8551503717899323
+1773762007.3004794,train_step,30,1,train/lr,9.975508273693644e-05
+1773762007.3004794,train_step,30,1,perf/step_duration_sec,5.69609066285193
+1773762007.3004794,train_step,30,1,perf/samples_per_sec,5.617888108539722
+1773762007.3004794,train_step,30,1,perf/tokens_per_sec,4432.689276641233
+1773762007.3004794,train_step,30,1,perf/logical_batch_size,32.0
+1773762007.3004794,train_step,30,1,perf/logical_token_count,25249.0
+1773762007.3004794,train_step,30,1,perf/gradient_accumulation_steps,4.0
+1773762007.3004794,train_step,30,1,system/cuda_memory_allocated_gb,15.915565013885498
+1773762007.3004794,train_step,30,1,system/cuda_max_memory_allocated_gb,94.4762544631958
+1773762065.1568909,train_step,40,1,train/step_loss,0.8950656801462173
+1773762065.1568909,train_step,40,1,train/step_real_loss,0.8950656801462173
+1773762065.1568909,train_step,40,1,train/lr,9.910929512300672e-05
+1773762065.1568909,train_step,40,1,perf/step_duration_sec,6.2338299779221416
+1773762065.1568909,train_step,40,1,perf/samples_per_sec,5.133280842328368
+1773762065.1568909,train_step,40,1,perf/tokens_per_sec,4016.4714290693023
+1773762065.1568909,train_step,40,1,perf/logical_batch_size,32.0
+1773762065.1568909,train_step,40,1,perf/logical_token_count,25038.0
+1773762065.1568909,train_step,40,1,perf/gradient_accumulation_steps,4.0
+1773762065.1568909,train_step,40,1,system/cuda_memory_allocated_gb,15.915565013885498
+1773762065.1568909,train_step,40,1,system/cuda_max_memory_allocated_gb,94.4762544631958
+1773762121.2732794,train_step,50,1,train/step_loss,0.8450518101453781
+1773762121.2732794,train_step,50,1,train/step_real_loss,0.8450518101453781
+1773762121.2732794,train_step,50,1,train/lr,9.806584072891234e-05
+1773762121.2732794,train_step,50,1,perf/step_duration_sec,5.423216213937849
+1773762121.2732794,train_step,50,1,perf/samples_per_sec,5.900557664980961
+1773762121.2732794,train_step,50,1,perf/tokens_per_sec,5482.724425329497
+1773762121.2732794,train_step,50,1,perf/logical_batch_size,32.0
+1773762121.2732794,train_step,50,1,perf/logical_token_count,29734.0
+1773762121.2732794,train_step,50,1,perf/gradient_accumulation_steps,4.0
+1773762121.2732794,train_step,50,1,system/cuda_memory_allocated_gb,15.915565013885498
+1773762121.2732794,train_step,50,1,system/cuda_max_memory_allocated_gb,94.4762544631958
+1773762134.073646,eval_step,50,1,eval/loss,0.8533683589253671
+1773762134.073646,eval_step,50,1,eval/duration_sec,12.798461285419762
+1773762190.2760499,train_step,60,1,train/step_loss,1.114606170943289
+1773762190.2760499,train_step,60,1,train/step_real_loss,0.8427969664335251
+1773762190.2760499,train_step,60,1,train/lr,9.663316901718597e-05
+1773762190.2760499,train_step,60,1,train/step_canary_loss,9.8125
+1773762190.2760499,train_step,60,1,perf/step_duration_sec,5.968041606713086
+1773762190.2760499,train_step,60,1,perf/samples_per_sec,5.5294520673046765
+1773762190.2760499,train_step,60,1,perf/tokens_per_sec,4259.353683360148
+1773762190.2760499,train_step,60,1,perf/logical_batch_size,33.0
+1773762190.2760499,train_step,60,1,perf/logical_token_count,25420.0
+1773762190.2760499,train_step,60,1,perf/gradient_accumulation_steps,4.0
+1773762190.2760499,train_step,60,1,system/cuda_memory_allocated_gb,15.915565013885498
+1773762190.2760499,train_step,60,1,system/cuda_max_memory_allocated_gb,94.4762544631958
+1773762249.207255,train_step,70,1,train/step_loss,1.1624658794114084
+1773762249.207255,train_step,70,1,train/step_real_loss,0.8745741844177246
+1773762249.207255,train_step,70,1,train/lr,9.48228811713756e-05
+1773762249.207255,train_step,70,1,train/step_canary_loss,10.375
+1773762249.207255,train_step,70,1,perf/step_duration_sec,6.334186799824238
+1773762249.207255,train_step,70,1,perf/samples_per_sec,5.209824250986045
+1773762249.207255,train_step,70,1,perf/tokens_per_sec,3829.378697936894
+1773762249.207255,train_step,70,1,perf/logical_batch_size,33.0
+1773762249.207255,train_step,70,1,perf/logical_token_count,24256.0
+1773762249.207255,train_step,70,1,perf/gradient_accumulation_steps,4.0
+1773762249.207255,train_step,70,1,system/cuda_memory_allocated_gb,16.205660820007324
+1773762249.207255,train_step,70,1,system/cuda_max_memory_allocated_gb,94.4762544631958
+1773762305.719841,train_step,80,1,train/step_loss,1.157600255573497
+1773762305.719841,train_step,80,1,train/step_real_loss,0.8803409039974213
+1773762305.719841,train_step,80,1,train/lr,9.26496361544538e-05
+1773762305.719841,train_step,80,1,train/step_canary_loss,5.59375
+1773762305.719841,train_step,80,1,perf/step_duration_sec,5.699764240998775
+1773762305.719841,train_step,80,1,perf/samples_per_sec,5.9651590070052
+1773762305.719841,train_step,80,1,perf/tokens_per_sec,4690.720329743854
+1773762305.719841,train_step,80,1,perf/logical_batch_size,34.0
+1773762305.719841,train_step,80,1,perf/logical_token_count,26736.0
+1773762305.719841,train_step,80,1,perf/gradient_accumulation_steps,4.0
+1773762305.719841,train_step,80,1,system/cuda_memory_allocated_gb,16.205660820007324
+1773762305.719841,train_step,80,1,system/cuda_max_memory_allocated_gb,94.4762544631958
+1773762363.2025864,train_step,90,1,train/step_loss,0.8746908158063889
+1773762363.2025864,train_step,90,1,train/step_real_loss,0.8746908158063889
+1773762363.2025864,train_step,90,1,train/lr,9.013103200659241e-05
+1773762363.2025864,train_step,90,1,perf/step_duration_sec,5.422750173136592
+1773762363.2025864,train_step,90,1,perf/samples_per_sec,5.901064769408466
+1773762363.2025864,train_step,90,1,perf/tokens_per_sec,4211.147346069116
+1773762363.2025864,train_step,90,1,perf/logical_batch_size,32.0
+1773762363.2025864,train_step,90,1,perf/logical_token_count,22836.0
+1773762363.2025864,train_step,90,1,perf/gradient_accumulation_steps,4.0
+1773762363.2025864,train_step,90,1,system/cuda_memory_allocated_gb,15.915565013885498
+1773762363.2025864,train_step,90,1,system/cuda_max_memory_allocated_gb,94.4762544631958
+1773762419.340239,train_step,100,1,train/step_loss,1.227062124194521
+1773762419.340239,train_step,100,1,train/step_real_loss,0.9060328304767609
+1773762419.340239,train_step,100,1,train/lr,8.728746334350483e-05
+1773762419.340239,train_step,100,1,train/step_canary_loss,11.5
+1773762419.340239,train_step,100,1,perf/step_duration_sec,5.688522285781801
+1773762419.340239,train_step,100,1,perf/samples_per_sec,5.801155087760134
+1773762419.340239,train_step,100,1,perf/tokens_per_sec,4329.068036096396
+1773762419.340239,train_step,100,1,perf/logical_batch_size,33.0
+1773762419.340239,train_step,100,1,perf/logical_token_count,24626.0
+1773762419.340239,train_step,100,1,perf/gradient_accumulation_steps,4.0
+1773762419.340239,train_step,100,1,system/cuda_memory_allocated_gb,16.205660820007324
+1773762419.340239,train_step,100,1,system/cuda_max_memory_allocated_gb,94.4762544631958
+1773762432.1276581,eval_step,100,1,eval/loss,0.829470864473245
+1773762432.1276581,eval_step,100,1,eval/duration_sec,12.785310188308358
+1773762489.3303852,train_step,110,1,train/step_loss,1.011215921604272
+1773762489.3303852,train_step,110,1,train/step_real_loss,0.9119570553302765
+1773762489.3303852,train_step,110,1,train/lr,8.414195620927492e-05
+1773762489.3303852,train_step,110,1,train/step_canary_loss,4.1875
+1773762489.3303852,train_step,110,1,perf/step_duration_sec,5.6974792359396815
+1773762489.3303852,train_step,110,1,perf/samples_per_sec,5.792035149831895
+1773762489.3303852,train_step,110,1,perf/tokens_per_sec,4259.603062159705
+1773762489.3303852,train_step,110,1,perf/logical_batch_size,33.0
+1773762489.3303852,train_step,110,1,perf/logical_token_count,24269.0
+1773762489.3303852,train_step,110,1,perf/gradient_accumulation_steps,4.0
+1773762489.3303852,train_step,110,1,system/cuda_memory_allocated_gb,15.915565013885498
+1773762489.3303852,train_step,110,1,system/cuda_max_memory_allocated_gb,94.4762544631958
+1773762544.9202547,train_step,120,1,train/step_loss,0.7180200964212418
+1773762544.9202547,train_step,120,1,train/step_real_loss,0.7180200964212418
+1773762544.9202547,train_step,120,1,train/lr,8.071998162096612e-05
+1773762544.9202547,train_step,120,1,perf/step_duration_sec,5.694211829919368
+1773762544.9202547,train_step,120,1,perf/samples_per_sec,5.619741758088605
+1773762544.9202547,train_step,120,1,perf/tokens_per_sec,4598.00245969612
+1773762544.9202547,train_step,120,1,perf/logical_batch_size,32.0
+1773762544.9202547,train_step,120,1,perf/logical_token_count,26182.0
+1773762544.9202547,train_step,120,1,perf/gradient_accumulation_steps,4.0
+1773762544.9202547,train_step,120,1,system/cuda_memory_allocated_gb,15.915565013885498
+1773762544.9202547,train_step,120,1,system/cuda_max_memory_allocated_gb,94.4762544631958
+1773762601.86658,train_step,130,1,train/step_loss,0.8463245183229446
+1773762601.86658,train_step,130,1,train/step_real_loss,0.8463245183229446
+1773762601.86658,train_step,130,1,train/lr,7.704924931484997e-05
+1773762601.86658,train_step,130,1,perf/step_duration_sec,5.1536158989183605
+1773762601.86658,train_step,130,1,perf/samples_per_sec,6.209232629602092
+1773762601.86658,train_step,130,1,perf/tokens_per_sec,4267.683201733388
+1773762601.86658,train_step,130,1,perf/logical_batch_size,32.0
+1773762601.86658,train_step,130,1,perf/logical_token_count,21994.0
+1773762601.86658,train_step,130,1,perf/gradient_accumulation_steps,4.0
+1773762601.86658,train_step,130,1,system/cuda_memory_allocated_gb,15.915565013885498
+1773762601.86658,train_step,130,1,system/cuda_max_memory_allocated_gb,101.70386934280396
+1773762659.5638738,train_step,140,1,train/step_loss,0.9742496162652969
+1773762659.5638738,train_step,140,1,train/step_real_loss,0.9742496162652969
+1773762659.5638738,train_step,140,1,train/lr,7.315948336441117e-05
+1773762659.5638738,train_step,140,1,perf/step_duration_sec,5.969049285165966
+1773762659.5638738,train_step,140,1,perf/samples_per_sec,5.360987733762741
+1773762659.5638738,train_step,140,1,perf/tokens_per_sec,3927.091045847888
+1773762659.5638738,train_step,140,1,perf/logical_batch_size,32.0
+1773762659.5638738,train_step,140,1,perf/logical_token_count,23441.0
+1773762659.5638738,train_step,140,1,perf/gradient_accumulation_steps,4.0
+1773762659.5638738,train_step,140,1,system/cuda_memory_allocated_gb,15.915565013885498
+1773762659.5638738,train_step,140,1,system/cuda_max_memory_allocated_gb,101.70386934280396
+1773762717.6212678,train_step,150,1,train/step_loss,0.914526179432869
+1773762717.6212678,train_step,150,1,train/step_real_loss,0.914526179432869
+1773762717.6212678,train_step,150,1,train/lr,6.908218148708247e-05
+1773762717.6212678,train_step,150,1,perf/step_duration_sec,5.703813333064318
+1773762717.6212678,train_step,150,1,perf/samples_per_sec,5.610281776666824
+1773762717.6212678,train_step,150,1,perf/tokens_per_sec,4645.31331108013
+1773762717.6212678,train_step,150,1,perf/logical_batch_size,32.0
+1773762717.6212678,train_step,150,1,perf/logical_token_count,26496.0
+1773762717.6212678,train_step,150,1,perf/gradient_accumulation_steps,4.0
+1773762717.6212678,train_step,150,1,system/cuda_memory_allocated_gb,15.915565013885498
+1773762717.6212678,train_step,150,1,system/cuda_max_memory_allocated_gb,101.70386934280396
+1773762730.4167268,eval_step,150,1,eval/loss,0.8174186625923866
+1773762730.4167268,eval_step,150,1,eval/duration_sec,12.793565314263105
+1773762785.9700294,train_step,160,1,train/step_loss,0.9113822728395462
+1773762785.9700294,train_step,160,1,train/step_real_loss,0.9113822728395462
+1773762785.9700294,train_step,160,1,train/lr,6.485035998874356e-05
+1773762785.9700294,train_step,160,1,perf/step_duration_sec,5.425368802621961
+1773762785.9700294,train_step,160,1,perf/samples_per_sec,5.898216538668322
+1773762785.9700294,train_step,160,1,perf/tokens_per_sec,4679.128907832313
+1773762785.9700294,train_step,160,1,perf/logical_batch_size,32.0
+1773762785.9700294,train_step,160,1,perf/logical_token_count,25386.0
+1773762785.9700294,train_step,160,1,perf/gradient_accumulation_steps,4.0
+1773762785.9700294,train_step,160,1,system/cuda_memory_allocated_gb,15.915565013885498
+1773762785.9700294,train_step,160,1,system/cuda_max_memory_allocated_gb,101.70386934280396
+1773762841.0786932,train_step,170,1,train/step_loss,0.8910564256436897
+1773762841.0786932,train_step,170,1,train/step_real_loss,0.8441949039697647
+1773762841.0786932,train_step,170,1,train/lr,6.049828641131825e-05
+1773762841.0786932,train_step,170,1,train/step_canary_loss,2.390625
+1773762841.0786932,train_step,170,1,perf/step_duration_sec,5.690398690290749
+1773762841.0786932,train_step,170,1,perf/samples_per_sec,5.799242161415912
+1773762841.0786932,train_step,170,1,perf/tokens_per_sec,4355.582332445254
+1773762841.0786932,train_step,170,1,perf/logical_batch_size,33.0
+1773762841.0786932,train_step,170,1,perf/logical_token_count,24785.0
+1773762841.0786932,train_step,170,1,perf/gradient_accumulation_steps,4.0
+1773762841.0786932,train_step,170,1,system/cuda_memory_allocated_gb,16.205660820007324
+1773762841.0786932,train_step,170,1,system/cuda_max_memory_allocated_gb,101.70386934280396
+1773762898.8441253,train_step,180,1,train/step_loss,1.0836328773787527
+1773762898.8441253,train_step,180,1,train/step_real_loss,0.8538245260715485
+1773762898.8441253,train_step,180,1,train/lr,5.6061202048379124e-05
+1773762898.8441253,train_step,180,1,train/step_canary_loss,8.4375
+1773762898.8441253,train_step,180,1,perf/step_duration_sec,5.692985306028277
+1773762898.8441253,train_step,180,1,perf/samples_per_sec,5.7966072677293665
+1773762898.8441253,train_step,180,1,perf/tokens_per_sec,4476.7373583439585
+1773762898.8441253,train_step,180,1,perf/logical_batch_size,33.0
+1773762898.8441253,train_step,180,1,perf/logical_token_count,25486.0
+1773762898.8441253,train_step,180,1,perf/gradient_accumulation_steps,4.0
+1773762898.8441253,train_step,180,1,system/cuda_memory_allocated_gb,15.915565013885498
+1773762898.8441253,train_step,180,1,system/cuda_max_memory_allocated_gb,101.70386934280396
+1773762934.6675656,train_epoch,184,1,train/epoch_loss,0.9805822407983543
+1773762934.6675656,train_epoch,184,1,train/epoch_real_loss,0.9071368900552877
+1773762934.6675656,train_epoch,184,1,train/epoch_canary_loss,8.42140839386602
+1773762934.6675656,train_epoch,184,1,perf/epoch_duration_sec,1085.6805565529503
+1773762934.6675656,train_epoch,184,1,perf/epoch_samples_per_sec,43.826887856473604
+1773762934.6675656,train_epoch,184,1,perf/epoch_tokens_per_sec,34574.90951038253
+1773762934.6675656,train_epoch,184,1,perf/epoch_samples,47582.0
+1773762934.6675656,train_epoch,184,1,perf/epoch_tokens,37537307.0
+1773762934.6675656,train_epoch,184,1,system/cuda_epoch_peak_memory_gb,101.70386934280396
+1773762934.6675656,train_epoch,184,1,eval/loss,0.8121764394335258
+1773762934.6675656,train_epoch,184,1,eval/duration_sec,12.826699289958924
+1773762949.1512606,audit_epoch,184,1,audit/delta,1e-05
+1773762949.1512606,audit_epoch,184,1,audit/num_canaries,500.0
+1773762949.1512606,audit_epoch,184,1,audit/num_members,250.0
+1773762949.1512606,audit_epoch,184,1,audit/paper_guess_fraction,0.2
+1773762949.1512606,audit_epoch,184,1,audit/paper_guess_steps,20.0
+1773762949.1512606,audit_epoch,184,1,audit/loss/auc,0.907944
+1773762949.1512606,audit_epoch,184,1,audit/loss/empirical_epsilon/0.05,3.4791953936219215
+1773762949.1512606,audit_epoch,184,1,audit/loss/empirical_epsilon/0.01,3.023197554051876
+1773762949.1512606,audit_epoch,184,1,audit/loss/empirical_epsilon_details/0.05/epsilon,3.4791953936219215
+1773762949.1512606,audit_epoch,184,1,audit/loss/empirical_epsilon_details/0.05/num_guesses,100.0
+1773762949.1512606,audit_epoch,184,1,audit/loss/empirical_epsilon_details/0.05/correct_guesses,100.0
+1773762949.1512606,audit_epoch,184,1,audit/loss/empirical_epsilon_details/0.01/epsilon,3.023197554051876
+1773762949.1512606,audit_epoch,184,1,audit/loss/empirical_epsilon_details/0.01/num_guesses,100.0
+1773762949.1512606,audit_epoch,184,1,audit/loss/empirical_epsilon_details/0.01/correct_guesses,100.0
+1773762949.1512606,audit_epoch,184,1,audit/embedding/auc,0.876048
+1773762949.1512606,audit_epoch,184,1,audit/embedding/empirical_epsilon/0.05,3.4791953936219215
+1773762949.1512606,audit_epoch,184,1,audit/embedding/empirical_epsilon/0.01,3.023197554051876
+1773762949.1512606,audit_epoch,184,1,audit/embedding/empirical_epsilon_details/0.05/epsilon,3.4791953936219215
+1773762949.1512606,audit_epoch,184,1,audit/embedding/empirical_epsilon_details/0.05/num_guesses,100.0
+1773762949.1512606,audit_epoch,184,1,audit/embedding/empirical_epsilon_details/0.05/correct_guesses,100.0
+1773762949.1512606,audit_epoch,184,1,audit/embedding/empirical_epsilon_details/0.01/epsilon,3.023197554051876
+1773762949.1512606,audit_epoch,184,1,audit/embedding/empirical_epsilon_details/0.01/num_guesses,100.0
+1773762949.1512606,audit_epoch,184,1,audit/embedding/empirical_epsilon_details/0.01/correct_guesses,100.0
+1773762949.1512606,audit_epoch,184,1,perf/audit_duration_sec,8.130579099990427
+1773762984.332577,train_step,190,2,train/step_loss,0.8655764758586884
+1773762984.332577,train_step,190,2,train/step_real_loss,0.8655764758586884
+1773762984.332577,train_step,190,2,train/lr,5.157503657571385e-05
+1773762984.332577,train_step,190,2,perf/step_duration_sec,5.690848938189447
+1773762984.332577,train_step,190,2,perf/samples_per_sec,5.623062630472995
+1773762984.332577,train_step,190,2,perf/tokens_per_sec,4602.301042334944
+1773762984.332577,train_step,190,2,perf/logical_batch_size,32.0
+1773762984.332577,train_step,190,2,perf/logical_token_count,26191.0
+1773762984.332577,train_step,190,2,perf/gradient_accumulation_steps,4.0
+1773762984.332577,train_step,190,2,system/cuda_memory_allocated_gb,15.915565013885498
+1773762984.332577,train_step,190,2,system/cuda_max_memory_allocated_gb,87.30217599868774
+1773763040.7157884,train_step,200,2,train/step_loss,0.8589679941986547
+1773763040.7157884,train_step,200,2,train/step_real_loss,0.8308791071176529
+1773763040.7157884,train_step,200,2,train/lr,4.7076117107656534e-05
+1773763040.7157884,train_step,200,2,train/step_canary_loss,1.7578125
+1773763040.7157884,train_step,200,2,perf/step_duration_sec,5.69332688068971
+1773763040.7157884,train_step,200,2,perf/samples_per_sec,5.796259496697344
+1773763040.7157884,train_step,200,2,perf/tokens_per_sec,5059.0806752537455
+1773763040.7157884,train_step,200,2,perf/logical_batch_size,33.0
+1773763040.7157884,train_step,200,2,perf/logical_token_count,28803.0
+1773763040.7157884,train_step,200,2,perf/gradient_accumulation_steps,4.0
+1773763040.7157884,train_step,200,2,system/cuda_memory_allocated_gb,15.915565013885498
+1773763040.7157884,train_step,200,2,system/cuda_max_memory_allocated_gb,87.30217599868774
+1773763053.5208263,eval_step,200,2,eval/loss,0.8110936123591204
+1773763053.5208263,eval_step,200,2,eval/duration_sec,12.803085402119905
+1773763110.1544352,train_step,210,2,train/step_loss,0.7684839069843292
+1773763110.1544352,train_step,210,2,train/step_real_loss,0.7684839069843292
+1773763110.1544352,train_step,210,2,train/lr,4.2600874035126046e-05
+1773763110.1544352,train_step,210,2,perf/step_duration_sec,5.4159107422456145
+1773763110.1544352,train_step,210,2,perf/samples_per_sec,5.908516872405425
+1773763110.1544352,train_step,210,2,perf/tokens_per_sec,4630.061534138701
+1773763110.1544352,train_step,210,2,perf/logical_batch_size,32.0
+1773763110.1544352,train_step,210,2,perf/logical_token_count,25076.0
+1773763110.1544352,train_step,210,2,perf/gradient_accumulation_steps,4.0
+1773763110.1544352,train_step,210,2,system/cuda_memory_allocated_gb,15.915565013885498
+1773763110.1544352,train_step,210,2,system/cuda_max_memory_allocated_gb,87.30229806900024
+1773763168.1105735,train_step,220,2,train/step_loss,0.8040641099214554
+1773763168.1105735,train_step,220,2,train/step_real_loss,0.8040641099214554
+1773763168.1105735,train_step,220,2,train/lr,3.818554602737332e-05
+1773763168.1105735,train_step,220,2,perf/step_duration_sec,5.967140641994774
+1773763168.1105735,train_step,220,2,perf/samples_per_sec,5.36270249351834
+1773763168.1105735,train_step,220,2,perf/tokens_per_sec,4430.932935269528
+1773763168.1105735,train_step,220,2,perf/logical_batch_size,32.0
+1773763168.1105735,train_step,220,2,perf/logical_token_count,26440.0
+1773763168.1105735,train_step,220,2,perf/gradient_accumulation_steps,4.0
+1773763168.1105735,train_step,220,2,system/cuda_memory_allocated_gb,15.915565013885498
+1773763168.1105735,train_step,220,2,system/cuda_max_memory_allocated_gb,87.30229806900024
+1773763224.221138,train_step,230,2,train/step_loss,0.806927278637886
+1773763224.221138,train_step,230,2,train/step_real_loss,0.806927278637886
+1773763224.221138,train_step,230,2,train/lr,3.386588658621128e-05
+1773763224.221138,train_step,230,2,perf/step_duration_sec,5.424597659613937
+1773763224.221138,train_step,230,2,perf/samples_per_sec,5.899055009782496
+1773763224.221138,train_step,230,2,perf/tokens_per_sec,4482.175734620363
+1773763224.221138,train_step,230,2,perf/logical_batch_size,32.0
+1773763224.221138,train_step,230,2,perf/logical_token_count,24314.0
+1773763224.221138,train_step,230,2,perf/gradient_accumulation_steps,4.0
+1773763224.221138,train_step,230,2,system/cuda_memory_allocated_gb,15.915565013885498
+1773763224.221138,train_step,230,2,system/cuda_max_memory_allocated_gb,87.30229806900024
+1773763282.06351,train_step,240,2,train/step_loss,0.9124463796615601
+1773763282.06351,train_step,240,2,train/step_real_loss,0.9124463796615601
+1773763282.06351,train_step,240,2,train/lr,2.967687452893051e-05
+1773763282.06351,train_step,240,2,perf/step_duration_sec,5.692487298045307
+1773763282.06351,train_step,240,2,perf/samples_per_sec,5.621444251792743
+1773763282.06351,train_step,240,2,perf/tokens_per_sec,4766.282044988772
+1773763282.06351,train_step,240,2,perf/logical_batch_size,32.0
+1773763282.06351,train_step,240,2,perf/logical_token_count,27132.0
+1773763282.06351,train_step,240,2,perf/gradient_accumulation_steps,4.0
+1773763282.06351,train_step,240,2,system/cuda_memory_allocated_gb,15.915565013885498
+1773763282.06351,train_step,240,2,system/cuda_max_memory_allocated_gb,94.47624206542969
+1773763337.7170568,train_step,250,2,train/step_loss,0.7962393760681152
+1773763337.7170568,train_step,250,2,train/step_real_loss,0.7962393760681152
+1773763337.7170568,train_step,250,2,train/lr,2.5652430744289756e-05
+1773763337.7170568,train_step,250,2,perf/step_duration_sec,5.419141778722405
+1773763337.7170568,train_step,250,2,perf/samples_per_sec,5.904994057480479
+1773763337.7170568,train_step,250,2,perf/tokens_per_sec,4963.885629569528
+1773763337.7170568,train_step,250,2,perf/logical_batch_size,32.0
+1773763337.7170568,train_step,250,2,perf/logical_token_count,26900.0
+1773763337.7170568,train_step,250,2,perf/gradient_accumulation_steps,4.0
+1773763337.7170568,train_step,250,2,system/cuda_memory_allocated_gb,15.915565013885498
+1773763337.7170568,train_step,250,2,system/cuda_max_memory_allocated_gb,94.47624206542969
+1773763350.5270941,eval_step,250,2,eval/loss,0.8086002505360506
+1773763350.5270941,eval_step,250,2,eval/duration_sec,12.808165564201772
+1773763406.908792,train_step,260,2,train/step_loss,0.7549401223659515
+1773763406.908792,train_step,260,2,train/step_real_loss,0.7549401223659515
+1773763406.908792,train_step,260,2,train/lr,2.1825143515174878e-05
+1773763406.908792,train_step,260,2,perf/step_duration_sec,5.96535021904856
+1773763406.908792,train_step,260,2,perf/samples_per_sec,5.364312039520762
+1773763406.908792,train_step,260,2,perf/tokens_per_sec,4828.551374573625
+1773763406.908792,train_step,260,2,perf/logical_batch_size,32.0
+1773763406.908792,train_step,260,2,perf/logical_token_count,28804.0
+1773763406.908792,train_step,260,2,perf/gradient_accumulation_steps,4.0
+1773763406.908792,train_step,260,2,system/cuda_memory_allocated_gb,15.915565013885498
+1773763406.908792,train_step,260,2,system/cuda_max_memory_allocated_gb,94.47624206542969
+1773763462.8593152,train_step,270,2,train/step_loss,0.8835187554359436
+1773763462.8593152,train_step,270,2,train/step_real_loss,0.8835187554359436
+1773763462.8593152,train_step,270,2,train/lr,1.822600463214922e-05
+1773763462.8593152,train_step,270,2,perf/step_duration_sec,6.238990655634552
+1773763462.8593152,train_step,270,2,perf/samples_per_sec,5.129034769606553
+1773763462.8593152,train_step,270,2,perf/tokens_per_sec,4065.8820312690445
+1773763462.8593152,train_step,270,2,perf/logical_batch_size,32.0
+1773763462.8593152,train_step,270,2,perf/logical_token_count,25367.0
+1773763462.8593152,train_step,270,2,perf/gradient_accumulation_steps,4.0
+1773763462.8593152,train_step,270,2,system/cuda_memory_allocated_gb,15.915565013885498
+1773763462.8593152,train_step,270,2,system/cuda_max_memory_allocated_gb,94.47624206542969
+1773763519.993293,train_step,280,2,train/step_loss,0.9217604398727417
+1773763519.993293,train_step,280,2,train/step_real_loss,0.9217604398727417
+1773763519.993293,train_step,280,2,train/lr,1.488415843473942e-05
+1773763519.993293,train_step,280,2,perf/step_duration_sec,5.145696292165667
+1773763519.993293,train_step,280,2,perf/samples_per_sec,6.2187890973511335
+1773763519.993293,train_step,280,2,perf/tokens_per_sec,4732.109828765628
+1773763519.993293,train_step,280,2,perf/logical_batch_size,32.0
+1773763519.993293,train_step,280,2,perf/logical_token_count,24350.0
+1773763519.993293,train_step,280,2,perf/gradient_accumulation_steps,4.0
+1773763519.993293,train_step,280,2,system/cuda_memory_allocated_gb,15.915565013885498
+1773763519.993293,train_step,280,2,system/cuda_max_memory_allocated_gb,94.47624206542969
+1773763577.139687,train_step,290,2,train/step_loss,0.9289029836654663
+1773763577.139687,train_step,290,2,train/step_real_loss,0.9481655806303024
+1773763577.139687,train_step,290,2,train/lr,1.1826665812616183e-05
+1773763577.139687,train_step,290,2,train/step_canary_loss,0.3125
+1773763577.139687,train_step,290,2,perf/step_duration_sec,5.694677841849625
+1773763577.139687,train_step,290,2,perf/samples_per_sec,5.794884437094274
+1773763577.139687,train_step,290,2,perf/tokens_per_sec,4061.3359776095867
+1773763577.139687,train_step,290,2,perf/logical_batch_size,33.0
+1773763577.139687,train_step,290,2,perf/logical_token_count,23128.0
+1773763577.139687,train_step,290,2,perf/gradient_accumulation_steps,4.0
+1773763577.139687,train_step,290,2,system/cuda_memory_allocated_gb,15.915565013885498
+1773763577.139687,train_step,290,2,system/cuda_max_memory_allocated_gb,94.47624206542969
+1773763634.8883243,train_step,300,2,train/step_loss,0.9466440713766849
+1773763634.8883243,train_step,300,2,train/step_real_loss,0.8473204374313354
+1773763634.8883243,train_step,300,2,train/lr,9.078285077691178e-06
+1773763634.8883243,train_step,300,2,train/step_canary_loss,4.125
+1773763634.8883243,train_step,300,2,perf/step_duration_sec,6.2339927861467
+1773763634.8883243,train_step,300,2,perf/samples_per_sec,5.293557617412269
+1773763634.8883243,train_step,300,2,perf/tokens_per_sec,4104.752905210987
+1773763634.8883243,train_step,300,2,perf/logical_batch_size,33.0
+1773763634.8883243,train_step,300,2,perf/logical_token_count,25589.0
+1773763634.8883243,train_step,300,2,perf/gradient_accumulation_steps,4.0
+1773763634.8883243,train_step,300,2,system/cuda_memory_allocated_gb,15.915565013885498
+1773763634.8883243,train_step,300,2,system/cuda_max_memory_allocated_gb,94.47624206542969
+1773763647.6901762,eval_step,300,2,eval/loss,0.8076710475560945
+1773763647.6901762,eval_step,300,2,eval/duration_sec,12.799929299857467
+1773763704.81611,train_step,310,2,train/step_loss,0.9043312668800354
+1773763704.81611,train_step,310,2,train/step_real_loss,0.9043312668800354
+1773763704.81611,train_step,310,2,train/lr,6.661271481537157e-06
+1773763704.81611,train_step,310,2,perf/step_duration_sec,5.423097257036716
+1773763704.81611,train_step,310,2,perf/samples_per_sec,5.900687095087322
+1773763704.81611,train_step,310,2,perf/tokens_per_sec,4037.360748341778
+1773763704.81611,train_step,310,2,perf/logical_batch_size,32.0
+1773763704.81611,train_step,310,2,perf/logical_token_count,21895.0
+1773763704.81611,train_step,310,2,perf/gradient_accumulation_steps,4.0
+1773763704.81611,train_step,310,2,system/cuda_memory_allocated_gb,15.915565013885498
+1773763704.81611,train_step,310,2,system/cuda_max_memory_allocated_gb,94.47624206542969
+1773763760.6698298,train_step,320,2,train/step_loss,0.8735850304365158
+1773763760.6698298,train_step,320,2,train/step_real_loss,0.8735850304365158
+1773763760.6698298,train_step,320,2,train/lr,4.595197001556562e-06
+1773763760.6698298,train_step,320,2,perf/step_duration_sec,5.15081740077585
+1773763760.6698298,train_step,320,2,perf/samples_per_sec,6.212606176872034
+1773763760.6698298,train_step,320,2,perf/tokens_per_sec,4842.532370928723
+1773763760.6698298,train_step,320,2,perf/logical_batch_size,32.0
+1773763760.6698298,train_step,320,2,perf/logical_token_count,24943.0
+1773763760.6698298,train_step,320,2,perf/gradient_accumulation_steps,4.0
+1773763760.6698298,train_step,320,2,system/cuda_memory_allocated_gb,15.915565013885498
+1773763760.6698298,train_step,320,2,system/cuda_max_memory_allocated_gb,94.47624206542969
+1773763818.6010072,train_step,330,2,train/step_loss,0.7620985209941864
+1773763818.6010072,train_step,330,2,train/step_real_loss,0.7620985209941864
+1773763818.6010072,train_step,330,2,train/lr,2.8967918551955297e-06
+1773763818.6010072,train_step,330,2,perf/step_duration_sec,6.058840225916356
+1773763818.6010072,train_step,330,2,perf/samples_per_sec,5.281538843543317
+1773763818.6010072,train_step,330,2,perf/tokens_per_sec,4943.025213289962
+1773763818.6010072,train_step,330,2,perf/logical_batch_size,32.0
+1773763818.6010072,train_step,330,2,perf/logical_token_count,29949.0
+1773763818.6010072,train_step,330,2,perf/gradient_accumulation_steps,4.0
+1773763818.6010072,train_step,330,2,system/cuda_memory_allocated_gb,15.915565013885498
+1773763818.6010072,train_step,330,2,system/cuda_max_memory_allocated_gb,94.47624206542969
+1773763875.2762098,train_step,340,2,train/step_loss,0.9000806212425232
+1773763875.2762098,train_step,340,2,train/step_real_loss,0.9000806212425232
+1773763875.2762098,train_step,340,2,train/lr,1.5798090255558617e-06
+1773763875.2762098,train_step,340,2,perf/step_duration_sec,5.154371100012213
+1773763875.2762098,train_step,340,2,perf/samples_per_sec,6.2083228737496565
+1773763875.2762098,train_step,340,2,perf/tokens_per_sec,4376.285595724094
+1773763875.2762098,train_step,340,2,perf/logical_batch_size,32.0
+1773763875.2762098,train_step,340,2,perf/logical_token_count,22557.0
+1773763875.2762098,train_step,340,2,perf/gradient_accumulation_steps,4.0
+1773763875.2762098,train_step,340,2,system/cuda_memory_allocated_gb,15.915565013885498
+1773763875.2762098,train_step,340,2,system/cuda_max_memory_allocated_gb,94.47624206542969
+1773763932.3082285,train_step,350,2,train/step_loss,0.7404757142066956
+1773763932.3082285,train_step,350,2,train/step_real_loss,0.7404757142066956
+1773763932.3082285,train_step,350,2,train/lr,6.54912895420573e-07
+1773763932.3082285,train_step,350,2,perf/step_duration_sec,5.694617530796677
+1773763932.3082285,train_step,350,2,perf/samples_per_sec,5.619341391575985
+1773763932.3082285,train_step,350,2,perf/tokens_per_sec,4990.501968974935
+1773763932.3082285,train_step,350,2,perf/logical_batch_size,32.0
+1773763932.3082285,train_step,350,2,perf/logical_token_count,28419.0
+1773763932.3082285,train_step,350,2,perf/gradient_accumulation_steps,4.0
+1773763932.3082285,train_step,350,2,system/cuda_memory_allocated_gb,15.915565013885498
+1773763932.3082285,train_step,350,2,system/cuda_max_memory_allocated_gb,94.47624206542969
+1773763945.1132307,eval_step,350,2,eval/loss,0.807514699605795
+1773763945.1132307,eval_step,350,2,eval/duration_sec,12.803151289001107
+1773764000.5064592,train_step,360,2,train/step_loss,0.8157700151205063
+1773764000.5064592,train_step,360,2,train/step_real_loss,0.8157700151205063
+1773764000.5064592,train_step,360,2,train/lr,1.295928914885336e-07
+1773764000.5064592,train_step,360,2,perf/step_duration_sec,5.695594378747046
+1773764000.5064592,train_step,360,2,perf/samples_per_sec,5.618377621729371
+1773764000.5064592,train_step,360,2,perf/tokens_per_sec,4667.116060650316
+1773764000.5064592,train_step,360,2,perf/logical_batch_size,32.0
+1773764000.5064592,train_step,360,2,perf/logical_token_count,26582.0
+1773764000.5064592,train_step,360,2,perf/gradient_accumulation_steps,4.0
+1773764000.5064592,train_step,360,2,system/cuda_memory_allocated_gb,15.915565013885498
+1773764000.5064592,train_step,360,2,system/cuda_max_memory_allocated_gb,94.47624206542969
+1773764058.7973218,train_epoch,368,2,train/epoch_loss,0.856036927981092
+1773764058.7973218,train_epoch,368,2,train/epoch_real_loss,0.8280306565727147
+1773764058.7973218,train_epoch,368,2,train/epoch_canary_loss,3.6806401156922846
+1773764058.7973218,train_epoch,368,2,perf/epoch_duration_sec,1096.7485609338619
+1773764058.7973218,train_epoch,368,2,perf/epoch_samples_per_sec,43.38460217306761
+1773764058.7973218,train_epoch,368,2,perf/epoch_tokens_per_sec,34225.847506959835
+1773764058.7973218,train_epoch,368,2,perf/epoch_samples,47582.0
+1773764058.7973218,train_epoch,368,2,perf/epoch_tokens,37537149.0
+1773764058.7973218,train_epoch,368,2,system/cuda_epoch_peak_memory_gb,94.47624206542969
+1773764058.7973218,train_epoch,368,2,eval/loss,0.8075161480750794
+1773764058.7973218,train_epoch,368,2,eval/duration_sec,12.857198356185108
+1773764072.6744637,audit_epoch,368,2,audit/delta,1e-05
+1773764072.6744637,audit_epoch,368,2,audit/num_canaries,500.0
+1773764072.6744637,audit_epoch,368,2,audit/num_members,250.0
+1773764072.6744637,audit_epoch,368,2,audit/paper_guess_fraction,0.2
+1773764072.6744637,audit_epoch,368,2,audit/paper_guess_steps,20.0
+1773764072.6744637,audit_epoch,368,2,audit/loss/auc,0.968584
+1773764072.6744637,audit_epoch,368,2,audit/loss/empirical_epsilon/0.05,3.4791953936219215
+1773764072.6744637,audit_epoch,368,2,audit/loss/empirical_epsilon/0.01,3.023197554051876
+1773764072.6744637,audit_epoch,368,2,audit/loss/empirical_epsilon_details/0.05/epsilon,3.4791953936219215
+1773764072.6744637,audit_epoch,368,2,audit/loss/empirical_epsilon_details/0.05/num_guesses,100.0
+1773764072.6744637,audit_epoch,368,2,audit/loss/empirical_epsilon_details/0.05/correct_guesses,100.0
+1773764072.6744637,audit_epoch,368,2,audit/loss/empirical_epsilon_details/0.01/epsilon,3.023197554051876
+1773764072.6744637,audit_epoch,368,2,audit/loss/empirical_epsilon_details/0.01/num_guesses,100.0
+1773764072.6744637,audit_epoch,368,2,audit/loss/empirical_epsilon_details/0.01/correct_guesses,100.0
+1773764072.6744637,audit_epoch,368,2,audit/embedding/auc,0.883776
+1773764072.6744637,audit_epoch,368,2,audit/embedding/empirical_epsilon/0.05,3.4791953936219215
+1773764072.6744637,audit_epoch,368,2,audit/embedding/empirical_epsilon/0.01,3.023197554051876
+1773764072.6744637,audit_epoch,368,2,audit/embedding/empirical_epsilon_details/0.05/epsilon,3.4791953936219215
+1773764072.6744637,audit_epoch,368,2,audit/embedding/empirical_epsilon_details/0.05/num_guesses,100.0
+1773764072.6744637,audit_epoch,368,2,audit/embedding/empirical_epsilon_details/0.05/correct_guesses,100.0
+1773764072.6744637,audit_epoch,368,2,audit/embedding/empirical_epsilon_details/0.01/epsilon,3.023197554051876
+1773764072.6744637,audit_epoch,368,2,audit/embedding/empirical_epsilon_details/0.01/num_guesses,100.0
+1773764072.6744637,audit_epoch,368,2,audit/embedding/empirical_epsilon_details/0.01/correct_guesses,100.0
+1773764072.6744637,audit_epoch,368,2,perf/audit_duration_sec,7.556974642910063
+1773764086.367738,audit_final,368,2,audit/delta,1e-05
+1773764086.367738,audit_final,368,2,audit/num_canaries,500.0
+1773764086.367738,audit_final,368,2,audit/num_members,250.0
+1773764086.367738,audit_final,368,2,audit/paper_guess_fraction,0.2
+1773764086.367738,audit_final,368,2,audit/paper_guess_steps,20.0
+1773764086.367738,audit_final,368,2,audit/loss/auc,0.968584
+1773764086.367738,audit_final,368,2,audit/loss/empirical_epsilon/0.05,3.4791953936219215
+1773764086.367738,audit_final,368,2,audit/loss/empirical_epsilon/0.01,3.023197554051876
+1773764086.367738,audit_final,368,2,audit/loss/empirical_epsilon_details/0.05/epsilon,3.4791953936219215
+1773764086.367738,audit_final,368,2,audit/loss/empirical_epsilon_details/0.05/num_guesses,100.0
+1773764086.367738,audit_final,368,2,audit/loss/empirical_epsilon_details/0.05/correct_guesses,100.0
+1773764086.367738,audit_final,368,2,audit/loss/empirical_epsilon_details/0.01/epsilon,3.023197554051876
+1773764086.367738,audit_final,368,2,audit/loss/empirical_epsilon_details/0.01/num_guesses,100.0
+1773764086.367738,audit_final,368,2,audit/loss/empirical_epsilon_details/0.01/correct_guesses,100.0
+1773764086.367738,audit_final,368,2,audit/embedding/auc,0.883776
+1773764086.367738,audit_final,368,2,audit/embedding/empirical_epsilon/0.05,3.4791953936219215
+1773764086.367738,audit_final,368,2,audit/embedding/empirical_epsilon/0.01,3.023197554051876
+1773764086.367738,audit_final,368,2,audit/embedding/empirical_epsilon_details/0.05/epsilon,3.4791953936219215
+1773764086.367738,audit_final,368,2,audit/embedding/empirical_epsilon_details/0.05/num_guesses,100.0
+1773764086.367738,audit_final,368,2,audit/embedding/empirical_epsilon_details/0.05/correct_guesses,100.0
+1773764086.367738,audit_final,368,2,audit/embedding/empirical_epsilon_details/0.01/epsilon,3.023197554051876
+1773764086.367738,audit_final,368,2,audit/embedding/empirical_epsilon_details/0.01/num_guesses,100.0
+1773764086.367738,audit_final,368,2,audit/embedding/empirical_epsilon_details/0.01/correct_guesses,100.0
+1773764086.9161372,energy_final,368,,energy/codecarbon/duration,2345.9966679112986
+1773764086.9161372,energy_final,368,,energy/codecarbon/emissions,0.09022432714096462
+1773764086.9161372,energy_final,368,,energy/codecarbon/emissions_rate,3.8458847096868924e-05
+1773764086.9161372,energy_final,368,,energy/codecarbon/cpu_power,72.02285277932866
+1773764086.9161372,energy_final,368,,energy/codecarbon/gpu_power,3280.290622412428
+1773764086.9161372,energy_final,368,,energy/codecarbon/ram_power,54.0
+1773764086.9161372,energy_final,368,,energy/codecarbon/cpu_energy,0.045218986505725985
+1773764086.9161372,energy_final,368,,energy/codecarbon/gpu_energy,2.137964725370466
+1773764086.9161372,energy_final,368,,energy/codecarbon/ram_energy,0.03390259211605879
+1773764086.9161372,energy_final,368,,energy/codecarbon/energy_consumed,2.2170863039922497
+1773764086.9161372,energy_final,368,,energy/codecarbon/water_consumed,0.0
+1773764086.9161372,energy_final,368,,energy/codecarbon/cpu_count,256.0
+1773764086.9161372,energy_final,368,,energy/codecarbon/gpu_count,8.0
+1773764086.9161372,energy_final,368,,energy/codecarbon/longitude,16.1885
+1773764086.9161372,energy_final,368,,energy/codecarbon/latitude,58.594
+1773764086.9161372,energy_final,368,,energy/codecarbon/ram_total_size,1511.49019241333
+1773764086.9161372,energy_final,368,,energy/codecarbon/cpu_utilization_percent,3.3142796066695253
+1773764086.9161372,energy_final,368,,energy/codecarbon/gpu_utilization_percent,88.58721675929884
+1773764086.9161372,energy_final,368,,energy/codecarbon/ram_utilization_percent,5.287772552372644
+1773764086.9161372,energy_final,368,,energy/codecarbon/ram_used_gb,79.7947571596665
+1773764086.9161372,energy_final,368,,energy/codecarbon/pue,1.0
+1773764086.9161372,energy_final,368,,energy/codecarbon/wue,0.0

qwen3-4b-instruct/base/summary.json

@@ -0,0 +1,71 @@
+{
+  "audit/delta": 1e-05,
+  "audit/embedding/auc": 0.883776,
+  "audit/embedding/empirical_epsilon/0.01": 3.023197554051876,
+  "audit/embedding/empirical_epsilon/0.05": 3.4791953936219215,
+  "audit/embedding/empirical_epsilon_details/0.01/correct_guesses": 100.0,
+  "audit/embedding/empirical_epsilon_details/0.01/epsilon": 3.023197554051876,
+  "audit/embedding/empirical_epsilon_details/0.01/num_guesses": 100.0,
+  "audit/embedding/empirical_epsilon_details/0.05/correct_guesses": 100.0,
+  "audit/embedding/empirical_epsilon_details/0.05/epsilon": 3.4791953936219215,
+  "audit/embedding/empirical_epsilon_details/0.05/num_guesses": 100.0,
+  "audit/loss/auc": 0.968584,
+  "audit/loss/empirical_epsilon/0.01": 3.023197554051876,
+  "audit/loss/empirical_epsilon/0.05": 3.4791953936219215,
+  "audit/loss/empirical_epsilon_details/0.01/correct_guesses": 100.0,
+  "audit/loss/empirical_epsilon_details/0.01/epsilon": 3.023197554051876,
+  "audit/loss/empirical_epsilon_details/0.01/num_guesses": 100.0,
+  "audit/loss/empirical_epsilon_details/0.05/correct_guesses": 100.0,
+  "audit/loss/empirical_epsilon_details/0.05/epsilon": 3.4791953936219215,
+  "audit/loss/empirical_epsilon_details/0.05/num_guesses": 100.0,
+  "audit/num_canaries": 500.0,
+  "audit/num_members": 250.0,
+  "audit/paper_guess_fraction": 0.2,
+  "audit/paper_guess_steps": 20.0,
+  "energy/codecarbon/cpu_count": 256.0,
+  "energy/codecarbon/cpu_energy": 0.045218986505725985,
+  "energy/codecarbon/cpu_power": 72.02285277932866,
+  "energy/codecarbon/cpu_utilization_percent": 3.3142796066695253,
+  "energy/codecarbon/duration": 2345.9966679112986,
+  "energy/codecarbon/emissions": 0.09022432714096462,
+  "energy/codecarbon/emissions_rate": 3.8458847096868924e-05,
+  "energy/codecarbon/energy_consumed": 2.2170863039922497,
+  "energy/codecarbon/gpu_count": 8.0,
+  "energy/codecarbon/gpu_energy": 2.137964725370466,
+  "energy/codecarbon/gpu_power": 3280.290622412428,
+  "energy/codecarbon/gpu_utilization_percent": 88.58721675929884,
+  "energy/codecarbon/latitude": 58.594,
+  "energy/codecarbon/longitude": 16.1885,
+  "energy/codecarbon/pue": 1.0,
+  "energy/codecarbon/ram_energy": 0.03390259211605879,
+  "energy/codecarbon/ram_power": 54.0,
+  "energy/codecarbon/ram_total_size": 1511.49019241333,
+  "energy/codecarbon/ram_used_gb": 79.7947571596665,
+  "energy/codecarbon/ram_utilization_percent": 5.287772552372644,
+  "energy/codecarbon/water_consumed": 0.0,
+  "energy/codecarbon/wue": 0.0,
+  "eval/duration_sec": 12.857198356185108,
+  "eval/loss": 0.8075161480750794,
+  "perf/audit_duration_sec": 7.556974642910063,
+  "perf/epoch_duration_sec": 1096.7485609338619,
+  "perf/epoch_samples": 47582.0,
+  "perf/epoch_samples_per_sec": 43.38460217306761,
+  "perf/epoch_tokens": 37537149.0,
+  "perf/epoch_tokens_per_sec": 34225.847506959835,
+  "perf/gradient_accumulation_steps": 4.0,
+  "perf/logical_batch_size": 32.0,
+  "perf/logical_token_count": 26582.0,
+  "perf/samples_per_sec": 5.618377621729371,
+  "perf/step_duration_sec": 5.695594378747046,
+  "perf/tokens_per_sec": 4667.116060650316,
+  "system/cuda_epoch_peak_memory_gb": 94.47624206542969,
+  "system/cuda_max_memory_allocated_gb": 94.47624206542969,
+  "system/cuda_memory_allocated_gb": 15.915565013885498,
+  "train/epoch_canary_loss": 3.6806401156922846,
+  "train/epoch_loss": 0.856036927981092,
+  "train/epoch_real_loss": 0.8280306565727147,
+  "train/lr": 1.295928914885336e-07,
+  "train/step_canary_loss": 4.125,
+  "train/step_loss": 0.8157700151205063,
+  "train/step_real_loss": 0.8157700151205063
+}

qwen3-4b-instruct/base/tensorboard/events.out.tfevents.1773761739.7b654b6988b0.32156.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b55227175e1ecbcf79b12550edd079956b7869b97adc97b1aadf0a02287d39a
+size 36378

qwen3-4b-instruct/base/tokenizer/chat_template.jinja

@@ -0,0 +1,61 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if message.content is string %}
+        {%- set content = message.content %}
+    {%- else %}
+        {%- set content = '' %}
+    {%- endif %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

qwen3-4b-instruct/base/tokenizer/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0e9c8aef460c70c1e1c32afe895f455856c0075e5706f06e6d80b2f581137715
+size 11517150

qwen3-4b-instruct/base/tokenizer/tokenizer_config.json

@@ -0,0 +1,516 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "865331112869",
+    "569765693871",
+    "485177821815",
+    "135441121756",
+    "367459894796",
+    "877482678543",
+    "457919547633",
+    "765474393376",
+    "114848338811",
+    "746285987371",
+    "649291669397",
+    "927914615679",
+    "445925149649",
+    "691587454538",
+    "143777992227",
+    "997981281989",
+    "425949483533",
+    "982993456429",
+    "718726519731",
+    "172599315861",
+    "643489267333",
+    "282322838685",
+    "781653545886",
+    "796415361892",
+    "841991688488",
+    "211411365397",
+    "698218415444",
+    "355977139358",
+    "682564697312",
+    "383837596997",
+    "689362171782",
+    "749966767285",
+    "753159165157",
+    "795693824762",
+    "669689115557",
+    "327491773134",
+    "983569279932",
+    "612128769512",
+    "374327157578",
+    "311632789559",
+    "523918658846",
+    "765981581453",
+    "794825141891",
+    "873898736873",
+    "447445629421",
+    "473822473819",
+    "181439694557",
+    "592538279337",
+    "668134915514",
+    "643692393748",
+    "696651276628",
+    "853859348234",
+    "778466723723",
+    "929826356991",
+    "272362973463",
+    "694235616268",
+    "281673864127",
+    "479676316326",
+    "646979124677",
+    "922327493433",
+    "883685933161",
+    "264259917554",
+    "836746273134",
+    "658481324922",
+    "481884157827",
+    "587787496812",
+    "579184949249",
+    "912193598348",
+    "529679678956",
+    "795838284624",
+    "159337222655",
+    "173781362446",
+    "773687856563",
+    "535787224917",
+    "351885857332",
+    "578827344666",
+    "198462689911",
+    "722618266242",
+    "952872416512",
+    "517778845323",
+    "749665846687",
+    "661436365453",
+    "259666844669",
+    "242851284913",
+    "514532995959",
+    "161588262349",
+    "742765629356",
+    "225164373623",
+    "676539973863",
+    "826214551218",
+    "182345464792",
+    "232776999554",
+    "337326533813",
+    "676676697292",
+    "929185622831",
+    "545512344383",
+    "499444466686",
+    "314697386682",
+    "517379856925",
+    "379557332953",
+    "614797267726",
+    "429781429464",
+    "922466849763",
+    "721737645236",
+    "479227349997",
+    "136931728327",
+    "259533577263",
+    "488538864842",
+    "937495658852",
+    "489991411364",
+    "499148455254",
+    "441373944925",
+    "899151413682",
+    "467893531755",
+    "527117488925",
+    "928335588653",
+    "374439448821",
+    "879425227932",
+    "867678158885",
+    "399749397872",
+    "129693547287",
+    "689285841825",
+    "771619544974",
+    "724883568652",
+    "516968424863",
+    "733737988257",
+    "852347289392",
+    "296953381169",
+    "377273562477",
+    "262296912232",
+    "547149832394",
+    "298464134954",
+    "216667245274",
+    "843998562287",
+    "572154333646",
+    "124589118494",
+    "841824384614",
+    "232896526252",
+    "295448593321",
+    "123741461297",
+    "653573457168",
+    "196735786156",
+    "377338713663",
+    "964342468552",
+    "586855179568",
+    "484773717614",
+    "894885246797",
+    "677896358599",
+    "848845611563",
+    "851852651677",
+    "398549545767",
+    "454244839926",
+    "799364566435",
+    "967114116556",
+    "817378986438",
+    "233795848681",
+    "824387273757",
+    "916198946615",
+    "563117729724",
+    "951794811935",
+    "374598961236",
+    "922867396683",
+    "765737843639",
+    "175469284871",
+    "231853711778",
+    "662426712668",
+    "711412347158",
+    "753466987363",
+    "513361312532",
+    "712992815957",
+    "971621888444",
+    "829235161526",
+    "585544633356",
+    "582471228164",
+    "678666359123",
+    "557533689478",
+    "632962475133",
+    "484489193824",
+    "489562189822",
+    "589547936288",
+    "363214487524",
+    "244885399387",
+    "431751228368",
+    "433581868192",
+    "486391569221",
+    "185438575221",
+    "126574388585",
+    "741757479784",
+    "529854679937",
+    "996116119839",
+    "616248973917",
+    "763531783491",
+    "955456118295",
+    "364196983365",
+    "195792996468",
+    "151859598873",
+    "399223169721",
+    "938488813964",
+    "961981959227",
+    "183368827562",
+    "533417736566",
+    "786391632558",
+    "665661658354",
+    "693281533643",
+    "475794684356",
+    "652154162978",
+    "753233719644",
+    "668514843129",
+    "819162623892",
+    "941169431859",
+    "877385381798",
+    "752644929761",
+    "881136466196",
+    "275597777299",
+    "731681792655",
+    "961133895172",
+    "864718285734",
+    "963852916563",
+    "319584985416",
+    "563365646341",
+    "811371928234",
+    "837131396371",
+    "267514771964",
+    "944513428457",
+    "117298239631",
+    "158142752582",
+    "252867443568",
+    "839269684865",
+    "612788593128",
+    "145669731981",
+    "121557291859",
+    "245416776926",
+    "799417897197",
+    "997958836435",
+    "892336777248",
+    "158929292238",
+    "581976444672",
+    "897784492783",
+    "492373714791",
+    "512659818733",
+    "881112998642",
+    "619454958782",
+    "431149748713",
+    "624221476921",
+    "125866399464",
+    "339882449689",
+    "186198784585",
+    "943193294691",
+    "955668961269",
+    "232787996724",
+    "215671314196",
+    "286173241916",
+    "745977673725",
+    "556976448182",
+    "599961512792",
+    "766294538337",
+    "934912591213",
+    "295118729589",
+    "529455466433",
+    "196119929397",
+    "379571934299",
+    "251789649997",
+    "564544131355",
+    "244371196654",
+    "384598329253",
+    "887753195844",
+    "364947325679",
+    "655517954651",
+    "673948786567",
+    "857231548835",
+    "816115936673",
+    "644234165531",
+    "182782912224",
+    "234316622259",
+    "421369185549",
+    "434632855397",
+    "921889371893",
+    "415956914763",
+    "598916996413",
+    "773671349113",
+    "952465217972",
+    "117657531962",
+    "729825168745",
+    "691315125346",
+    "768461952319",
+    "664847713559",
+    "953267689786",
+    "886464195129",
+    "824488329416",
+    "837873762491",
+    "532833541879",
+    "669183782449",
+    "941976537588",
+    "739394546916",
+    "267954879268",
+    "637551427887",
+    "217756494954",
+    "524444658383",
+    "117783274348",
+    "138218735276",
+    "814611949491",
+    "711641973413",
+    "499156317423",
+    "515856611931",
+    "454164859837",
+    "345271433112",
+    "462294118988",
+    "511785788222",
+    "497294727353",
+    "866519986723",
+    "334513529294",
+    "549946382131",
+    "284445431422",
+    "396521188476",
+    "421435255895",
+    "133373659361",
+    "322683334381",
+    "228358422847",
+    "291762694874",
+    "143182978129",
+    "511923256573",
+    "327158398268",
+    "879764613759",
+    "564395222747",
+    "451161679736",
+    "538631466654",
+    "221762325616",
+    "218391991184",
+    "322589379462",
+    "876537814263",
+    "152676556624",
+    "332522971941",
+    "884354318946",
+    "513349618943",
+    "116639746413",
+    "635185846287",
+    "993832498489",
+    "813981174797",
+    "438745114173",
+    "983493951323",
+    "724492262421",
+    "622553389126",
+    "889965243135",
+    "364492359246",
+    "154962668224",
+    "179564995814",
+    "418412875665",
+    "718951851413",
+    "699446724178",
+    "624266421831",
+    "815458725125",
+    "455423278865",
+    "393741199486",
+    "328552864359",
+    "211662639865",
+    "218784516525",
+    "762486672996",
+    "142799718159",
+    "858146415154",
+    "767858144912",
+    "571317457151",
+    "635127952696",
+    "116427191984",
+    "268921994538",
+    "523937669294",
+    "165429152138",
+    "739246183345",
+    "591464355756",
+    "212985874612",
+    "191887635211",
+    "967214577653",
+    "119342152414",
+    "946444632795",
+    "618423867817",
+    "228565148417",
+    "729116422489",
+    "527874729936",
+    "739784153482",
+    "387763951128",
+    "331369926711",
+    "562716493614",
+    "739667844957",
+    "562389434565",
+    "256497188281",
+    "859927364588",
+    "417668946583",
+    "357621613582",
+    "438435178228",
+    "485692541169",
+    "825815739116",
+    "342221452223",
+    "697747991249",
+    "716763689965",
+    "141499982867",
+    "818479319499",
+    "336813343298",
+    "594688742928",
+    "472129283475",
+    "514354144759",
+    "349249721685",
+    "546276298359",
+    "353755529131",
+    "315534574435",
+    "523723475786",
+    "215826764872",
+    "367968398551",
+    "569853653352",
+    "389715484387",
+    "293847485454",
+    "714738141818",
+    "178478368922",
+    "581493616981",
+    "589439538674",
+    "846657726193",
+    "722339992679",
+    "138154781148",
+    "757785319772",
+    "492516914298",
+    "919181521716",
+    "985781138935",
+    "476969195485",
+    "313145133463",
+    "758963111966",
+    "147541537162",
+    "557163366873",
+    "144373897488",
+    "522515164754",
+    "724964923582",
+    "284776712475",
+    "375429755114",
+    "181233596124",
+    "948585673431",
+    "243165586174",
+    "396847976144",
+    "997724962668",
+    "558837194455",
+    "163165456396",
+    "378749551722",
+    "161238482259",
+    "754978243758",
+    "195388849133",
+    "229775525672",
+    "262437452884",
+    "441377892146",
+    "451885565366",
+    "981277526855",
+    "762495822823",
+    "368763327262",
+    "757422791351",
+    "636324136426",
+    "214193645583",
+    "412843856172",
+    "179386156569",
+    "756916173536",
+    "892697125149",
+    "625334487352",
+    "941861857715",
+    "887417525236",
+    "649516938598",
+    "717628619782",
+    "438124184139",
+    "547563892268",
+    "856317483891",
+    "313313831273",
+    "371496153876",
+    "587541149322",
+    "265847332563",
+    "449549215429",
+    "163497196769",
+    "861342291298",
+    "268433315926",
+    "774679513717",
+    "851254219729",
+    "583527834464",
+    "488496781997",
+    "556814553861",
+    "482829231639",
+    "618878266619",
+    "147444452794",
+    "949235426629",
+    "357299947518",
+    "175528632226",
+    "645527857972",
+    "186872457894",
+    "552738847828",
+    "626748382482",
+    "921894985642",
+    "943878645871",
+    "859289776479",
+    "614583493135",
+    "933775286797",
+    "332234613346",
+    "325196781219",
+    "142526557681",
+    "356722692178",
+    "449318681694",
+    "687284547244",
+    "947262995132",
+    "893974619684",
+    "797238311233"
+  ],
+  "is_local": false,
+  "model_max_length": 1010000,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

qwen3-4b-instruct/base/train.log

@@ -0,0 +1,43 @@
+2026-03-17 15:38:14,810 [INFO] new_opacus_codex.train_steps: epoch=1 step=10 loss=1.2991
+2026-03-17 15:39:10,798 [INFO] new_opacus_codex.train_steps: epoch=1 step=20 loss=1.1180
+2026-03-17 15:40:07,300 [INFO] new_opacus_codex.train_steps: epoch=1 step=30 loss=1.1701
+2026-03-17 15:41:05,156 [INFO] new_opacus_codex.train_steps: epoch=1 step=40 loss=1.0721
+2026-03-17 15:42:01,272 [INFO] new_opacus_codex.train_steps: epoch=1 step=50 loss=0.9707
+2026-03-17 15:42:14,073 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=50 eval_loss=0.8534 duration_sec=12.80
+2026-03-17 15:43:10,275 [INFO] new_opacus_codex.train_steps: epoch=1 step=60 loss=1.0486
+2026-03-17 15:44:09,206 [INFO] new_opacus_codex.train_steps: epoch=1 step=70 loss=0.9598
+2026-03-17 15:45:05,719 [INFO] new_opacus_codex.train_steps: epoch=1 step=80 loss=0.9536
+2026-03-17 15:46:03,202 [INFO] new_opacus_codex.train_steps: epoch=1 step=90 loss=0.8977
+2026-03-17 15:46:59,339 [INFO] new_opacus_codex.train_steps: epoch=1 step=100 loss=0.9494
+2026-03-17 15:47:12,127 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=100 eval_loss=0.8295 duration_sec=12.79
+2026-03-17 15:48:09,329 [INFO] new_opacus_codex.train_steps: epoch=1 step=110 loss=0.9597
+2026-03-17 15:49:04,919 [INFO] new_opacus_codex.train_steps: epoch=1 step=120 loss=0.9211
+2026-03-17 15:50:01,866 [INFO] new_opacus_codex.train_steps: epoch=1 step=130 loss=0.9734
+2026-03-17 15:50:59,563 [INFO] new_opacus_codex.train_steps: epoch=1 step=140 loss=0.9642
+2026-03-17 15:51:57,620 [INFO] new_opacus_codex.train_steps: epoch=1 step=150 loss=0.9605
+2026-03-17 15:52:10,416 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=150 eval_loss=0.8174 duration_sec=12.79
+2026-03-17 15:53:05,969 [INFO] new_opacus_codex.train_steps: epoch=1 step=160 loss=0.8810
+2026-03-17 15:54:01,078 [INFO] new_opacus_codex.train_steps: epoch=1 step=170 loss=0.9470
+2026-03-17 15:54:58,843 [INFO] new_opacus_codex.train_steps: epoch=1 step=180 loss=0.8985
+2026-03-17 15:56:24,331 [INFO] new_opacus_codex.train_steps: epoch=2 step=190 loss=0.8604
+2026-03-17 15:57:20,715 [INFO] new_opacus_codex.train_steps: epoch=2 step=200 loss=0.8534
+2026-03-17 15:57:33,520 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=200 eval_loss=0.8111 duration_sec=12.80
+2026-03-17 15:58:30,153 [INFO] new_opacus_codex.train_steps: epoch=2 step=210 loss=0.8451
+2026-03-17 15:59:28,110 [INFO] new_opacus_codex.train_steps: epoch=2 step=220 loss=0.8697
+2026-03-17 16:00:24,220 [INFO] new_opacus_codex.train_steps: epoch=2 step=230 loss=0.8622
+2026-03-17 16:01:22,063 [INFO] new_opacus_codex.train_steps: epoch=2 step=240 loss=0.8586
+2026-03-17 16:02:17,716 [INFO] new_opacus_codex.train_steps: epoch=2 step=250 loss=0.8370
+2026-03-17 16:02:30,526 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=250 eval_loss=0.8086 duration_sec=12.81
+2026-03-17 16:03:26,908 [INFO] new_opacus_codex.train_steps: epoch=2 step=260 loss=0.8445
+2026-03-17 16:04:22,858 [INFO] new_opacus_codex.train_steps: epoch=2 step=270 loss=0.8807
+2026-03-17 16:05:19,992 [INFO] new_opacus_codex.train_steps: epoch=2 step=280 loss=0.8859
+2026-03-17 16:06:17,139 [INFO] new_opacus_codex.train_steps: epoch=2 step=290 loss=0.8174
+2026-03-17 16:07:14,887 [INFO] new_opacus_codex.train_steps: epoch=2 step=300 loss=0.8824
+2026-03-17 16:07:27,689 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=300 eval_loss=0.8077 duration_sec=12.80
+2026-03-17 16:08:24,815 [INFO] new_opacus_codex.train_steps: epoch=2 step=310 loss=0.8392
+2026-03-17 16:09:20,669 [INFO] new_opacus_codex.train_steps: epoch=2 step=320 loss=0.8540
+2026-03-17 16:10:18,600 [INFO] new_opacus_codex.train_steps: epoch=2 step=330 loss=0.8799
+2026-03-17 16:11:15,275 [INFO] new_opacus_codex.train_steps: epoch=2 step=340 loss=0.8421
+2026-03-17 16:12:12,307 [INFO] new_opacus_codex.train_steps: epoch=2 step=350 loss=0.9073
+2026-03-17 16:12:25,113 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=350 eval_loss=0.8075 duration_sec=12.80
+2026-03-17 16:13:20,505 [INFO] new_opacus_codex.train_steps: epoch=2 step=360 loss=0.8343