Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,9 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
+wandb/run-20250508_192410-bvjuhkx9/run-bvjuhkx9.wandb filter=lfs diff=lfs merge=lfs -text
+wandb/run-20250508_195305-eg5dmumm/run-eg5dmumm.wandb filter=lfs diff=lfs merge=lfs -text
+wandb/run-20250508_200345-11tfsh2x/run-11tfsh2x.wandb filter=lfs diff=lfs merge=lfs -text
+wandb/run-20250508_200812-r9ll1pfp/run-r9ll1pfp.wandb filter=lfs diff=lfs merge=lfs -text
+wandb/run-20250508_212154-is8630ik/run-is8630ik.wandb filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,202 @@
----
-license: apache-2.0
----
+---
+base_model: meta-llama/Llama-3.1-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2

adapter_config.json

@@ -0,0 +1,43 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "LlamaForCausalLM",
+    "parent_library": "transformers.models.llama.modeling_llama"
+  },
+  "base_model_name_or_path": "meta-llama/Llama-3.1-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "up_proj",
+    "k_proj",
+    "v_proj",
+    "o_proj",
+    "lm_head",
+    "q_proj",
+    "down_proj"
+  ],
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f80f97ba737f3c031e76d69dccad49a09d735bfb2e9091172f15f50637eb165
+size 1138873304

arguments.json

@@ -0,0 +1,60 @@
+{
+    "model_name_or_path": "meta-llama/Llama-3.1-8B-Instruct",
+    "recompute_baseline": false,
+    "cache_dir": "/home/panda/pda-llm/cache/sft-tools",
+    "max_length": 2048,
+    "trust_remote_code": true,
+    "train_datasets": [
+        [
+            "tools",
+            {
+                "proportion": 1.0
+            }
+        ]
+    ],
+    "eval_datasets": null,
+    "safety_ratio_tol": 10.0,
+    "important_sft": false,
+    "resilient_coeff": 1.0,
+    "epochs": 3,
+    "per_device_train_batch_size": 1,
+    "per_device_eval_batch_size": 1,
+    "gradient_accumulation_steps": 48,
+    "gradient_checkpointing": true,
+    "lr": 0.0001,
+    "lr_scheduler_type": "cosine",
+    "lr_warmup_ratio": 0.1,
+    "weight_decay": 0.0,
+    "seed": 42,
+    "fp16": false,
+    "bf16": true,
+    "tf32": false,
+    "lora_r": 16,
+    "lora_alpha": 32,
+    "lora_dropout": 0.05,
+    "eval_strategy": "epoch",
+    "eval_interval": 1000000,
+    "need_eval": true,
+    "eval_split_ratio": null,
+    "output_dir": "/home/panda/pda-llm/output/sft-tools/run-false-1-10",
+    "log_type": "wandb",
+    "log_dir": "/home/panda/pda-llm/output/sft-tools/run-false-1-10",
+    "log_project": "TOOLS-SFT",
+    "log_run_name": "tools-sft-2025-05-08-21-21-54",
+    "save_16bit": false,
+    "save_interval": 1000000,
+    "local_rank": 0,
+    "zero_stage": 0,
+    "offload": "none",
+    "deepspeed": false,
+    "deepspeed_config": null,
+    "deepscale": false,
+    "deepscale_config": null,
+    "global_rank": 0,
+    "device": {
+        "type": "torch.device",
+        "repr": "device(type='cuda', index=0)"
+    },
+    "num_update_steps_per_epoch": 118,
+    "total_training_steps": 354
+}

arguments.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fafb0bdf1ddeef98c143db9a23a20856b243c468bf8542e9bda3541f03d46598
+size 1225

config.json

@@ -0,0 +1,37 @@
+{
+  "_attn_implementation_autoset": true,
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 128000,
+  "eos_token_id": 128009,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 131072,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 128256,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": {
+    "factor": 8.0,
+    "high_freq_factor": 4.0,
+    "low_freq_factor": 1.0,
+    "original_max_position_embeddings": 8192,
+    "rope_type": "llama3"
+  },
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.51.3",
+  "use_cache": true,
+  "vocab_size": 128258
+}

environ.txt

@@ -0,0 +1,58 @@
+BROWSER=/home/panda/.vscode-server/cli/servers/Stable-4949701c880d4bdb949e3c0e6b400288da7f474b/server/bin/helpers/browser.sh
+COLORTERM=truecolor
+CONDA_DEFAULT_ENV=pda
+CONDA_EXE=/home/panda/miniconda3/bin/conda
+CONDA_PREFIX=/home/panda/miniconda3/envs/pda
+CONDA_PREFIX_1=/home/panda/miniconda3
+CONDA_PROMPT_MODIFIER=(pda) 
+CONDA_PYTHON_EXE=/home/panda/miniconda3/bin/python
+CONDA_SHLVL=2
+CROSS_RANK=0
+CROSS_SIZE=1
+CUDA_MODULE_LOADING=LAZY
+CUDA_VISIBLE_DEVICES=0,1
+DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1008/bus
+GIT_ASKPASS=/home/panda/.vscode-server/cli/servers/Stable-4949701c880d4bdb949e3c0e6b400288da7f474b/server/extensions/git/dist/askpass.sh
+HOME=/home/panda
+LANG=en_US.UTF-8
+LOCAL_RANK=0
+LOCAL_SIZE=2
+LOGLEVEL=WARNING
+LOGNAME=panda
+MASTER_ADDR=127.0.0.1
+MASTER_PORT=35777
+MOTD_SHOWN=pam
+OLDPWD=/home/panda
+PATH=/home/panda/miniconda3/envs/pda/bin:/home/panda/miniconda3/condabin:/home/panda/.vscode-server/cli/servers/Stable-4949701c880d4bdb949e3c0e6b400288da7f474b/server/bin/remote-cli:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/panda/.vscode-server/data/User/globalStorage/github.copilot-chat/debugCommand
+PWD=/home/panda/pda-llm/scripts
+PYTHONHASHSEED=42
+PYTHONPATH=/home/panda/pda-llm
+RANK=0
+SHELL=/bin/sh
+SHLVL=4
+SSH_CLIENT=10.100.129.163 58321 22
+SSH_CONNECTION=10.100.129.163 58321 158.130.110.127 22
+SSL_CERT_DIR=/usr/lib/ssl/certs
+SSL_CERT_FILE=/usr/lib/ssl/certs/ca-certificates.crt
+TERM=screen
+TERM_PROGRAM=vscode
+TERM_PROGRAM_VERSION=1.99.2
+TMUX=/tmp//tmux-1008/default,750869,0
+TMUX_PANE=%0
+USER=panda
+VSCODE_GIT_ASKPASS_EXTRA_ARGS=
+VSCODE_GIT_ASKPASS_MAIN=/home/panda/.vscode-server/cli/servers/Stable-4949701c880d4bdb949e3c0e6b400288da7f474b/server/extensions/git/dist/askpass-main.js
+VSCODE_GIT_ASKPASS_NODE=/home/panda/.vscode-server/cli/servers/Stable-4949701c880d4bdb949e3c0e6b400288da7f474b/server/node
+VSCODE_GIT_IPC_HANDLE=/run/user/1008/vscode-git-3d8733097b.sock
+VSCODE_IPC_HOOK_CLI=/run/user/1008/vscode-ipc-6dd97596-0fde-4c4b-bbcb-61896b21f983.sock
+WANDB_API_KEY=6a71e7fad84fe1aa8f6ccaa01e4e02fcf4c7ffb4
+WANDB_ENTITY=alelab
+WANDB_MODE=online
+WANDB_SERVICE=2-1548317-tcp-localhost-45581
+WORLD_SIZE=2
+XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop
+XDG_RUNTIME_DIR=/run/user/1008
+XDG_SESSION_CLASS=user
+XDG_SESSION_ID=1945
+XDG_SESSION_TYPE=tty
+_=/home/panda/miniconda3/envs/pda/bin/deepspeed

script.sh

@@ -0,0 +1,197 @@
+#!/usr/bin/env bash
+#
+# Copyright 2023-2024 PKU-Alignment Team. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ==============================================================================
+
+if [ -z "${BASH_VERSION}" ]; then
+	echo "Please use bash to run this script." >&2
+	exit 1
+fi
+
+set -x
+
+SCRIPT_DIR="$(cd "$(dirname "$0")" &>/dev/null && pwd)"
+ROOT_DIR="$(dirname "${SCRIPT_DIR}")"
+export PYTHONPATH="${ROOT_DIR}${PYTHONPATH:+:${PYTHONPATH}}"
+export LOGLEVEL="${LOGLEVEL:-WARNING}"
+export WANDB_MODE=online
+export WANDB_ENTITY="alelab"
+
+MODEL_NAME_OR_PATH="meta-llama/Llama-3.1-8B-Instruct"
+timestamp="$(date +%Y%m%d-%H%M%S)"
+unset HOSTFILE
+ZERO_STAGE=0
+OFFLOAD="none"
+IMPORTANT_SFT=False
+# GRIDSEARCH PARAMS
+SAFETY_RATIO_TOL=10
+RESILIENT_COEFF=1
+LEARNING_RATE=1e-4
+EPOCHS=3
+while [[ "$#" -gt 0 ]]; do
+	arg="$1"
+	shift
+	case "${arg}" in
+		--model_name_or_path)
+			MODEL_NAME_OR_PATH="$1"
+			shift
+			;;
+		--model_name_or_path=*)
+			MODEL_NAME_OR_PATH="${arg#*=}"
+			;;
+		--output_dir)
+			OUTPUT_DIR="$1"
+			shift
+			;;
+		--output_dir=*)
+			OUTPUT_DIR="${arg#*=}"
+			;;
+		--hostfile)
+			HOSTFILE="$1"
+			shift
+			;;
+		--hostfile=*)
+			HOSTFILE="${arg#*=}"
+			;;
+		--zero_stage)
+			ZERO_STAGE="$1"
+			shift
+			;;
+		--zero_stage=*)
+			ZERO_STAGE="${arg#*=}"
+			;;
+		--offload)
+			OFFLOAD="$1"
+			shift
+			;;
+		--offload=*)
+			OFFLOAD="${arg#*=}"
+			;;
+		--safety_ratio_tol)
+			SAFETY_RATIO_TOL="$1"
+			shift
+			;;
+		--safety_ratio_tol=*)
+			SAFETY_RATIO_TOL="${arg#*=}"
+			;;
+		--resilient_coeff)
+			RESILIENT_COEFF="$1"
+			shift
+			;;
+		--resilient_coeff=*)
+			RESILIENT_COEFF="${arg#*=}"
+			;;
+		--learning_rate)
+			LEARNING_RATE="$1"
+			shift
+			;;
+		--learning_rate=*)
+			LEARNING_RATE="${arg#*=}"
+			;;
+		--epochs)
+			EPOCHS="$1"
+			shift
+			;;
+		--epochs=*)
+			EPOCHS="${arg#*=}"
+			;;
+		--important_sft)
+			IMPORTANT_SFT="$1"
+			shift
+			;;
+		--important_sft=*)
+			IMPORTANT_SFT="${arg#*=}"
+			;;
+		*)
+			echo "Unknown parameter passed: '${arg}'" >&2
+			exit 1
+			;;
+	esac
+done
+
+OUTPUT_DIR="${ROOT_DIR}/output/sft-tools/run-${IMPORTANT_SFT}-${RESILIENT_COEFF}-${SAFETY_RATIO_TOL}"
+mkdir -p "${OUTPUT_DIR}"
+OUTPUT_DIR="$(cd "${OUTPUT_DIR}" &>/dev/null && pwd)"
+if [[ ! -f "${OUTPUT_DIR}/.gitignore" ]]; then
+	echo '*' >"${OUTPUT_DIR}/.gitignore"
+fi
+
+cp -f "$0" "${OUTPUT_DIR}/script.sh"
+
+export WANDB_API_KEY="6a71e7fad84fe1aa8f6ccaa01e4e02fcf4c7ffb4"
+if [[ -z "${WANDB_API_KEY}" ]]; then
+	export WANDB_MODE="offline"
+fi
+
+MASTER_PORT_START=10000
+MASTER_PORT_END=65535
+MASTER_PORT="$(
+	comm -23 \
+		<(seq "${MASTER_PORT_START}" "${MASTER_PORT_END}" | sort) \
+		<(ss -Htan | awk '{ print $4 }' | awk -F ':' '{ print $NF }' | sort -u) |
+		shuf | head -n 1
+)"
+
+DEEPSPEED_ARGS=()
+if [[ -n "${HOSTFILE+x}" ]]; then
+	DEEPSPEED_ARGS+=("--hostfile" "${HOSTFILE}")
+fi
+DEEPSPEED_ARGS+=("--master_port" "${MASTER_PORT}")
+
+exec 1> >(tee "${OUTPUT_DIR}/stdout.log" >&1) 2> >(tee "${OUTPUT_DIR}/stderr.log" >&2)
+echo "--------- Environment sanity check ---------"
+echo "shell:        $0 running under bash $BASH_VERSION"
+echo "conda env:    ${CONDA_DEFAULT_ENV:-<none>}"
+echo "python:       $(which python)"
+python - <<'PY'
+import sys, os
+print("sys.executable :", sys.executable)
+print("python version :", sys.version.split()[0])
+print("CONDA_PREFIX   :", os.environ.get("CONDA_PREFIX"))
+PY
+echo "deepspeed:    $(command -v deepspeed)"
+echo "--------------------------------------------"
+
+CUDA_VISIBLE_DEVICES=0,1 deepspeed "${DEEPSPEED_ARGS[@]}" \
+	--module safe_rlhf.algorithms.tools_ft \
+	--train_datasets "tools" \
+	--model_name_or_path "${MODEL_NAME_OR_PATH}" \
+	--cache_dir "${ROOT_DIR}/cache/sft-tools" \
+	--important_sft "${IMPORTANT_SFT}"	 \
+	--max_length 2048 \
+	--trust_remote_code True \
+	--epochs "${EPOCHS}"  \
+	--per_device_train_batch_size 1 \
+	--per_device_eval_batch_size 1 \
+	--gradient_accumulation_steps 48 \
+	--gradient_checkpointing \
+	--learning_rate  "${LEARNING_RATE}" \
+	--lr_scheduler_type cosine \
+	--lr_warmup_ratio 0.1 \
+	--weight_decay 0.0 \
+	--seed 42 \
+	--output_dir "${OUTPUT_DIR}" \
+	--log_type wandb \
+	--log_project TOOLS-SFT \
+	--zero_stage "${ZERO_STAGE}" \
+	--offload "${OFFLOAD}" \
+	--safety_ratio_tol "${SAFETY_RATIO_TOL}" \
+	--resilient_coeff "${RESILIENT_COEFF}" \
+	--lora_r "16" \
+	--lora_alpha "32" \
+	--lora_dropout "0.05" \
+	--bf16 True \
+	--fp16 False \
+	--tf32 False

special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<|begin_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|eot_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

stdout.log

@@ -0,0 +1,50 @@
+--------- Environment sanity check ---------
+shell:        ./sft-tools.sh running under bash 5.0.17(1)-release
+conda env:    pda
+python:       /home/panda/miniconda3/envs/pda/bin/python
+sys.executable : /home/panda/miniconda3/envs/pda/bin/python
+python version : 3.11.11
+CONDA_PREFIX   : /home/panda/miniconda3/envs/pda
+deepspeed:    /home/panda/miniconda3/envs/pda/bin/deepspeed
+--------------------------------------------
+[2025-05-08 20:43:00,175] [INFO] [real_accelerator.py:239:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-05-08 20:43:02,239] [WARNING] [runner.py:215:fetch_hostfile] Unable to find hostfile, will proceed with training with local resources only.
+Detected VISIBLE_DEVICES=0,1: setting --include=localhost:0,1
+[2025-05-08 20:43:02,239] [INFO] [runner.py:605:main] cmd = /home/panda/miniconda3/envs/pda/bin/python -u -m deepspeed.launcher.launch --world_info=eyJsb2NhbGhvc3QiOiBbMCwgMV19 --master_addr=127.0.0.1 --master_port=35777 --module --enable_each_rank_log=None safe_rlhf.algorithms.tools_ft --train_datasets tools --model_name_or_path meta-llama/Llama-3.1-8B-Instruct --cache_dir /home/panda/pda-llm/cache/sft-tools --important_sft false --max_length 2048 --trust_remote_code True --epochs 3 --per_device_train_batch_size 1 --per_device_eval_batch_size 1 --gradient_accumulation_steps 48 --gradient_checkpointing --learning_rate 1e-4 --lr_scheduler_type cosine --lr_warmup_ratio 0.1 --weight_decay 0.0 --seed 42 --output_dir /home/panda/pda-llm/output/sft-tools/run-false-1-10 --log_type wandb --log_project TOOLS-SFT --zero_stage 0 --offload none --safety_ratio_tol 10 --resilient_coeff 1 --lora_r 16 --lora_alpha 32 --lora_dropout 0.05 --bf16 True --fp16 False --tf32 False
+[2025-05-08 20:43:03,384] [INFO] [real_accelerator.py:239:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-05-08 20:43:05,396] [INFO] [launch.py:146:main] WORLD INFO DICT: {'localhost': [0, 1]}
+[2025-05-08 20:43:05,396] [INFO] [launch.py:152:main] nnodes=1, num_local_procs=2, node_rank=0
+[2025-05-08 20:43:05,396] [INFO] [launch.py:163:main] global_rank_mapping=defaultdict(<class 'list'>, {'localhost': [0, 1]})
+[2025-05-08 20:43:05,396] [INFO] [launch.py:164:main] dist_world_size=2
+[2025-05-08 20:43:05,396] [INFO] [launch.py:168:main] Setting CUDA_VISIBLE_DEVICES=0,1
+[2025-05-08 20:43:05,396] [INFO] [launch.py:256:main] process 1548317 spawned with command: ['/home/panda/miniconda3/envs/pda/bin/python', '-u', '-m', 'safe_rlhf.algorithms.tools_ft', '--local_rank=0', '--train_datasets', 'tools', '--model_name_or_path', 'meta-llama/Llama-3.1-8B-Instruct', '--cache_dir', '/home/panda/pda-llm/cache/sft-tools', '--important_sft', 'false', '--max_length', '2048', '--trust_remote_code', 'True', '--epochs', '3', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '1', '--gradient_accumulation_steps', '48', '--gradient_checkpointing', '--learning_rate', '1e-4', '--lr_scheduler_type', 'cosine', '--lr_warmup_ratio', '0.1', '--weight_decay', '0.0', '--seed', '42', '--output_dir', '/home/panda/pda-llm/output/sft-tools/run-false-1-10', '--log_type', 'wandb', '--log_project', 'TOOLS-SFT', '--zero_stage', '0', '--offload', 'none', '--safety_ratio_tol', '10', '--resilient_coeff', '1', '--lora_r', '16', '--lora_alpha', '32', '--lora_dropout', '0.05', '--bf16', 'True', '--fp16', 'False', '--tf32', 'False']
+[2025-05-08 20:43:05,397] [INFO] [launch.py:256:main] process 1548318 spawned with command: ['/home/panda/miniconda3/envs/pda/bin/python', '-u', '-m', 'safe_rlhf.algorithms.tools_ft', '--local_rank=1', '--train_datasets', 'tools', '--model_name_or_path', 'meta-llama/Llama-3.1-8B-Instruct', '--cache_dir', '/home/panda/pda-llm/cache/sft-tools', '--important_sft', 'false', '--max_length', '2048', '--trust_remote_code', 'True', '--epochs', '3', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '1', '--gradient_accumulation_steps', '48', '--gradient_checkpointing', '--learning_rate', '1e-4', '--lr_scheduler_type', 'cosine', '--lr_warmup_ratio', '0.1', '--weight_decay', '0.0', '--seed', '42', '--output_dir', '/home/panda/pda-llm/output/sft-tools/run-false-1-10', '--log_type', 'wandb', '--log_project', 'TOOLS-SFT', '--zero_stage', '0', '--offload', 'none', '--safety_ratio_tol', '10', '--resilient_coeff', '1', '--lora_r', '16', '--lora_alpha', '32', '--lora_dropout', '0.05', '--bf16', 'True', '--fp16', 'False', '--tf32', 'False']
+[2025-05-08 20:43:06,560] [INFO] [real_accelerator.py:239:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-05-08 20:43:06,564] [INFO] [real_accelerator.py:239:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-05-08 20:43:09,694] [INFO] [comm.py:669:init_distributed] cdb=None
+[2025-05-08 20:43:09,706] [INFO] [comm.py:669:init_distributed] cdb=None
+[2025-05-08 20:43:09,707] [INFO] [comm.py:700:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl
+Set logger level to WARNING.
+calculating baseline ...
+calculating baseline ...
+Computing baseline logprobs...
+ninja: no work to do.
+Time to load fused_adam op: 0.032095909118652344 seconds
+Saving computed baseline logprobs to /home/panda/pda-llm/cache/sft-tools/cached_baseline_logprobs.pt
+Saved baseline logprobs successfully
+ninja: no work to do.
+Time to load fused_adam op: 0.034185171127319336 seconds
+***** Running training *****
+
+***** Evaluating at the beginning *****
+
+***** Evaluating at epoch 1/3 *****
+
+***** Evaluating at epoch 2/3 *****
+
+***** Evaluating at epoch 3/3 *****
+Saving model to "/home/panda/pda-llm/output/sft-tools/run-false-1-10" ...
+Saving Hugging Face Checkpoints...
+[2025-05-09 00:28:41,058] [INFO] [launch.py:351:main] Process 1548318 exits successfully.
+Model saved!
+[2025-05-09 00:28:45,059] [INFO] [launch.py:351:main] Process 1548317 exits successfully.

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:65b66ccdde0ce45c83b06f31e9f11272cade6aff26ec3cc9d6c49ede82b3ee2d
+size 17210383

tokenizer_config.json

@@ -0,0 +1,2082 @@
+{
+  "added_tokens_decoder": {
+    "128000": {
+      "content": "<|begin_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128001": {
+      "content": "<|end_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128002": {
+      "content": "<|reserved_special_token_0|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128003": {
+      "content": "<|reserved_special_token_1|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128004": {
+      "content": "<|finetune_right_pad_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128005": {
+      "content": "<|reserved_special_token_2|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128006": {
+      "content": "<|start_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128007": {
+      "content": "<|end_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128008": {
+      "content": "<|eom_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128009": {
+      "content": "<|eot_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128010": {
+      "content": "<|python_tag|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128011": {
+      "content": "<|reserved_special_token_3|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128012": {
+      "content": "<|reserved_special_token_4|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128013": {
+      "content": "<|reserved_special_token_5|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128014": {
+      "content": "<|reserved_special_token_6|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128015": {
+      "content": "<|reserved_special_token_7|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128016": {
+      "content": "<|reserved_special_token_8|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128017": {
+      "content": "<|reserved_special_token_9|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128018": {
+      "content": "<|reserved_special_token_10|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128019": {
+      "content": "<|reserved_special_token_11|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128020": {
+      "content": "<|reserved_special_token_12|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128021": {
+      "content": "<|reserved_special_token_13|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128022": {
+      "content": "<|reserved_special_token_14|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128023": {
+      "content": "<|reserved_special_token_15|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128024": {
+      "content": "<|reserved_special_token_16|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128025": {
+      "content": "<|reserved_special_token_17|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128026": {
+      "content": "<|reserved_special_token_18|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128027": {
+      "content": "<|reserved_special_token_19|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128028": {
+      "content": "<|reserved_special_token_20|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128029": {
+      "content": "<|reserved_special_token_21|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128030": {
+      "content": "<|reserved_special_token_22|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128031": {
+      "content": "<|reserved_special_token_23|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128032": {
+      "content": "<|reserved_special_token_24|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128033": {
+      "content": "<|reserved_special_token_25|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128034": {
+      "content": "<|reserved_special_token_26|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128035": {
+      "content": "<|reserved_special_token_27|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128036": {
+      "content": "<|reserved_special_token_28|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128037": {
+      "content": "<|reserved_special_token_29|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128038": {
+      "content": "<|reserved_special_token_30|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128039": {
+      "content": "<|reserved_special_token_31|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128040": {
+      "content": "<|reserved_special_token_32|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128041": {
+      "content": "<|reserved_special_token_33|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128042": {
+      "content": "<|reserved_special_token_34|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128043": {
+      "content": "<|reserved_special_token_35|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128044": {
+      "content": "<|reserved_special_token_36|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128045": {
+      "content": "<|reserved_special_token_37|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128046": {
+      "content": "<|reserved_special_token_38|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128047": {
+      "content": "<|reserved_special_token_39|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128048": {
+      "content": "<|reserved_special_token_40|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128049": {
+      "content": "<|reserved_special_token_41|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128050": {
+      "content": "<|reserved_special_token_42|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128051": {
+      "content": "<|reserved_special_token_43|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128052": {
+      "content": "<|reserved_special_token_44|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128053": {
+      "content": "<|reserved_special_token_45|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128054": {
+      "content": "<|reserved_special_token_46|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128055": {
+      "content": "<|reserved_special_token_47|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128056": {
+      "content": "<|reserved_special_token_48|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128057": {
+      "content": "<|reserved_special_token_49|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128058": {
+      "content": "<|reserved_special_token_50|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128059": {
+      "content": "<|reserved_special_token_51|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128060": {
+      "content": "<|reserved_special_token_52|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128061": {
+      "content": "<|reserved_special_token_53|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128062": {
+      "content": "<|reserved_special_token_54|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128063": {
+      "content": "<|reserved_special_token_55|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128064": {
+      "content": "<|reserved_special_token_56|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128065": {
+      "content": "<|reserved_special_token_57|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128066": {
+      "content": "<|reserved_special_token_58|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128067": {
+      "content": "<|reserved_special_token_59|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128068": {
+      "content": "<|reserved_special_token_60|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128069": {
+      "content": "<|reserved_special_token_61|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128070": {
+      "content": "<|reserved_special_token_62|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128071": {
+      "content": "<|reserved_special_token_63|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128072": {
+      "content": "<|reserved_special_token_64|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128073": {
+      "content": "<|reserved_special_token_65|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128074": {
+      "content": "<|reserved_special_token_66|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128075": {
+      "content": "<|reserved_special_token_67|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128076": {
+      "content": "<|reserved_special_token_68|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128077": {
+      "content": "<|reserved_special_token_69|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128078": {
+      "content": "<|reserved_special_token_70|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128079": {
+      "content": "<|reserved_special_token_71|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128080": {
+      "content": "<|reserved_special_token_72|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128081": {
+      "content": "<|reserved_special_token_73|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128082": {
+      "content": "<|reserved_special_token_74|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128083": {
+      "content": "<|reserved_special_token_75|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128084": {
+      "content": "<|reserved_special_token_76|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128085": {
+      "content": "<|reserved_special_token_77|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128086": {
+      "content": "<|reserved_special_token_78|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128087": {
+      "content": "<|reserved_special_token_79|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128088": {
+      "content": "<|reserved_special_token_80|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128089": {
+      "content": "<|reserved_special_token_81|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128090": {
+      "content": "<|reserved_special_token_82|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128091": {
+      "content": "<|reserved_special_token_83|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128092": {
+      "content": "<|reserved_special_token_84|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128093": {
+      "content": "<|reserved_special_token_85|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128094": {
+      "content": "<|reserved_special_token_86|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128095": {
+      "content": "<|reserved_special_token_87|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128096": {
+      "content": "<|reserved_special_token_88|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128097": {
+      "content": "<|reserved_special_token_89|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128098": {
+      "content": "<|reserved_special_token_90|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128099": {
+      "content": "<|reserved_special_token_91|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128100": {
+      "content": "<|reserved_special_token_92|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128101": {
+      "content": "<|reserved_special_token_93|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128102": {
+      "content": "<|reserved_special_token_94|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128103": {
+      "content": "<|reserved_special_token_95|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128104": {
+      "content": "<|reserved_special_token_96|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128105": {
+      "content": "<|reserved_special_token_97|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128106": {
+      "content": "<|reserved_special_token_98|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128107": {
+      "content": "<|reserved_special_token_99|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128108": {
+      "content": "<|reserved_special_token_100|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128109": {
+      "content": "<|reserved_special_token_101|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128110": {
+      "content": "<|reserved_special_token_102|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128111": {
+      "content": "<|reserved_special_token_103|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128112": {
+      "content": "<|reserved_special_token_104|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128113": {
+      "content": "<|reserved_special_token_105|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128114": {
+      "content": "<|reserved_special_token_106|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128115": {
+      "content": "<|reserved_special_token_107|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128116": {
+      "content": "<|reserved_special_token_108|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128117": {
+      "content": "<|reserved_special_token_109|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128118": {
+      "content": "<|reserved_special_token_110|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128119": {
+      "content": "<|reserved_special_token_111|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128120": {
+      "content": "<|reserved_special_token_112|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128121": {
+      "content": "<|reserved_special_token_113|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128122": {
+      "content": "<|reserved_special_token_114|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128123": {
+      "content": "<|reserved_special_token_115|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128124": {
+      "content": "<|reserved_special_token_116|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128125": {
+      "content": "<|reserved_special_token_117|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128126": {
+      "content": "<|reserved_special_token_118|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128127": {
+      "content": "<|reserved_special_token_119|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128128": {
+      "content": "<|reserved_special_token_120|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128129": {
+      "content": "<|reserved_special_token_121|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128130": {
+      "content": "<|reserved_special_token_122|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128131": {
+      "content": "<|reserved_special_token_123|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128132": {
+      "content": "<|reserved_special_token_124|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128133": {
+      "content": "<|reserved_special_token_125|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128134": {
+      "content": "<|reserved_special_token_126|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128135": {
+      "content": "<|reserved_special_token_127|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128136": {
+      "content": "<|reserved_special_token_128|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128137": {
+      "content": "<|reserved_special_token_129|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128138": {
+      "content": "<|reserved_special_token_130|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128139": {
+      "content": "<|reserved_special_token_131|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128140": {
+      "content": "<|reserved_special_token_132|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128141": {
+      "content": "<|reserved_special_token_133|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128142": {
+      "content": "<|reserved_special_token_134|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128143": {
+      "content": "<|reserved_special_token_135|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128144": {
+      "content": "<|reserved_special_token_136|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128145": {
+      "content": "<|reserved_special_token_137|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128146": {
+      "content": "<|reserved_special_token_138|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128147": {
+      "content": "<|reserved_special_token_139|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128148": {
+      "content": "<|reserved_special_token_140|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128149": {
+      "content": "<|reserved_special_token_141|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128150": {
+      "content": "<|reserved_special_token_142|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128151": {
+      "content": "<|reserved_special_token_143|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128152": {
+      "content": "<|reserved_special_token_144|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128153": {
+      "content": "<|reserved_special_token_145|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128154": {
+      "content": "<|reserved_special_token_146|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128155": {
+      "content": "<|reserved_special_token_147|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128156": {
+      "content": "<|reserved_special_token_148|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128157": {
+      "content": "<|reserved_special_token_149|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128158": {
+      "content": "<|reserved_special_token_150|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128159": {
+      "content": "<|reserved_special_token_151|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128160": {
+      "content": "<|reserved_special_token_152|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128161": {
+      "content": "<|reserved_special_token_153|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128162": {
+      "content": "<|reserved_special_token_154|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128163": {
+      "content": "<|reserved_special_token_155|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128164": {
+      "content": "<|reserved_special_token_156|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128165": {
+      "content": "<|reserved_special_token_157|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128166": {
+      "content": "<|reserved_special_token_158|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128167": {
+      "content": "<|reserved_special_token_159|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128168": {
+      "content": "<|reserved_special_token_160|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128169": {
+      "content": "<|reserved_special_token_161|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128170": {
+      "content": "<|reserved_special_token_162|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128171": {
+      "content": "<|reserved_special_token_163|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128172": {
+      "content": "<|reserved_special_token_164|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128173": {
+      "content": "<|reserved_special_token_165|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128174": {
+      "content": "<|reserved_special_token_166|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128175": {
+      "content": "<|reserved_special_token_167|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128176": {
+      "content": "<|reserved_special_token_168|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128177": {
+      "content": "<|reserved_special_token_169|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128178": {
+      "content": "<|reserved_special_token_170|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128179": {
+      "content": "<|reserved_special_token_171|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128180": {
+      "content": "<|reserved_special_token_172|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128181": {
+      "content": "<|reserved_special_token_173|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128182": {
+      "content": "<|reserved_special_token_174|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128183": {
+      "content": "<|reserved_special_token_175|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128184": {
+      "content": "<|reserved_special_token_176|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128185": {
+      "content": "<|reserved_special_token_177|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128186": {
+      "content": "<|reserved_special_token_178|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128187": {
+      "content": "<|reserved_special_token_179|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128188": {
+      "content": "<|reserved_special_token_180|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128189": {
+      "content": "<|reserved_special_token_181|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128190": {
+      "content": "<|reserved_special_token_182|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128191": {
+      "content": "<|reserved_special_token_183|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128192": {
+      "content": "<|reserved_special_token_184|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128193": {
+      "content": "<|reserved_special_token_185|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128194": {
+      "content": "<|reserved_special_token_186|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128195": {
+      "content": "<|reserved_special_token_187|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128196": {
+      "content": "<|reserved_special_token_188|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128197": {
+      "content": "<|reserved_special_token_189|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128198": {
+      "content": "<|reserved_special_token_190|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128199": {
+      "content": "<|reserved_special_token_191|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128200": {
+      "content": "<|reserved_special_token_192|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128201": {
+      "content": "<|reserved_special_token_193|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128202": {
+      "content": "<|reserved_special_token_194|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128203": {
+      "content": "<|reserved_special_token_195|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128204": {
+      "content": "<|reserved_special_token_196|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128205": {
+      "content": "<|reserved_special_token_197|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128206": {
+      "content": "<|reserved_special_token_198|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128207": {
+      "content": "<|reserved_special_token_199|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128208": {
+      "content": "<|reserved_special_token_200|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128209": {
+      "content": "<|reserved_special_token_201|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128210": {
+      "content": "<|reserved_special_token_202|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128211": {
+      "content": "<|reserved_special_token_203|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128212": {
+      "content": "<|reserved_special_token_204|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128213": {
+      "content": "<|reserved_special_token_205|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128214": {
+      "content": "<|reserved_special_token_206|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128215": {
+      "content": "<|reserved_special_token_207|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128216": {
+      "content": "<|reserved_special_token_208|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128217": {
+      "content": "<|reserved_special_token_209|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128218": {
+      "content": "<|reserved_special_token_210|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128219": {
+      "content": "<|reserved_special_token_211|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128220": {
+      "content": "<|reserved_special_token_212|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128221": {
+      "content": "<|reserved_special_token_213|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128222": {
+      "content": "<|reserved_special_token_214|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128223": {
+      "content": "<|reserved_special_token_215|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128224": {
+      "content": "<|reserved_special_token_216|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128225": {
+      "content": "<|reserved_special_token_217|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128226": {
+      "content": "<|reserved_special_token_218|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128227": {
+      "content": "<|reserved_special_token_219|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128228": {
+      "content": "<|reserved_special_token_220|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128229": {
+      "content": "<|reserved_special_token_221|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128230": {
+      "content": "<|reserved_special_token_222|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128231": {
+      "content": "<|reserved_special_token_223|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128232": {
+      "content": "<|reserved_special_token_224|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128233": {
+      "content": "<|reserved_special_token_225|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128234": {
+      "content": "<|reserved_special_token_226|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128235": {
+      "content": "<|reserved_special_token_227|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128236": {
+      "content": "<|reserved_special_token_228|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128237": {
+      "content": "<|reserved_special_token_229|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128238": {
+      "content": "<|reserved_special_token_230|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128239": {
+      "content": "<|reserved_special_token_231|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128240": {
+      "content": "<|reserved_special_token_232|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128241": {
+      "content": "<|reserved_special_token_233|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128242": {
+      "content": "<|reserved_special_token_234|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128243": {
+      "content": "<|reserved_special_token_235|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128244": {
+      "content": "<|reserved_special_token_236|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128245": {
+      "content": "<|reserved_special_token_237|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128246": {
+      "content": "<|reserved_special_token_238|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128247": {
+      "content": "<|reserved_special_token_239|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128248": {
+      "content": "<|reserved_special_token_240|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128249": {
+      "content": "<|reserved_special_token_241|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128250": {
+      "content": "<|reserved_special_token_242|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128251": {
+      "content": "<|reserved_special_token_243|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128252": {
+      "content": "<|reserved_special_token_244|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128253": {
+      "content": "<|reserved_special_token_245|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128254": {
+      "content": "<|reserved_special_token_246|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128255": {
+      "content": "<|reserved_special_token_247|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128256": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128257": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|begin_of_text|>",
+  "chat_template": "{{- bos_token }}\n{%- if custom_tools is defined %}\n    {%- set tools = custom_tools %}\n{%- endif %}\n{%- if not tools_in_user_message is defined %}\n    {%- set tools_in_user_message = true %}\n{%- endif %}\n{%- if not date_string is defined %}\n    {%- set date_string = \"26 Jul 2024\" %}\n{%- endif %}\n{%- if not tools is defined %}\n    {%- set tools = none %}\n{%- endif %}\n\n{#- This block extracts the system message, so we can slot it into the right place. #}\n{%- if messages[0]['role'] == 'system' %}\n    {%- set system_message = messages[0]['content']|trim %}\n    {%- set messages = messages[1:] %}\n{%- else %}\n    {%- set system_message = \"\" %}\n{%- endif %}\n\n{#- System message + builtin tools #}\n{{- \"<|start_header_id|>system<|end_header_id|>\\n\\n\" }}\n{%- if builtin_tools is defined or tools is not none %}\n    {{- \"Environment: ipython\\n\" }}\n{%- endif %}\n{%- if builtin_tools is defined %}\n    {{- \"Tools: \" + builtin_tools | reject('equalto', 'code_interpreter') | join(\", \") + \"\\n\\n\"}}\n{%- endif %}\n{{- \"Cutting Knowledge Date: December 2023\\n\" }}\n{{- \"Today Date: \" + date_string + \"\\n\\n\" }}\n{%- if tools is not none and not tools_in_user_message %}\n    {{- \"You have access to the following functions. To call a function, please respond with JSON for a function call.\" }}\n    {{- 'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.' }}\n    {{- \"Do not use variables.\\n\\n\" }}\n    {%- for t in tools %}\n        {{- t | tojson(indent=4) }}\n        {{- \"\\n\\n\" }}\n    {%- endfor %}\n{%- endif %}\n{{- system_message }}\n{{- \"<|eot_id|>\" }}\n\n{#- Custom tools are passed in a user message with some extra guidance #}\n{%- if tools_in_user_message and not tools is none %}\n    {#- Extract the first user message so we can plug it in here #}\n    {%- if messages | length != 0 %}\n        {%- set first_user_message = messages[0]['content']|trim %}\n        {%- set messages = messages[1:] %}\n    {%- else %}\n        {{- raise_exception(\"Cannot put tools in the first user message when there's no first user message!\") }}\n{%- endif %}\n    {{- '<|start_header_id|>user<|end_header_id|>\\n\\n' -}}\n    {{- \"Given the following functions, please respond with a JSON for a function call \" }}\n    {{- \"with its proper arguments that best answers the given prompt.\\n\\n\" }}\n    {{- 'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.' }}\n    {{- \"Do not use variables.\\n\\n\" }}\n    {%- for t in tools %}\n        {{- t | tojson(indent=4) }}\n        {{- \"\\n\\n\" }}\n    {%- endfor %}\n    {{- first_user_message + \"<|eot_id|>\"}}\n{%- endif %}\n\n{%- for message in messages %}\n    {%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}\n        {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\\n\\n'+ message['content'] | trim + '<|eot_id|>' }}\n    {%- elif 'tool_calls' in message %}\n        {%- if not message.tool_calls|length == 1 %}\n            {{- raise_exception(\"This model only supports single tool-calls at once!\") }}\n        {%- endif %}\n        {%- set tool_call = message.tool_calls[0].function %}\n        {%- if builtin_tools is defined and tool_call.name in builtin_tools %}\n            {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n            {{- \"<|python_tag|>\" + tool_call.name + \".call(\" }}\n            {%- for arg_name, arg_val in tool_call.arguments | items %}\n                {{- arg_name + '=\"' + arg_val + '\"' }}\n                {%- if not loop.last %}\n                    {{- \", \" }}\n                {%- endif %}\n                {%- endfor %}\n            {{- \")\" }}\n        {%- else  %}\n            {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n            {{- '{\"name\": \"' + tool_call.name + '\", ' }}\n            {{- '\"parameters\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- \"}\" }}\n        {%- endif %}\n        {%- if builtin_tools is defined %}\n            {#- This means we're in ipython mode #}\n            {{- \"<|eom_id|>\" }}\n        {%- else %}\n            {{- \"<|eot_id|>\" }}\n        {%- endif %}\n    {%- elif message.role == \"tool\" or message.role == \"ipython\" %}\n        {{- \"<|start_header_id|>ipython<|end_header_id|>\\n\\n\" }}\n        {%- if message.content is mapping or message.content is iterable %}\n            {{- message.content | tojson }}\n        {%- else %}\n            {{- message.content }}\n        {%- endif %}\n        {{- \"<|eot_id|>\" }}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|eot_id|>",
+  "extra_special_tokens": {},
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 2048,
+  "pad_token": "<pad>",
+  "padding_side": "right",
+  "tokenizer_class": "PreTrainedTokenizer",
+  "unk_token": "<unk>"
+}

wandb/debug-internal.log

@@ -0,0 +1,18 @@
+{"time":"2025-05-08T21:21:54.561866061-04:00","level":"INFO","msg":"stream: starting","core version":"0.19.10","symlink path":"/home/panda/pda-llm/output/sft-tools/run-false-1-10/wandb/run-20250508_212154-is8630ik/logs/debug-core.log"}
+{"time":"2025-05-08T21:21:54.740453474-04:00","level":"INFO","msg":"created new stream","id":"is8630ik"}
+{"time":"2025-05-08T21:21:54.740487984-04:00","level":"INFO","msg":"stream: started","id":"is8630ik"}
+{"time":"2025-05-08T21:21:54.740504345-04:00","level":"INFO","msg":"writer: Do: started","stream_id":"is8630ik"}
+{"time":"2025-05-08T21:21:54.740556706-04:00","level":"INFO","msg":"handler: started","stream_id":"is8630ik"}
+{"time":"2025-05-08T21:21:54.740592257-04:00","level":"INFO","msg":"sender: started","stream_id":"is8630ik"}
+{"time":"2025-05-08T21:21:54.884261607-04:00","level":"INFO","msg":"Starting system monitor"}
+{"time":"2025-05-08T21:21:54.884293377-04:00","level":"WARN","msg":"handleCodeSave: program relative path is empty"}
+{"time":"2025-05-08T21:21:54.884348759-04:00","level":"ERROR","msg":"git repo not found","error":"repository does not exist"}
+{"time":"2025-05-09T00:28:41.16342178-04:00","level":"INFO","msg":"Stopping system monitor"}
+{"time":"2025-05-09T00:28:41.163499301-04:00","level":"INFO","msg":"Stopped system monitor"}
+{"time":"2025-05-09T00:28:41.635621981-04:00","level":"INFO","msg":"fileTransfer: Close: file transfer manager closed"}
+{"time":"2025-05-09T00:28:41.760403458-04:00","level":"INFO","msg":"handler: operation stats","stats":{}}
+{"time":"2025-05-09T00:28:41.773696754-04:00","level":"INFO","msg":"stream: closing","id":"is8630ik"}
+{"time":"2025-05-09T00:28:41.773713434-04:00","level":"INFO","msg":"handler: closed","stream_id":"is8630ik"}
+{"time":"2025-05-09T00:28:41.773723895-04:00","level":"INFO","msg":"writer: Close: closed","stream_id":"is8630ik"}
+{"time":"2025-05-09T00:28:41.773731765-04:00","level":"INFO","msg":"sender: closed","stream_id":"is8630ik"}
+{"time":"2025-05-09T00:28:41.773786876-04:00","level":"INFO","msg":"stream: closed","id":"is8630ik"}

wandb/debug.log

@@ -0,0 +1,29 @@
+2025-05-08 21:21:54,556 INFO    MainThread:1548317 [wandb_setup.py:_flush():68] Current SDK version is 0.19.10
+2025-05-08 21:21:54,556 INFO    MainThread:1548317 [wandb_setup.py:_flush():68] Configure stats pid to 1548317
+2025-05-08 21:21:54,556 INFO    MainThread:1548317 [wandb_setup.py:_flush():68] Loading settings from /home/panda/.config/wandb/settings
+2025-05-08 21:21:54,556 INFO    MainThread:1548317 [wandb_setup.py:_flush():68] Loading settings from /home/panda/pda-llm/scripts/wandb/settings
+2025-05-08 21:21:54,556 INFO    MainThread:1548317 [wandb_setup.py:_flush():68] Loading settings from environment variables
+2025-05-08 21:21:54,556 INFO    MainThread:1548317 [wandb_init.py:setup_run_log_directory():724] Logging user logs to /home/panda/pda-llm/output/sft-tools/run-false-1-10/wandb/run-20250508_212154-is8630ik/logs/debug.log
+2025-05-08 21:21:54,556 INFO    MainThread:1548317 [wandb_init.py:setup_run_log_directory():725] Logging internal logs to /home/panda/pda-llm/output/sft-tools/run-false-1-10/wandb/run-20250508_212154-is8630ik/logs/debug-internal.log
+2025-05-08 21:21:54,556 INFO    MainThread:1548317 [wandb_init.py:init():852] calling init triggers
+2025-05-08 21:21:54,556 INFO    MainThread:1548317 [wandb_init.py:init():857] wandb.init called with sweep_config: {}
+config: {'model_name_or_path': 'meta-llama/Llama-3.1-8B-Instruct', 'recompute_baseline': False, 'cache_dir': '/home/panda/pda-llm/cache/sft-tools', 'max_length': 2048, 'trust_remote_code': True, 'train_datasets': [('tools', {'proportion': 1.0})], 'eval_datasets': None, 'safety_ratio_tol': 10.0, 'important_sft': False, 'resilient_coeff': 1.0, 'epochs': 3, 'per_device_train_batch_size': 1, 'per_device_eval_batch_size': 1, 'gradient_accumulation_steps': 48, 'gradient_checkpointing': True, 'lr': 0.0001, 'lr_scheduler_type': <SchedulerType.COSINE: 'cosine'>, 'lr_warmup_ratio': 0.1, 'weight_decay': 0.0, 'seed': 42, 'fp16': False, 'bf16': True, 'tf32': False, 'lora_r': 16, 'lora_alpha': 32, 'lora_dropout': 0.05, 'eval_strategy': 'epoch', 'eval_interval': 1000000, 'need_eval': True, 'eval_split_ratio': None, 'output_dir': '/home/panda/pda-llm/output/sft-tools/run-false-1-10', 'log_type': 'wandb', 'log_dir': '/home/panda/pda-llm/output/sft-tools/run-false-1-10', 'log_project': 'TOOLS-SFT', 'log_run_name': 'tools-sft-2025-05-08-21-21-54', 'save_16bit': False, 'save_interval': 1000000, 'local_rank': 0, 'zero_stage': 0, 'offload': 'none', 'deepspeed': False, 'deepspeed_config': None, 'deepscale': False, 'deepscale_config': None, 'global_rank': 0, 'device': device(type='cuda', index=0), 'num_update_steps_per_epoch': 118, 'total_training_steps': 354, '_wandb': {}}
+2025-05-08 21:21:54,556 INFO    MainThread:1548317 [wandb_init.py:init():893] starting backend
+2025-05-08 21:21:54,556 INFO    MainThread:1548317 [wandb_init.py:init():897] sending inform_init request
+2025-05-08 21:21:54,558 INFO    MainThread:1548317 [backend.py:_multiprocessing_setup():101] multiprocessing start_methods=fork,spawn,forkserver, using: spawn
+2025-05-08 21:21:54,558 INFO    MainThread:1548317 [wandb_init.py:init():907] backend started and connected
+2025-05-08 21:21:54,559 INFO    MainThread:1548317 [wandb_init.py:init():1002] updated telemetry
+2025-05-08 21:21:54,564 INFO    MainThread:1548317 [wandb_init.py:init():1026] communicating run to backend with 90.0 second timeout
+2025-05-08 21:21:54,882 INFO    MainThread:1548317 [wandb_init.py:init():1101] starting run threads in backend
+2025-05-08 21:21:54,947 INFO    MainThread:1548317 [wandb_run.py:_console_start():2566] atexit reg
+2025-05-08 21:21:54,947 INFO    MainThread:1548317 [wandb_run.py:_redirect():2414] redirect: wrap_raw
+2025-05-08 21:21:54,947 INFO    MainThread:1548317 [wandb_run.py:_redirect():2483] Wrapping output streams.
+2025-05-08 21:21:54,947 INFO    MainThread:1548317 [wandb_run.py:_redirect():2506] Redirects installed.
+2025-05-08 21:21:54,948 INFO    MainThread:1548317 [wandb_init.py:init():1147] run started, returning control to user process
+2025-05-09 00:28:41,160 INFO    MainThread:1548317 [wandb_run.py:_finish():2314] finishing run alelab/TOOLS-SFT/is8630ik
+2025-05-09 00:28:41,162 INFO    MainThread:1548317 [wandb_run.py:_atexit_cleanup():2531] got exitcode: 0
+2025-05-09 00:28:41,162 INFO    MainThread:1548317 [wandb_run.py:_restore():2513] restore
+2025-05-09 00:28:41,162 INFO    MainThread:1548317 [wandb_run.py:_restore():2519] restore done
+2025-05-09 00:28:41,762 INFO    MainThread:1548317 [wandb_run.py:_footer_history_summary_info():4160] rendering history
+2025-05-09 00:28:41,765 INFO    MainThread:1548317 [wandb_run.py:_footer_history_summary_info():4192] rendering summary
+2025-05-09 00:28:41,773 INFO    MainThread:1548317 [wandb_run.py:_footer_sync_info():4121] logging synced files

wandb/run-20250508_005928-1jhxkbtl/files/config.yaml

@@ -0,0 +1,133 @@
+_wandb:
+    value:
+        cli_version: 0.19.10
+        m: []
+        python_version: 3.11.11
+        t:
+            "1":
+                - 1
+                - 11
+                - 49
+                - 51
+                - 55
+                - 71
+                - 98
+            "2":
+                - 1
+                - 11
+                - 49
+                - 51
+                - 55
+                - 71
+                - 98
+            "3":
+                - 2
+                - 13
+                - 16
+                - 23
+                - 55
+            "4": 3.11.11
+            "5": 0.19.10
+            "6": 4.49.0
+            "8":
+                - 5
+            "12": 0.19.10
+            "13": linux-x86_64
+bf16:
+    value: false
+cache_dir:
+    value: /home/panda/pda-llm/cache/sft-tools
+deepscale:
+    value: false
+deepscale_config:
+    value: null
+deepspeed:
+    value: false
+deepspeed_config:
+    value: null
+device:
+    value: cuda:0
+epochs:
+    value: 3
+eval_datasets:
+    value: null
+eval_interval:
+    value: 1000000
+eval_split_ratio:
+    value: null
+eval_strategy:
+    value: epoch
+fp16:
+    value: true
+global_rank:
+    value: 0
+gradient_accumulation_steps:
+    value: 12
+gradient_checkpointing:
+    value: true
+important_sft:
+    value: false
+local_rank:
+    value: 0
+log_dir:
+    value: /home/panda/pda-llm/output/sft-tools/run-false-1-10
+log_project:
+    value: TOOLS-SFT
+log_run_name:
+    value: tools-sft-2025-05-08-00-59-27
+log_type:
+    value: wandb
+lora_alpha:
+    value: 32
+lora_dropout:
+    value: 0.05
+lora_r:
+    value: 16
+lr:
+    value: 0.0001
+lr_scheduler_type:
+    value: COSINE
+lr_warmup_ratio:
+    value: 0.1
+max_length:
+    value: 4096
+model_name_or_path:
+    value: meta-llama/Llama-3.1-8B-Instruct
+need_eval:
+    value: true
+num_update_steps_per_epoch:
+    value: 236
+offload:
+    value: none
+output_dir:
+    value: /home/panda/pda-llm/output/sft-tools/run-false-1-10
+per_device_eval_batch_size:
+    value: 4
+per_device_train_batch_size:
+    value: 4
+recompute_baseline:
+    value: false
+resilient_coeff:
+    value: 1
+safety_ratio_tol:
+    value: 10
+save_16bit:
+    value: false
+save_interval:
+    value: 1000000
+seed:
+    value: 42
+tf32:
+    value: true
+total_training_steps:
+    value: 708
+train_datasets:
+    value:
+        - - tools
+          - proportion: 1
+trust_remote_code:
+    value: true
+weight_decay:
+    value: 0
+zero_stage:
+    value: 0

wandb/run-20250508_005928-1jhxkbtl/files/output.log

@@ -0,0 +1,33 @@
+***** Running training *****
+Training 1/3 epoch:   0%|          | 0/8475 [00:00<?, ?it/s]Traceback (most recent call last):
+
+***** Evaluating at the beginning *****
+  File "<frozen runpy>", line 198, in _run_module_as_main
+  File "<frozen runpy>", line 88, in _run_code
+  File "/home/panda/pda-llm/safe_rlhf/algorithms/tools_ft/__main__.py", line 23, in <module>
+    sys.exit(main())
+             ^^^^^^
+  File "/home/panda/pda-llm/safe_rlhf/algorithms/tools_ft/main.py", line 366, in main
+    trainer.train()
+  File "/home/panda/pda-llm/safe_rlhf/trainers/supervised_tools_trainer.py", line 292, in train
+    self.logger.log(self.eval(), step=0)
+                    ^^^^^^^^^^^
+  File "/home/panda/pda-llm/safe_rlhf/trainers/supervised_tools_trainer.py", line 344, in eval
+    is_important = batch['is_important']
+                   ~~~~~^^^^^^^^^^^^^^^^
+KeyError: 'is_important'
+[rank0]: Traceback (most recent call last):
+[rank0]:   File "<frozen runpy>", line 198, in _run_module_as_main
+[rank0]:   File "<frozen runpy>", line 88, in _run_code
+[rank0]:   File "/home/panda/pda-llm/safe_rlhf/algorithms/tools_ft/__main__.py", line 23, in <module>
+[rank0]:     sys.exit(main())
+[rank0]:              ^^^^^^
+[rank0]:   File "/home/panda/pda-llm/safe_rlhf/algorithms/tools_ft/main.py", line 366, in main
+[rank0]:     trainer.train()
+[rank0]:   File "/home/panda/pda-llm/safe_rlhf/trainers/supervised_tools_trainer.py", line 292, in train
+[rank0]:     self.logger.log(self.eval(), step=0)
+[rank0]:                     ^^^^^^^^^^^
+[rank0]:   File "/home/panda/pda-llm/safe_rlhf/trainers/supervised_tools_trainer.py", line 344, in eval
+[rank0]:     is_important = batch['is_important']
+[rank0]:                    ~~~~~^^^^^^^^^^^^^^^^
+[rank0]: KeyError: 'is_important'

wandb/run-20250508_005928-1jhxkbtl/files/requirements.txt

@@ -0,0 +1,108 @@
+PySocks==1.7.1
+certifi==2025.4.26
+nvidia-ml-py==12.535.133
+mkl_fft==1.3.11
+urllib3==2.3.0
+charset-normalizer==3.3.2
+smmap==4.0.0
+xxhash==3.5.0
+tzdata==2025.2
+deepspeed==0.16.7
+gmpy2==2.2.1
+pydantic==2.10.3
+tqdm==4.67.1
+pyparsing==3.2.0
+Markdown==3.8
+pydantic_core==2.27.1
+hjson==3.1.0
+mkl_random==1.2.8
+sentry-sdk==2.18.0
+tokenizers==0.21.0
+wandb==0.19.10
+einops==0.8.1
+Bottleneck==1.4.2
+msgpack==1.1.0
+fsspec==2024.12.0
+mdurl==0.1.0
+wheel==0.45.1
+networkx==3.4.2
+nvitop==1.5.0
+sympy==1.13.1
+MarkupSafe==3.0.2
+numexpr==2.10.1
+tensorboard_data_server==0.7.0
+markdown-it-py==2.2.0
+six==1.17.0
+idna==3.7
+docker-pycreds==0.4.0
+GitPython==3.1.43
+ninja==1.11.1.4
+fonttools==4.55.3
+annotated-types==0.6.0
+cycler==0.11.0
+py-cpuinfo==9.0.0
+accelerate==1.6.0
+filelock==3.17.0
+pytz==2024.1
+rich==13.9.4
+appdirs==1.4.4
+click==8.1.8
+Jinja2==3.1.6
+unicodedata2==15.1.0
+pillow==11.1.0
+transformers==4.49.0
+peft==0.15.2
+python-dateutil==2.9.0.post0
+requests==2.32.3
+Werkzeug==3.1.3
+setuptools==78.1.1
+gitdb==4.0.7
+torch==2.5.1
+scipy==1.15.2
+sentencepiece==0.2.0
+dill==0.3.8
+psutil==5.9.0
+contourpy==1.3.1
+grpcio==1.71.0
+optree==0.14.1
+safetensors==0.5.3
+mpmath==1.3.0
+packaging==24.2
+pyarrow==19.0.0
+PyYAML==6.0.2
+pandas==2.2.3
+tensorboard==2.19.0
+regex==2024.11.6
+pip==25.1
+Brotli==1.0.9
+mkl-service==2.4.0
+matplotlib==3.10.0
+Pygments==2.19.1
+absl-py==2.1.0
+triton==3.1.0
+multiprocess==0.70.15
+platformdirs==4.3.7
+numpy==2.0.1
+protobuf==5.29.3
+datasets==3.6.0
+eval_type_backport==0.2.2
+typing_extensions==4.12.2
+kiwisolver==1.4.8
+setproctitle==1.2.2
+huggingface_hub==0.29.2
+jaraco.text==3.12.1
+backports.tarfile==1.2.0
+importlib_metadata==8.0.0
+jaraco.context==5.3.0
+more-itertools==10.3.0
+wheel==0.45.1
+jaraco.collections==5.1.0
+platformdirs==4.2.2
+autocommand==2.2.2
+zipp==3.19.2
+tomli==2.0.1
+typeguard==4.3.0
+packaging==24.2
+inflect==7.3.1
+typing_extensions==4.12.2
+jaraco.functools==4.0.1

wandb/run-20250508_005928-1jhxkbtl/files/wandb-metadata.json

@@ -0,0 +1,106 @@
+{
+  "os":  "Linux-5.15.0-138-generic-x86_64-with-glibc2.31",
+  "python":  "CPython 3.11.11",
+  "startedAt":  "2025-05-08T04:59:28.129836Z",
+  "args":  [
+    "--local_rank=0",
+    "--train_datasets",
+    "tools",
+    "--model_name_or_path",
+    "meta-llama/Llama-3.1-8B-Instruct",
+    "--cache_dir",
+    "/home/panda/pda-llm/cache/sft-tools",
+    "--important_sft",
+    "false",
+    "--max_length",
+    "4096",
+    "--trust_remote_code",
+    "True",
+    "--epochs",
+    "3",
+    "--per_device_train_batch_size",
+    "4",
+    "--per_device_eval_batch_size",
+    "4",
+    "--gradient_accumulation_steps",
+    "12",
+    "--gradient_checkpointing",
+    "--learning_rate",
+    "1e-4",
+    "--lr_scheduler_type",
+    "cosine",
+    "--lr_warmup_ratio",
+    "0.1",
+    "--weight_decay",
+    "0.0",
+    "--seed",
+    "42",
+    "--output_dir",
+    "/home/panda/pda-llm/output/sft-tools/run-false-1-10",
+    "--log_type",
+    "wandb",
+    "--log_project",
+    "TOOLS-SFT",
+    "--zero_stage",
+    "0",
+    "--offload",
+    "none",
+    "--safety_ratio_tol",
+    "10",
+    "--resilient_coeff",
+    "1",
+    "--lora_r",
+    "16",
+    "--lora_alpha",
+    "32",
+    "--lora_dropout",
+    "0.05",
+    "--bf16",
+    "False",
+    "--fp16",
+    "True",
+    "--tf32",
+    "True"
+  ],
+  "program":  "-m safe_rlhf.algorithms.tools_ft.__main__",
+  "git":  {
+    "remote":  "git@github.com:IgnacioBoero/pda-llm.git",
+    "commit":  "2f331e2abf3b7b76829bc02cfb20143a948f9298"
+  },
+  "email":  "iboero@upenn.edu",
+  "root":  "/home/panda/pda-llm/output/sft-tools/run-false-1-10",
+  "host":  "plaza",
+  "executable":  "/home/panda/miniconda3/envs/pda/bin/python",
+  "cpu_count":  24,
+  "cpu_count_logical":  48,
+  "gpu":  "NVIDIA RTX A6000",
+  "gpu_count":  2,
+  "disk":  {
+    "/":  {
+      "total":  "982820896768",
+      "used":  "886362767360"
+    }
+  },
+  "memory":  {
+    "total":  "270090010624"
+  },
+  "cpu":  {
+    "count":  24,
+    "countLogical":  48
+  },
+  "gpu_nvidia":  [
+    {
+      "name":  "NVIDIA RTX A6000",
+      "memoryTotal":  "51527024640",
+      "cudaCores":  10752,
+      "architecture":  "Ampere"
+    },
+    {
+      "name":  "NVIDIA RTX A6000",
+      "memoryTotal":  "51527024640",
+      "cudaCores":  10752,
+      "architecture":  "Ampere"
+    }
+  ],
+  "cudaVersion":  "12.4"
+}

wandb/run-20250508_005928-1jhxkbtl/files/wandb-summary.json

@@ -0,0 +1 @@
+{"_wandb":{"runtime":0}}

wandb/run-20250508_005928-1jhxkbtl/logs/debug-core.log

@@ -0,0 +1,15 @@
+{"time":"2025-05-08T00:59:27.792254317-04:00","level":"INFO","msg":"main: starting server","port-filename":"/tmp/tmpgz00y30n/port-770208.txt","pid":770208,"log-level":0,"disable-analytics":false,"shutdown-on-parent-exit":false}
+{"time":"2025-05-08T00:59:27.798415148-04:00","level":"INFO","msg":"Will exit if parent process dies.","ppid":770208}
+{"time":"2025-05-08T00:59:27.798422588-04:00","level":"INFO","msg":"server is running","addr":{"IP":"127.0.0.1","Port":34881,"Zone":""}}
+{"time":"2025-05-08T00:59:27.976670485-04:00","level":"INFO","msg":"connection: ManageConnectionData: new connection created","id":"127.0.0.1:41134"}
+{"time":"2025-05-08T00:59:28.131725052-04:00","level":"INFO","msg":"handleInformInit: received","streamId":"1jhxkbtl","id":"127.0.0.1:41134"}
+{"time":"2025-05-08T00:59:28.326665772-04:00","level":"INFO","msg":"handleInformInit: stream started","streamId":"1jhxkbtl","id":"127.0.0.1:41134"}
+{"time":"2025-05-08T00:59:29.41791717-04:00","level":"INFO","msg":"handleInformFinish: finish message received","streamId":"1jhxkbtl","id":"127.0.0.1:41134"}
+{"time":"2025-05-08T00:59:29.418040741-04:00","level":"INFO","msg":"handleInformFinish: stream closed","streamId":"1jhxkbtl","id":"127.0.0.1:41134"}
+{"time":"2025-05-08T00:59:30.418475132-04:00","level":"INFO","msg":"handleInformTeardown: server teardown initiated","id":"127.0.0.1:41134"}
+{"time":"2025-05-08T00:59:30.418514823-04:00","level":"INFO","msg":"handleInformTeardown: server shutdown complete","id":"127.0.0.1:41134"}
+{"time":"2025-05-08T00:59:30.418524663-04:00","level":"INFO","msg":"server is shutting down"}
+{"time":"2025-05-08T00:59:30.418559823-04:00","level":"INFO","msg":"connection: closing","id":"127.0.0.1:41134"}
+{"time":"2025-05-08T00:59:30.418645274-04:00","level":"INFO","msg":"connection: closed successfully","id":"127.0.0.1:41134"}
+{"time":"2025-05-08T00:59:30.418654894-04:00","level":"INFO","msg":"connection: ManageConnectionData: connection closed","id":"127.0.0.1:41134"}
+{"time":"2025-05-08T00:59:30.418665734-04:00","level":"INFO","msg":"server is closed"}

wandb/run-20250508_005928-1jhxkbtl/logs/debug-internal.log

@@ -0,0 +1,18 @@
+{"time":"2025-05-08T00:59:28.132017565-04:00","level":"INFO","msg":"stream: starting","core version":"0.19.10","symlink path":"/home/panda/pda-llm/output/sft-tools/run-false-1-10/wandb/run-20250508_005928-1jhxkbtl/logs/debug-core.log"}
+{"time":"2025-05-08T00:59:28.326622421-04:00","level":"INFO","msg":"created new stream","id":"1jhxkbtl"}
+{"time":"2025-05-08T00:59:28.326657422-04:00","level":"INFO","msg":"stream: started","id":"1jhxkbtl"}
+{"time":"2025-05-08T00:59:28.326676812-04:00","level":"INFO","msg":"writer: Do: started","stream_id":"1jhxkbtl"}
+{"time":"2025-05-08T00:59:28.326745383-04:00","level":"INFO","msg":"sender: started","stream_id":"1jhxkbtl"}
+{"time":"2025-05-08T00:59:28.326827564-04:00","level":"INFO","msg":"handler: started","stream_id":"1jhxkbtl"}
+{"time":"2025-05-08T00:59:28.55136741-04:00","level":"INFO","msg":"Starting system monitor"}
+{"time":"2025-05-08T00:59:28.551407061-04:00","level":"WARN","msg":"handleCodeSave: program relative path is empty"}
+{"time":"2025-05-08T00:59:28.551479361-04:00","level":"ERROR","msg":"git repo not found","error":"repository does not exist"}
+{"time":"2025-05-08T00:59:28.645082906-04:00","level":"INFO","msg":"Stopping system monitor"}
+{"time":"2025-05-08T00:59:28.645130137-04:00","level":"INFO","msg":"Stopped system monitor"}
+{"time":"2025-05-08T00:59:29.314315271-04:00","level":"INFO","msg":"fileTransfer: Close: file transfer manager closed"}
+{"time":"2025-05-08T00:59:29.414164881-04:00","level":"INFO","msg":"handler: operation stats","stats":{}}
+{"time":"2025-05-08T00:59:29.417948-04:00","level":"INFO","msg":"stream: closing","id":"1jhxkbtl"}
+{"time":"2025-05-08T00:59:29.41796369-04:00","level":"INFO","msg":"handler: closed","stream_id":"1jhxkbtl"}
+{"time":"2025-05-08T00:59:29.41797832-04:00","level":"INFO","msg":"sender: closed","stream_id":"1jhxkbtl"}
+{"time":"2025-05-08T00:59:29.41797466-04:00","level":"INFO","msg":"writer: Close: closed","stream_id":"1jhxkbtl"}
+{"time":"2025-05-08T00:59:29.418031771-04:00","level":"INFO","msg":"stream: closed","id":"1jhxkbtl"}

wandb/run-20250508_005928-1jhxkbtl/logs/debug.log

@@ -0,0 +1,29 @@
+2025-05-08 00:59:28,127 INFO    MainThread:770208 [wandb_setup.py:_flush():68] Current SDK version is 0.19.10
+2025-05-08 00:59:28,127 INFO    MainThread:770208 [wandb_setup.py:_flush():68] Configure stats pid to 770208
+2025-05-08 00:59:28,127 INFO    MainThread:770208 [wandb_setup.py:_flush():68] Loading settings from /home/panda/.config/wandb/settings
+2025-05-08 00:59:28,127 INFO    MainThread:770208 [wandb_setup.py:_flush():68] Loading settings from /home/panda/pda-llm/scripts/wandb/settings
+2025-05-08 00:59:28,127 INFO    MainThread:770208 [wandb_setup.py:_flush():68] Loading settings from environment variables
+2025-05-08 00:59:28,127 INFO    MainThread:770208 [wandb_init.py:setup_run_log_directory():724] Logging user logs to /home/panda/pda-llm/output/sft-tools/run-false-1-10/wandb/run-20250508_005928-1jhxkbtl/logs/debug.log
+2025-05-08 00:59:28,127 INFO    MainThread:770208 [wandb_init.py:setup_run_log_directory():725] Logging internal logs to /home/panda/pda-llm/output/sft-tools/run-false-1-10/wandb/run-20250508_005928-1jhxkbtl/logs/debug-internal.log
+2025-05-08 00:59:28,127 INFO    MainThread:770208 [wandb_init.py:init():852] calling init triggers
+2025-05-08 00:59:28,127 INFO    MainThread:770208 [wandb_init.py:init():857] wandb.init called with sweep_config: {}
+config: {'model_name_or_path': 'meta-llama/Llama-3.1-8B-Instruct', 'recompute_baseline': False, 'cache_dir': '/home/panda/pda-llm/cache/sft-tools', 'max_length': 4096, 'trust_remote_code': True, 'train_datasets': [('tools', {'proportion': 1.0})], 'eval_datasets': None, 'safety_ratio_tol': 10.0, 'important_sft': False, 'resilient_coeff': 1.0, 'epochs': 3, 'per_device_train_batch_size': 4, 'per_device_eval_batch_size': 4, 'gradient_accumulation_steps': 12, 'gradient_checkpointing': True, 'lr': 0.0001, 'lr_scheduler_type': <SchedulerType.COSINE: 'cosine'>, 'lr_warmup_ratio': 0.1, 'weight_decay': 0.0, 'seed': 42, 'fp16': True, 'bf16': False, 'tf32': True, 'lora_r': 16, 'lora_alpha': 32, 'lora_dropout': 0.05, 'eval_strategy': 'epoch', 'eval_interval': 1000000, 'need_eval': True, 'eval_split_ratio': None, 'output_dir': '/home/panda/pda-llm/output/sft-tools/run-false-1-10', 'log_type': 'wandb', 'log_dir': '/home/panda/pda-llm/output/sft-tools/run-false-1-10', 'log_project': 'TOOLS-SFT', 'log_run_name': 'tools-sft-2025-05-08-00-59-27', 'save_16bit': False, 'save_interval': 1000000, 'local_rank': 0, 'zero_stage': 0, 'offload': 'none', 'deepspeed': False, 'deepspeed_config': None, 'deepscale': False, 'deepscale_config': None, 'global_rank': 0, 'device': device(type='cuda', index=0), 'num_update_steps_per_epoch': 236, 'total_training_steps': 708, '_wandb': {}}
+2025-05-08 00:59:28,127 INFO    MainThread:770208 [wandb_init.py:init():893] starting backend
+2025-05-08 00:59:28,127 INFO    MainThread:770208 [wandb_init.py:init():897] sending inform_init request
+2025-05-08 00:59:28,129 INFO    MainThread:770208 [backend.py:_multiprocessing_setup():101] multiprocessing start_methods=fork,spawn,forkserver, using: spawn
+2025-05-08 00:59:28,129 INFO    MainThread:770208 [wandb_init.py:init():907] backend started and connected
+2025-05-08 00:59:28,131 INFO    MainThread:770208 [wandb_init.py:init():1002] updated telemetry
+2025-05-08 00:59:28,136 INFO    MainThread:770208 [wandb_init.py:init():1026] communicating run to backend with 90.0 second timeout
+2025-05-08 00:59:28,548 INFO    MainThread:770208 [wandb_init.py:init():1101] starting run threads in backend
+2025-05-08 00:59:28,603 INFO    MainThread:770208 [wandb_run.py:_console_start():2566] atexit reg
+2025-05-08 00:59:28,604 INFO    MainThread:770208 [wandb_run.py:_redirect():2414] redirect: wrap_raw
+2025-05-08 00:59:28,604 INFO    MainThread:770208 [wandb_run.py:_redirect():2483] Wrapping output streams.
+2025-05-08 00:59:28,604 INFO    MainThread:770208 [wandb_run.py:_redirect():2506] Redirects installed.
+2025-05-08 00:59:28,605 INFO    MainThread:770208 [wandb_init.py:init():1147] run started, returning control to user process
+2025-05-08 00:59:28,613 INFO    MainThread:770208 [wandb_run.py:_finish():2314] finishing run alelab/TOOLS-SFT/1jhxkbtl
+2025-05-08 00:59:28,613 INFO    MainThread:770208 [wandb_run.py:_atexit_cleanup():2531] got exitcode: 0
+2025-05-08 00:59:28,613 INFO    MainThread:770208 [wandb_run.py:_restore():2513] restore
+2025-05-08 00:59:28,613 INFO    MainThread:770208 [wandb_run.py:_restore():2519] restore done
+2025-05-08 00:59:29,417 INFO    MainThread:770208 [wandb_run.py:_footer_history_summary_info():4160] rendering history
+2025-05-08 00:59:29,417 INFO    MainThread:770208 [wandb_run.py:_footer_history_summary_info():4192] rendering summary
+2025-05-08 00:59:29,417 INFO    MainThread:770208 [wandb_run.py:_footer_sync_info():4121] logging synced files

wandb/run-20250508_160817-0lem0nck/files/output.log

@@ -0,0 +1,2 @@
+***** Running training *****
+Training 1/3 epoch (loss 0.1149):   0%|          | 10/16950 [00:05<2:23:52,  1.96it/s]

wandb/run-20250508_160817-0lem0nck/files/requirements.txt

@@ -0,0 +1,137 @@
+PySocks==1.7.1
+pip==25.1.1
+certifi==2025.4.26
+parso==0.8.4
+wcwidth==0.2.13
+nvidia-ml-py==12.535.133
+mkl_fft==1.3.11
+urllib3==2.3.0
+charset-normalizer==3.3.2
+transformers==4.51.3
+smmap==4.0.0
+xxhash==3.5.0
+platformdirs==4.3.8
+tzdata==2025.2
+ipython==9.2.0
+deepspeed==0.16.7
+gmpy2==2.2.1
+pydantic==2.10.3
+tqdm==4.67.1
+pyparsing==3.2.0
+exceptiongroup==1.2.2
+tokenizers==0.21.1
+Markdown==3.8
+matplotlib-inline==0.1.7
+jupyter_core==5.7.2
+pydantic_core==2.27.1
+hjson==3.1.0
+mkl_random==1.2.8
+ipykernel==6.29.5
+sentry-sdk==2.18.0
+wandb==0.19.10
+einops==0.8.1
+prompt_toolkit==3.0.51
+Bottleneck==1.4.2
+msgpack==1.1.0
+fsspec==2024.12.0
+mdurl==0.1.0
+executing==2.2.0
+wheel==0.45.1
+bitsandbytes==0.45.5
+networkx==3.4.2
+comm==0.2.2
+asttokens==3.0.0
+nvitop==1.5.0
+sympy==1.13.1
+MarkupSafe==3.0.2
+numexpr==2.10.1
+tensorboard_data_server==0.7.0
+markdown-it-py==2.2.0
+six==1.17.0
+idna==3.7
+docker-pycreds==0.4.0
+jedi==0.19.2
+GitPython==3.1.43
+pure_eval==0.2.3
+ninja==1.11.1.4
+setuptools==80.1.0
+fonttools==4.55.3
+annotated-types==0.6.0
+psutil==7.0.0
+cycler==0.11.0
+py-cpuinfo==9.0.0
+accelerate==1.6.0
+filelock==3.17.0
+pytz==2024.1
+rich==13.9.4
+appdirs==1.4.4
+click==8.1.8
+Jinja2==3.1.6
+unicodedata2==15.1.0
+pillow==11.1.0
+traitlets==5.14.3
+stack_data==0.6.3
+peft==0.15.2
+python-dateutil==2.9.0.post0
+requests==2.32.3
+Werkzeug==3.1.3
+gitdb==4.0.7
+torch==2.5.1
+scipy==1.15.2
+sentencepiece==0.2.0
+huggingface-hub==0.31.1
+dill==0.3.8
+pexpect==4.9.0
+pickleshare==0.7.5
+ptyprocess==0.7.0
+contourpy==1.3.1
+grpcio==1.71.0
+optree==0.14.1
+safetensors==0.5.3
+mpmath==1.3.0
+nest_asyncio==1.6.0
+pyarrow==19.0.0
+PyYAML==6.0.2
+decorator==5.2.1
+pandas==2.2.3
+tensorboard==2.19.0
+zipp==3.21.0
+regex==2024.11.6
+Brotli==1.0.9
+mkl-service==2.4.0
+matplotlib==3.10.0
+Pygments==2.19.1
+absl-py==2.1.0
+hf-xet==1.1.0
+packaging==25.0
+jupyter_client==8.6.3
+triton==3.1.0
+multiprocess==0.70.15
+debugpy==1.8.14
+numpy==2.0.1
+protobuf==5.29.3
+tornado==6.4.2
+datasets==3.6.0
+eval_type_backport==0.2.2
+typing_extensions==4.12.2
+kiwisolver==1.4.8
+pyzmq==26.4.0
+ipython_pygments_lexers==1.1.1
+setproctitle==1.2.2
+importlib_metadata==8.6.1
+jaraco.text==3.12.1
+backports.tarfile==1.2.0
+importlib_metadata==8.0.0
+jaraco.context==5.3.0
+more-itertools==10.3.0
+wheel==0.45.1
+jaraco.collections==5.1.0
+platformdirs==4.2.2
+autocommand==2.2.2
+zipp==3.19.2
+tomli==2.0.1
+typeguard==4.3.0
+packaging==24.2
+inflect==7.3.1
+typing_extensions==4.12.2
+jaraco.functools==4.0.1

wandb/run-20250508_160817-0lem0nck/files/wandb-metadata.json

@@ -0,0 +1,106 @@
+{
+  "os":  "Linux-5.15.0-138-generic-x86_64-with-glibc2.31",
+  "python":  "CPython 3.11.11",
+  "startedAt":  "2025-05-08T20:08:17.504166Z",
+  "args":  [
+    "--local_rank=0",
+    "--train_datasets",
+    "tools",
+    "--model_name_or_path",
+    "meta-llama/Llama-3.1-8B-Instruct",
+    "--cache_dir",
+    "/home/panda/pda-llm/cache/sft-tools",
+    "--important_sft",
+    "false",
+    "--max_length",
+    "4096",
+    "--trust_remote_code",
+    "True",
+    "--epochs",
+    "3",
+    "--per_device_train_batch_size",
+    "1",
+    "--per_device_eval_batch_size",
+    "1",
+    "--gradient_accumulation_steps",
+    "48",
+    "--gradient_checkpointing",
+    "--learning_rate",
+    "1e-4",
+    "--lr_scheduler_type",
+    "cosine",
+    "--lr_warmup_ratio",
+    "0.1",
+    "--weight_decay",
+    "0.0",
+    "--seed",
+    "42",
+    "--output_dir",
+    "/home/panda/pda-llm/output/sft-tools/run-false-1-10",
+    "--log_type",
+    "wandb",
+    "--log_project",
+    "TOOLS-SFT",
+    "--zero_stage",
+    "0",
+    "--offload",
+    "none",
+    "--safety_ratio_tol",
+    "10",
+    "--resilient_coeff",
+    "1",
+    "--lora_r",
+    "16",
+    "--lora_alpha",
+    "32",
+    "--lora_dropout",
+    "0.05",
+    "--bf16",
+    "False",
+    "--fp16",
+    "True",
+    "--tf32",
+    "False"
+  ],
+  "program":  "-m safe_rlhf.algorithms.tools_ft.__main__",
+  "git":  {
+    "remote":  "git@github.com:IgnacioBoero/pda-llm.git",
+    "commit":  "3e25c5a329b7cfcf30b4af51d00c42fd38478aba"
+  },
+  "email":  "iboero@upenn.edu",
+  "root":  "/home/panda/pda-llm/output/sft-tools/run-false-1-10",
+  "host":  "plaza",
+  "executable":  "/home/panda/miniconda3/envs/pda/bin/python",
+  "cpu_count":  24,
+  "cpu_count_logical":  48,
+  "gpu":  "NVIDIA RTX A6000",
+  "gpu_count":  2,
+  "disk":  {
+    "/":  {
+      "total":  "982820896768",
+      "used":  "888964751360"
+    }
+  },
+  "memory":  {
+    "total":  "270090010624"
+  },
+  "cpu":  {
+    "count":  24,
+    "countLogical":  48
+  },
+  "gpu_nvidia":  [
+    {
+      "name":  "NVIDIA RTX A6000",
+      "memoryTotal":  "51527024640",
+      "cudaCores":  10752,
+      "architecture":  "Ampere"
+    },
+    {
+      "name":  "NVIDIA RTX A6000",
+      "memoryTotal":  "51527024640",
+      "cudaCores":  10752,
+      "architecture":  "Ampere"
+    }
+  ],
+  "cudaVersion":  "12.4"
+}

wandb/run-20250508_160817-0lem0nck/logs/debug-core.log

@@ -0,0 +1,7 @@
+{"time":"2025-05-08T16:08:17.190213216-04:00","level":"INFO","msg":"main: starting server","port-filename":"/tmp/tmpmnui6ecd/port-1356767.txt","pid":1356767,"log-level":0,"disable-analytics":false,"shutdown-on-parent-exit":false}
+{"time":"2025-05-08T16:08:17.196180277-04:00","level":"INFO","msg":"Will exit if parent process dies.","ppid":1356767}
+{"time":"2025-05-08T16:08:17.196169706-04:00","level":"INFO","msg":"server is running","addr":{"IP":"127.0.0.1","Port":33129,"Zone":""}}
+{"time":"2025-05-08T16:08:17.376078695-04:00","level":"INFO","msg":"connection: ManageConnectionData: new connection created","id":"127.0.0.1:43214"}
+{"time":"2025-05-08T16:08:17.506096175-04:00","level":"INFO","msg":"handleInformInit: received","streamId":"0lem0nck","id":"127.0.0.1:43214"}
+{"time":"2025-05-08T16:08:17.699304314-04:00","level":"INFO","msg":"handleInformInit: stream started","streamId":"0lem0nck","id":"127.0.0.1:43214"}
+{"time":"2025-05-08T16:08:23.606079392-04:00","level":"INFO","msg":"received shutdown signal","signal":15}

wandb/run-20250508_160817-0lem0nck/logs/debug-internal.log

@@ -0,0 +1,9 @@
+{"time":"2025-05-08T16:08:17.506377451-04:00","level":"INFO","msg":"stream: starting","core version":"0.19.10","symlink path":"/home/panda/pda-llm/output/sft-tools/run-false-1-10/wandb/run-20250508_160817-0lem0nck/logs/debug-core.log"}
+{"time":"2025-05-08T16:08:17.699250973-04:00","level":"INFO","msg":"created new stream","id":"0lem0nck"}
+{"time":"2025-05-08T16:08:17.699295584-04:00","level":"INFO","msg":"stream: started","id":"0lem0nck"}
+{"time":"2025-05-08T16:08:17.699316065-04:00","level":"INFO","msg":"writer: Do: started","stream_id":"0lem0nck"}
+{"time":"2025-05-08T16:08:17.699320815-04:00","level":"INFO","msg":"handler: started","stream_id":"0lem0nck"}
+{"time":"2025-05-08T16:08:17.699350985-04:00","level":"INFO","msg":"sender: started","stream_id":"0lem0nck"}
+{"time":"2025-05-08T16:08:17.867900772-04:00","level":"INFO","msg":"Starting system monitor"}
+{"time":"2025-05-08T16:08:17.867932482-04:00","level":"WARN","msg":"handleCodeSave: program relative path is empty"}
+{"time":"2025-05-08T16:08:17.867991013-04:00","level":"ERROR","msg":"git repo not found","error":"repository does not exist"}

wandb/run-20250508_160817-0lem0nck/logs/debug.log

@@ -0,0 +1,22 @@
+2025-05-08 16:08:17,501 INFO    MainThread:1356767 [wandb_setup.py:_flush():68] Current SDK version is 0.19.10
+2025-05-08 16:08:17,501 INFO    MainThread:1356767 [wandb_setup.py:_flush():68] Configure stats pid to 1356767
+2025-05-08 16:08:17,501 INFO    MainThread:1356767 [wandb_setup.py:_flush():68] Loading settings from /home/panda/.config/wandb/settings
+2025-05-08 16:08:17,501 INFO    MainThread:1356767 [wandb_setup.py:_flush():68] Loading settings from /home/panda/pda-llm/scripts/wandb/settings
+2025-05-08 16:08:17,501 INFO    MainThread:1356767 [wandb_setup.py:_flush():68] Loading settings from environment variables
+2025-05-08 16:08:17,501 INFO    MainThread:1356767 [wandb_init.py:setup_run_log_directory():724] Logging user logs to /home/panda/pda-llm/output/sft-tools/run-false-1-10/wandb/run-20250508_160817-0lem0nck/logs/debug.log
+2025-05-08 16:08:17,501 INFO    MainThread:1356767 [wandb_init.py:setup_run_log_directory():725] Logging internal logs to /home/panda/pda-llm/output/sft-tools/run-false-1-10/wandb/run-20250508_160817-0lem0nck/logs/debug-internal.log
+2025-05-08 16:08:17,501 INFO    MainThread:1356767 [wandb_init.py:init():852] calling init triggers
+2025-05-08 16:08:17,502 INFO    MainThread:1356767 [wandb_init.py:init():857] wandb.init called with sweep_config: {}
+config: {'model_name_or_path': 'meta-llama/Llama-3.1-8B-Instruct', 'recompute_baseline': False, 'cache_dir': '/home/panda/pda-llm/cache/sft-tools', 'max_length': 4096, 'trust_remote_code': True, 'train_datasets': [('tools', {'proportion': 1.0})], 'eval_datasets': None, 'safety_ratio_tol': 10.0, 'important_sft': False, 'resilient_coeff': 1.0, 'epochs': 3, 'per_device_train_batch_size': 1, 'per_device_eval_batch_size': 1, 'gradient_accumulation_steps': 48, 'gradient_checkpointing': True, 'lr': 0.0001, 'lr_scheduler_type': <SchedulerType.COSINE: 'cosine'>, 'lr_warmup_ratio': 0.1, 'weight_decay': 0.0, 'seed': 42, 'fp16': True, 'bf16': False, 'tf32': False, 'lora_r': 16, 'lora_alpha': 32, 'lora_dropout': 0.05, 'eval_strategy': 'epoch', 'eval_interval': 1000000, 'need_eval': False, 'eval_split_ratio': None, 'output_dir': '/home/panda/pda-llm/output/sft-tools/run-false-1-10', 'log_type': 'wandb', 'log_dir': '/home/panda/pda-llm/output/sft-tools/run-false-1-10', 'log_project': 'TOOLS-SFT', 'log_run_name': 'tools-sft-2025-05-08-16-08-17', 'save_16bit': False, 'save_interval': 1000000, 'local_rank': 0, 'zero_stage': 0, 'offload': 'none', 'deepspeed': False, 'deepspeed_config': None, 'deepscale': False, 'deepscale_config': None, 'global_rank': 0, 'device': device(type='cuda', index=0), 'num_update_steps_per_epoch': 118, 'total_training_steps': 354, '_wandb': {}}
+2025-05-08 16:08:17,502 INFO    MainThread:1356767 [wandb_init.py:init():893] starting backend
+2025-05-08 16:08:17,502 INFO    MainThread:1356767 [wandb_init.py:init():897] sending inform_init request
+2025-05-08 16:08:17,503 INFO    MainThread:1356767 [backend.py:_multiprocessing_setup():101] multiprocessing start_methods=fork,spawn,forkserver, using: spawn
+2025-05-08 16:08:17,504 INFO    MainThread:1356767 [wandb_init.py:init():907] backend started and connected
+2025-05-08 16:08:17,505 INFO    MainThread:1356767 [wandb_init.py:init():1002] updated telemetry
+2025-05-08 16:08:17,510 INFO    MainThread:1356767 [wandb_init.py:init():1026] communicating run to backend with 90.0 second timeout
+2025-05-08 16:08:17,865 INFO    MainThread:1356767 [wandb_init.py:init():1101] starting run threads in backend
+2025-05-08 16:08:17,927 INFO    MainThread:1356767 [wandb_run.py:_console_start():2566] atexit reg
+2025-05-08 16:08:17,927 INFO    MainThread:1356767 [wandb_run.py:_redirect():2414] redirect: wrap_raw
+2025-05-08 16:08:17,927 INFO    MainThread:1356767 [wandb_run.py:_redirect():2483] Wrapping output streams.
+2025-05-08 16:08:17,927 INFO    MainThread:1356767 [wandb_run.py:_redirect():2506] Redirects installed.
+2025-05-08 16:08:17,929 INFO    MainThread:1356767 [wandb_init.py:init():1147] run started, returning control to user process

wandb/run-20250508_160902-sm1tkrdz/files/output.log

@@ -0,0 +1,2 @@
+***** Running training *****
+Training 1/3 epoch (loss 0.0966):   0%|          | 48/16950 [00:31<3:03:57,  1.53it/s]

wandb/run-20250508_160902-sm1tkrdz/files/requirements.txt

@@ -0,0 +1,137 @@
+PySocks==1.7.1
+pip==25.1.1
+certifi==2025.4.26
+parso==0.8.4
+wcwidth==0.2.13
+nvidia-ml-py==12.535.133
+mkl_fft==1.3.11
+urllib3==2.3.0
+charset-normalizer==3.3.2
+transformers==4.51.3
+smmap==4.0.0
+xxhash==3.5.0
+platformdirs==4.3.8
+tzdata==2025.2
+ipython==9.2.0
+deepspeed==0.16.7
+gmpy2==2.2.1
+pydantic==2.10.3
+tqdm==4.67.1
+pyparsing==3.2.0
+exceptiongroup==1.2.2
+tokenizers==0.21.1
+Markdown==3.8
+matplotlib-inline==0.1.7
+jupyter_core==5.7.2
+pydantic_core==2.27.1
+hjson==3.1.0
+mkl_random==1.2.8
+ipykernel==6.29.5
+sentry-sdk==2.18.0
+wandb==0.19.10
+einops==0.8.1
+prompt_toolkit==3.0.51
+Bottleneck==1.4.2
+msgpack==1.1.0
+fsspec==2024.12.0
+mdurl==0.1.0
+executing==2.2.0
+wheel==0.45.1
+bitsandbytes==0.45.5
+networkx==3.4.2
+comm==0.2.2
+asttokens==3.0.0
+nvitop==1.5.0
+sympy==1.13.1
+MarkupSafe==3.0.2
+numexpr==2.10.1
+tensorboard_data_server==0.7.0
+markdown-it-py==2.2.0
+six==1.17.0
+idna==3.7
+docker-pycreds==0.4.0
+jedi==0.19.2
+GitPython==3.1.43
+pure_eval==0.2.3
+ninja==1.11.1.4
+setuptools==80.1.0
+fonttools==4.55.3
+annotated-types==0.6.0
+psutil==7.0.0
+cycler==0.11.0
+py-cpuinfo==9.0.0
+accelerate==1.6.0
+filelock==3.17.0
+pytz==2024.1
+rich==13.9.4
+appdirs==1.4.4
+click==8.1.8
+Jinja2==3.1.6
+unicodedata2==15.1.0
+pillow==11.1.0
+traitlets==5.14.3
+stack_data==0.6.3
+peft==0.15.2
+python-dateutil==2.9.0.post0
+requests==2.32.3
+Werkzeug==3.1.3
+gitdb==4.0.7
+torch==2.5.1
+scipy==1.15.2
+sentencepiece==0.2.0
+huggingface-hub==0.31.1
+dill==0.3.8
+pexpect==4.9.0
+pickleshare==0.7.5
+ptyprocess==0.7.0
+contourpy==1.3.1
+grpcio==1.71.0
+optree==0.14.1
+safetensors==0.5.3
+mpmath==1.3.0
+nest_asyncio==1.6.0
+pyarrow==19.0.0
+PyYAML==6.0.2
+decorator==5.2.1
+pandas==2.2.3
+tensorboard==2.19.0
+zipp==3.21.0
+regex==2024.11.6
+Brotli==1.0.9
+mkl-service==2.4.0
+matplotlib==3.10.0
+Pygments==2.19.1
+absl-py==2.1.0
+hf-xet==1.1.0
+packaging==25.0
+jupyter_client==8.6.3
+triton==3.1.0
+multiprocess==0.70.15
+debugpy==1.8.14
+numpy==2.0.1
+protobuf==5.29.3
+tornado==6.4.2
+datasets==3.6.0
+eval_type_backport==0.2.2
+typing_extensions==4.12.2
+kiwisolver==1.4.8
+pyzmq==26.4.0
+ipython_pygments_lexers==1.1.1
+setproctitle==1.2.2
+importlib_metadata==8.6.1
+jaraco.text==3.12.1
+backports.tarfile==1.2.0
+importlib_metadata==8.0.0
+jaraco.context==5.3.0
+more-itertools==10.3.0
+wheel==0.45.1
+jaraco.collections==5.1.0
+platformdirs==4.2.2
+autocommand==2.2.2
+zipp==3.19.2
+tomli==2.0.1
+typeguard==4.3.0
+packaging==24.2
+inflect==7.3.1
+typing_extensions==4.12.2
+jaraco.functools==4.0.1

wandb/run-20250508_160902-sm1tkrdz/files/wandb-metadata.json

@@ -0,0 +1,106 @@
+{
+  "os":  "Linux-5.15.0-138-generic-x86_64-with-glibc2.31",
+  "python":  "CPython 3.11.11",
+  "startedAt":  "2025-05-08T20:09:02.642208Z",
+  "args":  [
+    "--local_rank=0",
+    "--train_datasets",
+    "tools",
+    "--model_name_or_path",
+    "meta-llama/Llama-3.1-8B-Instruct",
+    "--cache_dir",
+    "/home/panda/pda-llm/cache/sft-tools",
+    "--important_sft",
+    "false",
+    "--max_length",
+    "4096",
+    "--trust_remote_code",
+    "True",
+    "--epochs",
+    "3",
+    "--per_device_train_batch_size",
+    "1",
+    "--per_device_eval_batch_size",
+    "1",
+    "--gradient_accumulation_steps",
+    "48",
+    "--gradient_checkpointing",
+    "--learning_rate",
+    "1e-4",
+    "--lr_scheduler_type",
+    "cosine",
+    "--lr_warmup_ratio",
+    "0.1",
+    "--weight_decay",
+    "0.0",
+    "--seed",
+    "42",
+    "--output_dir",
+    "/home/panda/pda-llm/output/sft-tools/run-false-1-10",
+    "--log_type",
+    "wandb",
+    "--log_project",
+    "TOOLS-SFT",
+    "--zero_stage",
+    "0",
+    "--offload",
+    "none",
+    "--safety_ratio_tol",
+    "10",
+    "--resilient_coeff",
+    "1",
+    "--lora_r",
+    "16",
+    "--lora_alpha",
+    "32",
+    "--lora_dropout",
+    "0.05",
+    "--bf16",
+    "False",
+    "--fp16",
+    "True",
+    "--tf32",
+    "False"
+  ],
+  "program":  "-m safe_rlhf.algorithms.tools_ft.__main__",
+  "git":  {
+    "remote":  "git@github.com:IgnacioBoero/pda-llm.git",
+    "commit":  "3e25c5a329b7cfcf30b4af51d00c42fd38478aba"
+  },
+  "email":  "iboero@upenn.edu",
+  "root":  "/home/panda/pda-llm/output/sft-tools/run-false-1-10",
+  "host":  "plaza",
+  "executable":  "/home/panda/miniconda3/envs/pda/bin/python",
+  "cpu_count":  24,
+  "cpu_count_logical":  48,
+  "gpu":  "NVIDIA RTX A6000",
+  "gpu_count":  2,
+  "disk":  {
+    "/":  {
+      "total":  "982820896768",
+      "used":  "888964878336"
+    }
+  },
+  "memory":  {
+    "total":  "270090010624"
+  },
+  "cpu":  {
+    "count":  24,
+    "countLogical":  48
+  },
+  "gpu_nvidia":  [
+    {
+      "name":  "NVIDIA RTX A6000",
+      "memoryTotal":  "51527024640",
+      "cudaCores":  10752,
+      "architecture":  "Ampere"
+    },
+    {
+      "name":  "NVIDIA RTX A6000",
+      "memoryTotal":  "51527024640",
+      "cudaCores":  10752,
+      "architecture":  "Ampere"
+    }
+  ],
+  "cudaVersion":  "12.4"
+}

wandb/run-20250508_160902-sm1tkrdz/logs/debug-core.log

@@ -0,0 +1,7 @@
+{"time":"2025-05-08T16:09:02.351406897-04:00","level":"INFO","msg":"main: starting server","port-filename":"/tmp/tmpab3smczz/port-1359956.txt","pid":1359956,"log-level":0,"disable-analytics":false,"shutdown-on-parent-exit":false}
+{"time":"2025-05-08T16:09:02.357060512-04:00","level":"INFO","msg":"Will exit if parent process dies.","ppid":1359956}
+{"time":"2025-05-08T16:09:02.357067772-04:00","level":"INFO","msg":"server is running","addr":{"IP":"127.0.0.1","Port":34899,"Zone":""}}
+{"time":"2025-05-08T16:09:02.536932502-04:00","level":"INFO","msg":"connection: ManageConnectionData: new connection created","id":"127.0.0.1:52382"}
+{"time":"2025-05-08T16:09:02.644160501-04:00","level":"INFO","msg":"handleInformInit: received","streamId":"sm1tkrdz","id":"127.0.0.1:52382"}
+{"time":"2025-05-08T16:09:02.838054407-04:00","level":"INFO","msg":"handleInformInit: stream started","streamId":"sm1tkrdz","id":"127.0.0.1:52382"}
+{"time":"2025-05-08T16:09:35.696088794-04:00","level":"INFO","msg":"received shutdown signal","signal":15}

wandb/run-20250508_160902-sm1tkrdz/logs/debug-internal.log

@@ -0,0 +1,9 @@
+{"time":"2025-05-08T16:09:02.644440987-04:00","level":"INFO","msg":"stream: starting","core version":"0.19.10","symlink path":"/home/panda/pda-llm/output/sft-tools/run-false-1-10/wandb/run-20250508_160902-sm1tkrdz/logs/debug-core.log"}
+{"time":"2025-05-08T16:09:02.838002086-04:00","level":"INFO","msg":"created new stream","id":"sm1tkrdz"}
+{"time":"2025-05-08T16:09:02.838045347-04:00","level":"INFO","msg":"stream: started","id":"sm1tkrdz"}
+{"time":"2025-05-08T16:09:02.838062637-04:00","level":"INFO","msg":"writer: Do: started","stream_id":"sm1tkrdz"}
+{"time":"2025-05-08T16:09:02.838103338-04:00","level":"INFO","msg":"handler: started","stream_id":"sm1tkrdz"}
+{"time":"2025-05-08T16:09:02.838129438-04:00","level":"INFO","msg":"sender: started","stream_id":"sm1tkrdz"}
+{"time":"2025-05-08T16:09:02.986767932-04:00","level":"INFO","msg":"Starting system monitor"}
+{"time":"2025-05-08T16:09:02.986795302-04:00","level":"WARN","msg":"handleCodeSave: program relative path is empty"}
+{"time":"2025-05-08T16:09:02.986852694-04:00","level":"ERROR","msg":"git repo not found","error":"repository does not exist"}

wandb/run-20250508_160902-sm1tkrdz/logs/debug.log

@@ -0,0 +1,22 @@
+2025-05-08 16:09:02,639 INFO    MainThread:1359956 [wandb_setup.py:_flush():68] Current SDK version is 0.19.10
+2025-05-08 16:09:02,639 INFO    MainThread:1359956 [wandb_setup.py:_flush():68] Configure stats pid to 1359956
+2025-05-08 16:09:02,639 INFO    MainThread:1359956 [wandb_setup.py:_flush():68] Loading settings from /home/panda/.config/wandb/settings
+2025-05-08 16:09:02,639 INFO    MainThread:1359956 [wandb_setup.py:_flush():68] Loading settings from /home/panda/pda-llm/scripts/wandb/settings
+2025-05-08 16:09:02,639 INFO    MainThread:1359956 [wandb_setup.py:_flush():68] Loading settings from environment variables
+2025-05-08 16:09:02,639 INFO    MainThread:1359956 [wandb_init.py:setup_run_log_directory():724] Logging user logs to /home/panda/pda-llm/output/sft-tools/run-false-1-10/wandb/run-20250508_160902-sm1tkrdz/logs/debug.log
+2025-05-08 16:09:02,639 INFO    MainThread:1359956 [wandb_init.py:setup_run_log_directory():725] Logging internal logs to /home/panda/pda-llm/output/sft-tools/run-false-1-10/wandb/run-20250508_160902-sm1tkrdz/logs/debug-internal.log
+2025-05-08 16:09:02,639 INFO    MainThread:1359956 [wandb_init.py:init():852] calling init triggers
+2025-05-08 16:09:02,639 INFO    MainThread:1359956 [wandb_init.py:init():857] wandb.init called with sweep_config: {}
+config: {'model_name_or_path': 'meta-llama/Llama-3.1-8B-Instruct', 'recompute_baseline': False, 'cache_dir': '/home/panda/pda-llm/cache/sft-tools', 'max_length': 4096, 'trust_remote_code': True, 'train_datasets': [('tools', {'proportion': 1.0})], 'eval_datasets': None, 'safety_ratio_tol': 10.0, 'important_sft': False, 'resilient_coeff': 1.0, 'epochs': 3, 'per_device_train_batch_size': 1, 'per_device_eval_batch_size': 1, 'gradient_accumulation_steps': 48, 'gradient_checkpointing': True, 'lr': 0.0001, 'lr_scheduler_type': <SchedulerType.COSINE: 'cosine'>, 'lr_warmup_ratio': 0.1, 'weight_decay': 0.0, 'seed': 42, 'fp16': True, 'bf16': False, 'tf32': False, 'lora_r': 16, 'lora_alpha': 32, 'lora_dropout': 0.05, 'eval_strategy': 'epoch', 'eval_interval': 1000000, 'need_eval': False, 'eval_split_ratio': None, 'output_dir': '/home/panda/pda-llm/output/sft-tools/run-false-1-10', 'log_type': 'wandb', 'log_dir': '/home/panda/pda-llm/output/sft-tools/run-false-1-10', 'log_project': 'TOOLS-SFT', 'log_run_name': 'tools-sft-2025-05-08-16-09-02', 'save_16bit': False, 'save_interval': 1000000, 'local_rank': 0, 'zero_stage': 0, 'offload': 'none', 'deepspeed': False, 'deepspeed_config': None, 'deepscale': False, 'deepscale_config': None, 'global_rank': 0, 'device': device(type='cuda', index=0), 'num_update_steps_per_epoch': 118, 'total_training_steps': 354, '_wandb': {}}
+2025-05-08 16:09:02,639 INFO    MainThread:1359956 [wandb_init.py:init():893] starting backend
+2025-05-08 16:09:02,639 INFO    MainThread:1359956 [wandb_init.py:init():897] sending inform_init request
+2025-05-08 16:09:02,642 INFO    MainThread:1359956 [backend.py:_multiprocessing_setup():101] multiprocessing start_methods=fork,spawn,forkserver, using: spawn
+2025-05-08 16:09:02,642 INFO    MainThread:1359956 [wandb_init.py:init():907] backend started and connected
+2025-05-08 16:09:02,643 INFO    MainThread:1359956 [wandb_init.py:init():1002] updated telemetry
+2025-05-08 16:09:02,648 INFO    MainThread:1359956 [wandb_init.py:init():1026] communicating run to backend with 90.0 second timeout
+2025-05-08 16:09:02,984 INFO    MainThread:1359956 [wandb_init.py:init():1101] starting run threads in backend
+2025-05-08 16:09:03,046 INFO    MainThread:1359956 [wandb_run.py:_console_start():2566] atexit reg
+2025-05-08 16:09:03,046 INFO    MainThread:1359956 [wandb_run.py:_redirect():2414] redirect: wrap_raw
+2025-05-08 16:09:03,046 INFO    MainThread:1359956 [wandb_run.py:_redirect():2483] Wrapping output streams.
+2025-05-08 16:09:03,046 INFO    MainThread:1359956 [wandb_run.py:_redirect():2506] Redirects installed.
+2025-05-08 16:09:03,048 INFO    MainThread:1359956 [wandb_init.py:init():1147] run started, returning control to user process

wandb/run-20250508_183954-8zs3b4m6/files/config.yaml

@@ -0,0 +1,133 @@
+_wandb:
+    value:
+        cli_version: 0.19.10
+        m: []
+        python_version: 3.11.11
+        t:
+            "1":
+                - 1
+                - 11
+                - 49
+                - 51
+                - 55
+                - 71
+                - 98
+            "2":
+                - 1
+                - 11
+                - 49
+                - 51
+                - 55
+                - 71
+                - 98
+            "3":
+                - 2
+                - 13
+                - 16
+                - 23
+                - 55
+            "4": 3.11.11
+            "5": 0.19.10
+            "6": 4.51.3
+            "8":
+                - 5
+            "12": 0.19.10
+            "13": linux-x86_64
+bf16:
+    value: true
+cache_dir:
+    value: /home/panda/pda-llm/cache/sft-tools
+deepscale:
+    value: false
+deepscale_config:
+    value: null
+deepspeed:
+    value: false
+deepspeed_config:
+    value: null
+device:
+    value: cuda:0
+epochs:
+    value: 3
+eval_datasets:
+    value: null
+eval_interval:
+    value: 1000000
+eval_split_ratio:
+    value: null
+eval_strategy:
+    value: epoch
+fp16:
+    value: false
+global_rank:
+    value: 0
+gradient_accumulation_steps:
+    value: 24
+gradient_checkpointing:
+    value: true
+important_sft:
+    value: false
+local_rank:
+    value: 0
+log_dir:
+    value: /home/panda/pda-llm/output/sft-tools/run-false-1-10
+log_project:
+    value: TOOLS-SFT
+log_run_name:
+    value: tools-sft-2025-05-08-18-39-54
+log_type:
+    value: wandb
+lora_alpha:
+    value: 32
+lora_dropout:
+    value: 0.05
+lora_r:
+    value: 16
+lr:
+    value: 0.0001
+lr_scheduler_type:
+    value: COSINE
+lr_warmup_ratio:
+    value: 0.1
+max_length:
+    value: 4096
+model_name_or_path:
+    value: meta-llama/Llama-3.1-8B-Instruct
+need_eval:
+    value: true
+num_update_steps_per_epoch:
+    value: 118
+offload:
+    value: none
+output_dir:
+    value: /home/panda/pda-llm/output/sft-tools/run-false-1-10
+per_device_eval_batch_size:
+    value: 2
+per_device_train_batch_size:
+    value: 2
+recompute_baseline:
+    value: false
+resilient_coeff:
+    value: 1
+safety_ratio_tol:
+    value: 10
+save_16bit:
+    value: false
+save_interval:
+    value: 1000000
+seed:
+    value: 42
+tf32:
+    value: false
+total_training_steps:
+    value: 354
+train_datasets:
+    value:
+        - - tools
+          - proportion: 1
+trust_remote_code:
+    value: true
+weight_decay:
+    value: 0
+zero_stage:
+    value: 0

wandb/run-20250508_183954-8zs3b4m6/files/output.log

@@ -0,0 +1,33 @@
+***** Running training *****
+Training 1/3 epoch:   0%|          | 0/8475 [00:00<?, ?it/s]Traceback (most recent call last):
+
+***** Evaluating at the beginning *****
+  File "<frozen runpy>", line 198, in _run_module_as_main
+  File "<frozen runpy>", line 88, in _run_code
+  File "/home/panda/pda-llm/safe_rlhf/algorithms/tools_ft/__main__.py", line 23, in <module>
+    sys.exit(main())
+             ^^^^^^
+  File "/home/panda/pda-llm/safe_rlhf/algorithms/tools_ft/main.py", line 367, in main
+    trainer.train()
+  File "/home/panda/pda-llm/safe_rlhf/trainers/supervised_tools_trainer.py", line 292, in train
+    self.logger.log(self.eval(), step=0)
+                    ^^^^^^^^^^^
+  File "/home/panda/pda-llm/safe_rlhf/trainers/supervised_tools_trainer.py", line 361, in eval
+    safe_log_ratios = safe_log_ratios.cpu().numpy()
+                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+TypeError: Got unsupported ScalarType BFloat16
+[rank0]: Traceback (most recent call last):
+[rank0]:   File "<frozen runpy>", line 198, in _run_module_as_main
+[rank0]:   File "<frozen runpy>", line 88, in _run_code
+[rank0]:   File "/home/panda/pda-llm/safe_rlhf/algorithms/tools_ft/__main__.py", line 23, in <module>
+[rank0]:     sys.exit(main())
+[rank0]:              ^^^^^^
+[rank0]:   File "/home/panda/pda-llm/safe_rlhf/algorithms/tools_ft/main.py", line 367, in main
+[rank0]:     trainer.train()
+[rank0]:   File "/home/panda/pda-llm/safe_rlhf/trainers/supervised_tools_trainer.py", line 292, in train
+[rank0]:     self.logger.log(self.eval(), step=0)
+[rank0]:                     ^^^^^^^^^^^
+[rank0]:   File "/home/panda/pda-llm/safe_rlhf/trainers/supervised_tools_trainer.py", line 361, in eval
+[rank0]:     safe_log_ratios = safe_log_ratios.cpu().numpy()
+[rank0]:                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+[rank0]: TypeError: Got unsupported ScalarType BFloat16

wandb/run-20250508_183954-8zs3b4m6/files/requirements.txt

@@ -0,0 +1,137 @@
+PySocks==1.7.1
+pip==25.1.1
+certifi==2025.4.26
+parso==0.8.4
+wcwidth==0.2.13
+nvidia-ml-py==12.535.133
+mkl_fft==1.3.11
+urllib3==2.3.0
+charset-normalizer==3.3.2
+transformers==4.51.3
+smmap==4.0.0
+xxhash==3.5.0
+platformdirs==4.3.8
+tzdata==2025.2
+ipython==9.2.0
+deepspeed==0.16.7
+gmpy2==2.2.1
+pydantic==2.10.3
+tqdm==4.67.1
+pyparsing==3.2.0
+exceptiongroup==1.2.2
+tokenizers==0.21.1
+Markdown==3.8
+matplotlib-inline==0.1.7
+jupyter_core==5.7.2
+pydantic_core==2.27.1
+hjson==3.1.0
+mkl_random==1.2.8
+ipykernel==6.29.5
+sentry-sdk==2.18.0
+wandb==0.19.10
+einops==0.8.1
+prompt_toolkit==3.0.51
+Bottleneck==1.4.2
+msgpack==1.1.0
+fsspec==2024.12.0
+mdurl==0.1.0
+executing==2.2.0
+wheel==0.45.1
+bitsandbytes==0.45.5
+networkx==3.4.2
+comm==0.2.2
+asttokens==3.0.0
+nvitop==1.5.0
+sympy==1.13.1
+MarkupSafe==3.0.2
+numexpr==2.10.1
+tensorboard_data_server==0.7.0
+markdown-it-py==2.2.0
+six==1.17.0
+idna==3.7
+docker-pycreds==0.4.0
+jedi==0.19.2
+GitPython==3.1.43
+pure_eval==0.2.3
+ninja==1.11.1.4
+setuptools==80.1.0
+fonttools==4.55.3
+annotated-types==0.6.0
+psutil==7.0.0
+cycler==0.11.0
+py-cpuinfo==9.0.0
+accelerate==1.6.0
+filelock==3.17.0
+pytz==2024.1
+rich==13.9.4
+appdirs==1.4.4
+click==8.1.8
+Jinja2==3.1.6
+unicodedata2==15.1.0
+pillow==11.1.0
+traitlets==5.14.3
+stack_data==0.6.3
+peft==0.15.2
+python-dateutil==2.9.0.post0
+requests==2.32.3
+Werkzeug==3.1.3
+gitdb==4.0.7
+torch==2.5.1
+scipy==1.15.2
+sentencepiece==0.2.0
+huggingface-hub==0.31.1
+dill==0.3.8
+pexpect==4.9.0
+pickleshare==0.7.5
+ptyprocess==0.7.0
+contourpy==1.3.1
+grpcio==1.71.0
+optree==0.14.1
+safetensors==0.5.3
+mpmath==1.3.0
+nest_asyncio==1.6.0
+pyarrow==19.0.0
+PyYAML==6.0.2
+decorator==5.2.1
+pandas==2.2.3
+tensorboard==2.19.0
+zipp==3.21.0
+regex==2024.11.6
+Brotli==1.0.9
+mkl-service==2.4.0
+matplotlib==3.10.0
+Pygments==2.19.1
+absl-py==2.1.0
+hf-xet==1.1.0
+packaging==25.0
+jupyter_client==8.6.3
+triton==3.1.0
+multiprocess==0.70.15
+debugpy==1.8.14
+numpy==2.0.1
+protobuf==5.29.3
+tornado==6.4.2
+datasets==3.6.0
+eval_type_backport==0.2.2
+typing_extensions==4.12.2
+kiwisolver==1.4.8
+pyzmq==26.4.0
+ipython_pygments_lexers==1.1.1
+setproctitle==1.2.2
+importlib_metadata==8.6.1
+jaraco.text==3.12.1
+backports.tarfile==1.2.0
+importlib_metadata==8.0.0
+jaraco.context==5.3.0
+more-itertools==10.3.0
+wheel==0.45.1
+jaraco.collections==5.1.0
+platformdirs==4.2.2
+autocommand==2.2.2
+zipp==3.19.2
+tomli==2.0.1
+typeguard==4.3.0
+packaging==24.2
+inflect==7.3.1
+typing_extensions==4.12.2
+jaraco.functools==4.0.1

wandb/run-20250508_183954-8zs3b4m6/files/wandb-metadata.json

@@ -0,0 +1,106 @@
+{
+  "os":  "Linux-5.15.0-138-generic-x86_64-with-glibc2.31",
+  "python":  "CPython 3.11.11",
+  "startedAt":  "2025-05-08T22:39:54.527119Z",
+  "args":  [
+    "--local_rank=0",
+    "--train_datasets",
+    "tools",
+    "--model_name_or_path",
+    "meta-llama/Llama-3.1-8B-Instruct",
+    "--cache_dir",
+    "/home/panda/pda-llm/cache/sft-tools",
+    "--important_sft",
+    "false",
+    "--max_length",
+    "4096",
+    "--trust_remote_code",
+    "True",
+    "--epochs",
+    "3",
+    "--per_device_train_batch_size",
+    "2",
+    "--per_device_eval_batch_size",
+    "2",
+    "--gradient_accumulation_steps",
+    "24",
+    "--gradient_checkpointing",
+    "--learning_rate",
+    "1e-4",
+    "--lr_scheduler_type",
+    "cosine",
+    "--lr_warmup_ratio",
+    "0.1",
+    "--weight_decay",
+    "0.0",
+    "--seed",
+    "42",
+    "--output_dir",
+    "/home/panda/pda-llm/output/sft-tools/run-false-1-10",
+    "--log_type",
+    "wandb",
+    "--log_project",
+    "TOOLS-SFT",
+    "--zero_stage",
+    "0",
+    "--offload",
+    "none",
+    "--safety_ratio_tol",
+    "10",
+    "--resilient_coeff",
+    "1",
+    "--lora_r",
+    "16",
+    "--lora_alpha",
+    "32",
+    "--lora_dropout",
+    "0.05",
+    "--bf16",
+    "True",
+    "--fp16",
+    "False",
+    "--tf32",
+    "False"
+  ],
+  "program":  "-m safe_rlhf.algorithms.tools_ft.__main__",
+  "git":  {
+    "remote":  "git@github.com:IgnacioBoero/pda-llm.git",
+    "commit":  "3e25c5a329b7cfcf30b4af51d00c42fd38478aba"
+  },
+  "email":  "iboero@upenn.edu",
+  "root":  "/home/panda/pda-llm/output/sft-tools/run-false-1-10",
+  "host":  "plaza",
+  "executable":  "/home/panda/miniconda3/envs/pda/bin/python",
+  "cpu_count":  24,
+  "cpu_count_logical":  48,
+  "gpu":  "NVIDIA RTX A6000",
+  "gpu_count":  2,
+  "disk":  {
+    "/":  {
+      "total":  "982820896768",
+      "used":  "888969158656"
+    }
+  },
+  "memory":  {
+    "total":  "270090010624"
+  },
+  "cpu":  {
+    "count":  24,
+    "countLogical":  48
+  },
+  "gpu_nvidia":  [
+    {
+      "name":  "NVIDIA RTX A6000",
+      "memoryTotal":  "51527024640",
+      "cudaCores":  10752,
+      "architecture":  "Ampere"
+    },
+    {
+      "name":  "NVIDIA RTX A6000",
+      "memoryTotal":  "51527024640",
+      "cudaCores":  10752,
+      "architecture":  "Ampere"
+    }
+  ],
+  "cudaVersion":  "12.4"
+}

wandb/run-20250508_183954-8zs3b4m6/files/wandb-summary.json

@@ -0,0 +1 @@
+{"_wandb":{"runtime":234}}

wandb/run-20250508_183954-8zs3b4m6/logs/debug-core.log

@@ -0,0 +1,15 @@
+{"time":"2025-05-08T18:39:54.176871407-04:00","level":"INFO","msg":"main: starting server","port-filename":"/tmp/tmpm9rm19xp/port-1436037.txt","pid":1436037,"log-level":0,"disable-analytics":false,"shutdown-on-parent-exit":false}
+{"time":"2025-05-08T18:39:54.182885566-04:00","level":"INFO","msg":"Will exit if parent process dies.","ppid":1436037}
+{"time":"2025-05-08T18:39:54.182859406-04:00","level":"INFO","msg":"server is running","addr":{"IP":"127.0.0.1","Port":46483,"Zone":""}}
+{"time":"2025-05-08T18:39:54.363158642-04:00","level":"INFO","msg":"connection: ManageConnectionData: new connection created","id":"127.0.0.1:44756"}
+{"time":"2025-05-08T18:39:54.529022063-04:00","level":"INFO","msg":"handleInformInit: received","streamId":"8zs3b4m6","id":"127.0.0.1:44756"}
+{"time":"2025-05-08T18:39:54.721870969-04:00","level":"INFO","msg":"handleInformInit: stream started","streamId":"8zs3b4m6","id":"127.0.0.1:44756"}
+{"time":"2025-05-08T18:43:49.728612807-04:00","level":"INFO","msg":"handleInformFinish: finish message received","streamId":"8zs3b4m6","id":"127.0.0.1:44756"}
+{"time":"2025-05-08T18:43:49.72878048-04:00","level":"INFO","msg":"handleInformFinish: stream closed","streamId":"8zs3b4m6","id":"127.0.0.1:44756"}
+{"time":"2025-05-08T18:43:50.729045178-04:00","level":"INFO","msg":"handleInformTeardown: server teardown initiated","id":"127.0.0.1:44756"}
+{"time":"2025-05-08T18:43:50.729069669-04:00","level":"INFO","msg":"handleInformTeardown: server shutdown complete","id":"127.0.0.1:44756"}
+{"time":"2025-05-08T18:43:50.729078859-04:00","level":"INFO","msg":"server is shutting down"}
+{"time":"2025-05-08T18:43:50.729084919-04:00","level":"INFO","msg":"connection: closing","id":"127.0.0.1:44756"}
+{"time":"2025-05-08T18:43:50.72916792-04:00","level":"INFO","msg":"connection: closed successfully","id":"127.0.0.1:44756"}
+{"time":"2025-05-08T18:43:50.729198191-04:00","level":"INFO","msg":"connection: ManageConnectionData: connection closed","id":"127.0.0.1:44756"}
+{"time":"2025-05-08T18:43:50.729212691-04:00","level":"INFO","msg":"server is closed"}

wandb/run-20250508_183954-8zs3b4m6/logs/debug-internal.log

@@ -0,0 +1,18 @@
+{"time":"2025-05-08T18:39:54.529292648-04:00","level":"INFO","msg":"stream: starting","core version":"0.19.10","symlink path":"/home/panda/pda-llm/output/sft-tools/run-false-1-10/wandb/run-20250508_183954-8zs3b4m6/logs/debug-core.log"}
+{"time":"2025-05-08T18:39:54.721820928-04:00","level":"INFO","msg":"created new stream","id":"8zs3b4m6"}
+{"time":"2025-05-08T18:39:54.721862709-04:00","level":"INFO","msg":"stream: started","id":"8zs3b4m6"}
+{"time":"2025-05-08T18:39:54.72190933-04:00","level":"INFO","msg":"handler: started","stream_id":"8zs3b4m6"}
+{"time":"2025-05-08T18:39:54.7218919-04:00","level":"INFO","msg":"writer: Do: started","stream_id":"8zs3b4m6"}
+{"time":"2025-05-08T18:39:54.721943141-04:00","level":"INFO","msg":"sender: started","stream_id":"8zs3b4m6"}
+{"time":"2025-05-08T18:39:55.093280788-04:00","level":"INFO","msg":"Starting system monitor"}
+{"time":"2025-05-08T18:39:55.093320509-04:00","level":"WARN","msg":"handleCodeSave: program relative path is empty"}
+{"time":"2025-05-08T18:39:55.09338449-04:00","level":"ERROR","msg":"git repo not found","error":"repository does not exist"}
+{"time":"2025-05-08T18:43:49.04487936-04:00","level":"INFO","msg":"Stopping system monitor"}
+{"time":"2025-05-08T18:43:49.045105404-04:00","level":"INFO","msg":"Stopped system monitor"}
+{"time":"2025-05-08T18:43:49.452801239-04:00","level":"INFO","msg":"fileTransfer: Close: file transfer manager closed"}
+{"time":"2025-05-08T18:43:49.726313582-04:00","level":"INFO","msg":"handler: operation stats","stats":{}}
+{"time":"2025-05-08T18:43:49.728641487-04:00","level":"INFO","msg":"stream: closing","id":"8zs3b4m6"}
+{"time":"2025-05-08T18:43:49.728657388-04:00","level":"INFO","msg":"handler: closed","stream_id":"8zs3b4m6"}
+{"time":"2025-05-08T18:43:49.728667428-04:00","level":"INFO","msg":"writer: Close: closed","stream_id":"8zs3b4m6"}
+{"time":"2025-05-08T18:43:49.728696378-04:00","level":"INFO","msg":"sender: closed","stream_id":"8zs3b4m6"}
+{"time":"2025-05-08T18:43:49.72877225-04:00","level":"INFO","msg":"stream: closed","id":"8zs3b4m6"}

wandb/run-20250508_183954-8zs3b4m6/logs/debug.log

@@ -0,0 +1,29 @@
+2025-05-08 18:39:54,524 INFO    MainThread:1436037 [wandb_setup.py:_flush():68] Current SDK version is 0.19.10
+2025-05-08 18:39:54,524 INFO    MainThread:1436037 [wandb_setup.py:_flush():68] Configure stats pid to 1436037
+2025-05-08 18:39:54,524 INFO    MainThread:1436037 [wandb_setup.py:_flush():68] Loading settings from /home/panda/.config/wandb/settings
+2025-05-08 18:39:54,524 INFO    MainThread:1436037 [wandb_setup.py:_flush():68] Loading settings from /home/panda/pda-llm/scripts/wandb/settings
+2025-05-08 18:39:54,524 INFO    MainThread:1436037 [wandb_setup.py:_flush():68] Loading settings from environment variables
+2025-05-08 18:39:54,524 INFO    MainThread:1436037 [wandb_init.py:setup_run_log_directory():724] Logging user logs to /home/panda/pda-llm/output/sft-tools/run-false-1-10/wandb/run-20250508_183954-8zs3b4m6/logs/debug.log
+2025-05-08 18:39:54,524 INFO    MainThread:1436037 [wandb_init.py:setup_run_log_directory():725] Logging internal logs to /home/panda/pda-llm/output/sft-tools/run-false-1-10/wandb/run-20250508_183954-8zs3b4m6/logs/debug-internal.log
+2025-05-08 18:39:54,524 INFO    MainThread:1436037 [wandb_init.py:init():852] calling init triggers
+2025-05-08 18:39:54,524 INFO    MainThread:1436037 [wandb_init.py:init():857] wandb.init called with sweep_config: {}
+config: {'model_name_or_path': 'meta-llama/Llama-3.1-8B-Instruct', 'recompute_baseline': False, 'cache_dir': '/home/panda/pda-llm/cache/sft-tools', 'max_length': 4096, 'trust_remote_code': True, 'train_datasets': [('tools', {'proportion': 1.0})], 'eval_datasets': None, 'safety_ratio_tol': 10.0, 'important_sft': False, 'resilient_coeff': 1.0, 'epochs': 3, 'per_device_train_batch_size': 2, 'per_device_eval_batch_size': 2, 'gradient_accumulation_steps': 24, 'gradient_checkpointing': True, 'lr': 0.0001, 'lr_scheduler_type': <SchedulerType.COSINE: 'cosine'>, 'lr_warmup_ratio': 0.1, 'weight_decay': 0.0, 'seed': 42, 'fp16': False, 'bf16': True, 'tf32': False, 'lora_r': 16, 'lora_alpha': 32, 'lora_dropout': 0.05, 'eval_strategy': 'epoch', 'eval_interval': 1000000, 'need_eval': True, 'eval_split_ratio': None, 'output_dir': '/home/panda/pda-llm/output/sft-tools/run-false-1-10', 'log_type': 'wandb', 'log_dir': '/home/panda/pda-llm/output/sft-tools/run-false-1-10', 'log_project': 'TOOLS-SFT', 'log_run_name': 'tools-sft-2025-05-08-18-39-54', 'save_16bit': False, 'save_interval': 1000000, 'local_rank': 0, 'zero_stage': 0, 'offload': 'none', 'deepspeed': False, 'deepspeed_config': None, 'deepscale': False, 'deepscale_config': None, 'global_rank': 0, 'device': device(type='cuda', index=0), 'num_update_steps_per_epoch': 118, 'total_training_steps': 354, '_wandb': {}}
+2025-05-08 18:39:54,525 INFO    MainThread:1436037 [wandb_init.py:init():893] starting backend
+2025-05-08 18:39:54,525 INFO    MainThread:1436037 [wandb_init.py:init():897] sending inform_init request
+2025-05-08 18:39:54,526 INFO    MainThread:1436037 [backend.py:_multiprocessing_setup():101] multiprocessing start_methods=fork,spawn,forkserver, using: spawn
+2025-05-08 18:39:54,526 INFO    MainThread:1436037 [wandb_init.py:init():907] backend started and connected
+2025-05-08 18:39:54,528 INFO    MainThread:1436037 [wandb_init.py:init():1002] updated telemetry
+2025-05-08 18:39:54,534 INFO    MainThread:1436037 [wandb_init.py:init():1026] communicating run to backend with 90.0 second timeout
+2025-05-08 18:39:55,090 INFO    MainThread:1436037 [wandb_init.py:init():1101] starting run threads in backend
+2025-05-08 18:39:55,157 INFO    MainThread:1436037 [wandb_run.py:_console_start():2566] atexit reg
+2025-05-08 18:39:55,157 INFO    MainThread:1436037 [wandb_run.py:_redirect():2414] redirect: wrap_raw
+2025-05-08 18:39:55,157 INFO    MainThread:1436037 [wandb_run.py:_redirect():2483] Wrapping output streams.
+2025-05-08 18:39:55,157 INFO    MainThread:1436037 [wandb_run.py:_redirect():2506] Redirects installed.
+2025-05-08 18:39:55,158 INFO    MainThread:1436037 [wandb_init.py:init():1147] run started, returning control to user process
+2025-05-08 18:43:49,004 INFO    MainThread:1436037 [wandb_run.py:_finish():2314] finishing run alelab/TOOLS-SFT/8zs3b4m6
+2025-05-08 18:43:49,005 INFO    MainThread:1436037 [wandb_run.py:_atexit_cleanup():2531] got exitcode: 0
+2025-05-08 18:43:49,005 INFO    MainThread:1436037 [wandb_run.py:_restore():2513] restore
+2025-05-08 18:43:49,005 INFO    MainThread:1436037 [wandb_run.py:_restore():2519] restore done
+2025-05-08 18:43:49,727 INFO    MainThread:1436037 [wandb_run.py:_footer_history_summary_info():4160] rendering history
+2025-05-08 18:43:49,728 INFO    MainThread:1436037 [wandb_run.py:_footer_history_summary_info():4192] rendering summary
+2025-05-08 18:43:49,728 INFO    MainThread:1436037 [wandb_run.py:_footer_sync_info():4121] logging synced files

wandb/run-20250508_190248-2v7dhhsz/files/config.yaml

@@ -0,0 +1,134 @@
+_wandb:
+    value:
+        cli_version: 0.19.10
+        m: []
+        python_version: 3.11.11
+        t:
+            "1":
+                - 1
+                - 11
+                - 49
+                - 51
+                - 55
+                - 71
+                - 98
+            "2":
+                - 1
+                - 11
+                - 49
+                - 51
+                - 55
+                - 71
+                - 98
+            "3":
+                - 2
+                - 13
+                - 16
+                - 23
+                - 55
+                - 61
+            "4": 3.11.11
+            "5": 0.19.10
+            "6": 4.51.3
+            "8":
+                - 5
+            "12": 0.19.10
+            "13": linux-x86_64
+bf16:
+    value: true
+cache_dir:
+    value: /home/panda/pda-llm/cache/sft-tools
+deepscale:
+    value: false
+deepscale_config:
+    value: null
+deepspeed:
+    value: false
+deepspeed_config:
+    value: null
+device:
+    value: cuda:0
+epochs:
+    value: 3
+eval_datasets:
+    value: null
+eval_interval:
+    value: 1000000
+eval_split_ratio:
+    value: null
+eval_strategy:
+    value: epoch
+fp16:
+    value: false
+global_rank:
+    value: 0
+gradient_accumulation_steps:
+    value: 24
+gradient_checkpointing:
+    value: true
+important_sft:
+    value: false
+local_rank:
+    value: 0
+log_dir:
+    value: /home/panda/pda-llm/output/sft-tools/run-false-1-10
+log_project:
+    value: TOOLS-SFT
+log_run_name:
+    value: tools-sft-2025-05-08-19-02-47
+log_type:
+    value: wandb
+lora_alpha:
+    value: 32
+lora_dropout:
+    value: 0.05
+lora_r:
+    value: 16
+lr:
+    value: 0.0001
+lr_scheduler_type:
+    value: COSINE
+lr_warmup_ratio:
+    value: 0.1
+max_length:
+    value: 4096
+model_name_or_path:
+    value: meta-llama/Llama-3.1-8B-Instruct
+need_eval:
+    value: true
+num_update_steps_per_epoch:
+    value: 118
+offload:
+    value: none
+output_dir:
+    value: /home/panda/pda-llm/output/sft-tools/run-false-1-10
+per_device_eval_batch_size:
+    value: 2
+per_device_train_batch_size:
+    value: 2
+recompute_baseline:
+    value: false
+resilient_coeff:
+    value: 1
+safety_ratio_tol:
+    value: 10
+save_16bit:
+    value: false
+save_interval:
+    value: 1000000
+seed:
+    value: 42
+tf32:
+    value: false
+total_training_steps:
+    value: 354
+train_datasets:
+    value:
+        - - tools
+          - proportion: 1
+trust_remote_code:
+    value: true
+weight_decay:
+    value: 0
+zero_stage:
+    value: 0

wandb/run-20250508_190248-2v7dhhsz/files/output.log

@@ -0,0 +1,141 @@
+***** Running training *****
+Training 1/3 epoch:   0%|          | 0/8475 [00:00<?, ?it/s]Traceback (most recent call last):
+
+***** Evaluating at the beginning *****
+  File "<frozen runpy>", line 198, in _run_module_as_main
+  File "<frozen runpy>", line 88, in _run_code
+  File "/home/panda/pda-llm/safe_rlhf/algorithms/tools_ft/__main__.py", line 23, in <module>
+    sys.exit(main())
+             ^^^^^^
+  File "/home/panda/pda-llm/safe_rlhf/algorithms/tools_ft/main.py", line 367, in main
+    trainer.train()
+  File "/home/panda/pda-llm/safe_rlhf/trainers/supervised_tools_trainer.py", line 292, in train
+    self.logger.log(self.eval(), step=0)
+  File "/home/panda/pda-llm/safe_rlhf/utils.py", line 195, in wrapper
+    return func(*args, **kwargs)
+           ^^^^^^^^^^^^^^^^^^^^^
+  File "/home/panda/pda-llm/safe_rlhf/logger.py", line 158, in log
+    self.wandb.log(metrics, step=step)
+  File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/wandb_run.py", line 406, in wrapper
+    return func(self, *args, **kwargs)
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/wandb_run.py", line 503, in wrapper
+    return func(self, *args, **kwargs)
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/wandb_run.py", line 464, in wrapper_fn
+    return func(self, *args, **kwargs)
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/wandb_run.py", line 451, in wrapper
+    return func(self, *args, **kwargs)
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/wandb_run.py", line 2064, in log
+    self._log(data=data, step=step, commit=commit)
+  File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/wandb_run.py", line 1776, in _log
+    self._partial_history_callback(data, step, commit)
+  File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/wandb_run.py", line 406, in wrapper
+    return func(self, *args, **kwargs)
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/wandb_run.py", line 1603, in _partial_history_callback
+    self._backend.interface.publish_partial_history(
+  File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/interface/interface.py", line 674, in publish_partial_history
+    data = history_dict_to_json(run, data, step=user_step, ignore_copy_err=True)
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/data_types/utils.py", line 54, in history_dict_to_json
+    payload[key] = val_to_json(
+                   ^^^^^^^^^^^^
+  File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/data_types/utils.py", line 162, in val_to_json
+    val.bind_to_run(run, key, namespace)
+  File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/data_types/table.py", line 510, in bind_to_run
+    util.json_dump_safer(data, fp)
+  File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/util.py", line 812, in json_dump_safer
+    return dump(obj, fp, cls=WandBJSONEncoder, **kwargs)
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/panda/miniconda3/envs/pda/lib/python3.11/json/__init__.py", line 179, in dump
+    for chunk in iterable:
+  File "/home/panda/miniconda3/envs/pda/lib/python3.11/json/encoder.py", line 432, in _iterencode
+    yield from _iterencode_dict(o, _current_indent_level)
+  File "/home/panda/miniconda3/envs/pda/lib/python3.11/json/encoder.py", line 406, in _iterencode_dict
+    yield from chunks
+  File "/home/panda/miniconda3/envs/pda/lib/python3.11/json/encoder.py", line 326, in _iterencode_list
+    yield from chunks
+  File "/home/panda/miniconda3/envs/pda/lib/python3.11/json/encoder.py", line 326, in _iterencode_list
+    yield from chunks
+  File "/home/panda/miniconda3/envs/pda/lib/python3.11/json/encoder.py", line 439, in _iterencode
+    o = _default(o)
+        ^^^^^^^^^^^
+  File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/util.py", line 763, in default
+    tmp_obj, converted = json_friendly(obj)
+                         ^^^^^^^^^^^^^^^^^^
+  File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/util.py", line 595, in json_friendly
+    obj = obj.cpu().detach().numpy()
+          ^^^^^^^^^^^^^^^^^^^^^^^^^^
+TypeError: Got unsupported ScalarType BFloat16
+[rank0]: Traceback (most recent call last):
+[rank0]:   File "<frozen runpy>", line 198, in _run_module_as_main
+[rank0]:   File "<frozen runpy>", line 88, in _run_code
+[rank0]:   File "/home/panda/pda-llm/safe_rlhf/algorithms/tools_ft/__main__.py", line 23, in <module>
+[rank0]:     sys.exit(main())
+[rank0]:              ^^^^^^
+[rank0]:   File "/home/panda/pda-llm/safe_rlhf/algorithms/tools_ft/main.py", line 367, in main
+[rank0]:     trainer.train()
+[rank0]:   File "/home/panda/pda-llm/safe_rlhf/trainers/supervised_tools_trainer.py", line 292, in train
+[rank0]:     self.logger.log(self.eval(), step=0)
+[rank0]:   File "/home/panda/pda-llm/safe_rlhf/utils.py", line 195, in wrapper
+[rank0]:     return func(*args, **kwargs)
+[rank0]:            ^^^^^^^^^^^^^^^^^^^^^
+[rank0]:   File "/home/panda/pda-llm/safe_rlhf/logger.py", line 158, in log
+[rank0]:     self.wandb.log(metrics, step=step)
+[rank0]:   File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/wandb_run.py", line 406, in wrapper
+[rank0]:     return func(self, *args, **kwargs)
+[rank0]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
+[rank0]:   File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/wandb_run.py", line 503, in wrapper
+[rank0]:     return func(self, *args, **kwargs)
+[rank0]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
+[rank0]:   File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/wandb_run.py", line 464, in wrapper_fn
+[rank0]:     return func(self, *args, **kwargs)
+[rank0]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
+[rank0]:   File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/wandb_run.py", line 451, in wrapper
+[rank0]:     return func(self, *args, **kwargs)
+[rank0]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
+[rank0]:   File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/wandb_run.py", line 2064, in log
+[rank0]:     self._log(data=data, step=step, commit=commit)
+[rank0]:   File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/wandb_run.py", line 1776, in _log
+[rank0]:     self._partial_history_callback(data, step, commit)
+[rank0]:   File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/wandb_run.py", line 406, in wrapper
+[rank0]:     return func(self, *args, **kwargs)
+[rank0]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
+[rank0]:   File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/wandb_run.py", line 1603, in _partial_history_callback
+[rank0]:     self._backend.interface.publish_partial_history(
+[rank0]:   File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/interface/interface.py", line 674, in publish_partial_history
+[rank0]:     data = history_dict_to_json(run, data, step=user_step, ignore_copy_err=True)
+[rank0]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+[rank0]:   File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/data_types/utils.py", line 54, in history_dict_to_json
+[rank0]:     payload[key] = val_to_json(
+[rank0]:                    ^^^^^^^^^^^^
+[rank0]:   File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/data_types/utils.py", line 162, in val_to_json
+[rank0]:     val.bind_to_run(run, key, namespace)
+[rank0]:   File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/sdk/data_types/table.py", line 510, in bind_to_run
+[rank0]:     util.json_dump_safer(data, fp)
+[rank0]:   File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/util.py", line 812, in json_dump_safer
+[rank0]:     return dump(obj, fp, cls=WandBJSONEncoder, **kwargs)
+[rank0]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+[rank0]:   File "/home/panda/miniconda3/envs/pda/lib/python3.11/json/__init__.py", line 179, in dump
+[rank0]:     for chunk in iterable:
+[rank0]:   File "/home/panda/miniconda3/envs/pda/lib/python3.11/json/encoder.py", line 432, in _iterencode
+[rank0]:     yield from _iterencode_dict(o, _current_indent_level)
+[rank0]:   File "/home/panda/miniconda3/envs/pda/lib/python3.11/json/encoder.py", line 406, in _iterencode_dict
+[rank0]:     yield from chunks
+[rank0]:   File "/home/panda/miniconda3/envs/pda/lib/python3.11/json/encoder.py", line 326, in _iterencode_list
+[rank0]:     yield from chunks
+[rank0]:   File "/home/panda/miniconda3/envs/pda/lib/python3.11/json/encoder.py", line 326, in _iterencode_list
+[rank0]:     yield from chunks
+[rank0]:   File "/home/panda/miniconda3/envs/pda/lib/python3.11/json/encoder.py", line 439, in _iterencode
+[rank0]:     o = _default(o)
+[rank0]:         ^^^^^^^^^^^
+[rank0]:   File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/util.py", line 763, in default
+[rank0]:     tmp_obj, converted = json_friendly(obj)
+[rank0]:                          ^^^^^^^^^^^^^^^^^^
+[rank0]:   File "/home/panda/miniconda3/envs/pda/lib/python3.11/site-packages/wandb/util.py", line 595, in json_friendly
+[rank0]:     obj = obj.cpu().detach().numpy()
+[rank0]:           ^^^^^^^^^^^^^^^^^^^^^^^^^^
+[rank0]: TypeError: Got unsupported ScalarType BFloat16