2026-06-18 11:15:04,356 INFO MainThread:57523 [wandb_setup.py:_flush():81] Current SDK version is 0.27.2 2026-06-18 11:15:04,357 INFO MainThread:57523 [wandb_setup.py:_flush():81] Configure stats pid to 57523 2026-06-18 11:15:04,357 INFO MainThread:57523 [wandb_setup.py:_flush():81] Loading settings from environment variables 2026-06-18 11:15:04,357 INFO MainThread:57523 [wandb_init.py:setup_run_log_directory():723] Logging user logs to adapters_sweep/cot_lr5e-5_L16/wandb/run-20260618_111504-15jtjyrw/logs/debug.log 2026-06-18 11:15:04,357 INFO MainThread:57523 [wandb_init.py:setup_run_log_directory():724] Logging internal logs to adapters_sweep/cot_lr5e-5_L16/wandb/run-20260618_111504-15jtjyrw/logs/debug-internal.log 2026-06-18 11:15:04,357 INFO MainThread:57523 [wandb_init.py:init():766] calling init triggers 2026-06-18 11:15:04,358 INFO MainThread:57523 [wandb_init.py:init():771] wandb.init called with sweep_config: {} config: {'model': 'mlx-community/Llama-3.2-1B-Instruct-bf16', 'train': True, 'data': 'data/mlx_cot', 'fine_tune_type': 'lora', 'optimizer': 'adam', 'mask_prompt': True, 'num_layers': 16, 'batch_size': 2, 'iters': 1500, 'val_batches': 25, 'learning_rate': 5e-05, 'steps_per_report': 50, 'steps_per_eval': 300, 'grad_accumulation_steps': 1, 'resume_adapter_file': None, 'adapter_path': 'adapters_sweep/cot_lr5e-5_L16', 'save_every': 500, 'test': False, 'test_batches': 500, 'max_seq_length': 2048, 'config': None, 'grad_checkpoint': True, 'clear_cache_threshold': 0, 'report_to': 'wandb', 'project_name': 'text2sql-qwen-coder', 'seed': 0, 'optimizer_config': {'adam': {}, 'adamw': {}, 'muon': {}, 'sgd': {}, 'adafactor': {}}, 'lr_schedule': None, 'lora_parameters': {'rank': 8, 'dropout': 0.0, 'scale': 20.0}, '_wandb': {}} 2026-06-18 11:15:04,358 INFO MainThread:57523 [wandb_init.py:init():814] starting backend 2026-06-18 11:15:04,358 INFO MainThread:57523 [wandb_init.py:init():829] sending inform_init request 2026-06-18 11:15:05,103 INFO MainThread:57523 [wandb_init.py:init():834] backend started and connected 2026-06-18 11:15:05,115 INFO MainThread:57523 [wandb_init.py:init():904] updated telemetry 2026-06-18 11:15:05,149 INFO MainThread:57523 [wandb_init.py:init():927] communicating run to backend with 90.0 second timeout 2026-06-18 11:15:05,976 INFO MainThread:57523 [wandb_init.py:init():972] starting run threads in backend 2026-06-18 11:15:06,085 INFO MainThread:57523 [wandb_run.py:_console_start():2523] atexit reg 2026-06-18 11:15:06,085 INFO MainThread:57523 [wandb_run.py:_redirect():2373] redirect: wrap_raw 2026-06-18 11:15:06,086 INFO MainThread:57523 [wandb_run.py:_redirect():2442] Wrapping output streams. 2026-06-18 11:15:06,086 INFO MainThread:57523 [wandb_run.py:_redirect():2465] Redirects installed. 2026-06-18 11:15:06,087 INFO MainThread:57523 [wandb_init.py:init():1010] run started, returning control to user process 2026-06-18 11:57:06,270 INFO wandb-AsyncioManager-main:57523 [service_client.py:_forward_responses():122] [no run ID] Reached EOF. 2026-06-18 11:57:06,280 INFO wandb-AsyncioManager-main:57523 [mailbox.py:close():154] [no run ID] Closing mailbox, abandoning 2 handles. 2026-06-18 11:57:06,283 ERROR wandb-AsyncioManager-main:57523 [asyncio_manager.py:fn_wrap_exceptions():184] Uncaught exception in run_soon callback. Traceback (most recent call last): File "/Users/apple/Documents/work/Kd-project/Insufficient_Tokens/W5_Fine_Tune/.venv/lib/python3.11/site-packages/wandb/sdk/lib/asyncio_manager.py", line 182, in fn_wrap_exceptions await fn() File "/Users/apple/Documents/work/Kd-project/Insufficient_Tokens/W5_Fine_Tune/.venv/lib/python3.11/site-packages/wandb/sdk/lib/run_messages.py", line 90, in loop await asyncio_compat.race( File "/Users/apple/Documents/work/Kd-project/Insufficient_Tokens/W5_Fine_Tune/.venv/lib/python3.11/site-packages/wandb/sdk/lib/asyncio_compat.py", line 278, in race async with open_task_group(race=True) as tg: File "/opt/homebrew/Cellar/python@3.11/3.11.15/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py", line 217, in __aexit__ await anext(self.gen) File "/Users/apple/Documents/work/Kd-project/Insufficient_Tokens/W5_Fine_Tune/.venv/lib/python3.11/site-packages/wandb/sdk/lib/asyncio_compat.py", line 240, in open_task_group await task_group._wait_all(race=race, timeout=exit_timeout) File "/Users/apple/Documents/work/Kd-project/Insufficient_Tokens/W5_Fine_Tune/.venv/lib/python3.11/site-packages/wandb/sdk/lib/asyncio_compat.py", line 180, in _wait_all raise exc File "/Users/apple/Documents/work/Kd-project/Insufficient_Tokens/W5_Fine_Tune/.venv/lib/python3.11/site-packages/wandb/sdk/lib/run_messages.py", line 113, in _print_all result = await handle.wait_async(timeout=None) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/apple/Documents/work/Kd-project/Insufficient_Tokens/W5_Fine_Tune/.venv/lib/python3.11/site-packages/wandb/sdk/mailbox/mailbox_handle.py", line 127, in wait_async response = await self._handle.wait_async(timeout=timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/apple/Documents/work/Kd-project/Insufficient_Tokens/W5_Fine_Tune/.venv/lib/python3.11/site-packages/wandb/sdk/mailbox/response_handle.py", line 123, in wait_async raise HandleAbandonedError() wandb.sdk.mailbox.mailbox_handle.HandleAbandonedError 2026-06-18 11:57:08,150 ERROR wandb-AsyncioManager-main:57523 [asyncio_manager.py:fn_wrap_exceptions():184] Uncaught exception in run_soon callback. Traceback (most recent call last): File "/Users/apple/Documents/work/Kd-project/Insufficient_Tokens/W5_Fine_Tune/.venv/lib/python3.11/site-packages/wandb/sdk/lib/asyncio_manager.py", line 182, in fn_wrap_exceptions await fn() File "/Users/apple/Documents/work/Kd-project/Insufficient_Tokens/W5_Fine_Tune/.venv/lib/python3.11/site-packages/wandb/sdk/lib/run_stopping.py", line 87, in loop should_stop = await self._check_should_stop() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/apple/Documents/work/Kd-project/Insufficient_Tokens/W5_Fine_Tune/.venv/lib/python3.11/site-packages/wandb/sdk/lib/run_stopping.py", line 115, in _check_should_stop handle = await self._interface.deliver_async(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/apple/Documents/work/Kd-project/Insufficient_Tokens/W5_Fine_Tune/.venv/lib/python3.11/site-packages/wandb/sdk/interface/interface_sock.py", line 58, in deliver_async handle = await self._client.deliver(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/apple/Documents/work/Kd-project/Insufficient_Tokens/W5_Fine_Tune/.venv/lib/python3.11/site-packages/wandb/sdk/lib/service/service_client.py", line 57, in deliver handle = self._mailbox.require_response(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/apple/Documents/work/Kd-project/Insufficient_Tokens/W5_Fine_Tune/.venv/lib/python3.11/site-packages/wandb/sdk/mailbox/mailbox.py", line 89, in require_response raise MailboxClosedError() wandb.sdk.mailbox.mailbox.MailboxClosedError