/workspace/third_party/AnyBimanual/train.py:18: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_name="config", config_path="conf") /workspace/envs/rlbench/lib/python3.10/site-packages/hydra/_internal/defaults_list.py:251: UserWarning: In 'config': Defaults list is missing `_self_`. See https://hydra.cc/docs/1.2/upgrades/1.0_to_1.1/default_composition_order for more information warnings.warn(msg, UserWarning) /workspace/envs/rlbench/lib/python3.10/site-packages/hydra/core/default_element.py:124: UserWarning: In 'method/PERACT_BC': Usage of deprecated keyword in package header '# @package _group_'. See https://hydra.cc/docs/1.2/upgrades/1.0_to_1.1/changes_to_package_header for more information deprecation_warning( /workspace/envs/rlbench/lib/python3.10/site-packages/hydra/_internal/hydra.py:119: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default. See https://hydra.cc/docs/1.2/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information. ret = run_job( [2026-03-30 05:02:50,133][root][INFO] - method: name: PERACT_BC agent_type: leader_follower robot_name: bimanual image_crop_size: 64 bounds_offset: - 0.15 voxel_sizes: - 100 include_prev_layer: false num_latents: 2048 latent_dim: 512 transformer_depth: 6 transformer_iterations: 1 cross_heads: 1 cross_dim_head: 64 latent_heads: 8 latent_dim_head: 64 pos_encoding_with_lang: true conv_downsample: true lang_fusion_type: seq voxel_patch_size: 5 voxel_patch_stride: 5 final_dim: 64 low_dim_size: 4 input_dropout: 0.1 attn_dropout: 0.1 decoder_dropout: 0.0 lr: 0.0005 lr_scheduler: false num_warmup_steps: 3000 optimizer: lamb lambda_weight_l2: 1.0e-06 trans_loss_weight: 1.0 rot_loss_weight: 1.0 grip_loss_weight: 1.0 collision_loss_weight: 1.0 rotation_resolution: 5 activation: lrelu norm: None crop_augmentation: true transform_augmentation: apply_se3: true aug_xyz: - 0.125 - 0.125 - 0.125 aug_rpy: - 0.0 - 0.0 - 45.0 aug_rot_resolution: ${method.rotation_resolution} demo_augmentation: true demo_augmentation_every_n: 10 no_skip_connection: false no_perceiver: false no_language: false keypoint_method: heuristic ddp: master_addr: localhost master_port: 12992 num_devices: 1 rlbench: task_name: peract_bc_subset3_overlap_smoke200 tasks: - coordinated_push_box - coordinated_lift_ball - dual_push_buttons demos: 100 demo_path: /workspace/baselines/AnyBimanual_subset3_demo_root episode_length: 25 cameras: - over_shoulder_right - wrist_right - front - overhead - over_shoulder_left - wrist_left camera_resolution: - 256 - 256 scene_bounds: - -0.3 - -0.5 - 0.6 - 0.7 - 0.5 - 1.6 include_lang_goal_in_obs: true instructions: /path replay: batch_size: 2 timesteps: 1 prioritisation: false task_uniform: true use_disk: true path: /workspace/baselines/AnyBimanual_overlap_replay max_parallel_processes: 8 task_folder: multi framework: log_freq: 50 save_freq: 200 train_envs: 1 replay_ratio: ${replay.batch_size} transitions_before_train: 200 tensorboard_logging: true csv_logging: true training_iterations: 200 gpu: 0 env_gpu: 0 logdir: /workspace/baselines/AnyBimanual_overlap_runs logging_level: 20 seeds: 1 start_seed: 0 load_existing_weights: false num_weights_to_keep: 60 num_workers: 0 record_every_n: 5 checkpoint_name_prefix: checkpoint use_wandb: false wandb_project: '' wandb_group: peract_bc_subset3_overlap_smoke200 seed: 0 wandb_name: peract_bc_subset3_overlap_smoke200 use_skill: true use_pretrained: false use_prefix: false frozen: false anybimanual: true augmentation_type: ab [2026-03-30 05:02:50,135][root][INFO] - CWD:/workspace/baselines/AnyBimanual_overlap_runs/peract_bc_subset3_overlap_smoke200/PERACT_BC [2026-03-30 05:02:50,142][root][INFO] - Starting seed 0. /workspace/third_party/AnyBimanual/train.py:18: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_name="config", config_path="conf") [03/30/26 05:03:02] INFO INFO:root:Using method agent_factory.py:23 PERACT_BC with type leader_follower /workspace/baselines/AnyBimanual_overlap_replay/multi No replay files found. Creating replay... [03/30/26 05:03:05] INFO INFO:root:Creating a uniform_replay_buffer.py:156 TaskUniformReplayBuffe r replay memory with the following parameters: INFO INFO:root: uniform_replay_buffer.py:159 timesteps: 1 INFO INFO:root: uniform_replay_buffer.py:160 replay_capacity: 300000 INFO INFO:root: uniform_replay_buffer.py:161 batch_size: 2 INFO INFO:root: uniform_replay_buffer.py:162 update_horizon: 1 INFO INFO:root: uniform_replay_buffer.py:163 gamma: 0.990000 INFO INFO:root: uniform_replay_buffer.py:169 saving to disk: /workspace/baselines/A nyBimanual_overlap_rep lay/multi /workspace/third_party/AnyBimanual/train.py:18: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_name="config", config_path="conf") /workspace/third_party/AnyBimanual/train.py:18: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_name="config", config_path="conf") /workspace/third_party/AnyBimanual/train.py:18: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_name="config", config_path="conf") /workspace/third_party/AnyBimanual/train.py:18: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_name="config", config_path="conf") Found 4 keypoints. [50, 82, 84, 95] Found 4 keypoints. [49, 95, 97, 118] Found 4 keypoints. [65, 85, 87, 103] Found 4 keypoints. [48, 86, 88, 98] Found 4 keypoints. [63, 100, 102, 120] Found 4 keypoints. [60, 86, 88, 100] Found 4 keypoints. [47, 94, 96, 108] Found 4 keypoints. [48, 88, 90, 100] Found 4 keypoints. [48, 90, 92, 103] Found 4 keypoints. [60, 88, 90, 102] Found 4 keypoints. [49, 94, 96, 108] Found 4 keypoints. [48, 85, 87, 97] Found 4 keypoints. [50, 78, 80, 91] Found 4 keypoints. [48, 87, 89, 100] Found 4 keypoints. [53, 86, 88, 99] Found 4 keypoints. [47, 96, 98, 113] Found 4 keypoints. [53, 92, 94, 109] Found 4 keypoints. [62, 87, 89, 103] Found 4 keypoints. [52, 86, 88, 99] Found 4 keypoints. [61, 99, 101, 122] Found 4 keypoints. [56, 88, 90, 101] Found 4 keypoints. [58, 93, 95, 108] Found 4 keypoints. [67, 85, 87, 106] Found 4 keypoints. [48, 91, 93, 105] Found 4 keypoints. [67, 91, 93, 109] Found 4 keypoints. [53, 85, 87, 98] Found 4 keypoints. [49, 96, 98, 110] Found 4 keypoints. [50, 86, 88, 99] Found 4 keypoints. [49, 83, 85, 96] Found 4 keypoints. [51, 95, 97, 110] Found 4 keypoints. [51, 97, 99, 118] Found 4 keypoints. [64, 77, 79, 95] Found 4 keypoints. [49, 82, 84, 94] Found 4 keypoints. [49, 93, 95, 106] Found 4 keypoints. [56, 98, 100, 118] Found 4 keypoints. [62, 93, 95, 110] Found 4 keypoints. [56, 91, 93, 105] Found 4 keypoints. [55, 87, 89, 100] Found 4 keypoints. [54, 81, 83, 102] Found 4 keypoints. [65, 94, 96, 112] Found 4 keypoints. [68, 100, 102, 164] Found 4 keypoints. [47, 81, 83, 93] Found 4 keypoints. [56, 88, 90, 102] Found 4 keypoints. [67, 96, 98, 118] Found 4 keypoints. [61, 88, 90, 103] Found 4 keypoints. [59, 93, 95, 109] Found 4 keypoints. [47, 87, 89, 99] Found 4 keypoints. [67, 87, 89, 107] Found 4 keypoints. [50, 87, 89, 99] Found 4 keypoints. [68, 82, 84, 104] Found 4 keypoints. [49, 94, 96, 108] Found 4 keypoints. [57, 88, 90, 101] Found 4 keypoints. [64, 73, 75, 91] Found 4 keypoints. [50, 84, 86, 96] Found 4 keypoints. [65, 89, 91, 108] Found 4 keypoints. [48, 83, 85, 95] Found 4 keypoints. [66, 76, 78, 93] Found 4 keypoints. [56, 81, 83, 95] Found 4 keypoints. [60, 80, 82, 95] Found 4 keypoints. [66, 96, 98, 121] Found 4 keypoints. [61, 87, 89, 102] Found 4 keypoints. [66, 94, 96, 115] Found 4 keypoints. [65, 85, 87, 101] Found 4 keypoints. [64, 98, 100, 115] Found 4 keypoints. [57, 91, 93, 105] Found 4 keypoints. [50, 86, 88, 98] Found 4 keypoints. [48, 83, 85, 95] Found 4 keypoints. [51, 77, 79, 90] Found 4 keypoints. [55, 83, 85, 97] Found 4 keypoints. [47, 89, 91, 103] Found 4 keypoints. [48, 99, 101, 116] Found 4 keypoints. [52, 83, 85, 95] Found 4 keypoints. [56, 97, 99, 122] Found 4 keypoints. [47, 102, 104, 130] Found 4 keypoints. [47, 83, 85, 95] Found 4 keypoints. [48, 89, 91, 103] Found 4 keypoints. [50, 84, 86, 97] Found 4 keypoints. [55, 91, 93, 105] Found 4 keypoints. [63, 90, 92, 106] Found 4 keypoints. [59, 87, 89, 101] Found 4 keypoints. [68, 72, 74, 96] Found 4 keypoints. [68, 86, 88, 108] Found 4 keypoints. [64, 86, 88, 103] Found 4 keypoints. [65, 82, 84, 100] Found 4 keypoints. [54, 87, 89, 101] Found 4 keypoints. [55, 95, 97, 109] Found 4 keypoints. [52, 90, 92, 105] Found 4 keypoints. [57, 90, 92, 105] Found 4 keypoints. [66, 93, 95, 112] Found 4 keypoints. [48, 92, 94, 105] Found 4 keypoints. [48, 81, 83, 93] Found 4 keypoints. [54, 84, 86, 97] Found 5 keypoints. [48, 78, 80, 94, 96] Found 4 keypoints. [66, 91, 93, 111] Found 4 keypoints. [66, 88, 90, 107] Found 4 keypoints. [54, 99, 101, 121] Found 4 keypoints. [54, 89, 91, 102] Found 4 keypoints. [51, 88, 90, 101] Found 4 keypoints. [61, 94, 96, 112] Found 4 keypoints. [47, 84, 86, 97] Found 2 keypoints. [101, 157] Found 1 keypoints. [131] Found 2 keypoints. [94, 146] Found 2 keypoints. [81, 131] Found 2 keypoints. [84, 136] Found 2 keypoints. [93, 141] Found 2 keypoints. [92, 143] Found 2 keypoints. [94, 149] Found 2 keypoints. [86, 135] Found 2 keypoints. [99, 153] Found 2 keypoints. [96, 147] Found 2 keypoints. [102, 154] Found 2 keypoints. [86, 134] Found 2 keypoints. [83, 136] Found 2 keypoints. [86, 134] Found 2 keypoints. [87, 135] Found 2 keypoints. [96, 147] Found 2 keypoints. [88, 135] Found 2 keypoints. [86, 136] Found 2 keypoints. [86, 135] Found 2 keypoints. [85, 136] Found 2 keypoints. [85, 134] Found 2 keypoints. [92, 143] Found 2 keypoints. [96, 149] Found 2 keypoints. [100, 157] Found 2 keypoints. [93, 142] Found 2 keypoints. [84, 137] Found 2 keypoints. [97, 154] Found 2 keypoints. [86, 134] Found 2 keypoints. [91, 138] Found 2 keypoints. [95, 146] Found 2 keypoints. [86, 135] Found 2 keypoints. [85, 135] Found 2 keypoints. [94, 150] Found 2 keypoints. [89, 136] Found 2 keypoints. [89, 139] Found 2 keypoints. [95, 151] Found 2 keypoints. [90, 137] Found 2 keypoints. [103, 155] Found 2 keypoints. [89, 140] Found 2 keypoints. [85, 134] Found 2 keypoints. [91, 139] Found 2 keypoints. [89, 139] Found 2 keypoints. [87, 137] Found 2 keypoints. [104, 155] Found 2 keypoints. [91, 141] Found 2 keypoints. [87, 138] Found 2 keypoints. [96, 148] Found 2 keypoints. [91, 138] Found 2 keypoints. [104, 162] Found 2 keypoints. [87, 136] Found 2 keypoints. [88, 134] Found 2 keypoints. [87, 135] Found 2 keypoints. [86, 135] Found 2 keypoints. [90, 135] Found 2 keypoints. [86, 135] Found 2 keypoints. [86, 135] Found 2 keypoints. [76, 129] Found 2 keypoints. [87, 137] Found 2 keypoints. [90, 139] Found 2 keypoints. [88, 139] Found 2 keypoints. [94, 148] Found 2 keypoints. [104, 154] Found 2 keypoints. [92, 139] Found 2 keypoints. [99, 153] Found 2 keypoints. [96, 151] Found 2 keypoints. [87, 137] Found 2 keypoints. [92, 146] Found 2 keypoints. [78, 125] Found 2 keypoints. [86, 136] Found 2 keypoints. [78, 132] Found 2 keypoints. [97, 150] Found 2 keypoints. [89, 135] Found 2 keypoints. [83, 132] Found 2 keypoints. [85, 134] Found 2 keypoints. [91, 141] Found 2 keypoints. [87, 135] Found 2 keypoints. [91, 145] Found 2 keypoints. [92, 141] Found 2 keypoints. [93, 142] Found 2 keypoints. [88, 134] Found 2 keypoints. [87, 135] Found 2 keypoints. [92, 138] Found 1 keypoints. [127] Found 2 keypoints. [88, 139] Found 2 keypoints. [93, 144] Found 2 keypoints. [85, 136] Found 2 keypoints. [85, 135] Found 2 keypoints. [84, 137] Found 2 keypoints. [92, 142] Found 2 keypoints. [85, 136] Found 2 keypoints. [93, 143] Found 2 keypoints. [81, 129] Found 2 keypoints. [84, 136] Found 2 keypoints. [85, 135] Found 2 keypoints. [89, 139] Found 2 keypoints. [90, 142] Found 2 keypoints. [86, 137] Found 2 keypoints. [82, 130] Found 2 keypoints. [93, 146] Found 4 keypoints. [73, 89, 97, 129] Found 4 keypoints. [76, 92, 100, 125] Found 4 keypoints. [67, 83, 90, 116] Found 4 keypoints. [65, 81, 89, 120] Found 4 keypoints. [79, 112, 120, 162] Found 4 keypoints. [66, 83, 91, 119] Found 4 keypoints. [80, 96, 104, 131] Found 4 keypoints. [78, 97, 105, 142] Found 4 keypoints. [72, 89, 97, 132] Found 4 keypoints. [76, 93, 102, 129] Found 4 keypoints. [68, 84, 92, 116] Found 4 keypoints. [70, 86, 94, 119] Found 4 keypoints. [73, 91, 98, 134] Found 4 keypoints. [73, 88, 96, 126] Found 4 keypoints. [84, 103, 111, 164] Found 4 keypoints. [80, 98, 106, 143] Found 4 keypoints. [77, 94, 102, 147] Found 4 keypoints. [73, 93, 102, 133] Found 4 keypoints. [70, 87, 95, 121] Found 4 keypoints. [72, 90, 99, 129] Found 4 keypoints. [68, 85, 94, 121] Found 4 keypoints. [82, 99, 106, 145] Found 4 keypoints. [74, 90, 98, 128] Found 4 keypoints. [74, 91, 99, 136] Found 4 keypoints. [65, 83, 92, 122] Found 4 keypoints. [65, 80, 89, 117] Found 4 keypoints. [78, 96, 105, 140] Found 4 keypoints. [80, 99, 106, 156] Found 4 keypoints. [68, 84, 92, 120] Found 4 keypoints. [76, 93, 102, 129] Found 4 keypoints. [80, 102, 109, 137] Found 4 keypoints. [79, 103, 110, 155] Found 4 keypoints. [78, 95, 104, 148] Found 4 keypoints. [74, 91, 99, 138] Found 4 keypoints. [74, 91, 99, 131] Found 4 keypoints. [82, 99, 108, 145] Found 4 keypoints. [70, 86, 94, 125] Found 4 keypoints. [66, 82, 90, 120] Found 4 keypoints. [65, 83, 92, 118] Found 4 keypoints. [73, 97, 104, 141] Found 4 keypoints. [78, 106, 114, 145] Found 4 keypoints. [72, 87, 95, 118] Found 7 keypoints. [63, 66, 73, 74, 99, 107, 143] Found 4 keypoints. [76, 93, 100, 141] Found 4 keypoints. [72, 88, 95, 134] Found 4 keypoints. [86, 103, 111, 148] Found 4 keypoints. [78, 100, 108, 135] Found 4 keypoints. [70, 86, 94, 125] Found 4 keypoints. [76, 92, 99, 138] Found 4 keypoints. [69, 85, 93, 125] Found 4 keypoints. [69, 85, 93, 126] Found 4 keypoints. [78, 95, 102, 140] Found 4 keypoints. [69, 86, 94, 122] Found 4 keypoints. [78, 99, 109, 144] Found 4 keypoints. [68, 89, 98, 123] Found 4 keypoints. [77, 95, 102, 129] Found 4 keypoints. [79, 99, 108, 135] Found 4 keypoints. [82, 99, 108, 140] Found 4 keypoints. [68, 85, 93, 119] Found 4 keypoints. [74, 90, 98, 133] Found 4 keypoints. [82, 99, 108, 143] Found 4 keypoints. [79, 95, 103, 135] Found 4 keypoints. [75, 89, 97, 130] Found 4 keypoints. [77, 94, 103, 139] Found 4 keypoints. [73, 91, 97, 142] Found 4 keypoints. [76, 92, 101, 131] Found 4 keypoints. [76, 93, 102, 130] Found 4 keypoints. [80, 98, 107, 148] Found 4 keypoints. [79, 105, 113, 146] Found 4 keypoints. [67, 84, 93, 121] Found 4 keypoints. [76, 93, 102, 137] Found 4 keypoints. [74, 90, 98, 128] Found 4 keypoints. [73, 89, 97, 133] Found 4 keypoints. [77, 94, 102, 140] Found 4 keypoints. [72, 95, 102, 141] Found 4 keypoints. [67, 83, 91, 125] Found 4 keypoints. [73, 89, 98, 123] Found 4 keypoints. [76, 92, 100, 136] Found 4 keypoints. [78, 96, 105, 139] Found 4 keypoints. [79, 95, 103, 137] Found 4 keypoints. [70, 86, 95, 119] Found 4 keypoints. [68, 84, 91, 126] Found 4 keypoints. [67, 85, 94, 119] Found 4 keypoints. [66, 82, 90, 118] Found 4 keypoints. [75, 92, 100, 124] Found 4 keypoints. [71, 88, 96, 128] Found 4 keypoints. [67, 82, 91, 118] Found 4 keypoints. [81, 101, 110, 145] Found 4 keypoints. [79, 95, 103, 139] Found 4 keypoints. [70, 111, 120, 150] Found 4 keypoints. [65, 82, 91, 118] Found 4 keypoints. [74, 91, 99, 143] Found 4 keypoints. [79, 112, 121, 148] Found 4 keypoints. [81, 97, 104, 136] Found 4 keypoints. [75, 92, 100, 136] Found 4 keypoints. [73, 88, 96, 121] Found 4 keypoints. [72, 89, 98, 123] Found 4 keypoints. [77, 94, 103, 129] Found 4 keypoints. [83, 101, 110, 147] Found 4 keypoints. [64, 83, 91, 117] [03/30/26 07:29:49] INFO INFO:root:# Q qattention_peract_bc_agent.py:315 Params: 44677231 INFO INFO:root:# Q qattention_peract_bc_agent.py:315 Params: 44677743 Error executing job with overrides: ['method=PERACT_BC', 'ddp.num_devices=1', 'ddp.master_port=12992', 'framework.logdir=/workspace/baselines/AnyBimanual_overlap_runs', 'framework.start_seed=0', 'framework.use_wandb=False', 'framework.wandb_project=', 'framework.wandb_group=peract_bc_subset3_overlap_smoke200', 'framework.wandb_name=peract_bc_subset3_overlap_smoke200', 'framework.gpu=0', 'framework.env_gpu=0', 'framework.anybimanual=True', 'framework.augmentation_type=ab', 'framework.training_iterations=200', 'framework.save_freq=200', 'framework.log_freq=50', 'framework.seeds=1', 'framework.num_workers=0', 'replay.batch_size=2', 'replay.path=/workspace/baselines/AnyBimanual_overlap_replay', 'replay.task_folder=multi', 'replay.max_parallel_processes=8', 'rlbench.task_name=peract_bc_subset3_overlap_smoke200', 'rlbench.tasks=[coordinated_push_box,coordinated_lift_ball,dual_push_buttons]', 'rlbench.demo_path=/workspace/baselines/AnyBimanual_subset3_demo_root', 'rlbench.demos=100', 'rlbench.episode_length=25'] Traceback (most recent call last): File "/workspace/third_party/AnyBimanual/train.py", line 98, in main mp.spawn( File "/workspace/envs/rlbench/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 340, in spawn return start_processes(fn, args, nprocs, join, daemon, start_method="spawn") File "/workspace/envs/rlbench/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 296, in start_processes while not context.join(): File "/workspace/envs/rlbench/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 211, in join raise ProcessRaisedException(msg, error_index, failed_process.pid) torch.multiprocessing.spawn.ProcessRaisedException: -- Process 0 terminated with the following error: Traceback (most recent call last): File "/workspace/envs/rlbench/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 87, in _wrap fn(i, *args) File "/workspace/third_party/AnyBimanual/run_seed_fn.py", line 238, in run_seed train_runner.start() File "/workspace/third_party/AnyBimanual/third_party/YARR/yarr/runners/offline_train_runner.py", line 125, in start self._agent.load_weights(file_path) File "/workspace/third_party/AnyBimanual/third_party/YARR/yarr/agents/agent.py", line 337, in load_weights self.leader_agent.load_weights(savedir) File "/workspace/third_party/AnyBimanual/helpers/preprocess_agent.py", line 125, in load_weights self._pose_agent.load_weights(savedir) File "/workspace/third_party/AnyBimanual/agents/peract_bc/qattention_stack_agent.py", line 128, in load_weights qa.load_weights(savedir) File "/workspace/third_party/AnyBimanual/agents/peract_bc/qattention_peract_bc_agent.py", line 861, in load_weights state_dict = torch.load(weight_file, map_location=device) File "/workspace/envs/rlbench/lib/python3.10/site-packages/torch/serialization.py", line 1530, in load with _open_file_like(f, "rb") as opened_file: File "/workspace/envs/rlbench/lib/python3.10/site-packages/torch/serialization.py", line 795, in _open_file_like return _open_file(name_or_buffer, mode) File "/workspace/envs/rlbench/lib/python3.10/site-packages/torch/serialization.py", line 776, in __init__ super().__init__(open(name, mode)) # noqa: SIM115 FileNotFoundError: [Errno 2] No such file or directory: '/path/to/your/pre-trained ckpts/PERACT_BC/checkpoint_peract_bc_leader_layer_0.pt' Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.