/usr/local/lib/python3.10/dist-packages/torch/distributed/launch.py:181: FutureWarning: The module torch.distributed.launch is deprecated and will be removed in future. Use torchrun. Note that --use-env is set by default in torchrun. If your script expects `--local-rank` argument to be set, please change it to read from `os.environ['LOCAL_RANK']` instead. See https://pytorch.org/docs/stable/distributed.html#launch-utility for further instructions warnings.warn( [2025-02-19 11:03:16,865] torch.distributed.run: [WARNING] [2025-02-19 11:03:16,865] torch.distributed.run: [WARNING] ***************************************** [2025-02-19 11:03:16,865] torch.distributed.run: [WARNING] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. [2025-02-19 11:03:16,865] torch.distributed.run: [WARNING] ***************************************** [2025-02-19 11:03:23,750] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2025-02-19 11:03:23,757] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2025-02-19 11:03:23,758] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2025-02-19 11:03:23,762] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2025-02-19 11:03:23,764] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2025-02-19 11:03:23,765] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2025-02-19 11:03:23,766] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2025-02-19 11:03:23,767] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect) df: /home/users/xuewu.lin/.triton/autotune: No such file or directory  [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.  [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.  [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.  [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.  [WARNING]  async_io: please install the libaio-dev package with apt  [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.  [WARNING]  async_io: please install the libaio-dev package with apt [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH  [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.  [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH  [WARNING]  async_io: please install the libaio-dev package with apt  [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.  [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH  [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.  [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.  [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.  [WARNING]  async_io: please install the libaio-dev package with apt  [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.  [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.  [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH  [WARNING]  async_io: please install the libaio-dev package with apt  [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.  [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH  [WARNING]  async_io: please install the libaio-dev package with apt  [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.  [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH  [WARNING]  async_io: please install the libaio-dev package with apt  [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.  [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH  [WARNING]  async_io: please install the libaio-dev package with apt  [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.  [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH  [WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.1  [WARNING]  using untested triton version (2.1.0), only 1.0.0 is known to be compatible  [WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.1  [WARNING]  using untested triton version (2.1.0), only 1.0.0 is known to be compatible  [WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.1  [WARNING]  using untested triton version (2.1.0), only 1.0.0 is known to be compatible  [WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.1  [WARNING]  using untested triton version (2.1.0), only 1.0.0 is known to be compatible  [WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.1  [WARNING]  using untested triton version (2.1.0), only 1.0.0 is known to be compatible  [WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.1  [WARNING]  using untested triton version (2.1.0), only 1.0.0 is known to be compatible  [WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.1  [WARNING]  using untested triton version (2.1.0), only 1.0.0 is known to be compatible  [WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.1  [WARNING]  using untested triton version (2.1.0), only 1.0.0 is known to be compatible /usr/local/lib/python3.10/dist-packages/mmengine/utils/dl_utils/setup_env.py:56: UserWarning: Setting MKL_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. warnings.warn( /usr/local/lib/python3.10/dist-packages/mmengine/utils/dl_utils/setup_env.py:56: UserWarning: Setting MKL_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. warnings.warn( /usr/local/lib/python3.10/dist-packages/mmengine/utils/dl_utils/setup_env.py:56: UserWarning: Setting MKL_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. warnings.warn( /usr/local/lib/python3.10/dist-packages/mmengine/utils/dl_utils/setup_env.py:56: UserWarning: Setting MKL_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. warnings.warn( /usr/local/lib/python3.10/dist-packages/mmengine/utils/dl_utils/setup_env.py:56: UserWarning: Setting MKL_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. warnings.warn( /usr/local/lib/python3.10/dist-packages/mmengine/utils/dl_utils/setup_env.py:56: UserWarning: Setting MKL_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. warnings.warn( /usr/local/lib/python3.10/dist-packages/mmengine/utils/dl_utils/setup_env.py:56: UserWarning: Setting MKL_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. warnings.warn( /usr/local/lib/python3.10/dist-packages/mmengine/utils/dl_utils/setup_env.py:56: UserWarning: Setting MKL_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. warnings.warn( 02/19 11:03:26 - mmengine - WARNING - Failed to search registry with scope "bip3d" in the "log_processor" registry tree. As a workaround, the current "log_processor" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "bip3d" is a correct scope, or whether the registry is initialized. 02/19 11:03:26 - mmengine - INFO - ------------------------------------------------------------ System environment: sys.platform: linux Python: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] CUDA available: True MUSA available: False numpy_random_seed: 0 GPU 0,1,2,3,4,5,6,7: NVIDIA GeForce RTX 4090 CUDA_HOME: /usr/local/cuda NVCC: Cuda compilation tools, release 11.8, V11.8.89 GCC: x86_64-linux-gnu-gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 PyTorch: 2.1.0+cu118 PyTorch compiling details: PyTorch built with: - GCC 9.3 - C++ Version: 201703 - Intel(R) oneAPI Math Kernel Library Version 2022.2-Product Build 20220804 for Intel(R) 64 architecture applications - Intel(R) MKL-DNN v3.1.1 (Git Hash 64f6bcbcbab628e96f33a62c3e975f8535a7bde4) - OpenMP 201511 (a.k.a. OpenMP 4.5) - LAPACK is enabled (usually provided by MKL) - NNPACK is enabled - CPU capability usage: AVX512 - CUDA Runtime 11.8 - NVCC architecture flags: -gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_90,code=sm_90 - CuDNN 8.9.4 - Built with CuDNN 8.7 - Magma 2.6.1 - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.8, CUDNN_VERSION=8.7.0, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 -fabi-version=11 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-invalid-partial-specialization -Wno-unused-private-field -Wno-aligned-allocation-unavailable -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_DISABLE_GPU_ASSERTS=ON, TORCH_VERSION=2.1.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, TorchVision: 0.16.0+cu118 OpenCV: 4.10.0 MMEngine: 0.10.4 Runtime environment: cudnn_benchmark: False mp_cfg: {'mp_start_method': 'fork', 'opencv_num_threads': 0} dist_cfg: {'backend': 'nccl'} seed: 0 Distributed launcher: pytorch Distributed training: True GPU number: 8 ------------------------------------------------------------ 02/19 11:03:28 - mmengine - INFO - Config: backend_args = None cam_standardization = True class_names = ( 'adhesive tape', 'air conditioner', 'alarm', 'album', 'arch', 'backpack', 'bag', 'balcony', 'ball', 'banister', 'bar', 'barricade', 'baseboard', 'basin', 'basket', 'bathtub', 'beam', 'beanbag', 'bed', 'bench', 'bicycle', 'bidet', 'bin', 'blackboard', 'blanket', 'blinds', 'board', 'body loofah', 'book', 'boots', 'bottle', 'bowl', 'box', 'bread', 'broom', 'brush', 'bucket', 'cabinet', 'calendar', 'camera', 'can', 'candle', 'candlestick', 'cap', 'car', 'carpet', 'cart', 'case', 'chair', 'chandelier', 'cleanser', 'clock', 'clothes', 'clothes dryer', 'coat hanger', 'coffee maker', 'coil', 'column', 'commode', 'computer', 'conducting wire', 'container', 'control', 'copier', 'cosmetics', 'couch', 'counter', 'countertop', 'crate', 'crib', 'cube', 'cup', 'curtain', 'cushion', 'decoration', 'desk', 'detergent', 'device', 'dish rack', 'dishwasher', 'dispenser', 'divider', 'door', 'door knob', 'doorframe', 'doorway', 'drawer', 'dress', 'dresser', 'drum', 'duct', 'dumbbell', 'dustpan', 'dvd', 'eraser', 'excercise equipment', 'fan', 'faucet', 'fence', 'file', 'fire extinguisher', 'fireplace', 'flowerpot', 'flush', 'folder', 'food', 'footstool', 'frame', 'fruit', 'furniture', 'garage door', 'garbage', 'glass', 'globe', 'glove', 'grab bar', 'grass', 'guitar', 'hair dryer', 'hamper', 'handle', 'hanger', 'hat', 'headboard', 'headphones', 'heater', 'helmets', 'holder', 'hook', 'humidifier', 'ironware', 'jacket', 'jalousie', 'jar', 'kettle', 'keyboard', 'kitchen island', 'kitchenware', 'knife', 'label', 'ladder', 'lamp', 'laptop', 'ledge', 'letter', 'light', 'luggage', 'machine', 'magazine', 'mailbox', 'map', 'mask', 'mat', 'mattress', 'menu', 'microwave', 'mirror', 'molding', 'monitor', 'mop', 'mouse', 'napkins', 'notebook', 'ottoman', 'oven', 'pack', 'package', 'pad', 'pan', 'panel', 'paper', 'paper cutter', 'partition', 'pedestal', 'pen', 'person', 'piano', 'picture', 'pillar', 'pillow', 'pipe', 'pitcher', 'plant', 'plate', 'player', 'plug', 'plunger', 'pool', 'pool table', 'poster', 'pot', 'price tag', 'printer', 'projector', 'purse', 'rack', 'radiator', 'radio', 'rail', 'range hood', 'refrigerator', 'remote control', 'ridge', 'rod', 'roll', 'roof', 'rope', 'sack', 'salt', 'scale', 'scissors', 'screen', 'seasoning', 'shampoo', 'sheet', 'shelf', 'shirt', 'shoe', 'shovel', 'shower', 'sign', 'sink', 'soap', 'soap dish', 'soap dispenser', 'socket', 'speaker', 'sponge', 'spoon', 'stairs', 'stall', 'stand', 'stapler', 'statue', 'steps', 'stick', 'stool', 'stopcock', 'stove', 'structure', 'sunglasses', 'support', 'switch', 'table', 'tablet', 'teapot', 'telephone', 'thermostat', 'tissue', 'tissue box', 'toaster', 'toilet', 'toilet paper', 'toiletry', 'tool', 'toothbrush', 'toothpaste', 'towel', 'toy', 'tray', 'treadmill', 'trophy', 'tube', 'tv', 'umbrella', 'urn', 'utensil', 'vacuum cleaner', 'vanity', 'vase', 'vent', 'ventilation', 'wardrobe', 'washbasin', 'washing machine', 'water cooler', 'water heater', 'window', 'window frame', 'windowsill', 'wine', 'wire', 'wood', 'wrap', ) common_labels = [ 189, 164, 101, 205, 273, 233, 131, 180, 86, 220, 67, 268, 224, 270, 53, 203, 237, 226, 10, 133, 248, 41, 55, 16, 199, 134, 99, 185, 2, 20, 234, 194, 253, 35, 174, 8, 223, 13, 91, 262, 230, 121, 49, 63, 119, 162, 79, 168, 245, 267, 122, 104, 100, 1, 176, 280, 140, 209, 259, 143, 165, 147, 117, 85, 105, 95, 109, 207, 68, 175, 106, 60, 4, 46, 171, 204, 111, 211, 108, 120, 157, 222, 17, 264, 151, 98, 38, 261, 123, 78, 118, 127, 240, 124, ] custom_hooks = [ dict(after_iter=False, type='EmptyCacheHook'), ] data_root = 'data' data_version = 'v1' dataset_type = 'EmbodiedScanDetGroundingDataset' default_hooks = dict( checkpoint=dict(interval=1, max_keep_ckpts=3, type='CheckpointHook'), logger=dict(interval=25, type='LoggerHook'), param_scheduler=dict(type='ParamSchedulerHook'), sampler_seed=dict(type='DistSamplerSeedHook'), timer=dict(type='IterTimerHook')) default_scope = 'bip3d' depth = True depth_loss = True env_cfg = dict( cudnn_benchmark=False, dist_cfg=dict(backend='nccl'), mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0)) find_unused_parameters = False head_labels = [ 48, 177, 82, 179, 37, 243, 28, 277, 32, 84, 215, 145, 182, 170, 22, 72, 30, 141, 65, 257, 221, 225, 52, 75, 231, 158, 236, 156, 47, 74, 6, 18, 71, 242, 217, 251, 66, 263, 5, 45, 14, 73, 278, 198, 24, 23, 196, 252, 19, 135, 26, 229, 183, 200, 107, 272, 246, 269, 125, 59, 279, 15, 163, 258, 57, 195, 51, 88, 97, 58, 102, 36, 137, 31, 80, 160, 155, 61, 238, 96, 190, 25, 219, 152, 142, 201, 274, 249, 178, 192, ] if_cluster = True image_wh = ( 512, 512, ) launcher = 'pytorch' load_from = 'http://svcspawner.tcloud.hobot.cc/user/homespace/xuewu.lin/plat_gpu/bip3d_det_rgb_withdepth-20250217-155444.364390/output/work_dirs/epoch_24.pth' log_level = 'INFO' log_processor = dict(by_epoch=True, type='LogProcessor', window_size=50) lr = 0.0002 max_depth = 10 max_epochs = 2 metainfo = dict( box_type_3d='euler-depth', classes=( 'adhesive tape', 'air conditioner', 'alarm', 'album', 'arch', 'backpack', 'bag', 'balcony', 'ball', 'banister', 'bar', 'barricade', 'baseboard', 'basin', 'basket', 'bathtub', 'beam', 'beanbag', 'bed', 'bench', 'bicycle', 'bidet', 'bin', 'blackboard', 'blanket', 'blinds', 'board', 'body loofah', 'book', 'boots', 'bottle', 'bowl', 'box', 'bread', 'broom', 'brush', 'bucket', 'cabinet', 'calendar', 'camera', 'can', 'candle', 'candlestick', 'cap', 'car', 'carpet', 'cart', 'case', 'chair', 'chandelier', 'cleanser', 'clock', 'clothes', 'clothes dryer', 'coat hanger', 'coffee maker', 'coil', 'column', 'commode', 'computer', 'conducting wire', 'container', 'control', 'copier', 'cosmetics', 'couch', 'counter', 'countertop', 'crate', 'crib', 'cube', 'cup', 'curtain', 'cushion', 'decoration', 'desk', 'detergent', 'device', 'dish rack', 'dishwasher', 'dispenser', 'divider', 'door', 'door knob', 'doorframe', 'doorway', 'drawer', 'dress', 'dresser', 'drum', 'duct', 'dumbbell', 'dustpan', 'dvd', 'eraser', 'excercise equipment', 'fan', 'faucet', 'fence', 'file', 'fire extinguisher', 'fireplace', 'flowerpot', 'flush', 'folder', 'food', 'footstool', 'frame', 'fruit', 'furniture', 'garage door', 'garbage', 'glass', 'globe', 'glove', 'grab bar', 'grass', 'guitar', 'hair dryer', 'hamper', 'handle', 'hanger', 'hat', 'headboard', 'headphones', 'heater', 'helmets', 'holder', 'hook', 'humidifier', 'ironware', 'jacket', 'jalousie', 'jar', 'kettle', 'keyboard', 'kitchen island', 'kitchenware', 'knife', 'label', 'ladder', 'lamp', 'laptop', 'ledge', 'letter', 'light', 'luggage', 'machine', 'magazine', 'mailbox', 'map', 'mask', 'mat', 'mattress', 'menu', 'microwave', 'mirror', 'molding', 'monitor', 'mop', 'mouse', 'napkins', 'notebook', 'ottoman', 'oven', 'pack', 'package', 'pad', 'pan', 'panel', 'paper', 'paper cutter', 'partition', 'pedestal', 'pen', 'person', 'piano', 'picture', 'pillar', 'pillow', 'pipe', 'pitcher', 'plant', 'plate', 'player', 'plug', 'plunger', 'pool', 'pool table', 'poster', 'pot', 'price tag', 'printer', 'projector', 'purse', 'rack', 'radiator', 'radio', 'rail', 'range hood', 'refrigerator', 'remote control', 'ridge', 'rod', 'roll', 'roof', 'rope', 'sack', 'salt', 'scale', 'scissors', 'screen', 'seasoning', 'shampoo', 'sheet', 'shelf', 'shirt', 'shoe', 'shovel', 'shower', 'sign', 'sink', 'soap', 'soap dish', 'soap dispenser', 'socket', 'speaker', 'sponge', 'spoon', 'stairs', 'stall', 'stand', 'stapler', 'statue', 'steps', 'stick', 'stool', 'stopcock', 'stove', 'structure', 'sunglasses', 'support', 'switch', 'table', 'tablet', 'teapot', 'telephone', 'thermostat', 'tissue', 'tissue box', 'toaster', 'toilet', 'toilet paper', 'toiletry', 'tool', 'toothbrush', 'toothpaste', 'towel', 'toy', 'tray', 'treadmill', 'trophy', 'tube', 'tv', 'umbrella', 'urn', 'utensil', 'vacuum cleaner', 'vanity', 'vase', 'vent', 'ventilation', 'wardrobe', 'washbasin', 'washing machine', 'water cooler', 'water heater', 'window', 'window frame', 'windowsill', 'wine', 'wire', 'wood', 'wrap', ), classes_split=( [ 48, 177, 82, 179, 37, 243, 28, 277, 32, 84, 215, 145, 182, 170, 22, 72, 30, 141, 65, 257, 221, 225, 52, 75, 231, 158, 236, 156, 47, 74, 6, 18, 71, 242, 217, 251, 66, 263, 5, 45, 14, 73, 278, 198, 24, 23, 196, 252, 19, 135, 26, 229, 183, 200, 107, 272, 246, 269, 125, 59, 279, 15, 163, 258, 57, 195, 51, 88, 97, 58, 102, 36, 137, 31, 80, 160, 155, 61, 238, 96, 190, 25, 219, 152, 142, 201, 274, 249, 178, 192, ], [ 189, 164, 101, 205, 273, 233, 131, 180, 86, 220, 67, 268, 224, 270, 53, 203, 237, 226, 10, 133, 248, 41, 55, 16, 199, 134, 99, 185, 2, 20, 234, 194, 253, 35, 174, 8, 223, 13, 91, 262, 230, 121, 49, 63, 119, 162, 79, 168, 245, 267, 122, 104, 100, 1, 176, 280, 140, 209, 259, 143, 165, 147, 117, 85, 105, 95, 109, 207, 68, 175, 106, 60, 4, 46, 171, 204, 111, 211, 108, 120, 157, 222, 17, 264, 151, 98, 38, 261, 123, 78, 118, 127, 240, 124, ], [ 76, 149, 173, 250, 275, 255, 34, 77, 266, 283, 112, 115, 186, 136, 256, 40, 254, 172, 9, 212, 213, 181, 154, 94, 191, 193, 3, 130, 146, 70, 128, 167, 126, 81, 7, 11, 148, 228, 239, 247, 21, 42, 89, 153, 161, 244, 110, 0, 29, 114, 132, 159, 218, 232, 260, 56, 92, 116, 282, 33, 113, 138, 12, 188, 44, 150, 197, 271, 169, 206, 90, 235, 103, 281, 184, 208, 216, 202, 214, 241, 129, 210, 276, 64, 27, 87, 139, 227, 187, 62, 43, 50, 69, 93, 144, 166, 265, 54, 83, 39, ], )) min_depth = 0.25 model = dict( backbone=dict( attn_drop_rate=0.0, convert_weights=False, depths=[ 2, 2, 6, 2, ], drop_path_rate=0.2, drop_rate=0.0, embed_dims=96, mlp_ratio=4, num_heads=[ 3, 6, 12, 24, ], out_indices=( 1, 2, 3, ), patch_norm=True, qk_scale=None, qkv_bias=True, type='mmdet.SwinTransformer', window_size=7, with_cp=True), backbone_3d=None, data_preprocessor=dict( bgr_to_rgb=True, mean=[ 123.675, 116.28, 103.53, ], pad_size_divisor=32, std=[ 58.395, 57.12, 57.375, ], type='CustomDet3DDataPreprocessor'), decoder=dict( anchor_encoder=dict(embed_dims=256, rot_dims=3, type='DoF9BoxEncoder'), deformable_model=dict( embed_dims=256, filter_outlier=True, kps_generator=dict( fix_scale=[ [ 0, 0, 0, ], [ 0.45, 0, 0, ], [ -0.45, 0, 0, ], [ 0, 0.45, 0, ], [ 0, -0.45, 0, ], [ 0, 0, 0.45, ], [ 0, 0, -0.45, ], ], num_learnable_pts=9, type='SparseBox3DKeyPointsGenerator'), max_depth=10, min_depth=0.25, num_groups=8, num_levels=4, type='DeformableFeatureAggregation', use_camera_embed=True, use_deformable_func=True, with_depth=True, with_value_proj=True), ffn=dict( embed_dims=256, feedforward_channels=2048, ffn_drop=0.0, type='FFN'), graph_model=dict( batch_first=True, dropout=0.0, embed_dims=256, num_heads=8, type='MultiheadAttention'), gt_cls_key='tokens_positive', gt_reg_key='gt_bboxes_3d', instance_bank=dict( anchor='anchor_files/embodiedscan_kmeans_det_cam_log_z-0.2-3.npy', anchor_in_camera=True, embed_dims=256, num_anchor=50, type='InstanceBank'), look_forward_twice=True, loss_cls=dict( alpha=0.25, gamma=2.0, loss_weight=1.0, type='mmdet.FocalLoss', use_sigmoid=True), loss_reg=dict( loss_weight_cd=0.8, loss_weight_pcd=0.0, loss_weight_wd=1.0, type='DoF9BoxLoss'), norm_layer=dict(normalized_shape=256, type='LN'), post_processor=dict(num_output=1000, type='GroundingBox3DPostProcess'), refine_layer=dict( cls_bias=True, embed_dims=256, output_dim=9, type='GroundingRefineClsHead'), sampler=dict( box_weight=1.0, cls_weight=1.0, cost_weight_cd=0.8, cost_weight_pcd=0.0, cost_weight_wd=1.0, embed_dims=256, num_classes=284, num_dn=100, type='Grounding3DTarget', with_dn_query=True), text_cross_attn=dict( batch_first=True, dropout=0.0, embed_dims=256, num_heads=8, type='MultiheadAttention'), type='BBox3DDecoder', with_instance_id=False), feature_enhancer=dict( embed_dims=256, img_attn_block=dict( ffn_cfg=dict( embed_dims=256, feedforward_channels=2048, ffn_drop=0.0), self_attn_cfg=dict( dropout=0.0, embed_dims=256, im2col_step=64, num_levels=4)), num_feature_levels=4, num_layers=6, positional_encoding=dict( normalize=True, num_feats=128, offset=0.0, temperature=20), text_attn_block=dict( ffn_cfg=dict( embed_dims=256, feedforward_channels=1024, ffn_drop=0.0), self_attn_cfg=dict(dropout=0.0, embed_dims=256, num_heads=4)), text_img_attn_block=dict( embed_dim=1024, init_values=0.0001, l_dim=256, num_heads=4, v_dim=256), type='TextImageDeformable2DEnhancer'), input_3d='depth_img', neck=dict( act_cfg=None, bias=True, in_channels=[ 192, 384, 768, ], kernel_size=1, norm_cfg=dict(num_groups=32, type='GN'), num_outs=4, out_channels=256, type='mmdet.ChannelMapper'), neck_3d=None, spatial_enhancer=dict( embed_dims=256, feature_3d_dim=32, loss_depth_weight=1.0, max_depth=10, min_depth=0.25, num_depth=64, num_depth_layers=2, type='DepthFusionSpatialEnhancer', with_feature_3d=False), text_encoder=dict( add_pooling_layer=False, max_tokens=768, name='./ckpt/bert-base-uncased', pad_to_max=False, return_tokenized=True, special_tokens_list=[ '[CLS]', '[SEP]', ], type='BertModel', use_checkpoint=True, use_sub_sentence_represent=True), type='BIP3D', use_depth_grid_mask=True) num_depth = 64 optim_wrapper = dict( clip_grad=dict(max_norm=10, norm_type=2), optimizer=dict(lr=0.0002, type='AdamW', weight_decay=0.0005), paramwise_cfg=dict( custom_keys=dict({ 'absolute_pos_embed': dict(decay_mult=0.0), 'backbone.': dict(lr_mult=0.1) })), type='OptimWrapper') param_scheduler = [ dict( begin=0, by_epoch=False, end=500, start_factor=0.001, type='LinearLR'), dict( begin=0, by_epoch=False, end=62000, gamma=0.1, milestones=[ 40000, 55000, ], type='MultiStepLR'), ] randomness = dict(seed=0) resize = dict( dst_intrinsic=[ [ 432.57943431339237, 0.0, 256, ], [ 0.0, 539.8570854208559, 256, ], [ 0.0, 0.0, 1.0, ], ], dst_wh=( 512, 512, ), type='CamIntrisicStandardization') resume = False rotate_3rscan = True sep_token = '[SEP]' tail_labels = [ 76, 149, 173, 250, 275, 255, 34, 77, 266, 283, 112, 115, 186, 136, 256, 40, 254, 172, 9, 212, 213, 181, 154, 94, 191, 193, 3, 130, 146, 70, 128, 167, 126, 81, 7, 11, 148, 228, 239, 247, 21, 42, 89, 153, 161, 244, 110, 0, 29, 114, 132, 159, 218, 232, 260, 56, 92, 116, 282, 33, 113, 138, 12, 188, 44, 150, 197, 271, 169, 206, 90, 235, 103, 281, 184, 208, 216, 202, 214, 241, 129, 210, 276, 64, 27, 87, 139, 227, 187, 62, 43, 50, 69, 93, 144, 166, 265, 54, 83, 39, ] test_cfg = dict(type='TestLoop') test_dataloader = dict( batch_size=1, dataset=dict( ann_file='embodiedscan/embodiedscan_infos_val.pkl', box_type_3d='Euler-Depth', data_root='data', filter_empty_gt=True, metainfo=dict( box_type_3d='euler-depth', classes=( 'adhesive tape', 'air conditioner', 'alarm', 'album', 'arch', 'backpack', 'bag', 'balcony', 'ball', 'banister', 'bar', 'barricade', 'baseboard', 'basin', 'basket', 'bathtub', 'beam', 'beanbag', 'bed', 'bench', 'bicycle', 'bidet', 'bin', 'blackboard', 'blanket', 'blinds', 'board', 'body loofah', 'book', 'boots', 'bottle', 'bowl', 'box', 'bread', 'broom', 'brush', 'bucket', 'cabinet', 'calendar', 'camera', 'can', 'candle', 'candlestick', 'cap', 'car', 'carpet', 'cart', 'case', 'chair', 'chandelier', 'cleanser', 'clock', 'clothes', 'clothes dryer', 'coat hanger', 'coffee maker', 'coil', 'column', 'commode', 'computer', 'conducting wire', 'container', 'control', 'copier', 'cosmetics', 'couch', 'counter', 'countertop', 'crate', 'crib', 'cube', 'cup', 'curtain', 'cushion', 'decoration', 'desk', 'detergent', 'device', 'dish rack', 'dishwasher', 'dispenser', 'divider', 'door', 'door knob', 'doorframe', 'doorway', 'drawer', 'dress', 'dresser', 'drum', 'duct', 'dumbbell', 'dustpan', 'dvd', 'eraser', 'excercise equipment', 'fan', 'faucet', 'fence', 'file', 'fire extinguisher', 'fireplace', 'flowerpot', 'flush', 'folder', 'food', 'footstool', 'frame', 'fruit', 'furniture', 'garage door', 'garbage', 'glass', 'globe', 'glove', 'grab bar', 'grass', 'guitar', 'hair dryer', 'hamper', 'handle', 'hanger', 'hat', 'headboard', 'headphones', 'heater', 'helmets', 'holder', 'hook', 'humidifier', 'ironware', 'jacket', 'jalousie', 'jar', 'kettle', 'keyboard', 'kitchen island', 'kitchenware', 'knife', 'label', 'ladder', 'lamp', 'laptop', 'ledge', 'letter', 'light', 'luggage', 'machine', 'magazine', 'mailbox', 'map', 'mask', 'mat', 'mattress', 'menu', 'microwave', 'mirror', 'molding', 'monitor', 'mop', 'mouse', 'napkins', 'notebook', 'ottoman', 'oven', 'pack', 'package', 'pad', 'pan', 'panel', 'paper', 'paper cutter', 'partition', 'pedestal', 'pen', 'person', 'piano', 'picture', 'pillar', 'pillow', 'pipe', 'pitcher', 'plant', 'plate', 'player', 'plug', 'plunger', 'pool', 'pool table', 'poster', 'pot', 'price tag', 'printer', 'projector', 'purse', 'rack', 'radiator', 'radio', 'rail', 'range hood', 'refrigerator', 'remote control', 'ridge', 'rod', 'roll', 'roof', 'rope', 'sack', 'salt', 'scale', 'scissors', 'screen', 'seasoning', 'shampoo', 'sheet', 'shelf', 'shirt', 'shoe', 'shovel', 'shower', 'sign', 'sink', 'soap', 'soap dish', 'soap dispenser', 'socket', 'speaker', 'sponge', 'spoon', 'stairs', 'stall', 'stand', 'stapler', 'statue', 'steps', 'stick', 'stool', 'stopcock', 'stove', 'structure', 'sunglasses', 'support', 'switch', 'table', 'tablet', 'teapot', 'telephone', 'thermostat', 'tissue', 'tissue box', 'toaster', 'toilet', 'toilet paper', 'toiletry', 'tool', 'toothbrush', 'toothpaste', 'towel', 'toy', 'tray', 'treadmill', 'trophy', 'tube', 'tv', 'umbrella', 'urn', 'utensil', 'vacuum cleaner', 'vanity', 'vase', 'vent', 'ventilation', 'wardrobe', 'washbasin', 'washing machine', 'water cooler', 'water heater', 'window', 'window frame', 'windowsill', 'wine', 'wire', 'wood', 'wrap', ), classes_split=( [ 48, 177, 82, 179, 37, 243, 28, 277, 32, 84, 215, 145, 182, 170, 22, 72, 30, 141, 65, 257, 221, 225, 52, 75, 231, 158, 236, 156, 47, 74, 6, 18, 71, 242, 217, 251, 66, 263, 5, 45, 14, 73, 278, 198, 24, 23, 196, 252, 19, 135, 26, 229, 183, 200, 107, 272, 246, 269, 125, 59, 279, 15, 163, 258, 57, 195, 51, 88, 97, 58, 102, 36, 137, 31, 80, 160, 155, 61, 238, 96, 190, 25, 219, 152, 142, 201, 274, 249, 178, 192, ], [ 189, 164, 101, 205, 273, 233, 131, 180, 86, 220, 67, 268, 224, 270, 53, 203, 237, 226, 10, 133, 248, 41, 55, 16, 199, 134, 99, 185, 2, 20, 234, 194, 253, 35, 174, 8, 223, 13, 91, 262, 230, 121, 49, 63, 119, 162, 79, 168, 245, 267, 122, 104, 100, 1, 176, 280, 140, 209, 259, 143, 165, 147, 117, 85, 105, 95, 109, 207, 68, 175, 106, 60, 4, 46, 171, 204, 111, 211, 108, 120, 157, 222, 17, 264, 151, 98, 38, 261, 123, 78, 118, 127, 240, 124, ], [ 76, 149, 173, 250, 275, 255, 34, 77, 266, 283, 112, 115, 186, 136, 256, 40, 254, 172, 9, 212, 213, 181, 154, 94, 191, 193, 3, 130, 146, 70, 128, 167, 126, 81, 7, 11, 148, 228, 239, 247, 21, 42, 89, 153, 161, 244, 110, 0, 29, 114, 132, 159, 218, 232, 260, 56, 92, 116, 282, 33, 113, 138, 12, 188, 44, 150, 197, 271, 169, 206, 90, 235, 103, 281, 184, 208, 216, 202, 214, 241, 129, 210, 276, 64, 27, 87, 139, 227, 187, 62, 43, 50, 69, 93, 144, 166, 265, 54, 83, 39, ], )), mode='grounding', pipeline=[ dict(type='LoadAnnotations3D'), dict( max_n_images=50, n_images=1, ordered=True, rotate_3rscan=True, transforms=[ dict(backend_args=None, type='LoadImageFromFile'), dict(backend_args=None, type='LoadDepthFromFile'), dict( dst_intrinsic=[ [ 432.57943431339237, 0.0, 256, ], [ 0.0, 539.8570854208559, 256, ], [ 0.0, 0.0, 1.0, ], ], dst_wh=( 512, 512, ), type='CamIntrisicStandardization'), ], type='MultiViewPipeline'), dict( keys=[ 'img', 'depth_img', 'gt_bboxes_3d', 'gt_labels_3d', ], type='Pack3DDetInputs'), ], test_mode=True, type='EmbodiedScanDetGroundingDataset', vg_file='embodiedscan/embodiedscan_val_vg_all.json'), drop_last=False, num_workers=4, persistent_workers=False, sampler=dict(shuffle=False, type='DefaultSampler')) test_evaluator = dict(type='GroundingMetric') test_pipeline = [ dict(type='LoadAnnotations3D'), dict( max_n_images=50, n_images=1, ordered=True, rotate_3rscan=True, transforms=[ dict(backend_args=None, type='LoadImageFromFile'), dict(backend_args=None, type='LoadDepthFromFile'), dict( dst_intrinsic=[ [ 432.57943431339237, 0.0, 256, ], [ 0.0, 539.8570854208559, 256, ], [ 0.0, 0.0, 1.0, ], ], dst_wh=( 512, 512, ), type='CamIntrisicStandardization'), ], type='MultiViewPipeline'), dict( keys=[ 'img', 'depth_img', 'gt_bboxes_3d', 'gt_labels_3d', ], type='Pack3DDetInputs'), ] train_ann_file = 'embodiedscan/embodiedscan_infos_train.pkl' train_cfg = dict(max_epochs=2, type='EpochBasedTrainLoop', val_interval=1) train_dataloader = dict( batch_size=1, dataset=dict( ann_file='embodiedscan/embodiedscan_infos_train.pkl', box_type_3d='Euler-Depth', data_root='data', filter_empty_gt=True, metainfo=dict( box_type_3d='euler-depth', classes=( 'adhesive tape', 'air conditioner', 'alarm', 'album', 'arch', 'backpack', 'bag', 'balcony', 'ball', 'banister', 'bar', 'barricade', 'baseboard', 'basin', 'basket', 'bathtub', 'beam', 'beanbag', 'bed', 'bench', 'bicycle', 'bidet', 'bin', 'blackboard', 'blanket', 'blinds', 'board', 'body loofah', 'book', 'boots', 'bottle', 'bowl', 'box', 'bread', 'broom', 'brush', 'bucket', 'cabinet', 'calendar', 'camera', 'can', 'candle', 'candlestick', 'cap', 'car', 'carpet', 'cart', 'case', 'chair', 'chandelier', 'cleanser', 'clock', 'clothes', 'clothes dryer', 'coat hanger', 'coffee maker', 'coil', 'column', 'commode', 'computer', 'conducting wire', 'container', 'control', 'copier', 'cosmetics', 'couch', 'counter', 'countertop', 'crate', 'crib', 'cube', 'cup', 'curtain', 'cushion', 'decoration', 'desk', 'detergent', 'device', 'dish rack', 'dishwasher', 'dispenser', 'divider', 'door', 'door knob', 'doorframe', 'doorway', 'drawer', 'dress', 'dresser', 'drum', 'duct', 'dumbbell', 'dustpan', 'dvd', 'eraser', 'excercise equipment', 'fan', 'faucet', 'fence', 'file', 'fire extinguisher', 'fireplace', 'flowerpot', 'flush', 'folder', 'food', 'footstool', 'frame', 'fruit', 'furniture', 'garage door', 'garbage', 'glass', 'globe', 'glove', 'grab bar', 'grass', 'guitar', 'hair dryer', 'hamper', 'handle', 'hanger', 'hat', 'headboard', 'headphones', 'heater', 'helmets', 'holder', 'hook', 'humidifier', 'ironware', 'jacket', 'jalousie', 'jar', 'kettle', 'keyboard', 'kitchen island', 'kitchenware', 'knife', 'label', 'ladder', 'lamp', 'laptop', 'ledge', 'letter', 'light', 'luggage', 'machine', 'magazine', 'mailbox', 'map', 'mask', 'mat', 'mattress', 'menu', 'microwave', 'mirror', 'molding', 'monitor', 'mop', 'mouse', 'napkins', 'notebook', 'ottoman', 'oven', 'pack', 'package', 'pad', 'pan', 'panel', 'paper', 'paper cutter', 'partition', 'pedestal', 'pen', 'person', 'piano', 'picture', 'pillar', 'pillow', 'pipe', 'pitcher', 'plant', 'plate', 'player', 'plug', 'plunger', 'pool', 'pool table', 'poster', 'pot', 'price tag', 'printer', 'projector', 'purse', 'rack', 'radiator', 'radio', 'rail', 'range hood', 'refrigerator', 'remote control', 'ridge', 'rod', 'roll', 'roof', 'rope', 'sack', 'salt', 'scale', 'scissors', 'screen', 'seasoning', 'shampoo', 'sheet', 'shelf', 'shirt', 'shoe', 'shovel', 'shower', 'sign', 'sink', 'soap', 'soap dish', 'soap dispenser', 'socket', 'speaker', 'sponge', 'spoon', 'stairs', 'stall', 'stand', 'stapler', 'statue', 'steps', 'stick', 'stool', 'stopcock', 'stove', 'structure', 'sunglasses', 'support', 'switch', 'table', 'tablet', 'teapot', 'telephone', 'thermostat', 'tissue', 'tissue box', 'toaster', 'toilet', 'toilet paper', 'toiletry', 'tool', 'toothbrush', 'toothpaste', 'towel', 'toy', 'tray', 'treadmill', 'trophy', 'tube', 'tv', 'umbrella', 'urn', 'utensil', 'vacuum cleaner', 'vanity', 'vase', 'vent', 'ventilation', 'wardrobe', 'washbasin', 'washing machine', 'water cooler', 'water heater', 'window', 'window frame', 'windowsill', 'wine', 'wire', 'wood', 'wrap', ), classes_split=( [ 48, 177, 82, 179, 37, 243, 28, 277, 32, 84, 215, 145, 182, 170, 22, 72, 30, 141, 65, 257, 221, 225, 52, 75, 231, 158, 236, 156, 47, 74, 6, 18, 71, 242, 217, 251, 66, 263, 5, 45, 14, 73, 278, 198, 24, 23, 196, 252, 19, 135, 26, 229, 183, 200, 107, 272, 246, 269, 125, 59, 279, 15, 163, 258, 57, 195, 51, 88, 97, 58, 102, 36, 137, 31, 80, 160, 155, 61, 238, 96, 190, 25, 219, 152, 142, 201, 274, 249, 178, 192, ], [ 189, 164, 101, 205, 273, 233, 131, 180, 86, 220, 67, 268, 224, 270, 53, 203, 237, 226, 10, 133, 248, 41, 55, 16, 199, 134, 99, 185, 2, 20, 234, 194, 253, 35, 174, 8, 223, 13, 91, 262, 230, 121, 49, 63, 119, 162, 79, 168, 245, 267, 122, 104, 100, 1, 176, 280, 140, 209, 259, 143, 165, 147, 117, 85, 105, 95, 109, 207, 68, 175, 106, 60, 4, 46, 171, 204, 111, 211, 108, 120, 157, 222, 17, 264, 151, 98, 38, 261, 123, 78, 118, 127, 240, 124, ], [ 76, 149, 173, 250, 275, 255, 34, 77, 266, 283, 112, 115, 186, 136, 256, 40, 254, 172, 9, 212, 213, 181, 154, 94, 191, 193, 3, 130, 146, 70, 128, 167, 126, 81, 7, 11, 148, 228, 239, 247, 21, 42, 89, 153, 161, 244, 110, 0, 29, 114, 132, 159, 218, 232, 260, 56, 92, 116, 282, 33, 113, 138, 12, 188, 44, 150, 197, 271, 169, 206, 90, 235, 103, 281, 184, 208, 216, 202, 214, 241, 129, 210, 276, 64, 27, 87, 139, 227, 187, 62, 43, 50, 69, 93, 144, 166, 265, 54, 83, 39, ], )), mode='grounding', num_text=10, pipeline=[ dict(type='LoadAnnotations3D'), dict( max_n_images=18, n_images=1, ordered=True, rotate_3rscan=True, transforms=[ dict(backend_args=None, type='LoadImageFromFile'), dict(backend_args=None, type='LoadDepthFromFile'), dict( dst_intrinsic=[ [ 432.57943431339237, 0.0, 256, ], [ 0.0, 539.8570854208559, 256, ], [ 0.0, 0.0, 1.0, ], ], dst_wh=( 512, 512, ), type='CamIntrisicStandardization'), dict( data_aug_conf=dict( H=512, W=512, bot_pct_lim=( 0.0, 0.05, ), final_dim=( 512, 512, ), rand_flip=False, resize_lim=( 1.0, 1.0, ), rot_lim=( 0, 0, )), type='ResizeCropFlipImage'), ], type='MultiViewPipeline'), dict( keys=[ 'img', 'depth_img', 'gt_bboxes_3d', 'gt_labels_3d', ], type='Pack3DDetInputs'), dict( max_depth=10, min_depth=0.25, num_depth=64, origin_stride=4, type='DepthProbLabelGenerator'), ], sep_token='[SEP]', test_mode=False, type='EmbodiedScanDetGroundingDataset', vg_file='embodiedscan/embodiedscan_train_vg_all.json'), num_workers=4, persistent_workers=False, sampler=dict(shuffle=True, type='DefaultSampler')) train_dataset = dict( ann_file='embodiedscan/embodiedscan_infos_train.pkl', box_type_3d='Euler-Depth', data_root='data', filter_empty_gt=True, metainfo=dict( box_type_3d='euler-depth', classes=( 'adhesive tape', 'air conditioner', 'alarm', 'album', 'arch', 'backpack', 'bag', 'balcony', 'ball', 'banister', 'bar', 'barricade', 'baseboard', 'basin', 'basket', 'bathtub', 'beam', 'beanbag', 'bed', 'bench', 'bicycle', 'bidet', 'bin', 'blackboard', 'blanket', 'blinds', 'board', 'body loofah', 'book', 'boots', 'bottle', 'bowl', 'box', 'bread', 'broom', 'brush', 'bucket', 'cabinet', 'calendar', 'camera', 'can', 'candle', 'candlestick', 'cap', 'car', 'carpet', 'cart', 'case', 'chair', 'chandelier', 'cleanser', 'clock', 'clothes', 'clothes dryer', 'coat hanger', 'coffee maker', 'coil', 'column', 'commode', 'computer', 'conducting wire', 'container', 'control', 'copier', 'cosmetics', 'couch', 'counter', 'countertop', 'crate', 'crib', 'cube', 'cup', 'curtain', 'cushion', 'decoration', 'desk', 'detergent', 'device', 'dish rack', 'dishwasher', 'dispenser', 'divider', 'door', 'door knob', 'doorframe', 'doorway', 'drawer', 'dress', 'dresser', 'drum', 'duct', 'dumbbell', 'dustpan', 'dvd', 'eraser', 'excercise equipment', 'fan', 'faucet', 'fence', 'file', 'fire extinguisher', 'fireplace', 'flowerpot', 'flush', 'folder', 'food', 'footstool', 'frame', 'fruit', 'furniture', 'garage door', 'garbage', 'glass', 'globe', 'glove', 'grab bar', 'grass', 'guitar', 'hair dryer', 'hamper', 'handle', 'hanger', 'hat', 'headboard', 'headphones', 'heater', 'helmets', 'holder', 'hook', 'humidifier', 'ironware', 'jacket', 'jalousie', 'jar', 'kettle', 'keyboard', 'kitchen island', 'kitchenware', 'knife', 'label', 'ladder', 'lamp', 'laptop', 'ledge', 'letter', 'light', 'luggage', 'machine', 'magazine', 'mailbox', 'map', 'mask', 'mat', 'mattress', 'menu', 'microwave', 'mirror', 'molding', 'monitor', 'mop', 'mouse', 'napkins', 'notebook', 'ottoman', 'oven', 'pack', 'package', 'pad', 'pan', 'panel', 'paper', 'paper cutter', 'partition', 'pedestal', 'pen', 'person', 'piano', 'picture', 'pillar', 'pillow', 'pipe', 'pitcher', 'plant', 'plate', 'player', 'plug', 'plunger', 'pool', 'pool table', 'poster', 'pot', 'price tag', 'printer', 'projector', 'purse', 'rack', 'radiator', 'radio', 'rail', 'range hood', 'refrigerator', 'remote control', 'ridge', 'rod', 'roll', 'roof', 'rope', 'sack', 'salt', 'scale', 'scissors', 'screen', 'seasoning', 'shampoo', 'sheet', 'shelf', 'shirt', 'shoe', 'shovel', 'shower', 'sign', 'sink', 'soap', 'soap dish', 'soap dispenser', 'socket', 'speaker', 'sponge', 'spoon', 'stairs', 'stall', 'stand', 'stapler', 'statue', 'steps', 'stick', 'stool', 'stopcock', 'stove', 'structure', 'sunglasses', 'support', 'switch', 'table', 'tablet', 'teapot', 'telephone', 'thermostat', 'tissue', 'tissue box', 'toaster', 'toilet', 'toilet paper', 'toiletry', 'tool', 'toothbrush', 'toothpaste', 'towel', 'toy', 'tray', 'treadmill', 'trophy', 'tube', 'tv', 'umbrella', 'urn', 'utensil', 'vacuum cleaner', 'vanity', 'vase', 'vent', 'ventilation', 'wardrobe', 'washbasin', 'washing machine', 'water cooler', 'water heater', 'window', 'window frame', 'windowsill', 'wine', 'wire', 'wood', 'wrap', ), classes_split=( [ 48, 177, 82, 179, 37, 243, 28, 277, 32, 84, 215, 145, 182, 170, 22, 72, 30, 141, 65, 257, 221, 225, 52, 75, 231, 158, 236, 156, 47, 74, 6, 18, 71, 242, 217, 251, 66, 263, 5, 45, 14, 73, 278, 198, 24, 23, 196, 252, 19, 135, 26, 229, 183, 200, 107, 272, 246, 269, 125, 59, 279, 15, 163, 258, 57, 195, 51, 88, 97, 58, 102, 36, 137, 31, 80, 160, 155, 61, 238, 96, 190, 25, 219, 152, 142, 201, 274, 249, 178, 192, ], [ 189, 164, 101, 205, 273, 233, 131, 180, 86, 220, 67, 268, 224, 270, 53, 203, 237, 226, 10, 133, 248, 41, 55, 16, 199, 134, 99, 185, 2, 20, 234, 194, 253, 35, 174, 8, 223, 13, 91, 262, 230, 121, 49, 63, 119, 162, 79, 168, 245, 267, 122, 104, 100, 1, 176, 280, 140, 209, 259, 143, 165, 147, 117, 85, 105, 95, 109, 207, 68, 175, 106, 60, 4, 46, 171, 204, 111, 211, 108, 120, 157, 222, 17, 264, 151, 98, 38, 261, 123, 78, 118, 127, 240, 124, ], [ 76, 149, 173, 250, 275, 255, 34, 77, 266, 283, 112, 115, 186, 136, 256, 40, 254, 172, 9, 212, 213, 181, 154, 94, 191, 193, 3, 130, 146, 70, 128, 167, 126, 81, 7, 11, 148, 228, 239, 247, 21, 42, 89, 153, 161, 244, 110, 0, 29, 114, 132, 159, 218, 232, 260, 56, 92, 116, 282, 33, 113, 138, 12, 188, 44, 150, 197, 271, 169, 206, 90, 235, 103, 281, 184, 208, 216, 202, 214, 241, 129, 210, 276, 64, 27, 87, 139, 227, 187, 62, 43, 50, 69, 93, 144, 166, 265, 54, 83, 39, ], )), mode='grounding', num_text=10, pipeline=[ dict(type='LoadAnnotations3D'), dict( max_n_images=18, n_images=1, ordered=True, rotate_3rscan=True, transforms=[ dict(backend_args=None, type='LoadImageFromFile'), dict(backend_args=None, type='LoadDepthFromFile'), dict( dst_intrinsic=[ [ 432.57943431339237, 0.0, 256, ], [ 0.0, 539.8570854208559, 256, ], [ 0.0, 0.0, 1.0, ], ], dst_wh=( 512, 512, ), type='CamIntrisicStandardization'), dict( data_aug_conf=dict( H=512, W=512, bot_pct_lim=( 0.0, 0.05, ), final_dim=( 512, 512, ), rand_flip=False, resize_lim=( 1.0, 1.0, ), rot_lim=( 0, 0, )), type='ResizeCropFlipImage'), ], type='MultiViewPipeline'), dict( keys=[ 'img', 'depth_img', 'gt_bboxes_3d', 'gt_labels_3d', ], type='Pack3DDetInputs'), dict( max_depth=10, min_depth=0.25, num_depth=64, origin_stride=4, type='DepthProbLabelGenerator'), ], sep_token='[SEP]', test_mode=False, type='EmbodiedScanDetGroundingDataset', vg_file='embodiedscan/embodiedscan_train_vg_all.json') train_pipeline = [ dict(type='LoadAnnotations3D'), dict( max_n_images=18, n_images=1, ordered=True, rotate_3rscan=True, transforms=[ dict(backend_args=None, type='LoadImageFromFile'), dict(backend_args=None, type='LoadDepthFromFile'), dict( dst_intrinsic=[ [ 432.57943431339237, 0.0, 256, ], [ 0.0, 539.8570854208559, 256, ], [ 0.0, 0.0, 1.0, ], ], dst_wh=( 512, 512, ), type='CamIntrisicStandardization'), dict( data_aug_conf=dict( H=512, W=512, bot_pct_lim=( 0.0, 0.05, ), final_dim=( 512, 512, ), rand_flip=False, resize_lim=( 1.0, 1.0, ), rot_lim=( 0, 0, )), type='ResizeCropFlipImage'), ], type='MultiViewPipeline'), dict( keys=[ 'img', 'depth_img', 'gt_bboxes_3d', 'gt_labels_3d', ], type='Pack3DDetInputs'), dict( max_depth=10, min_depth=0.25, num_depth=64, origin_stride=4, type='DepthProbLabelGenerator'), ] train_vg_file = 'embodiedscan/embodiedscan_train_vg_all.json' trainval = False val_ann_file = 'embodiedscan/embodiedscan_infos_val.pkl' val_cfg = dict(type='ValLoop') val_dataloader = dict( batch_size=1, dataset=dict( ann_file='embodiedscan/embodiedscan_infos_val.pkl', box_type_3d='Euler-Depth', data_root='data', filter_empty_gt=True, metainfo=dict( box_type_3d='euler-depth', classes=( 'adhesive tape', 'air conditioner', 'alarm', 'album', 'arch', 'backpack', 'bag', 'balcony', 'ball', 'banister', 'bar', 'barricade', 'baseboard', 'basin', 'basket', 'bathtub', 'beam', 'beanbag', 'bed', 'bench', 'bicycle', 'bidet', 'bin', 'blackboard', 'blanket', 'blinds', 'board', 'body loofah', 'book', 'boots', 'bottle', 'bowl', 'box', 'bread', 'broom', 'brush', 'bucket', 'cabinet', 'calendar', 'camera', 'can', 'candle', 'candlestick', 'cap', 'car', 'carpet', 'cart', 'case', 'chair', 'chandelier', 'cleanser', 'clock', 'clothes', 'clothes dryer', 'coat hanger', 'coffee maker', 'coil', 'column', 'commode', 'computer', 'conducting wire', 'container', 'control', 'copier', 'cosmetics', 'couch', 'counter', 'countertop', 'crate', 'crib', 'cube', 'cup', 'curtain', 'cushion', 'decoration', 'desk', 'detergent', 'device', 'dish rack', 'dishwasher', 'dispenser', 'divider', 'door', 'door knob', 'doorframe', 'doorway', 'drawer', 'dress', 'dresser', 'drum', 'duct', 'dumbbell', 'dustpan', 'dvd', 'eraser', 'excercise equipment', 'fan', 'faucet', 'fence', 'file', 'fire extinguisher', 'fireplace', 'flowerpot', 'flush', 'folder', 'food', 'footstool', 'frame', 'fruit', 'furniture', 'garage door', 'garbage', 'glass', 'globe', 'glove', 'grab bar', 'grass', 'guitar', 'hair dryer', 'hamper', 'handle', 'hanger', 'hat', 'headboard', 'headphones', 'heater', 'helmets', 'holder', 'hook', 'humidifier', 'ironware', 'jacket', 'jalousie', 'jar', 'kettle', 'keyboard', 'kitchen island', 'kitchenware', 'knife', 'label', 'ladder', 'lamp', 'laptop', 'ledge', 'letter', 'light', 'luggage', 'machine', 'magazine', 'mailbox', 'map', 'mask', 'mat', 'mattress', 'menu', 'microwave', 'mirror', 'molding', 'monitor', 'mop', 'mouse', 'napkins', 'notebook', 'ottoman', 'oven', 'pack', 'package', 'pad', 'pan', 'panel', 'paper', 'paper cutter', 'partition', 'pedestal', 'pen', 'person', 'piano', 'picture', 'pillar', 'pillow', 'pipe', 'pitcher', 'plant', 'plate', 'player', 'plug', 'plunger', 'pool', 'pool table', 'poster', 'pot', 'price tag', 'printer', 'projector', 'purse', 'rack', 'radiator', 'radio', 'rail', 'range hood', 'refrigerator', 'remote control', 'ridge', 'rod', 'roll', 'roof', 'rope', 'sack', 'salt', 'scale', 'scissors', 'screen', 'seasoning', 'shampoo', 'sheet', 'shelf', 'shirt', 'shoe', 'shovel', 'shower', 'sign', 'sink', 'soap', 'soap dish', 'soap dispenser', 'socket', 'speaker', 'sponge', 'spoon', 'stairs', 'stall', 'stand', 'stapler', 'statue', 'steps', 'stick', 'stool', 'stopcock', 'stove', 'structure', 'sunglasses', 'support', 'switch', 'table', 'tablet', 'teapot', 'telephone', 'thermostat', 'tissue', 'tissue box', 'toaster', 'toilet', 'toilet paper', 'toiletry', 'tool', 'toothbrush', 'toothpaste', 'towel', 'toy', 'tray', 'treadmill', 'trophy', 'tube', 'tv', 'umbrella', 'urn', 'utensil', 'vacuum cleaner', 'vanity', 'vase', 'vent', 'ventilation', 'wardrobe', 'washbasin', 'washing machine', 'water cooler', 'water heater', 'window', 'window frame', 'windowsill', 'wine', 'wire', 'wood', 'wrap', ), classes_split=( [ 48, 177, 82, 179, 37, 243, 28, 277, 32, 84, 215, 145, 182, 170, 22, 72, 30, 141, 65, 257, 221, 225, 52, 75, 231, 158, 236, 156, 47, 74, 6, 18, 71, 242, 217, 251, 66, 263, 5, 45, 14, 73, 278, 198, 24, 23, 196, 252, 19, 135, 26, 229, 183, 200, 107, 272, 246, 269, 125, 59, 279, 15, 163, 258, 57, 195, 51, 88, 97, 58, 102, 36, 137, 31, 80, 160, 155, 61, 238, 96, 190, 25, 219, 152, 142, 201, 274, 249, 178, 192, ], [ 189, 164, 101, 205, 273, 233, 131, 180, 86, 220, 67, 268, 224, 270, 53, 203, 237, 226, 10, 133, 248, 41, 55, 16, 199, 134, 99, 185, 2, 20, 234, 194, 253, 35, 174, 8, 223, 13, 91, 262, 230, 121, 49, 63, 119, 162, 79, 168, 245, 267, 122, 104, 100, 1, 176, 280, 140, 209, 259, 143, 165, 147, 117, 85, 105, 95, 109, 207, 68, 175, 106, 60, 4, 46, 171, 204, 111, 211, 108, 120, 157, 222, 17, 264, 151, 98, 38, 261, 123, 78, 118, 127, 240, 124, ], [ 76, 149, 173, 250, 275, 255, 34, 77, 266, 283, 112, 115, 186, 136, 256, 40, 254, 172, 9, 212, 213, 181, 154, 94, 191, 193, 3, 130, 146, 70, 128, 167, 126, 81, 7, 11, 148, 228, 239, 247, 21, 42, 89, 153, 161, 244, 110, 0, 29, 114, 132, 159, 218, 232, 260, 56, 92, 116, 282, 33, 113, 138, 12, 188, 44, 150, 197, 271, 169, 206, 90, 235, 103, 281, 184, 208, 216, 202, 214, 241, 129, 210, 276, 64, 27, 87, 139, 227, 187, 62, 43, 50, 69, 93, 144, 166, 265, 54, 83, 39, ], )), mode='grounding', pipeline=[ dict(type='LoadAnnotations3D'), dict( max_n_images=50, n_images=1, ordered=True, rotate_3rscan=True, transforms=[ dict(backend_args=None, type='LoadImageFromFile'), dict(backend_args=None, type='LoadDepthFromFile'), dict( dst_intrinsic=[ [ 432.57943431339237, 0.0, 256, ], [ 0.0, 539.8570854208559, 256, ], [ 0.0, 0.0, 1.0, ], ], dst_wh=( 512, 512, ), type='CamIntrisicStandardization'), ], type='MultiViewPipeline'), dict( keys=[ 'img', 'depth_img', 'gt_bboxes_3d', 'gt_labels_3d', ], type='Pack3DDetInputs'), ], test_mode=True, type='EmbodiedScanDetGroundingDataset', vg_file='embodiedscan/embodiedscan_val_vg_all.json'), drop_last=False, num_workers=4, persistent_workers=False, sampler=dict(shuffle=False, type='DefaultSampler')) val_evaluator = dict(type='GroundingMetric') val_vg_file = 'embodiedscan/embodiedscan_val_vg_all.json' vis_backends = [ dict(save_dir='/job_tboard', type='TensorboardVisBackend'), ] visualizer = dict( name='visualizer', type='Visualizer', vis_backends=[ dict(save_dir='/job_tboard', type='TensorboardVisBackend'), ]) work_dir = '/job_data/work_dirs' z_range = [ -0.2, 3, ] 02/19 11:03:28 - mmengine - WARNING - Failed to search registry with scope "bip3d" in the "visualizer" registry tree. As a workaround, the current "visualizer" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "bip3d" is a correct scope, or whether the registry is initialized. 02/19 11:03:28 - mmengine - WARNING - Failed to search registry with scope "bip3d" in the "vis_backend" registry tree. As a workaround, the current "vis_backend" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "bip3d" is a correct scope, or whether the registry is initialized. /usr/local/lib/python3.10/dist-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage() return self.fget.__get__(instance, owner)() /usr/local/lib/python3.10/dist-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage() return self.fget.__get__(instance, owner)() /usr/local/lib/python3.10/dist-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage() return self.fget.__get__(instance, owner)() /usr/local/lib/python3.10/dist-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage() return self.fget.__get__(instance, owner)() /usr/local/lib/python3.10/dist-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage() return self.fget.__get__(instance, owner)() /usr/local/lib/python3.10/dist-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage() return self.fget.__get__(instance, owner)() /usr/local/lib/python3.10/dist-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage() return self.fget.__get__(instance, owner)() /usr/local/lib/python3.10/dist-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage() return self.fget.__get__(instance, owner)() 02/19 11:03:35 - mmengine - WARNING - Failed to search registry with scope "bip3d" in the "hook" registry tree. As a workaround, the current "hook" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "bip3d" is a correct scope, or whether the registry is initialized. 02/19 11:03:35 - mmengine - INFO - Hooks will be executed in the following order: before_run: (VERY_HIGH ) RuntimeInfoHook (BELOW_NORMAL) LoggerHook -------------------- before_train: (VERY_HIGH ) RuntimeInfoHook (NORMAL ) IterTimerHook (VERY_LOW ) CheckpointHook -------------------- before_train_epoch: (VERY_HIGH ) RuntimeInfoHook (NORMAL ) IterTimerHook (NORMAL ) DistSamplerSeedHook (NORMAL ) EmptyCacheHook -------------------- before_train_iter: (VERY_HIGH ) RuntimeInfoHook (NORMAL ) IterTimerHook -------------------- after_train_iter: (VERY_HIGH ) RuntimeInfoHook (NORMAL ) IterTimerHook (NORMAL ) EmptyCacheHook (BELOW_NORMAL) LoggerHook (LOW ) ParamSchedulerHook (VERY_LOW ) CheckpointHook -------------------- after_train_epoch: (NORMAL ) IterTimerHook (NORMAL ) EmptyCacheHook (LOW ) ParamSchedulerHook (VERY_LOW ) CheckpointHook -------------------- before_val: (VERY_HIGH ) RuntimeInfoHook -------------------- before_val_epoch: (NORMAL ) IterTimerHook (NORMAL ) EmptyCacheHook -------------------- before_val_iter: (NORMAL ) IterTimerHook -------------------- after_val_iter: (NORMAL ) IterTimerHook (NORMAL ) EmptyCacheHook (BELOW_NORMAL) LoggerHook -------------------- after_val_epoch: (VERY_HIGH ) RuntimeInfoHook (NORMAL ) IterTimerHook (NORMAL ) EmptyCacheHook (BELOW_NORMAL) LoggerHook (LOW ) ParamSchedulerHook (VERY_LOW ) CheckpointHook -------------------- after_val: (VERY_HIGH ) RuntimeInfoHook -------------------- after_train: (VERY_HIGH ) RuntimeInfoHook (VERY_LOW ) CheckpointHook -------------------- before_test: (VERY_HIGH ) RuntimeInfoHook -------------------- before_test_epoch: (NORMAL ) IterTimerHook (NORMAL ) EmptyCacheHook -------------------- before_test_iter: (NORMAL ) IterTimerHook -------------------- after_test_iter: (NORMAL ) IterTimerHook (NORMAL ) EmptyCacheHook (BELOW_NORMAL) LoggerHook -------------------- after_test_epoch: (VERY_HIGH ) RuntimeInfoHook (NORMAL ) IterTimerHook (NORMAL ) EmptyCacheHook (BELOW_NORMAL) LoggerHook -------------------- after_test: (VERY_HIGH ) RuntimeInfoHook -------------------- after_run: (BELOW_NORMAL) LoggerHook -------------------- 02/19 11:03:36 - mmengine - WARNING - Failed to search registry with scope "bip3d" in the "loop" registry tree. As a workaround, the current "loop" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "bip3d" is a correct scope, or whether the registry is initialized. 02/19 11:03:36 - mmengine - WARNING - euler-depth is not a meta file, simply parsed as meta information Loading Train dataset: 0%| | 0/3113 [00:00