logs/compute_embeddings_satclip.log

2026-02-01 09:09:57,720 [INFO] ================================================================================
2026-02-01 09:09:57,720 [INFO] Computing SATCLIP embeddings
2026-02-01 09:09:57,720 [INFO] Timestamp: 2026-02-01T09:09:57.720447
2026-02-01 09:09:57,720 [INFO] Device: cuda:1
2026-02-01 09:09:57,720 [INFO] ================================================================================
2026-02-01 09:09:57,721 [INFO] Batch size: 128
2026-02-01 09:09:57,721 [INFO] Output path: /data1/zyj/EarthEmbeddings/Core-S2L2A-249k/satclip/SatCLIP_crop_384x384.parquet
2026-02-01 09:09:57,721 [INFO] Loading satclip model...
2026-02-01 09:09:57,727 [INFO] SatCLIP-MS model loaded on cuda:1
2026-02-01 09:09:57,728 [INFO] Found 1 input files
2026-02-01 09:10:21,830 [WARNING] No valid embeddings for batch_0001_384x384.parquet
2026-02-01 09:10:22,107 [ERROR] No data processed!
2026-02-01 09:39:17,993 [INFO] ================================================================================
2026-02-01 09:39:17,993 [INFO] Computing SATCLIP embeddings
2026-02-01 09:39:17,993 [INFO] Timestamp: 2026-02-01T09:39:17.993775
2026-02-01 09:39:17,993 [INFO] Device: cuda:1
2026-02-01 09:39:17,993 [INFO] ================================================================================
2026-02-01 09:39:17,994 [INFO] Batch size: 128
2026-02-01 09:39:17,994 [INFO] Output path: /data1/zyj/EarthEmbeddings/Core-S2L2A-249k/satclip/SatCLIP_crop_384x384.parquet
2026-02-01 09:39:17,994 [INFO] Loading satclip model...
2026-02-01 09:39:20,179 [INFO] SatCLIP-MS model loaded on cuda:1
2026-02-01 09:39:20,180 [INFO] Found 1 input files
2026-02-01 09:40:01,084 [INFO] [batch_0001_384x384.parquet] Processed 1996 rows
2026-02-01 09:40:01,084 [INFO] Merging all results...
2026-02-01 09:40:01,085 [INFO] Final columns: ['grid_cell', 'grid_row_u', 'grid_col_r', 'product_id', 'timestamp', 'centre_lat', 'centre_lon', 'utm_crs', 'parquet_url', 'parquet_row', 'pixel_bbox', 'embedding']
2026-02-01 09:40:01,085 [INFO] ✓ All unwanted columns removed
2026-02-01 09:40:01,085 [INFO] ✓ Column 'crs' renamed to 'utm_crs'
2026-02-01 09:40:01,085 [INFO] ✓ Column 'pixel_bbox' added
2026-02-01 09:40:01,085 [INFO] Saving to /data1/zyj/EarthEmbeddings/Core-S2L2A-249k/satclip/SatCLIP_crop_384x384.parquet...
2026-02-01 09:40:01,134 [INFO] ================================================================================
2026-02-01 09:40:01,134 [INFO] Processing complete!
2026-02-01 09:40:01,134 [INFO] Total rows: 1,996
2026-02-01 09:40:01,134 [INFO] Embedding dimension: 256
2026-02-01 09:40:01,134 [INFO] Output file: /data1/zyj/EarthEmbeddings/Core-S2L2A-249k/satclip/SatCLIP_crop_384x384.parquet
2026-02-01 09:40:01,134 [INFO] ================================================================================
2026-02-01 09:43:19,666 [INFO] ================================================================================
2026-02-01 09:43:19,666 [INFO] Computing SATCLIP embeddings
2026-02-01 09:43:19,666 [INFO] Timestamp: 2026-02-01T09:43:19.666577
2026-02-01 09:43:19,666 [INFO] Device: cuda:3
2026-02-01 09:43:19,666 [INFO] ================================================================================
2026-02-01 09:43:19,668 [INFO] Batch size: 128
2026-02-01 09:43:19,668 [INFO] Output path: /data1/zyj/EarthEmbeddings/Core-S2L2A-249k/satclip/SatCLIP_crop_384x384.parquet
2026-02-01 09:43:19,668 [INFO] Loading satclip model...
2026-02-01 09:43:21,344 [INFO] SatCLIP-MS model loaded on cuda:3
2026-02-01 09:43:21,345 [INFO] Found 125 input files
2026-02-01 09:44:03,000 [INFO] [batch_0001_384x384.parquet] Processed 1996 rows
2026-02-01 09:44:46,041 [INFO] [batch_0002_384x384.parquet] Processed 1990 rows
2026-02-01 09:45:27,652 [INFO] [batch_0003_384x384.parquet] Processed 1997 rows
2026-02-01 09:46:15,446 [INFO] [batch_0004_384x384.parquet] Processed 1992 rows
2026-02-01 09:47:09,769 [INFO] [batch_0005_384x384.parquet] Processed 1992 rows
2026-02-01 09:47:59,773 [INFO] [batch_0006_384x384.parquet] Processed 1992 rows
2026-02-01 09:48:51,057 [INFO] [batch_0007_384x384.parquet] Processed 1998 rows
2026-02-01 09:49:34,202 [INFO] [batch_0008_384x384.parquet] Processed 1994 rows
2026-02-01 09:50:25,944 [INFO] [batch_0009_384x384.parquet] Processed 1993 rows
2026-02-01 09:51:09,586 [INFO] [batch_0010_384x384.parquet] Processed 1993 rows
2026-02-01 09:51:56,545 [INFO] [batch_0011_384x384.parquet] Processed 1990 rows
2026-02-01 09:52:44,526 [INFO] [batch_0012_384x384.parquet] Processed 1993 rows
2026-02-01 09:53:32,729 [INFO] [batch_0013_384x384.parquet] Processed 1993 rows
2026-02-01 09:54:14,312 [INFO] [batch_0014_384x384.parquet] Processed 1991 rows
2026-02-01 09:55:05,975 [INFO] [batch_0015_384x384.parquet] Processed 1993 rows
2026-02-01 09:55:57,268 [INFO] [batch_0016_384x384.parquet] Processed 1990 rows
2026-02-01 09:57:00,591 [INFO] [batch_0017_384x384.parquet] Processed 1991 rows
2026-02-01 09:57:48,464 [INFO] [batch_0018_384x384.parquet] Processed 1991 rows
2026-02-01 09:58:52,420 [INFO] [batch_0019_384x384.parquet] Processed 1996 rows
2026-02-01 10:00:04,202 [INFO] [batch_0020_384x384.parquet] Processed 1994 rows
2026-02-01 10:01:10,309 [INFO] [batch_0021_384x384.parquet] Processed 1996 rows
2026-02-01 10:02:15,265 [INFO] [batch_0022_384x384.parquet] Processed 1995 rows
2026-02-01 10:03:31,554 [INFO] [batch_0023_384x384.parquet] Processed 1992 rows
2026-02-01 10:04:40,240 [INFO] [batch_0024_384x384.parquet] Processed 1989 rows
2026-02-01 10:05:55,812 [INFO] [batch_0025_384x384.parquet] Processed 1993 rows
2026-02-01 10:07:00,366 [INFO] [batch_0026_384x384.parquet] Processed 1995 rows
2026-02-01 10:08:10,532 [INFO] [batch_0027_384x384.parquet] Processed 1997 rows
2026-02-01 10:09:11,505 [INFO] [batch_0028_384x384.parquet] Processed 1991 rows
2026-02-01 10:10:21,951 [INFO] [batch_0029_384x384.parquet] Processed 1992 rows
2026-02-01 10:11:30,988 [INFO] [batch_0030_384x384.parquet] Processed 1993 rows
2026-02-01 10:12:26,034 [INFO] [batch_0031_384x384.parquet] Processed 1988 rows
2026-02-01 10:13:36,732 [INFO] [batch_0032_384x384.parquet] Processed 1994 rows
2026-02-01 10:14:36,787 [INFO] [batch_0033_384x384.parquet] Processed 1992 rows
2026-02-01 10:15:36,921 [INFO] [batch_0034_384x384.parquet] Processed 1994 rows
2026-02-01 10:16:38,623 [INFO] [batch_0035_384x384.parquet] Processed 1994 rows
2026-02-01 10:17:27,583 [INFO] [batch_0036_384x384.parquet] Processed 1996 rows
2026-02-01 10:18:29,976 [INFO] [batch_0037_384x384.parquet] Processed 1995 rows
2026-02-01 10:19:26,843 [INFO] [batch_0038_384x384.parquet] Processed 1993 rows
2026-02-01 10:20:14,532 [INFO] [batch_0039_384x384.parquet] Processed 1989 rows
2026-02-01 10:21:13,694 [INFO] [batch_0040_384x384.parquet] Processed 1990 rows
2026-02-01 10:22:05,858 [INFO] [batch_0041_384x384.parquet] Processed 1998 rows
2026-02-01 10:23:04,226 [INFO] [batch_0042_384x384.parquet] Processed 1997 rows
2026-02-01 10:23:56,641 [INFO] [batch_0043_384x384.parquet] Processed 1988 rows
2026-02-01 10:24:38,594 [INFO] [batch_0044_384x384.parquet] Processed 1991 rows
2026-02-01 10:25:42,517 [INFO] [batch_0045_384x384.parquet] Processed 1993 rows
2026-02-01 10:26:23,732 [INFO] [batch_0046_384x384.parquet] Processed 1994 rows
2026-02-01 10:27:39,298 [INFO] [batch_0047_384x384.parquet] Processed 1995 rows
2026-02-01 10:28:34,546 [INFO] [batch_0048_384x384.parquet] Processed 1992 rows
2026-02-01 10:29:35,568 [INFO] [batch_0049_384x384.parquet] Processed 1996 rows
2026-02-01 10:30:38,004 [INFO] [batch_0050_384x384.parquet] Processed 1991 rows
2026-02-01 10:31:50,544 [INFO] [batch_0051_384x384.parquet] Processed 1997 rows
2026-02-01 10:32:38,165 [INFO] [batch_0052_384x384.parquet] Processed 1993 rows
2026-02-01 10:33:54,330 [INFO] [batch_0053_384x384.parquet] Processed 1995 rows
2026-02-01 10:35:11,070 [INFO] [batch_0054_384x384.parquet] Processed 1997 rows
2026-02-01 10:36:06,495 [INFO] [batch_0055_384x384.parquet] Processed 1995 rows
2026-02-01 10:37:26,449 [INFO] [batch_0056_384x384.parquet] Processed 1997 rows
2026-02-01 10:38:40,433 [INFO] [batch_0057_384x384.parquet] Processed 1991 rows
2026-02-01 10:39:36,229 [INFO] [batch_0058_384x384.parquet] Processed 1994 rows
2026-02-01 10:40:50,558 [INFO] [batch_0059_384x384.parquet] Processed 1993 rows
2026-02-01 10:42:00,100 [INFO] [batch_0060_384x384.parquet] Processed 1995 rows
2026-02-01 10:42:53,440 [INFO] [batch_0061_384x384.parquet] Processed 1995 rows
2026-02-01 10:44:21,706 [INFO] [batch_0062_384x384.parquet] Processed 1998 rows
2026-02-01 10:45:56,656 [INFO] [batch_0063_384x384.parquet] Processed 1997 rows
2026-02-01 10:46:53,942 [INFO] [batch_0064_384x384.parquet] Processed 1992 rows
2026-02-01 10:47:47,760 [INFO] [batch_0065_384x384.parquet] Processed 1994 rows
2026-02-01 10:48:37,571 [INFO] [batch_0066_384x384.parquet] Processed 1992 rows
2026-02-01 10:50:00,819 [INFO] [batch_0067_384x384.parquet] Processed 1993 rows
2026-02-01 10:51:30,799 [INFO] [batch_0068_384x384.parquet] Processed 1994 rows
2026-02-01 10:52:28,413 [INFO] [batch_0069_384x384.parquet] Processed 1992 rows
2026-02-01 10:53:50,597 [INFO] [batch_0070_384x384.parquet] Processed 1997 rows
2026-02-01 10:55:01,173 [INFO] [batch_0071_384x384.parquet] Processed 1996 rows
2026-02-01 10:56:03,395 [INFO] [batch_0072_384x384.parquet] Processed 1992 rows
2026-02-01 10:57:10,601 [INFO] [batch_0073_384x384.parquet] Processed 1995 rows
2026-02-01 10:58:22,789 [INFO] [batch_0074_384x384.parquet] Processed 1992 rows
2026-02-01 10:59:39,697 [INFO] [batch_0075_384x384.parquet] Processed 1991 rows
2026-02-01 11:00:48,962 [INFO] [batch_0076_384x384.parquet] Processed 1998 rows
2026-02-01 11:01:59,729 [INFO] [batch_0077_384x384.parquet] Processed 1996 rows
2026-02-01 11:03:01,575 [INFO] [batch_0078_384x384.parquet] Processed 1992 rows
2026-02-01 11:04:15,721 [INFO] [batch_0079_384x384.parquet] Processed 1995 rows
2026-02-01 11:05:26,147 [INFO] [batch_0080_384x384.parquet] Processed 1993 rows
2026-02-01 11:06:21,742 [INFO] [batch_0081_384x384.parquet] Processed 1995 rows
2026-02-01 11:07:34,071 [INFO] [batch_0082_384x384.parquet] Processed 1989 rows
2026-02-01 11:08:51,443 [INFO] [batch_0083_384x384.parquet] Processed 1995 rows
2026-02-01 11:09:45,289 [INFO] [batch_0084_384x384.parquet] Processed 1996 rows
2026-02-01 11:10:59,507 [INFO] [batch_0085_384x384.parquet] Processed 1997 rows
2026-02-01 11:12:12,671 [INFO] [batch_0086_384x384.parquet] Processed 1996 rows
2026-02-01 11:13:16,945 [INFO] [batch_0087_384x384.parquet] Processed 1994 rows
2026-02-01 11:14:26,324 [INFO] [batch_0088_384x384.parquet] Processed 1992 rows
2026-02-01 11:15:25,871 [INFO] [batch_0089_384x384.parquet] Processed 1993 rows
2026-02-01 11:16:43,653 [INFO] [batch_0090_384x384.parquet] Processed 1993 rows
2026-02-01 11:17:52,205 [INFO] [batch_0091_384x384.parquet] Processed 1995 rows
2026-02-01 11:19:02,073 [INFO] [batch_0092_384x384.parquet] Processed 1994 rows
2026-02-01 11:20:14,843 [INFO] [batch_0093_384x384.parquet] Processed 1998 rows
2026-02-01 11:21:09,193 [INFO] [batch_0094_384x384.parquet] Processed 1993 rows
2026-02-01 11:22:03,303 [INFO] [batch_0095_384x384.parquet] Processed 1995 rows
2026-02-01 11:23:12,708 [INFO] [batch_0096_384x384.parquet] Processed 1997 rows
2026-02-01 11:24:18,831 [INFO] [batch_0097_384x384.parquet] Processed 1990 rows
2026-02-01 11:25:07,701 [INFO] [batch_0098_384x384.parquet] Processed 1995 rows
2026-02-01 11:26:18,306 [INFO] [batch_0099_384x384.parquet] Processed 1992 rows
2026-02-01 11:27:02,698 [INFO] [batch_0100_384x384.parquet] Processed 1993 rows
2026-02-01 11:28:08,644 [INFO] [batch_0101_384x384.parquet] Processed 1994 rows
2026-02-01 11:29:33,678 [INFO] [batch_0102_384x384.parquet] Processed 1991 rows
2026-02-01 11:30:25,760 [INFO] [batch_0103_384x384.parquet] Processed 1990 rows
2026-02-01 11:31:38,365 [INFO] [batch_0104_384x384.parquet] Processed 1995 rows
2026-02-01 11:33:06,206 [INFO] [batch_0105_384x384.parquet] Processed 1993 rows
2026-02-01 11:33:59,497 [INFO] [batch_0106_384x384.parquet] Processed 1988 rows
2026-02-01 11:35:04,565 [INFO] [batch_0107_384x384.parquet] Processed 1996 rows
2026-02-01 11:36:30,898 [INFO] [batch_0108_384x384.parquet] Processed 1992 rows
2026-02-01 11:37:34,766 [INFO] [batch_0109_384x384.parquet] Processed 1993 rows
2026-02-01 11:38:36,780 [INFO] [batch_0110_384x384.parquet] Processed 1996 rows
2026-02-01 11:39:53,826 [INFO] [batch_0111_384x384.parquet] Processed 1991 rows
2026-02-01 11:40:48,014 [INFO] [batch_0112_384x384.parquet] Processed 1994 rows
2026-02-01 11:41:49,113 [INFO] [batch_0113_384x384.parquet] Processed 1996 rows
2026-02-01 11:42:56,188 [INFO] [batch_0114_384x384.parquet] Processed 1997 rows
2026-02-01 11:43:43,288 [INFO] [batch_0115_384x384.parquet] Processed 1997 rows
2026-02-01 11:44:48,748 [INFO] [batch_0116_384x384.parquet] Processed 1992 rows
2026-02-01 11:45:54,394 [INFO] [batch_0117_384x384.parquet] Processed 1992 rows
2026-02-01 11:46:53,275 [INFO] [batch_0118_384x384.parquet] Processed 1993 rows
2026-02-01 11:48:08,611 [INFO] [batch_0119_384x384.parquet] Processed 1998 rows
2026-02-01 11:49:07,195 [INFO] [batch_0120_384x384.parquet] Processed 1995 rows
2026-02-01 11:50:22,347 [INFO] [batch_0121_384x384.parquet] Processed 1995 rows
2026-02-01 11:51:26,391 [INFO] [batch_0122_384x384.parquet] Processed 1992 rows
2026-02-01 11:52:22,734 [INFO] [batch_0123_384x384.parquet] Processed 1994 rows
2026-02-01 11:53:34,357 [INFO] [batch_0124_384x384.parquet] Processed 1995 rows
2026-02-01 11:54:05,024 [INFO] [batch_0125_384x384.parquet] Processed 1505 rows
2026-02-01 11:54:05,024 [INFO] Merging all results...
2026-02-01 11:54:05,057 [INFO] Final columns: ['grid_cell', 'grid_row_u', 'grid_col_r', 'product_id', 'timestamp', 'centre_lat', 'centre_lon', 'utm_crs', 'parquet_url', 'parquet_row', 'pixel_bbox', 'embedding']
2026-02-01 11:54:05,058 [INFO] ✓ All unwanted columns removed
2026-02-01 11:54:05,058 [INFO] ✓ Column 'crs' renamed to 'utm_crs'
2026-02-01 11:54:05,058 [INFO] ✓ Column 'pixel_bbox' added
2026-02-01 11:54:05,058 [INFO] Saving to /data1/zyj/EarthEmbeddings/Core-S2L2A-249k/satclip/SatCLIP_crop_384x384.parquet...
2026-02-01 11:54:06,861 [INFO] ================================================================================
2026-02-01 11:54:06,861 [INFO] Processing complete!
2026-02-01 11:54:06,861 [INFO] Total rows: 248,719
2026-02-01 11:54:06,862 [INFO] Embedding dimension: 256
2026-02-01 11:54:06,862 [INFO] Output file: /data1/zyj/EarthEmbeddings/Core-S2L2A-249k/satclip/SatCLIP_crop_384x384.parquet
2026-02-01 11:54:06,862 [INFO] ================================================================================