Spaces:
Build error
Build error
File size: 12,061 Bytes
eb1aec4 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 | 2026-02-01 09:54:48,604 [INFO] ================================================================================
2026-02-01 09:54:48,605 [INFO] Computing FARSLIP embeddings
2026-02-01 09:54:48,605 [INFO] Timestamp: 2026-02-01T09:54:48.605134
2026-02-01 09:54:48,605 [INFO] Device: cuda:4
2026-02-01 09:54:48,605 [INFO] ================================================================================
2026-02-01 09:54:48,606 [INFO] Batch size: 64
2026-02-01 09:54:48,607 [INFO] Output path: /data1/zyj/EarthEmbeddings/Core-S2L2A-249k/farslip/FarSLIP_crop_384x384.parquet
2026-02-01 09:54:48,607 [INFO] Loading farslip model...
2026-02-01 09:54:48,613 [INFO] Loaded ViT-B-16 model config.
2026-02-01 09:54:50,536 [INFO] Loading pretrained ViT-B-16 weights (/data1/zyj/checkpoints/FarSLIP/FarSLIP2_ViT-B-16.pt).
2026-02-01 09:54:51,666 [INFO] Missing keys: []
2026-02-01 09:54:51,745 [INFO] FarSLIP model loaded on cuda:4
2026-02-01 09:54:51,745 [INFO] Found 125 input files
2026-02-01 09:55:38,785 [INFO] [batch_0001_384x384.parquet] Processed 1996 rows
2026-02-01 09:56:18,239 [INFO] [batch_0002_384x384.parquet] Processed 1990 rows
2026-02-01 09:57:17,259 [INFO] [batch_0003_384x384.parquet] Processed 1997 rows
2026-02-01 09:58:08,339 [INFO] [batch_0004_384x384.parquet] Processed 1992 rows
2026-02-01 09:59:00,302 [INFO] [batch_0005_384x384.parquet] Processed 1992 rows
2026-02-01 10:00:15,416 [INFO] [batch_0006_384x384.parquet] Processed 1992 rows
2026-02-01 10:01:22,601 [INFO] [batch_0007_384x384.parquet] Processed 1998 rows
2026-02-01 10:02:25,131 [INFO] [batch_0008_384x384.parquet] Processed 1994 rows
2026-02-01 10:03:31,735 [INFO] [batch_0009_384x384.parquet] Processed 1993 rows
2026-02-01 10:04:47,342 [INFO] [batch_0010_384x384.parquet] Processed 1993 rows
2026-02-01 10:05:54,617 [INFO] [batch_0011_384x384.parquet] Processed 1990 rows
2026-02-01 10:06:58,372 [INFO] [batch_0012_384x384.parquet] Processed 1993 rows
2026-02-01 10:08:16,301 [INFO] [batch_0013_384x384.parquet] Processed 1993 rows
2026-02-01 10:09:11,722 [INFO] [batch_0014_384x384.parquet] Processed 1991 rows
2026-02-01 10:10:23,603 [INFO] [batch_0015_384x384.parquet] Processed 1993 rows
2026-02-01 10:11:38,047 [INFO] [batch_0016_384x384.parquet] Processed 1990 rows
2026-02-01 10:12:22,943 [INFO] [batch_0017_384x384.parquet] Processed 1991 rows
2026-02-01 10:13:41,095 [INFO] [batch_0018_384x384.parquet] Processed 1991 rows
2026-02-01 10:14:47,596 [INFO] [batch_0019_384x384.parquet] Processed 1996 rows
2026-02-01 10:15:40,983 [INFO] [batch_0020_384x384.parquet] Processed 1994 rows
2026-02-01 10:16:52,878 [INFO] [batch_0021_384x384.parquet] Processed 1996 rows
2026-02-01 10:17:43,460 [INFO] [batch_0022_384x384.parquet] Processed 1995 rows
2026-02-01 10:18:41,479 [INFO] [batch_0023_384x384.parquet] Processed 1992 rows
2026-02-01 10:19:40,728 [INFO] [batch_0024_384x384.parquet] Processed 1989 rows
2026-02-01 10:20:25,503 [INFO] [batch_0025_384x384.parquet] Processed 1993 rows
2026-02-01 10:21:27,428 [INFO] [batch_0026_384x384.parquet] Processed 1995 rows
2026-02-01 10:22:23,776 [INFO] [batch_0027_384x384.parquet] Processed 1997 rows
2026-02-01 10:23:16,992 [INFO] [batch_0028_384x384.parquet] Processed 1991 rows
2026-02-01 10:24:14,634 [INFO] [batch_0029_384x384.parquet] Processed 1992 rows
2026-02-01 10:24:55,464 [INFO] [batch_0030_384x384.parquet] Processed 1993 rows
2026-02-01 10:25:56,600 [INFO] [batch_0031_384x384.parquet] Processed 1988 rows
2026-02-01 10:26:40,392 [INFO] [batch_0032_384x384.parquet] Processed 1994 rows
2026-02-01 10:27:49,696 [INFO] [batch_0033_384x384.parquet] Processed 1992 rows
2026-02-01 10:28:49,831 [INFO] [batch_0034_384x384.parquet] Processed 1994 rows
2026-02-01 10:29:42,378 [INFO] [batch_0035_384x384.parquet] Processed 1994 rows
2026-02-01 10:30:48,969 [INFO] [batch_0036_384x384.parquet] Processed 1996 rows
2026-02-01 10:32:01,922 [INFO] [batch_0037_384x384.parquet] Processed 1995 rows
2026-02-01 10:32:47,057 [INFO] [batch_0038_384x384.parquet] Processed 1993 rows
2026-02-01 10:34:01,196 [INFO] [batch_0039_384x384.parquet] Processed 1989 rows
2026-02-01 10:35:19,501 [INFO] [batch_0040_384x384.parquet] Processed 1990 rows
2026-02-01 10:36:09,997 [INFO] [batch_0041_384x384.parquet] Processed 1998 rows
2026-02-01 10:37:25,589 [INFO] [batch_0042_384x384.parquet] Processed 1997 rows
2026-02-01 10:38:42,876 [INFO] [batch_0043_384x384.parquet] Processed 1988 rows
2026-02-01 10:39:31,979 [INFO] [batch_0044_384x384.parquet] Processed 1991 rows
2026-02-01 10:40:43,745 [INFO] [batch_0045_384x384.parquet] Processed 1993 rows
2026-02-01 10:41:59,576 [INFO] [batch_0046_384x384.parquet] Processed 1994 rows
2026-02-01 10:42:53,620 [INFO] [batch_0047_384x384.parquet] Processed 1995 rows
2026-02-01 10:44:25,584 [INFO] [batch_0048_384x384.parquet] Processed 1992 rows
2026-02-01 10:46:13,258 [INFO] [batch_0049_384x384.parquet] Processed 1996 rows
2026-02-01 10:47:13,109 [INFO] [batch_0050_384x384.parquet] Processed 1991 rows
2026-02-01 10:48:13,385 [INFO] [batch_0051_384x384.parquet] Processed 1997 rows
2026-02-01 10:49:48,140 [INFO] [batch_0052_384x384.parquet] Processed 1993 rows
2026-02-01 10:51:22,710 [INFO] [batch_0053_384x384.parquet] Processed 1995 rows
2026-02-01 10:52:23,823 [INFO] [batch_0054_384x384.parquet] Processed 1997 rows
2026-02-01 10:53:48,669 [INFO] [batch_0055_384x384.parquet] Processed 1995 rows
2026-02-01 10:55:03,785 [INFO] [batch_0056_384x384.parquet] Processed 1997 rows
2026-02-01 10:55:56,653 [INFO] [batch_0057_384x384.parquet] Processed 1991 rows
2026-02-01 10:56:50,364 [INFO] [batch_0058_384x384.parquet] Processed 1994 rows
2026-02-01 10:57:33,268 [INFO] [batch_0059_384x384.parquet] Processed 1993 rows
2026-02-01 10:58:36,103 [INFO] [batch_0060_384x384.parquet] Processed 1995 rows
2026-02-01 10:59:43,156 [INFO] [batch_0061_384x384.parquet] Processed 1995 rows
2026-02-01 11:00:45,280 [INFO] [batch_0062_384x384.parquet] Processed 1998 rows
2026-02-01 11:02:03,960 [INFO] [batch_0063_384x384.parquet] Processed 1997 rows
2026-02-01 11:03:01,993 [INFO] [batch_0064_384x384.parquet] Processed 1992 rows
2026-02-01 11:04:18,812 [INFO] [batch_0065_384x384.parquet] Processed 1994 rows
2026-02-01 11:05:34,954 [INFO] [batch_0066_384x384.parquet] Processed 1992 rows
2026-02-01 11:06:26,502 [INFO] [batch_0067_384x384.parquet] Processed 1993 rows
2026-02-01 11:07:42,754 [INFO] [batch_0068_384x384.parquet] Processed 1994 rows
2026-02-01 11:09:01,751 [INFO] [batch_0069_384x384.parquet] Processed 1992 rows
2026-02-01 11:09:49,394 [INFO] [batch_0070_384x384.parquet] Processed 1997 rows
2026-02-01 11:11:06,518 [INFO] [batch_0071_384x384.parquet] Processed 1996 rows
2026-02-01 11:12:22,688 [INFO] [batch_0072_384x384.parquet] Processed 1992 rows
2026-02-01 11:13:14,831 [INFO] [batch_0073_384x384.parquet] Processed 1995 rows
2026-02-01 11:14:14,879 [INFO] [batch_0074_384x384.parquet] Processed 1992 rows
2026-02-01 11:14:58,098 [INFO] [batch_0075_384x384.parquet] Processed 1991 rows
2026-02-01 11:15:43,764 [INFO] [batch_0076_384x384.parquet] Processed 1998 rows
2026-02-01 11:16:53,710 [INFO] [batch_0077_384x384.parquet] Processed 1996 rows
2026-02-01 11:17:51,040 [INFO] [batch_0078_384x384.parquet] Processed 1992 rows
2026-02-01 11:18:57,871 [INFO] [batch_0079_384x384.parquet] Processed 1995 rows
2026-02-01 11:20:06,930 [INFO] [batch_0080_384x384.parquet] Processed 1993 rows
2026-02-01 11:20:51,630 [INFO] [batch_0081_384x384.parquet] Processed 1995 rows
2026-02-01 11:21:43,270 [INFO] [batch_0082_384x384.parquet] Processed 1989 rows
2026-02-01 11:22:29,228 [INFO] [batch_0083_384x384.parquet] Processed 1995 rows
2026-02-01 11:23:23,236 [INFO] [batch_0084_384x384.parquet] Processed 1996 rows
2026-02-01 11:24:32,532 [INFO] [batch_0085_384x384.parquet] Processed 1997 rows
2026-02-01 11:25:20,336 [INFO] [batch_0086_384x384.parquet] Processed 1996 rows
2026-02-01 11:26:33,616 [INFO] [batch_0087_384x384.parquet] Processed 1994 rows
2026-02-01 11:27:24,449 [INFO] [batch_0088_384x384.parquet] Processed 1992 rows
2026-02-01 11:28:20,047 [INFO] [batch_0089_384x384.parquet] Processed 1993 rows
2026-02-01 11:29:43,109 [INFO] [batch_0090_384x384.parquet] Processed 1993 rows
2026-02-01 11:30:41,652 [INFO] [batch_0091_384x384.parquet] Processed 1995 rows
2026-02-01 11:31:43,751 [INFO] [batch_0092_384x384.parquet] Processed 1994 rows
2026-02-01 11:33:10,661 [INFO] [batch_0093_384x384.parquet] Processed 1998 rows
2026-02-01 11:34:12,721 [INFO] [batch_0094_384x384.parquet] Processed 1993 rows
2026-02-01 11:35:09,887 [INFO] [batch_0095_384x384.parquet] Processed 1995 rows
2026-02-01 11:36:36,141 [INFO] [batch_0096_384x384.parquet] Processed 1997 rows
2026-02-01 11:37:41,740 [INFO] [batch_0097_384x384.parquet] Processed 1990 rows
2026-02-01 11:38:40,066 [INFO] [batch_0098_384x384.parquet] Processed 1995 rows
2026-02-01 11:39:45,765 [INFO] [batch_0099_384x384.parquet] Processed 1992 rows
2026-02-01 11:40:40,739 [INFO] [batch_0100_384x384.parquet] Processed 1993 rows
2026-02-01 11:41:41,583 [INFO] [batch_0101_384x384.parquet] Processed 1994 rows
2026-02-01 11:42:47,504 [INFO] [batch_0102_384x384.parquet] Processed 1991 rows
2026-02-01 11:43:31,148 [INFO] [batch_0103_384x384.parquet] Processed 1990 rows
2026-02-01 11:44:38,070 [INFO] [batch_0104_384x384.parquet] Processed 1995 rows
2026-02-01 11:45:48,089 [INFO] [batch_0105_384x384.parquet] Processed 1993 rows
2026-02-01 11:46:47,156 [INFO] [batch_0106_384x384.parquet] Processed 1988 rows
2026-02-01 11:48:06,340 [INFO] [batch_0107_384x384.parquet] Processed 1996 rows
2026-02-01 11:49:08,016 [INFO] [batch_0108_384x384.parquet] Processed 1992 rows
2026-02-01 11:50:27,665 [INFO] [batch_0109_384x384.parquet] Processed 1993 rows
2026-02-01 11:51:38,073 [INFO] [batch_0110_384x384.parquet] Processed 1996 rows
2026-02-01 11:52:26,956 [INFO] [batch_0111_384x384.parquet] Processed 1991 rows
2026-02-01 11:53:44,395 [INFO] [batch_0112_384x384.parquet] Processed 1994 rows
2026-02-01 11:54:23,803 [INFO] [batch_0113_384x384.parquet] Processed 1996 rows
2026-02-01 11:55:07,867 [INFO] [batch_0114_384x384.parquet] Processed 1997 rows
2026-02-01 11:55:54,834 [INFO] [batch_0115_384x384.parquet] Processed 1997 rows
2026-02-01 11:56:36,849 [INFO] [batch_0116_384x384.parquet] Processed 1992 rows
2026-02-01 11:57:20,506 [INFO] [batch_0117_384x384.parquet] Processed 1992 rows
2026-02-01 11:57:58,985 [INFO] [batch_0118_384x384.parquet] Processed 1993 rows
2026-02-01 11:58:38,965 [INFO] [batch_0119_384x384.parquet] Processed 1998 rows
2026-02-01 11:59:16,459 [INFO] [batch_0120_384x384.parquet] Processed 1995 rows
2026-02-01 11:59:55,497 [INFO] [batch_0121_384x384.parquet] Processed 1995 rows
2026-02-01 12:00:33,857 [INFO] [batch_0122_384x384.parquet] Processed 1992 rows
2026-02-01 12:01:15,871 [INFO] [batch_0123_384x384.parquet] Processed 1994 rows
2026-02-01 12:01:53,537 [INFO] [batch_0124_384x384.parquet] Processed 1995 rows
2026-02-01 12:02:22,334 [INFO] [batch_0125_384x384.parquet] Processed 1505 rows
2026-02-01 12:02:22,334 [INFO] Merging all results...
2026-02-01 12:02:22,384 [INFO] Final columns: ['grid_cell', 'grid_row_u', 'grid_col_r', 'product_id', 'timestamp', 'centre_lat', 'centre_lon', 'utm_crs', 'parquet_url', 'parquet_row', 'pixel_bbox', 'embedding']
2026-02-01 12:02:22,384 [INFO] ✓ All unwanted columns removed
2026-02-01 12:02:22,384 [INFO] ✓ Column 'crs' renamed to 'utm_crs'
2026-02-01 12:02:22,384 [INFO] ✓ Column 'pixel_bbox' added
2026-02-01 12:02:22,384 [INFO] Saving to /data1/zyj/EarthEmbeddings/Core-S2L2A-249k/farslip/FarSLIP_crop_384x384.parquet...
2026-02-01 12:02:25,588 [INFO] ================================================================================
2026-02-01 12:02:25,588 [INFO] Processing complete!
2026-02-01 12:02:25,588 [INFO] Total rows: 248,719
2026-02-01 12:02:25,589 [INFO] Embedding dimension: 512
2026-02-01 12:02:25,589 [INFO] Output file: /data1/zyj/EarthEmbeddings/Core-S2L2A-249k/farslip/FarSLIP_crop_384x384.parquet
2026-02-01 12:02:25,589 [INFO] ================================================================================
|