mikonvergence commited on
Commit
2be166b
·
verified ·
1 Parent(s): bc66791

Sync from GitHub 2c58fb3

Browse files
Files changed (1) hide show
  1. README.md +34 -1
README.md CHANGED
@@ -10,7 +10,9 @@ short_description: Major TOM Core samples on a 3D globe, read in the browser
10
 
11
  > **CLAUDE-GENERATED DOCUMENT** — automated LLM-generated content. Verify before relying on it.
12
 
13
- > **Review status:** draft for review, 2026-11-02. Space created (private) and deployed 2026-11-02.
 
 
14
 
15
  # Major TOM Raster Viewer
16
 
@@ -45,6 +47,10 @@ AlphaEarth Embeddings (aef.js): footprints/AEF.parquet maps each Core-S2L2A cell
45
  proxy at 10 m (~15 s). A cell's window is joined from every tile it touches (24.6% need two to four). The COGs are
46
  stored south-up: origin and row direction from each file. View menu: single bands at 10 m. Downloads: the shown
47
  image (.png), the shown band or all 64 at 10 m (GeoTIFF written in the browser), Python (rasterio.merge)
 
 
 
 
48
  Downloads (download.js): the shown band as stored (.tif); every band, the thumbnail and metadata.json (.zip); a
49
  Python snippet that reads the same row group from Hugging Face
50
  in the background: the other datasets' footprints once the first is drawn; after a click, the same cell's samples
@@ -61,6 +67,10 @@ in the background: the other datasets' footprints once the first is drawn; after
61
  | `samples.js` | one sample's thumbnail from its image file |
62
  | `bands.js` | a sample's band columns: read, decode (geotiff.js), stretch, composites |
63
  | `download.js` | band GeoTIFFs as stored, all bands as a zip (fflate), the Python snippet |
 
 
 
 
64
  | `parquet.js` | range requests for hyparquet, with bytes and time counted; each file's footer read once |
65
  | `tiles.js`, `labels.js`, `places.json` | copied from the Climate Globe: cells and tiles, place names |
66
  | `build_index.py` | Core `metadata.parquet` -> `footprints/<name>.parquet` and `footprints/datasets.json` |
@@ -72,11 +82,25 @@ in the background: the other datasets' footprints once the first is drawn; after
72
  |---|---|---|
73
  | S2L2A footprints (coverage of 2,245,886 samples) | 3.9 MB in 37 requests | 0.7 s |
74
  | One S2L2A thumbnail, first from its image file (footer 1.5 MB) | 1.1 MB in 4 requests | 1.8 s |
 
 
 
 
75
 
76
  Reading each dataset's `metadata.parquet` from Hugging Face instead was measured first: S2L2A's has one row group
77
  per image file (4,492), so its positions took 462 s in 4,492 requests; S1RTC's and DEM's have two row groups, so
78
  one sample's row cost 23-27 MB. Hence the footprint files.
79
 
 
 
 
 
 
 
 
 
 
 
80
  ## Run locally
81
 
82
  ```
@@ -86,6 +110,15 @@ pytest -m network # se
86
 
87
  The page needs a server that answers HTTP range requests (the tests' server does; `python -m http.server` does not).
88
 
 
 
 
 
 
 
 
 
 
89
  ## Credit
90
 
91
  Developed by [Mikolaj Czerkawski](https://github.com/mikonvergence), [Asterisk Labs](https://asterisk.coop), as part of
 
10
 
11
  > **CLAUDE-GENERATED DOCUMENT** — automated LLM-generated content. Verify before relying on it.
12
 
13
+ > **Review status:** not yet reviewed. First version 2026-11-02; Space created (private) and deployed 2026-11-02.
14
+ > Changed 2026-11-04: Core-Weather panel in the flow; `weather.js`, `viirs.js`, `aef.js` and `zstd.js` in the
15
+ > Files and Reads tables; the private-Space import rule; the test files.
16
 
17
  # Major TOM Raster Viewer
18
 
 
47
  proxy at 10 m (~15 s). A cell's window is joined from every tile it touches (24.6% need two to four). The COGs are
48
  stored south-up: origin and row direction from each file. View menu: single bands at 10 m. Downloads: the shown
49
  image (.png), the shown band or all 64 at 10 m (GeoTIFF written in the browser), Python (rasterio.merge)
50
+ Core-Weather (weather.js): the sample's acquisition (S2L2A, S2L1C, S1RTC; for DEM, VIIRS and AlphaEarth the cell's
51
+ S2L2A acquisition, said in the panel) -> hourly/ and daily/ day files on source.coop major-tom/core-weather
52
+ -> grid_cell and acquired columns -> the sample's rows -> charts: hourly ±24 h (ERA5 temperature, precipitation,
53
+ cloud cover), daily ±30 d (ERA5 temperature and precipitation, CAMS PM2.5, Sentinel-5P NO₂)
54
  Downloads (download.js): the shown band as stored (.tif); every band, the thumbnail and metadata.json (.zip); a
55
  Python snippet that reads the same row group from Hugging Face
56
  in the background: the other datasets' footprints once the first is drawn; after a click, the same cell's samples
 
67
  | `samples.js` | one sample's thumbnail from its image file |
68
  | `bands.js` | a sample's band columns: read, decode (geotiff.js), stretch, composites |
69
  | `download.js` | band GeoTIFFs as stored, all bands as a zip (fflate), the Python snippet |
70
+ | `weather.js` | Core-Weather rows of one acquisition: hourly (±24 h) and daily (±30 d) tables, one SVG chart per variable |
71
+ | `viirs.js` | one VIIRS tile from its zip shard (one range request), radiance in inferno on a log scale 0.1-100 nW/cm²/sr, corners from the GeoTIFF bounds |
72
+ | `aef.js` | AlphaEarth window of a cell: its COG tiles per year, joined; PCA preview at 160 m, PCA proxy from 8 bands at 10 m, single bands |
73
+ | `zstd.js` | zstd decoder (fzstd) registered in geotiff.js for compression 50000 (VIIRS and AlphaEarth GeoTIFFs); reads nothing |
74
  | `parquet.js` | range requests for hyparquet, with bytes and time counted; each file's footer read once |
75
  | `tiles.js`, `labels.js`, `places.json` | copied from the Climate Globe: cells and tiles, place names |
76
  | `build_index.py` | Core `metadata.parquet` -> `footprints/<name>.parquet` and `footprints/datasets.json` |
 
82
  |---|---|---|
83
  | S2L2A footprints (coverage of 2,245,886 samples) | 3.9 MB in 37 requests | 0.7 s |
84
  | One S2L2A thumbnail, first from its image file (footer 1.5 MB) | 1.1 MB in 4 requests | 1.8 s |
85
+ | Core-Weather for one sample (`weather.js`): `hourly/` and `daily/` day files, source.coop `major-tom/core-weather` | 3.3 MB (LOG.md, 2026-11-03) | not measured |
86
+ | One VIIRS tile (`viirs.js`): the tile's bytes in its zip shard, Hugging Face `Major-TOM/Core-VIIRS-Nighttime-Light` | the tile's size, 1 request | not measured |
87
+ | AlphaEarth preview (`aef.js`): each tile's 160 m level, 64 bands, source.coop `tge-labs/aef` | ~10 MB, 1 request per tile | ~4.5 s |
88
+ | AlphaEarth 10 m PCA proxy (`aef.js`): 8 bands at 10 m, the tiles under the window | not measured | ~15 s |
89
 
90
  Reading each dataset's `metadata.parquet` from Hugging Face instead was measured first: S2L2A's has one row group
91
  per image file (4,492), so its positions took 462 s in 4,492 requests; S1RTC's and DEM's have two row groups, so
92
  one sample's row cost 23-27 MB. Hence the footprint files.
93
 
94
+ The AlphaEarth figures are from the comments in `aef.js` and the flow above, not a fresh measurement.
95
+
96
+ ## Private Space: every local module is imported by app.js
97
+
98
+ On the private Space the app runs in an iframe, where the login cookie is not sent. A file is served only if its
99
+ URL or its referer carries the `?__sign=` token. Only `index.html` and `app.js` carry it, so a module imported only
100
+ by another module gets a 401 and the app does not start. Every local module is therefore imported directly by
101
+ `app.js` (`app.js:18-29`; `zstd.js` is imported there as well as by `viirs.js` and `aef.js`).
102
+ `tests/test_imports.py` checks this without the network.
103
+
104
  ## Run locally
105
 
106
  ```
 
110
 
111
  The page needs a server that answers HTTP range requests (the tests' server does; `python -m http.server` does not).
112
 
113
+ ## Tests
114
+
115
+ | File | What it checks | Needs |
116
+ |---|---|---|
117
+ | `tests/test_imports.py` | every `./x.js` imported by another module is also imported by `app.js` | nothing (`pytest tests/test_imports.py`) |
118
+ | `tests/test_app.py` | headless Chromium (Playwright) at Loch Lomond (624U_29L): thumbnails, VIIRS, AlphaEarth PCA, cursor, background loads, full window, image on the map, Core-Weather, bands and composites, S1 and DEM views, downloads, loading flag (13 tests) | network, `footprints/` built (skipped otherwise) |
119
+
120
+ `pytest -m network` runs `test_app.py` only; `pytest` runs both.
121
+
122
  ## Credit
123
 
124
  Developed by [Mikolaj Czerkawski](https://github.com/mikonvergence), [Asterisk Labs](https://asterisk.coop), as part of