anima_closeout v10: latent caching died on its FIRST image read — the fork's ParquetImageReader whole-file column read cannot deliver >2GiB nested (struct) columns (ArrowNotImplementedError), and the 8192-row split monolith crosses that line. Fixed in the fork (8ed4b6b: per-row-group reads assembled into a ChunkedArray; proven against a 1200x2MB struct parquet reproducing the exact error). CELL 0 asserts the fixed reader pre-spend. No CELL 1 re-run needed: the monolith is readable as-is.
Browse files
colab/anima_closeout.ipynb
CHANGED
|
@@ -123,6 +123,11 @@
|
|
| 123 |
"# crash, ModuleNotFoundError: hyvideo \u2014 the recorded exp004 + smoke killer)\n",
|
| 124 |
"assert \"_HAS_HYVIDEO\" in (FORK / \"utils\" / \"patches.py\").read_text(), \\\n",
|
| 125 |
" \"fork main lacks the guarded hyvideo import (utils/patches.py)\"\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 126 |
"\n",
|
| 127 |
"if not ANIMA.get(\"DEPS_DONE\"):\n",
|
| 128 |
" sh(f\"pip install -q -r {FORK}/requirements.txt\")\n",
|
|
|
|
| 123 |
"# crash, ModuleNotFoundError: hyvideo \u2014 the recorded exp004 + smoke killer)\n",
|
| 124 |
"assert \"_HAS_HYVIDEO\" in (FORK / \"utils\" / \"patches.py\").read_text(), \\\n",
|
| 125 |
" \"fork main lacks the guarded hyvideo import (utils/patches.py)\"\n",
|
| 126 |
+
"# the 8192-row split monolith needs the per-row-group image reader (fork\n",
|
| 127 |
+
"# 8ed4b6b) \u2014 the whole-file read dies on >2GiB nested columns\n",
|
| 128 |
+
"assert \"pa.chunked_array(chunks)\" in \\\n",
|
| 129 |
+
" (FORK / \"utils\" / \"parquet_source.py\").read_text(), \\\n",
|
| 130 |
+
" \"fork main lacks the per-row-group ParquetImageReader (parquet_source.py)\"\n",
|
| 131 |
"\n",
|
| 132 |
"if not ANIMA.get(\"DEPS_DONE\"):\n",
|
| 133 |
" sh(f\"pip install -q -r {FORK}/requirements.txt\")\n",
|