anhtld commited on
Commit
c679031
·
verified ·
1 Parent(s): 090513a

sync final logs after metric artifact update

Browse files
workspace/outputs/hf_sync/hf_push_daemon_login.out CHANGED
@@ -4501,3 +4501,9 @@ Traceback (most recent call last):
4501
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4502
  OSError: I/O error: I/O error: Cannot send after transport endpoint shutdown (os error 108)
4503
  [2026-07-04T04:19:22Z] sync cycle failed with status=1
 
 
 
 
 
 
 
4501
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4502
  OSError: I/O error: I/O error: Cannot send after transport endpoint shutdown (os error 108)
4503
  [2026-07-04T04:19:22Z] sync cycle failed with status=1
4504
+ [2026-07-04T04:34:22Z] checking Hugging Face auth
4505
+ [2026-07-04T04:34:25Z] sync start repo=anhtld/vla type=model dry_run=0
4506
+ [2026-07-04T04:34:26Z] upload /lustre09/project/6037638/knguy52/vla -> anhtld/vla/workspace
4507
+ /lustre09/project/6037638/knguy52/vla/.venv/lib/python3.11/site-packages/huggingface_hub/hf_api.py:11299: UserWarning: Warnings while validating metadata in README.md:
4508
+ - empty or missing yaml metadata in repo card
4509
+ warnings.warn(f"Warnings while validating metadata in README.md:\n{message}")
workspace/outputs/hf_sync/hf_push_every_15m.setsid.log CHANGED
@@ -2158,3 +2158,46 @@ Uploading... 256/31,202 files checked, 0/0 uploaded (0.00B transferred), 0 commi
2158
  Uploading... 256/31,202 files checked, 0/0 uploaded (0.00B transferred), 0 committed in 0 commit(s)
2159
  Uploading... 256/31,202 files checked, 0/0 uploaded (0.00B transferred), 0 committed in 0 commit(s)
2160
  Uploading... 256/31,202 files checked, 0/0 uploaded (0.00B transferred), 0 committed in 0 commit(s)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2158
  Uploading... 256/31,202 files checked, 0/0 uploaded (0.00B transferred), 0 committed in 0 commit(s)
2159
  Uploading... 256/31,202 files checked, 0/0 uploaded (0.00B transferred), 0 committed in 0 commit(s)
2160
  Uploading... 256/31,202 files checked, 0/0 uploaded (0.00B transferred), 0 committed in 0 commit(s)
2161
+ Uploading... 25,856/31,202 files checked, 162/162 uploaded (0.00B transferred), 25,750 committed in 2 commit(s)
2162
+ Upload completed in 8 commits.
2163
+ url=https://huggingface.co/anhtld/vla/commit/52f42ba921bd701ba0bc4720826e314b4f7365fc
2164
+ [2026-07-04T04:33:15Z] upload /scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_runs -> anhtld/vla/scratch/experiments/dovla_h16_policy_ckpt_runs
2165
+ Found 1,276 files to upload
2166
+ Uploading... 256/1,276 files checked, 0/0 uploaded (0.00B transferred), 0 committed in 0 commit(s)
2167
+ No files have been modified since last commit. Skipping to prevent empty commit.
2168
+ url=https://huggingface.co/anhtld/vla/commit/f3fd10f6aa3790eb2df5acd9ec39d69ffd767a08
2169
+ [2026-07-04T04:33:47Z] upload /scratch/knguy52/dovla/experiments/dovla_h16_rollout_runs -> anhtld/vla/scratch/experiments/dovla_h16_rollout_runs
2170
+ Found 51 files to upload
2171
+ Uploading... 51/51 files checked, 0/0 uploaded (0.00B transferred), 0 committed in 0 commit(s)
2172
+ No files have been modified since last commit. Skipping to prevent empty commit.
2173
+ url=https://huggingface.co/anhtld/vla/commit/f3fd10f6aa3790eb2df5acd9ec39d69ffd767a08
2174
+ [2026-07-04T04:33:52Z] upload /scratch/knguy52/dovla/experiments/h16_merged_dataset -> anhtld/vla/scratch/experiments/h16_merged_dataset
2175
+ Found 49 files to upload
2176
+ Uploading... 49/49 files checked, 0/0 uploaded (0.00B transferred), 0 committed in 0 commit(s)
2177
+ No files have been modified since last commit. Skipping to prevent empty commit.
2178
+ url=https://huggingface.co/anhtld/vla/commit/f3fd10f6aa3790eb2df5acd9ec39d69ffd767a08
2179
+ [2026-07-04T04:33:56Z] upload /scratch/knguy52/dovla/experiments/six_task_h16_collection -> anhtld/vla/scratch/experiments/six_task_h16_collection
2180
+ Found 87 files to upload
2181
+ Uploading... 87/87 files checked, 0/0 uploaded (0.00B transferred), 0 committed in 0 commit(s)
2182
+ No files have been modified since last commit. Skipping to prevent empty commit.
2183
+ url=https://huggingface.co/anhtld/vla/commit/f3fd10f6aa3790eb2df5acd9ec39d69ffd767a08
2184
+ [2026-07-04T04:34:04Z] upload /scratch/knguy52/dovla/experiments/embedding_cache -> anhtld/vla/scratch/experiments/embedding_cache
2185
+ Found 0 files to upload
2186
+ No files have been modified since last commit. Skipping to prevent empty commit.
2187
+ url=https://huggingface.co/anhtld/vla/commit/f3fd10f6aa3790eb2df5acd9ec39d69ffd767a08
2188
+ [2026-07-04T04:34:04Z] upload /scratch/knguy52/dovla/cache -> anhtld/vla/scratch/cache
2189
+ Found 10 files to upload
2190
+ Uploading... 10/10 files checked, 0/0 uploaded (0.00B transferred), 0 committed in 0 commit(s)
2191
+ No files have been modified since last commit. Skipping to prevent empty commit.
2192
+ url=https://huggingface.co/anhtld/vla/commit/f3fd10f6aa3790eb2df5acd9ec39d69ffd767a08
2193
+ [2026-07-04T04:34:29Z] upload /scratch/knguy52/dovla/caches -> anhtld/vla/scratch/caches
2194
+ Found 3 files to upload
2195
+ Uploading... 3/3 files checked, 0/0 uploaded (0.00B transferred), 0 committed in 0 commit(s)
2196
+ No files have been modified since last commit. Skipping to prevent empty commit.
2197
+ url=https://huggingface.co/anhtld/vla/commit/f3fd10f6aa3790eb2df5acd9ec39d69ffd767a08
2198
+ [2026-07-04T04:34:42Z] upload /home/knguy52/links/projects/def-yalda/knguy52/vla/outputs/hf_sync/last_manifest.json -> anhtld/vla/manifests/hf_sync_last_manifest.json
2199
+ /lustre09/project/6037638/knguy52/vla/.venv/lib/python3.11/site-packages/huggingface_hub/cli/upload.py:189: UserWarning: Ignoring --exclude since a single file is uploaded.
2200
+ warnings.warn("Ignoring --exclude since a single file is uploaded.")
2201
+ url=https://huggingface.co/anhtld/vla/commit/090513a59075fc17c01bab4375ab722e1c81976f
2202
+ [2026-07-04T04:34:48Z] sync complete
2203
+ [2026-07-04T04:34:48Z] sync cycle succeeded
workspace/outputs/hf_sync/hf_sync.log CHANGED
@@ -1324,3 +1324,24 @@
1324
  [2026-07-04T04:30:21Z] checking Hugging Face auth
1325
  [2026-07-04T04:30:24Z] sync start repo=anhtld/vla type=model dry_run=0
1326
  [2026-07-04T04:30:25Z] upload /lustre09/project/6037638/knguy52/vla -> anhtld/vla/workspace
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1324
  [2026-07-04T04:30:21Z] checking Hugging Face auth
1325
  [2026-07-04T04:30:24Z] sync start repo=anhtld/vla type=model dry_run=0
1326
  [2026-07-04T04:30:25Z] upload /lustre09/project/6037638/knguy52/vla -> anhtld/vla/workspace
1327
+ [2026-07-04T04:33:15Z] upload /scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_runs -> anhtld/vla/scratch/experiments/dovla_h16_policy_ckpt_runs
1328
+ [2026-07-04T04:33:15Z] upload /scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_runs -> anhtld/vla/scratch/experiments/dovla_h16_policy_ckpt_runs
1329
+ [2026-07-04T04:33:47Z] upload /scratch/knguy52/dovla/experiments/dovla_h16_rollout_runs -> anhtld/vla/scratch/experiments/dovla_h16_rollout_runs
1330
+ [2026-07-04T04:33:47Z] upload /scratch/knguy52/dovla/experiments/dovla_h16_rollout_runs -> anhtld/vla/scratch/experiments/dovla_h16_rollout_runs
1331
+ [2026-07-04T04:33:52Z] upload /scratch/knguy52/dovla/experiments/h16_merged_dataset -> anhtld/vla/scratch/experiments/h16_merged_dataset
1332
+ [2026-07-04T04:33:52Z] upload /scratch/knguy52/dovla/experiments/h16_merged_dataset -> anhtld/vla/scratch/experiments/h16_merged_dataset
1333
+ [2026-07-04T04:33:56Z] upload /scratch/knguy52/dovla/experiments/six_task_h16_collection -> anhtld/vla/scratch/experiments/six_task_h16_collection
1334
+ [2026-07-04T04:33:56Z] upload /scratch/knguy52/dovla/experiments/six_task_h16_collection -> anhtld/vla/scratch/experiments/six_task_h16_collection
1335
+ [2026-07-04T04:34:03Z] upload /scratch/knguy52/dovla/experiments/embedding_cache -> anhtld/vla/scratch/experiments/embedding_cache
1336
+ [2026-07-04T04:34:04Z] upload /scratch/knguy52/dovla/experiments/embedding_cache -> anhtld/vla/scratch/experiments/embedding_cache
1337
+ [2026-07-04T04:34:04Z] upload /scratch/knguy52/dovla/cache -> anhtld/vla/scratch/cache
1338
+ [2026-07-04T04:34:04Z] upload /scratch/knguy52/dovla/cache -> anhtld/vla/scratch/cache
1339
+ [2026-07-04T04:34:22Z] checking Hugging Face auth
1340
+ [2026-07-04T04:34:25Z] sync start repo=anhtld/vla type=model dry_run=0
1341
+ [2026-07-04T04:34:26Z] upload /lustre09/project/6037638/knguy52/vla -> anhtld/vla/workspace
1342
+ [2026-07-04T04:34:29Z] upload /scratch/knguy52/dovla/caches -> anhtld/vla/scratch/caches
1343
+ [2026-07-04T04:34:29Z] upload /scratch/knguy52/dovla/caches -> anhtld/vla/scratch/caches
1344
+ [2026-07-04T04:34:42Z] upload /lustre09/project/6037638/knguy52/vla/outputs/hf_sync/last_manifest.json -> anhtld/vla/manifests/hf_sync_last_manifest.json
1345
+ [2026-07-04T04:34:42Z] upload /home/knguy52/links/projects/def-yalda/knguy52/vla/outputs/hf_sync/last_manifest.json -> anhtld/vla/manifests/hf_sync_last_manifest.json
1346
+ [2026-07-04T04:34:44Z] sync complete
1347
+ [2026-07-04T04:34:48Z] sync complete
workspace/outputs/hf_sync/hf_sync_daemon.log CHANGED
@@ -121,3 +121,4 @@
121
  [2026-07-04T03:44:08Z] sync cycle succeeded
122
  [2026-07-04T04:13:17Z] sync cycle succeeded
123
  [2026-07-04T04:19:22Z] sync cycle failed with status=1
 
 
121
  [2026-07-04T03:44:08Z] sync cycle succeeded
122
  [2026-07-04T04:13:17Z] sync cycle succeeded
123
  [2026-07-04T04:19:22Z] sync cycle failed with status=1
124
+ [2026-07-04T04:34:48Z] sync cycle succeeded
workspace/outputs/hf_sync/last_manifest.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "generated_utc": "2026-07-04T04:30:25.248708+00:00",
3
  "repo_id": "anhtld/vla",
4
  "project_dir": "/lustre09/project/6037638/knguy52/vla",
5
  "scratch_root": "/scratch/knguy52/dovla",
 
1
  {
2
+ "generated_utc": "2026-07-04T04:34:26.018788+00:00",
3
  "repo_id": "anhtld/vla",
4
  "project_dir": "/lustre09/project/6037638/knguy52/vla",
5
  "scratch_root": "/scratch/knguy52/dovla",