Spaces:

Dataset-Tools
/

pdf-to-page-images-dataset

Running

davanstrien HF Staff commited on Sep 19, 2024

Commit

ca4bcc4

1 Parent(s): 5192410

use large folder upload

Files changed (1) hide show

app.py CHANGED Viewed

@@ -118,11 +118,11 @@ def process_pdfs(
                     repo_type="dataset",
                     private=private_repo,
                 )
-                hf_api.upload_folder(
-                    folder_path=images_dir,
                     repo_id=hf_repo,
                     repo_type="dataset",
-                    path_in_repo="images",
                 )
                 # Determine size category

                     repo_type="dataset",
                     private=private_repo,
                 )
+                hf_api.upload_large_folder(
+                    folder_path=temp_dir,
                     repo_id=hf_repo,
                     repo_type="dataset",
+                    # path_in_repo="images",
                 )
                 # Determine size category