Hugging Face's logo

Spaces:
DataMuncher-Labs
/
AutoWS
Running

App Files Community
1
Roman190928 commited on
Commit
09a0929
·
verified ·
1 Parent(s): 60eb7e9

Raise max shard cap to 15 in crawler config

Browse files
Files changed (1) hide show
  1. crawler/config.py +1 -1
crawler/config.py CHANGED
@@ -6,7 +6,7 @@ from pathlib import Path
6
  NORMAL_TOTAL_WORKERS = 12
7
  SUPER_TOTAL_WORKERS = 24
8
  MAX_SHARD_ROWS = 15_000
9
- MAX_SHARDS = 10
10
 
11
 
12
  def validate_total_workers(total_workers: int) -> int:
 
6
  NORMAL_TOTAL_WORKERS = 12
7
  SUPER_TOTAL_WORKERS = 24
8
  MAX_SHARD_ROWS = 15_000
9
+ MAX_SHARDS = 15
10
 
11
 
12
  def validate_total_workers(total_workers: int) -> int: