Hugging Face's logo

Spaces:
kseniks
/
captcha_pixelplanet
Runtime error

App Files Community
kseniks sneedium commited on
Commit
7c2fae3
·
0 Parent(s):

Duplicate from sneedium/captcha_pixelplanet

Browse files

Co-authored-by: cope bro <sneedium@users.noreply.huggingface.co>

This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .config/.last_opt_in_prompt.yaml +1 -0
  2. .config/.last_survey_prompt.yaml +1 -0
  3. .config/.last_update_check.json +1 -0
  4. .config/active_config +1 -0
  5. .config/config_sentinel +0 -0
  6. .config/configurations/config_default +6 -0
  7. .config/gce +1 -0
  8. .config/logs/2022.09.14/13.43.12.857966.log +503 -0
  9. .config/logs/2022.09.14/13.43.34.754807.log +4 -0
  10. .config/logs/2022.09.14/13.43.54.433400.log +165 -0
  11. .config/logs/2022.09.14/13.44.02.480346.log +4 -0
  12. .config/logs/2022.09.14/13.44.22.887116.log +7 -0
  13. .config/logs/2022.09.14/13.44.23.609544.log +7 -0
  14. README.md +12 -0
  15. app.py +36 -0
  16. callbacks.py +360 -0
  17. configs/.gitattributes +31 -0
  18. configs/template.yaml +67 -0
  19. configs/train_iternet.yaml +65 -0
  20. data/charset_36.txt +36 -0
  21. dataset.py +278 -0
  22. demo.py +109 -0
  23. figs_captchas/1fkL.jpg +0 -0
  24. figs_captchas/4QS3.jpg +0 -0
  25. figs_captchas/4eLO.jpg +0 -0
  26. figs_captchas/5UZB.jpg +0 -0
  27. figs_captchas/6H3e.jpg +0 -0
  28. figs_captchas/8Kgm.jpg +0 -0
  29. figs_captchas/8sAX.jpg +0 -0
  30. figs_captchas/944m.jpg +0 -0
  31. figs_captchas/ArYq.jpg +0 -0
  32. figs_captchas/EH65.jpg +0 -0
  33. figs_captchas/GIQR.jpg +0 -0
  34. figs_captchas/GYyo.jpg +0 -0
  35. figs_captchas/H184.jpg +0 -0
  36. figs_captchas/HSv8.jpg +0 -0
  37. figs_captchas/JRRW.jpg +0 -0
  38. figs_captchas/L24g.jpg +0 -0
  39. figs_captchas/MIly.jpg +0 -0
  40. figs_captchas/NCpP.jpg +0 -0
  41. figs_captchas/NshS.jpg +0 -0
  42. figs_captchas/R0cv.jpg +0 -0
  43. figs_captchas/ReEH.jpg +0 -0
  44. figs_captchas/Sp87.jpg +0 -0
  45. figs_captchas/TJll.jpg +0 -0
  46. figs_captchas/TwO2.jpg +0 -0
  47. figs_captchas/Uk9I.jpg +0 -0
  48. figs_captchas/VDke.jpg +0 -0
  49. figs_captchas/VKft.jpg +0 -0
  50. figs_captchas/ZEmP.jpg +0 -0
.config/.last_opt_in_prompt.yaml ADDED
@@ -0,0 +1 @@
 
 
1
+ {}
.config/.last_survey_prompt.yaml ADDED
@@ -0,0 +1 @@
 
 
1
+ last_prompt_time: 1663163033.713858
.config/.last_update_check.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"last_update_check_time": 1663163041.8911226, "last_update_check_revision": 20220912133630, "notifications": [], "last_nag_times": {}}
.config/active_config ADDED
@@ -0,0 +1 @@
 
 
1
+ default
.config/config_sentinel ADDED
File without changes
.config/configurations/config_default ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ [component_manager]
2
+ disable_update_check = true
3
+
4
+ [compute]
5
+ gce_metadata_read_timeout_sec = 0
6
+
.config/gce ADDED
@@ -0,0 +1 @@
 
 
1
+ False
.config/logs/2022.09.14/13.43.12.857966.log ADDED
@@ -0,0 +1,503 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 2022-09-14 13:43:12,862 DEBUG root Loaded Command Group: ['gcloud', 'components']
2
+ 2022-09-14 13:43:12,866 DEBUG root Loaded Command Group: ['gcloud', 'components', 'update']
3
+ 2022-09-14 13:43:12,868 DEBUG root Running [gcloud.components.update] with arguments: [--allow-no-backup: "True", --quiet: "True", COMPONENT-IDS:7: "['core', 'gcloud-deps', 'bq', 'gcloud', 'gcloud-crc32c', 'gsutil', 'anthoscli']"]
4
+ 2022-09-14 13:43:12,875 INFO ___FILE_ONLY___ Beginning update. This process may take several minutes.
5
+
6
+ 2022-09-14 13:43:24,906 DEBUG urllib3.connectionpool Starting new HTTPS connection (1): dl.google.com:443
7
+ 2022-09-14 13:43:24,995 DEBUG urllib3.connectionpool https://dl.google.com:443 "GET /dl/cloudsdk/channels/rapid/components-2.json HTTP/1.1" 200 197751
8
+ 2022-09-14 13:43:25,045 INFO ___FILE_ONLY___
9
+
10
+ 2022-09-14 13:43:25,045 INFO ___FILE_ONLY___
11
+ Your current Google Cloud CLI version is: 402.0.0
12
+
13
+ 2022-09-14 13:43:25,045 INFO ___FILE_ONLY___ Installing components from version: 402.0.0
14
+
15
+ 2022-09-14 13:43:25,045 INFO ___FILE_ONLY___
16
+
17
+ 2022-09-14 13:43:25,052 INFO ___FILE_ONLY___ ┌─────────────────────────────────────────────────────────────────────────────┐
18
+ 2022-09-14 13:43:25,052 INFO ___FILE_ONLY___
19
+
20
+ 2022-09-14 13:43:25,052 INFO ___FILE_ONLY___ │ These components will be installed. │
21
+ 2022-09-14 13:43:25,052 INFO ___FILE_ONLY___
22
+
23
+ 2022-09-14 13:43:25,053 INFO ___FILE_ONLY___ ├─────────────────────────────────────────────────────┬────────────┬──────────┤
24
+ 2022-09-14 13:43:25,053 INFO ___FILE_ONLY___
25
+
26
+ 2022-09-14 13:43:25,053 INFO ___FILE_ONLY___ │ Name │ Version │ Size │
27
+ 2022-09-14 13:43:25,053 INFO ___FILE_ONLY___
28
+
29
+ 2022-09-14 13:43:25,053 INFO ___FILE_ONLY___ ├─────────────────────────────────────────────────────┼────────────┼──────────┤
30
+ 2022-09-14 13:43:25,053 INFO ___FILE_ONLY___
31
+
32
+ 2022-09-14 13:43:25,053 INFO ___FILE_ONLY___ │
33
+ 2022-09-14 13:43:25,053 INFO ___FILE_ONLY___ BigQuery Command Line Tool
34
+ 2022-09-14 13:43:25,053 INFO ___FILE_ONLY___
35
+ 2022-09-14 13:43:25,053 INFO ___FILE_ONLY___ │
36
+ 2022-09-14 13:43:25,053 INFO ___FILE_ONLY___ 2.0.75
37
+ 2022-09-14 13:43:25,053 INFO ___FILE_ONLY___
38
+ 2022-09-14 13:43:25,053 INFO ___FILE_ONLY___ │
39
+ 2022-09-14 13:43:25,053 INFO ___FILE_ONLY___ 1.6 MiB
40
+ 2022-09-14 13:43:25,054 INFO ___FILE_ONLY___
41
+ 2022-09-14 13:43:25,054 INFO ___FILE_ONLY___ │
42
+ 2022-09-14 13:43:25,054 INFO ___FILE_ONLY___
43
+
44
+ 2022-09-14 13:43:25,054 INFO ___FILE_ONLY___ │
45
+ 2022-09-14 13:43:25,054 INFO ___FILE_ONLY___ BigQuery Command Line Tool (Platform Specific)
46
+ 2022-09-14 13:43:25,054 INFO ___FILE_ONLY___
47
+ 2022-09-14 13:43:25,054 INFO ___FILE_ONLY___ │
48
+ 2022-09-14 13:43:25,054 INFO ___FILE_ONLY___ 2.0.75
49
+ 2022-09-14 13:43:25,054 INFO ___FILE_ONLY___
50
+ 2022-09-14 13:43:25,054 INFO ___FILE_ONLY___ │
51
+ 2022-09-14 13:43:25,054 INFO ___FILE_ONLY___ < 1 MiB
52
+ 2022-09-14 13:43:25,054 INFO ___FILE_ONLY___
53
+ 2022-09-14 13:43:25,054 INFO ___FILE_ONLY___ │
54
+ 2022-09-14 13:43:25,054 INFO ___FILE_ONLY___
55
+
56
+ 2022-09-14 13:43:25,055 INFO ___FILE_ONLY___ │
57
+ 2022-09-14 13:43:25,055 INFO ___FILE_ONLY___ Cloud Storage Command Line Tool
58
+ 2022-09-14 13:43:25,055 INFO ___FILE_ONLY___
59
+ 2022-09-14 13:43:25,055 INFO ___FILE_ONLY___ │
60
+ 2022-09-14 13:43:25,055 INFO ___FILE_ONLY___ 5.13
61
+ 2022-09-14 13:43:25,055 INFO ___FILE_ONLY___
62
+ 2022-09-14 13:43:25,055 INFO ___FILE_ONLY___ │
63
+ 2022-09-14 13:43:25,055 INFO ___FILE_ONLY___ 15.5 MiB
64
+ 2022-09-14 13:43:25,055 INFO ___FILE_ONLY___
65
+ 2022-09-14 13:43:25,055 INFO ___FILE_ONLY___ │
66
+ 2022-09-14 13:43:25,055 INFO ___FILE_ONLY___
67
+
68
+ 2022-09-14 13:43:25,055 INFO ___FILE_ONLY___ │
69
+ 2022-09-14 13:43:25,055 INFO ___FILE_ONLY___ Cloud Storage Command Line Tool (Platform Specific)
70
+ 2022-09-14 13:43:25,055 INFO ___FILE_ONLY___
71
+ 2022-09-14 13:43:25,056 INFO ___FILE_ONLY___ │
72
+ 2022-09-14 13:43:25,056 INFO ___FILE_ONLY___ 5.11
73
+ 2022-09-14 13:43:25,056 INFO ___FILE_ONLY___
74
+ 2022-09-14 13:43:25,056 INFO ___FILE_ONLY___ │
75
+ 2022-09-14 13:43:25,056 INFO ___FILE_ONLY___ < 1 MiB
76
+ 2022-09-14 13:43:25,056 INFO ___FILE_ONLY___
77
+ 2022-09-14 13:43:25,056 INFO ___FILE_ONLY___ │
78
+ 2022-09-14 13:43:25,056 INFO ___FILE_ONLY___
79
+
80
+ 2022-09-14 13:43:25,056 INFO ___FILE_ONLY___ │
81
+ 2022-09-14 13:43:25,056 INFO ___FILE_ONLY___ Google Cloud CLI Core Libraries (Platform Specific)
82
+ 2022-09-14 13:43:25,056 INFO ___FILE_ONLY___
83
+ 2022-09-14 13:43:25,056 INFO ___FILE_ONLY___ │
84
+ 2022-09-14 13:43:25,056 INFO ___FILE_ONLY___ 2022.08.05
85
+ 2022-09-14 13:43:25,056 INFO ___FILE_ONLY___
86
+ 2022-09-14 13:43:25,057 INFO ___FILE_ONLY___ │
87
+ 2022-09-14 13:43:25,057 INFO ___FILE_ONLY___ < 1 MiB
88
+ 2022-09-14 13:43:25,057 INFO ___FILE_ONLY___
89
+ 2022-09-14 13:43:25,057 INFO ___FILE_ONLY___ │
90
+ 2022-09-14 13:43:25,057 INFO ___FILE_ONLY___
91
+
92
+ 2022-09-14 13:43:25,057 INFO ___FILE_ONLY___ │
93
+ 2022-09-14 13:43:25,057 INFO ___FILE_ONLY___ Google Cloud CRC32C Hash Tool
94
+ 2022-09-14 13:43:25,057 INFO ___FILE_ONLY___
95
+ 2022-09-14 13:43:25,057 INFO ___FILE_ONLY___ │
96
+ 2022-09-14 13:43:25,057 INFO ___FILE_ONLY___ 1.0.0
97
+ 2022-09-14 13:43:25,057 INFO ___FILE_ONLY___
98
+ 2022-09-14 13:43:25,057 INFO ___FILE_ONLY___ │
99
+ 2022-09-14 13:43:25,057 INFO ___FILE_ONLY___ 1.2 MiB
100
+ 2022-09-14 13:43:25,057 INFO ___FILE_ONLY___
101
+ 2022-09-14 13:43:25,058 INFO ___FILE_ONLY___ │
102
+ 2022-09-14 13:43:25,058 INFO ___FILE_ONLY___
103
+
104
+ 2022-09-14 13:43:25,058 INFO ___FILE_ONLY___ │
105
+ 2022-09-14 13:43:25,058 INFO ___FILE_ONLY___ anthoscli
106
+ 2022-09-14 13:43:25,058 INFO ___FILE_ONLY___
107
+ 2022-09-14 13:43:25,058 INFO ___FILE_ONLY___ │
108
+ 2022-09-14 13:43:25,058 INFO ___FILE_ONLY___ 0.2.29
109
+ 2022-09-14 13:43:25,058 INFO ___FILE_ONLY___
110
+ 2022-09-14 13:43:25,058 INFO ___FILE_ONLY___ │
111
+ 2022-09-14 13:43:25,058 INFO ___FILE_ONLY___ 37.6 MiB
112
+ 2022-09-14 13:43:25,058 INFO ___FILE_ONLY___
113
+ 2022-09-14 13:43:25,058 INFO ___FILE_ONLY___ │
114
+ 2022-09-14 13:43:25,058 INFO ___FILE_ONLY___
115
+
116
+ 2022-09-14 13:43:25,058 INFO ___FILE_ONLY___ │
117
+ 2022-09-14 13:43:25,058 INFO ___FILE_ONLY___ gcloud cli dependencies
118
+ 2022-09-14 13:43:25,059 INFO ___FILE_ONLY___
119
+ 2022-09-14 13:43:25,059 INFO ___FILE_ONLY___ │
120
+ 2022-09-14 13:43:25,059 INFO ___FILE_ONLY___ 2021.04.16
121
+ 2022-09-14 13:43:25,059 INFO ___FILE_ONLY___
122
+ 2022-09-14 13:43:25,059 INFO ___FILE_ONLY___ │
123
+ 2022-09-14 13:43:25,059 INFO ___FILE_ONLY___ < 1 MiB
124
+ 2022-09-14 13:43:25,059 INFO ___FILE_ONLY___
125
+ 2022-09-14 13:43:25,059 INFO ___FILE_ONLY___ │
126
+ 2022-09-14 13:43:25,059 INFO ___FILE_ONLY___
127
+
128
+ 2022-09-14 13:43:25,059 INFO ___FILE_ONLY___ └─────────────────────────────────────────────────────┴────────────┴──────────┘
129
+ 2022-09-14 13:43:25,059 INFO ___FILE_ONLY___
130
+
131
+ 2022-09-14 13:43:25,059 INFO ___FILE_ONLY___
132
+
133
+ 2022-09-14 13:43:25,062 DEBUG urllib3.connectionpool Starting new HTTPS connection (1): dl.google.com:443
134
+ 2022-09-14 13:43:25,093 DEBUG urllib3.connectionpool https://dl.google.com:443 "GET /dl/cloudsdk/channels/rapid/RELEASE_NOTES HTTP/1.1" 200 867084
135
+ 2022-09-14 13:43:25,185 INFO ___FILE_ONLY___ For the latest full release notes, please visit:
136
+ https://cloud.google.com/sdk/release_notes
137
+
138
+
139
+ 2022-09-14 13:43:25,191 INFO ___FILE_ONLY___ ╔════════════════════════════════════════════════════════════╗
140
+
141
+ 2022-09-14 13:43:25,191 INFO ___FILE_ONLY___ ╠═ Creating update staging area ═╣
142
+
143
+ 2022-09-14 13:43:25,191 INFO ___FILE_ONLY___ ╚
144
+ 2022-09-14 13:43:25,192 INFO ___FILE_ONLY___ ══════
145
+ 2022-09-14 13:43:25,192 INFO ___FILE_ONLY___ ══════
146
+ 2022-09-14 13:43:25,192 INFO ___FILE_ONLY___ ══════
147
+ 2022-09-14 13:43:25,441 INFO ___FILE_ONLY___ ═
148
+ 2022-09-14 13:43:25,497 INFO ___FILE_ONLY___ ═
149
+ 2022-09-14 13:43:25,543 INFO ___FILE_ONLY___ ═
150
+ 2022-09-14 13:43:25,605 INFO ___FILE_ONLY___ ═
151
+ 2022-09-14 13:43:25,656 INFO ___FILE_ONLY___ ═
152
+ 2022-09-14 13:43:25,707 INFO ___FILE_ONLY___ ═
153
+ 2022-09-14 13:43:25,761 INFO ___FILE_ONLY___ ═
154
+ 2022-09-14 13:43:25,817 INFO ___FILE_ONLY___ ═
155
+ 2022-09-14 13:43:25,892 INFO ___FILE_ONLY___ ═
156
+ 2022-09-14 13:43:25,953 INFO ___FILE_ONLY___ ═
157
+ 2022-09-14 13:43:26,013 INFO ___FILE_ONLY___ ═
158
+ 2022-09-14 13:43:26,068 INFO ___FILE_ONLY___ ═
159
+ 2022-09-14 13:43:26,121 INFO ___FILE_ONLY___ ═
160
+ 2022-09-14 13:43:26,177 INFO ___FILE_ONLY___ ═
161
+ 2022-09-14 13:43:26,230 INFO ___FILE_ONLY___ ═
162
+ 2022-09-14 13:43:26,358 INFO ___FILE_ONLY___ ═
163
+ 2022-09-14 13:43:26,415 INFO ___FILE_ONLY___ ═
164
+ 2022-09-14 13:43:26,594 INFO ___FILE_ONLY___ ═
165
+ 2022-09-14 13:43:26,669 INFO ___FILE_ONLY___ ═
166
+ 2022-09-14 13:43:26,746 INFO ___FILE_ONLY___ ═
167
+ 2022-09-14 13:43:26,786 INFO ___FILE_ONLY___ ═
168
+ 2022-09-14 13:43:26,822 INFO ___FILE_ONLY___ ═
169
+ 2022-09-14 13:43:26,852 INFO ___FILE_ONLY___ ═
170
+ 2022-09-14 13:43:26,882 INFO ___FILE_ONLY___ ═
171
+ 2022-09-14 13:43:26,911 INFO ___FILE_ONLY___ ═
172
+ 2022-09-14 13:43:26,942 INFO ___FILE_ONLY___ ═
173
+ 2022-09-14 13:43:27,036 INFO ___FILE_ONLY___ ═
174
+ 2022-09-14 13:43:27,161 INFO ___FILE_ONLY___ ═
175
+ 2022-09-14 13:43:27,224 INFO ___FILE_ONLY___ ═
176
+ 2022-09-14 13:43:27,270 INFO ___FILE_ONLY___ ═
177
+ 2022-09-14 13:43:27,350 INFO ___FILE_ONLY___ ═
178
+ 2022-09-14 13:43:27,432 INFO ___FILE_ONLY___ ═
179
+ 2022-09-14 13:43:27,489 INFO ___FILE_ONLY___ ═
180
+ 2022-09-14 13:43:27,550 INFO ___FILE_ONLY___ ═
181
+ 2022-09-14 13:43:27,627 INFO ___FILE_ONLY___ ═
182
+ 2022-09-14 13:43:27,687 INFO ___FILE_ONLY___ ═
183
+ 2022-09-14 13:43:27,767 INFO ___FILE_ONLY___ ═
184
+ 2022-09-14 13:43:27,828 INFO ___FILE_ONLY___ ═
185
+ 2022-09-14 13:43:27,865 INFO ___FILE_ONLY___ ═
186
+ 2022-09-14 13:43:27,918 INFO ___FILE_ONLY___ ═
187
+ 2022-09-14 13:43:27,974 INFO ___FILE_ONLY___ ═
188
+ 2022-09-14 13:43:28,015 INFO ___FILE_ONLY___ ═
189
+ 2022-09-14 13:43:28,015 INFO ___FILE_ONLY___ ╝
190
+
191
+ 2022-09-14 13:43:28,116 INFO ___FILE_ONLY___ ╔════════════════════════════════════════════════════════════╗
192
+
193
+ 2022-09-14 13:43:28,116 INFO ___FILE_ONLY___ ╠═ Installing: BigQuery Command Line Tool ═╣
194
+
195
+ 2022-09-14 13:43:28,117 INFO ___FILE_ONLY___ ╚
196
+ 2022-09-14 13:43:28,121 DEBUG urllib3.connectionpool Starting new HTTPS connection (1): dl.google.com:443
197
+ 2022-09-14 13:43:28,216 DEBUG urllib3.connectionpool https://dl.google.com:443 "GET /dl/cloudsdk/channels/rapid/components/google-cloud-sdk-bq-20220826172526.tar.gz HTTP/1.1" 200 1649904
198
+ 2022-09-14 13:43:28,301 INFO ___FILE_ONLY___ ═
199
+ 2022-09-14 13:43:28,301 INFO ___FILE_ONLY___ ═
200
+ 2022-09-14 13:43:28,302 INFO ___FILE_ONLY___ ═
201
+ 2022-09-14 13:43:28,302 INFO ___FILE_ONLY___ ═
202
+ 2022-09-14 13:43:28,302 INFO ___FILE_ONLY___ ═
203
+ 2022-09-14 13:43:28,302 INFO ___FILE_ONLY___ ═
204
+ 2022-09-14 13:43:28,302 INFO ___FILE_ONLY___ ═
205
+ 2022-09-14 13:43:28,302 INFO ___FILE_ONLY___ ═
206
+ 2022-09-14 13:43:28,303 INFO ___FILE_ONLY___ ═
207
+ 2022-09-14 13:43:28,303 INFO ___FILE_ONLY___ ═
208
+ 2022-09-14 13:43:28,303 INFO ___FILE_ONLY___ ═
209
+ 2022-09-14 13:43:28,303 INFO ___FILE_ONLY___ ═
210
+ 2022-09-14 13:43:28,303 INFO ___FILE_ONLY___ ═
211
+ 2022-09-14 13:43:28,303 INFO ___FILE_ONLY___ ═
212
+ 2022-09-14 13:43:28,303 INFO ___FILE_ONLY___ ═
213
+ 2022-09-14 13:43:28,304 INFO ___FILE_ONLY___ ═
214
+ 2022-09-14 13:43:28,304 INFO ___FILE_ONLY___ ═
215
+ 2022-09-14 13:43:28,304 INFO ___FILE_ONLY___ ═
216
+ 2022-09-14 13:43:28,304 INFO ___FILE_ONLY___ ═
217
+ 2022-09-14 13:43:28,304 INFO ___FILE_ONLY___ ═
218
+ 2022-09-14 13:43:28,304 INFO ___FILE_ONLY___ ═
219
+ 2022-09-14 13:43:28,305 INFO ___FILE_ONLY___ ═
220
+ 2022-09-14 13:43:28,305 INFO ___FILE_ONLY___ ═
221
+ 2022-09-14 13:43:28,305 INFO ___FILE_ONLY___ ═
222
+ 2022-09-14 13:43:28,305 INFO ___FILE_ONLY___ ═
223
+ 2022-09-14 13:43:28,305 INFO ___FILE_ONLY___ ═
224
+ 2022-09-14 13:43:28,305 INFO ___FILE_ONLY___ ═
225
+ 2022-09-14 13:43:28,305 INFO ___FILE_ONLY___ ═
226
+ 2022-09-14 13:43:28,306 INFO ___FILE_ONLY___ ═
227
+ 2022-09-14 13:43:28,306 INFO ___FILE_ONLY___ ═
228
+ 2022-09-14 13:43:28,430 INFO ___FILE_ONLY___ ═
229
+ 2022-09-14 13:43:28,434 INFO ___FILE_ONLY___ ═
230
+ 2022-09-14 13:43:28,439 INFO ___FILE_ONLY___ ═
231
+ 2022-09-14 13:43:28,446 INFO ___FILE_ONLY___ ═
232
+ 2022-09-14 13:43:28,450 INFO ___FILE_ONLY___ ═
233
+ 2022-09-14 13:43:28,454 INFO ___FILE_ONLY___ ═
234
+ 2022-09-14 13:43:28,458 INFO ___FILE_ONLY___ ═
235
+ 2022-09-14 13:43:28,463 INFO ___FILE_ONLY___ ═
236
+ 2022-09-14 13:43:28,467 INFO ___FILE_ONLY___ ═
237
+ 2022-09-14 13:43:28,472 INFO ___FILE_ONLY___ ═
238
+ 2022-09-14 13:43:28,476 INFO ___FILE_ONLY___ ═
239
+ 2022-09-14 13:43:28,481 INFO ___FILE_ONLY___ ═
240
+ 2022-09-14 13:43:28,487 INFO ___FILE_ONLY___ ═
241
+ 2022-09-14 13:43:28,493 INFO ___FILE_ONLY___ ═
242
+ 2022-09-14 13:43:28,498 INFO ___FILE_ONLY___ ═
243
+ 2022-09-14 13:43:28,502 INFO ___FILE_ONLY___ ═
244
+ 2022-09-14 13:43:28,506 INFO ___FILE_ONLY___ ═
245
+ 2022-09-14 13:43:28,510 INFO ___FILE_ONLY___ ═
246
+ 2022-09-14 13:43:28,514 INFO ___FILE_ONLY___ ═
247
+ 2022-09-14 13:43:28,518 INFO ___FILE_ONLY___ ═
248
+ 2022-09-14 13:43:28,524 INFO ___FILE_ONLY___ ═
249
+ 2022-09-14 13:43:28,528 INFO ___FILE_ONLY___ ═
250
+ 2022-09-14 13:43:28,533 INFO ___FILE_ONLY___ ═
251
+ 2022-09-14 13:43:28,537 INFO ___FILE_ONLY___ ═
252
+ 2022-09-14 13:43:28,542 INFO ___FILE_ONLY___ ═
253
+ 2022-09-14 13:43:28,546 INFO ___FILE_ONLY___ ═
254
+ 2022-09-14 13:43:28,551 INFO ___FILE_ONLY___ ═
255
+ 2022-09-14 13:43:28,555 INFO ___FILE_ONLY___ ═
256
+ 2022-09-14 13:43:28,560 INFO ___FILE_ONLY___ ═
257
+ 2022-09-14 13:43:28,564 INFO ___FILE_ONLY___ ═
258
+ 2022-09-14 13:43:28,564 INFO ___FILE_ONLY___ ╝
259
+
260
+ 2022-09-14 13:43:28,581 INFO ___FILE_ONLY___ ╔════════════════════════════════════════════════════════════╗
261
+
262
+ 2022-09-14 13:43:28,581 INFO ___FILE_ONLY___ ╠═ Installing: BigQuery Command Line Tool (Platform Spec... ═╣
263
+
264
+ 2022-09-14 13:43:28,581 INFO ___FILE_ONLY___ ╚
265
+ 2022-09-14 13:43:28,586 DEBUG urllib3.connectionpool Starting new HTTPS connection (1): dl.google.com:443
266
+ 2022-09-14 13:43:28,671 DEBUG urllib3.connectionpool https://dl.google.com:443 "GET /dl/cloudsdk/channels/rapid/components/google-cloud-sdk-bq-nix-20220805142045.tar.gz HTTP/1.1" 200 1816
267
+ 2022-09-14 13:43:28,672 INFO ___FILE_ONLY___ ══════════════════════════════
268
+ 2022-09-14 13:43:28,673 INFO ___FILE_ONLY___ ══════════════════════════════
269
+ 2022-09-14 13:43:28,673 INFO ___FILE_ONLY___ ╝
270
+
271
+ 2022-09-14 13:43:28,685 INFO ___FILE_ONLY___ ╔════════════════════════════════════════════════════════════╗
272
+
273
+ 2022-09-14 13:43:28,685 INFO ___FILE_ONLY___ ╠═ Installing: Cloud Storage Command Line Tool ═╣
274
+
275
+ 2022-09-14 13:43:28,685 INFO ___FILE_ONLY___ ╚
276
+ 2022-09-14 13:43:28,689 DEBUG urllib3.connectionpool Starting new HTTPS connection (1): dl.google.com:443
277
+ 2022-09-14 13:43:28,785 DEBUG urllib3.connectionpool https://dl.google.com:443 "GET /dl/cloudsdk/channels/rapid/components/google-cloud-sdk-gsutil-20220912133630.tar.gz HTTP/1.1" 200 16234713
278
+ 2022-09-14 13:43:28,948 INFO ___FILE_ONLY___ ═
279
+ 2022-09-14 13:43:28,949 INFO ___FILE_ONLY___ ═
280
+ 2022-09-14 13:43:28,949 INFO ___FILE_ONLY___ ═
281
+ 2022-09-14 13:43:28,950 INFO ___FILE_ONLY___ ═
282
+ 2022-09-14 13:43:28,951 INFO ___FILE_ONLY___ ═
283
+ 2022-09-14 13:43:28,952 INFO ___FILE_ONLY___ ═
284
+ 2022-09-14 13:43:28,953 INFO ___FILE_ONLY___ ═
285
+ 2022-09-14 13:43:28,954 INFO ___FILE_ONLY___ ═
286
+ 2022-09-14 13:43:28,954 INFO ___FILE_ONLY___ ═
287
+ 2022-09-14 13:43:28,955 INFO ___FILE_ONLY___ ═
288
+ 2022-09-14 13:43:28,956 INFO ___FILE_ONLY___ ═
289
+ 2022-09-14 13:43:28,957 INFO ___FILE_ONLY___ ═
290
+ 2022-09-14 13:43:28,957 INFO ___FILE_ONLY___ ═
291
+ 2022-09-14 13:43:28,958 INFO ___FILE_ONLY___ ═
292
+ 2022-09-14 13:43:28,959 INFO ___FILE_ONLY___ ═
293
+ 2022-09-14 13:43:28,960 INFO ___FILE_ONLY___ ═
294
+ 2022-09-14 13:43:28,961 INFO ___FILE_ONLY___ ═
295
+ 2022-09-14 13:43:28,961 INFO ___FILE_ONLY___ ═
296
+ 2022-09-14 13:43:28,962 INFO ___FILE_ONLY___ ═
297
+ 2022-09-14 13:43:28,963 INFO ___FILE_ONLY___ ═
298
+ 2022-09-14 13:43:28,964 INFO ___FILE_ONLY___ ═
299
+ 2022-09-14 13:43:28,965 INFO ___FILE_ONLY___ ═
300
+ 2022-09-14 13:43:28,965 INFO ___FILE_ONLY___ ═
301
+ 2022-09-14 13:43:28,966 INFO ___FILE_ONLY___ ═
302
+ 2022-09-14 13:43:28,967 INFO ___FILE_ONLY___ ═
303
+ 2022-09-14 13:43:28,968 INFO ___FILE_ONLY___ ═
304
+ 2022-09-14 13:43:28,968 INFO ___FILE_ONLY___ ═
305
+ 2022-09-14 13:43:28,969 INFO ___FILE_ONLY___ ═
306
+ 2022-09-14 13:43:28,970 INFO ___FILE_ONLY___ ═
307
+ 2022-09-14 13:43:28,971 INFO ___FILE_ONLY___ ═
308
+ 2022-09-14 13:43:30,254 INFO ___FILE_ONLY___ ═
309
+ 2022-09-14 13:43:30,308 INFO ___FILE_ONLY___ ═
310
+ 2022-09-14 13:43:30,353 INFO ___FILE_ONLY___ ═
311
+ 2022-09-14 13:43:30,396 INFO ___FILE_ONLY___ ═
312
+ 2022-09-14 13:43:30,450 INFO ___FILE_ONLY___ ═
313
+ 2022-09-14 13:43:30,509 INFO ___FILE_ONLY___ ═
314
+ 2022-09-14 13:43:30,563 INFO ___FILE_ONLY___ ═
315
+ 2022-09-14 13:43:30,628 INFO ___FILE_ONLY___ ═
316
+ 2022-09-14 13:43:30,694 INFO ___FILE_ONLY___ ═
317
+ 2022-09-14 13:43:30,765 INFO ___FILE_ONLY___ ═
318
+ 2022-09-14 13:43:30,812 INFO ___FILE_ONLY___ ═
319
+ 2022-09-14 13:43:30,858 INFO ___FILE_ONLY___ ═
320
+ 2022-09-14 13:43:30,915 INFO ___FILE_ONLY___ ═
321
+ 2022-09-14 13:43:30,981 INFO ___FILE_ONLY___ ═
322
+ 2022-09-14 13:43:31,038 INFO ___FILE_ONLY___ ═
323
+ 2022-09-14 13:43:31,082 INFO ___FILE_ONLY___ ═
324
+ 2022-09-14 13:43:31,155 INFO ___FILE_ONLY___ ═
325
+ 2022-09-14 13:43:31,206 INFO ___FILE_ONLY___ ═
326
+ 2022-09-14 13:43:31,249 INFO ___FILE_ONLY___ ═
327
+ 2022-09-14 13:43:31,297 INFO ___FILE_ONLY___ ═
328
+ 2022-09-14 13:43:31,347 INFO ___FILE_ONLY___ ═
329
+ 2022-09-14 13:43:31,398 INFO ___FILE_ONLY___ ═
330
+ 2022-09-14 13:43:31,456 INFO ___FILE_ONLY___ ═
331
+ 2022-09-14 13:43:31,506 INFO ___FILE_ONLY___ ═
332
+ 2022-09-14 13:43:31,562 INFO ___FILE_ONLY___ ═
333
+ 2022-09-14 13:43:31,619 INFO ___FILE_ONLY___ ═
334
+ 2022-09-14 13:43:31,664 INFO ___FILE_ONLY___ ═
335
+ 2022-09-14 13:43:31,707 INFO ___FILE_ONLY___ ═
336
+ 2022-09-14 13:43:31,750 INFO ___FILE_ONLY___ ═
337
+ 2022-09-14 13:43:31,810 INFO ___FILE_ONLY___ ═
338
+ 2022-09-14 13:43:31,810 INFO ___FILE_ONLY___ ╝
339
+
340
+ 2022-09-14 13:43:31,908 INFO ___FILE_ONLY___ ╔════════════════════════════════════════════════════════════╗
341
+
342
+ 2022-09-14 13:43:31,909 INFO ___FILE_ONLY___ ╠═ Installing: Cloud Storage Command Line Tool (Platform... ═╣
343
+
344
+ 2022-09-14 13:43:31,909 INFO ___FILE_ONLY___ ╚
345
+ 2022-09-14 13:43:31,913 DEBUG urllib3.connectionpool Starting new HTTPS connection (1): dl.google.com:443
346
+ 2022-09-14 13:43:32,000 DEBUG urllib3.connectionpool https://dl.google.com:443 "GET /dl/cloudsdk/channels/rapid/components/google-cloud-sdk-gsutil-nix-20220805142045.tar.gz HTTP/1.1" 200 1831
347
+ 2022-09-14 13:43:32,001 INFO ___FILE_ONLY___ ══════════════════════════════
348
+ 2022-09-14 13:43:32,002 INFO ___FILE_ONLY___ ══════════════════════════════
349
+ 2022-09-14 13:43:32,002 INFO ___FILE_ONLY___ ╝
350
+
351
+ 2022-09-14 13:43:32,014 INFO ___FILE_ONLY___ ╔════════════════════════════════════════════════════════════╗
352
+
353
+ 2022-09-14 13:43:32,015 INFO ___FILE_ONLY___ ╠═ Installing: Default set of gcloud commands ═╣
354
+
355
+ 2022-09-14 13:43:32,015 INFO ___FILE_ONLY___ ╚
356
+ 2022-09-14 13:43:32,020 INFO ___FILE_ONLY___ ════════════════════════════════════════════════════════════
357
+ 2022-09-14 13:43:32,020 INFO ___FILE_ONLY___ ╝
358
+
359
+ 2022-09-14 13:43:32,025 INFO ___FILE_ONLY___ ╔════════════════════════════════════════════════════════════╗
360
+
361
+ 2022-09-14 13:43:32,025 INFO ___FILE_ONLY___ ╠═ Installing: Google Cloud CLI Core Libraries (Platform... ═╣
362
+
363
+ 2022-09-14 13:43:32,025 INFO ___FILE_ONLY___ ╚
364
+ 2022-09-14 13:43:32,029 DEBUG urllib3.connectionpool Starting new HTTPS connection (1): dl.google.com:443
365
+ 2022-09-14 13:43:32,118 DEBUG urllib3.connectionpool https://dl.google.com:443 "GET /dl/cloudsdk/channels/rapid/components/google-cloud-sdk-core-nix-20220805142045.tar.gz HTTP/1.1" 200 2202
366
+ 2022-09-14 13:43:32,119 INFO ___FILE_ONLY___ ══════════════════════════════
367
+ 2022-09-14 13:43:32,120 INFO ___FILE_ONLY___ ═══════════════
368
+ 2022-09-14 13:43:32,121 INFO ___FILE_ONLY___ ═══════════════
369
+ 2022-09-14 13:43:32,121 INFO ___FILE_ONLY___ ╝
370
+
371
+ 2022-09-14 13:43:32,132 INFO ___FILE_ONLY___ ╔════════════════════════════════════════════════════════════╗
372
+
373
+ 2022-09-14 13:43:32,133 INFO ___FILE_ONLY___ ╠═ Installing: Google Cloud CRC32C Hash Tool ═╣
374
+
375
+ 2022-09-14 13:43:32,133 INFO ___FILE_ONLY___ ╚
376
+ 2022-09-14 13:43:32,137 DEBUG urllib3.connectionpool Starting new HTTPS connection (1): dl.google.com:443
377
+ 2022-09-14 13:43:32,170 DEBUG urllib3.connectionpool https://dl.google.com:443 "GET /dl/cloudsdk/channels/rapid/components/google-cloud-sdk-gcloud-crc32c-linux-x86_64-20220729144039.tar.gz HTTP/1.1" 200 1243294
378
+ 2022-09-14 13:43:32,248 INFO ___FILE_ONLY___ ═
379
+ 2022-09-14 13:43:32,248 INFO ___FILE_ONLY___ ═
380
+ 2022-09-14 13:43:32,248 INFO ___FILE_ONLY___ ═
381
+ 2022-09-14 13:43:32,248 INFO ___FILE_ONLY___ ═
382
+ 2022-09-14 13:43:32,248 INFO ___FILE_ONLY___ ═
383
+ 2022-09-14 13:43:32,248 INFO ___FILE_ONLY___ ═
384
+ 2022-09-14 13:43:32,249 INFO ___FILE_ONLY___ ═
385
+ 2022-09-14 13:43:32,249 INFO ___FILE_ONLY___ ═
386
+ 2022-09-14 13:43:32,249 INFO ___FILE_ONLY___ ═
387
+ 2022-09-14 13:43:32,249 INFO ___FILE_ONLY___ ═
388
+ 2022-09-14 13:43:32,249 INFO ___FILE_ONLY___ ═
389
+ 2022-09-14 13:43:32,249 INFO ___FILE_ONLY___ ═
390
+ 2022-09-14 13:43:32,249 INFO ___FILE_ONLY___ ═
391
+ 2022-09-14 13:43:32,250 INFO ___FILE_ONLY___ ═
392
+ 2022-09-14 13:43:32,250 INFO ___FILE_ONLY___ ═
393
+ 2022-09-14 13:43:32,250 INFO ___FILE_ONLY___ ═
394
+ 2022-09-14 13:43:32,250 INFO ___FILE_ONLY___ ═
395
+ 2022-09-14 13:43:32,250 INFO ___FILE_ONLY___ ═
396
+ 2022-09-14 13:43:32,250 INFO ___FILE_ONLY___ ═
397
+ 2022-09-14 13:43:32,250 INFO ___FILE_ONLY___ ═
398
+ 2022-09-14 13:43:32,250 INFO ___FILE_ONLY___ ═
399
+ 2022-09-14 13:43:32,251 INFO ___FILE_ONLY___ ═
400
+ 2022-09-14 13:43:32,251 INFO ___FILE_ONLY___ ═
401
+ 2022-09-14 13:43:32,251 INFO ___FILE_ONLY___ ═
402
+ 2022-09-14 13:43:32,251 INFO ___FILE_ONLY___ ═
403
+ 2022-09-14 13:43:32,251 INFO ___FILE_ONLY___ ═
404
+ 2022-09-14 13:43:32,251 INFO ___FILE_ONLY___ ═
405
+ 2022-09-14 13:43:32,251 INFO ___FILE_ONLY___ ═
406
+ 2022-09-14 13:43:32,252 INFO ___FILE_ONLY___ ═
407
+ 2022-09-14 13:43:32,252 INFO ___FILE_ONLY___ ═
408
+ 2022-09-14 13:43:32,288 INFO ___FILE_ONLY___ ═══════════════
409
+ 2022-09-14 13:43:32,289 INFO ___FILE_ONLY___ ═══════════════
410
+ 2022-09-14 13:43:32,289 INFO ___FILE_ONLY___ ╝
411
+
412
+ 2022-09-14 13:43:32,301 INFO ___FILE_ONLY___ ╔════════════════════════════════════════════════════════════╗
413
+
414
+ 2022-09-14 13:43:32,302 INFO ___FILE_ONLY___ ╠═ Installing: Google Cloud CRC32C Hash Tool ═╣
415
+
416
+ 2022-09-14 13:43:32,302 INFO ___FILE_ONLY___ ╚
417
+ 2022-09-14 13:43:32,307 INFO ___FILE_ONLY___ ════════════════════════════════════════════════════════════
418
+ 2022-09-14 13:43:32,307 INFO ___FILE_ONLY___ ╝
419
+
420
+ 2022-09-14 13:43:32,311 INFO ___FILE_ONLY___ ╔════════════════════════════════════════════════════════════╗
421
+
422
+ 2022-09-14 13:43:32,312 INFO ___FILE_ONLY___ ╠═ Installing: anthoscli ═╣
423
+
424
+ 2022-09-14 13:43:32,312 INFO ___FILE_ONLY___ ╚
425
+ 2022-09-14 13:43:32,316 DEBUG urllib3.connectionpool Starting new HTTPS connection (1): dl.google.com:443
426
+ 2022-09-14 13:43:32,408 DEBUG urllib3.connectionpool https://dl.google.com:443 "GET /dl/cloudsdk/channels/rapid/components/google-cloud-sdk-anthoscli-linux-x86_64-20220826172526.tar.gz HTTP/1.1" 200 39436637
427
+ 2022-09-14 13:43:32,689 INFO ___FILE_ONLY___ ═
428
+ 2022-09-14 13:43:32,691 INFO ___FILE_ONLY___ ═
429
+ 2022-09-14 13:43:32,693 INFO ___FILE_ONLY___ ═
430
+ 2022-09-14 13:43:32,695 INFO ___FILE_ONLY___ ═
431
+ 2022-09-14 13:43:32,696 INFO ___FILE_ONLY___ ═
432
+ 2022-09-14 13:43:32,698 INFO ___FILE_ONLY___ ═
433
+ 2022-09-14 13:43:32,700 INFO ___FILE_ONLY___ ═
434
+ 2022-09-14 13:43:32,702 INFO ___FILE_ONLY___ ═
435
+ 2022-09-14 13:43:32,703 INFO ___FILE_ONLY___ ═
436
+ 2022-09-14 13:43:32,705 INFO ___FILE_ONLY___ ═
437
+ 2022-09-14 13:43:32,707 INFO ___FILE_ONLY___ ═
438
+ 2022-09-14 13:43:32,708 INFO ___FILE_ONLY___ ═
439
+ 2022-09-14 13:43:32,710 INFO ___FILE_ONLY___ ═
440
+ 2022-09-14 13:43:32,712 INFO ___FILE_ONLY___ ═
441
+ 2022-09-14 13:43:32,714 INFO ___FILE_ONLY___ ═
442
+ 2022-09-14 13:43:32,715 INFO ___FILE_ONLY___ ═
443
+ 2022-09-14 13:43:32,717 INFO ___FILE_ONLY___ ═
444
+ 2022-09-14 13:43:32,719 INFO ___FILE_ONLY___ ═
445
+ 2022-09-14 13:43:32,720 INFO ___FILE_ONLY___ ═
446
+ 2022-09-14 13:43:32,722 INFO ___FILE_ONLY___ ═
447
+ 2022-09-14 13:43:32,724 INFO ___FILE_ONLY___ ═
448
+ 2022-09-14 13:43:32,725 INFO ___FILE_ONLY___ ═
449
+ 2022-09-14 13:43:32,727 INFO ___FILE_ONLY___ ═
450
+ 2022-09-14 13:43:32,729 INFO ___FILE_ONLY___ ═
451
+ 2022-09-14 13:43:32,731 INFO ___FILE_ONLY___ ═
452
+ 2022-09-14 13:43:32,732 INFO ___FILE_ONLY___ ═
453
+ 2022-09-14 13:43:32,734 INFO ___FILE_ONLY___ ═
454
+ 2022-09-14 13:43:32,736 INFO ___FILE_ONLY___ ═
455
+ 2022-09-14 13:43:32,737 INFO ___FILE_ONLY___ ═
456
+ 2022-09-14 13:43:32,739 INFO ___FILE_ONLY___ ═
457
+ 2022-09-14 13:43:34,003 INFO ___FILE_ONLY___ ══════════
458
+ 2022-09-14 13:43:34,007 INFO ___FILE_ONLY___ ═════════
459
+ 2022-09-14 13:43:34,030 INFO ___FILE_ONLY___ ═══════════
460
+ 2022-09-14 13:43:34,030 INFO ___FILE_ONLY___ ╝
461
+
462
+ 2022-09-14 13:43:34,050 INFO ___FILE_ONLY___ ╔════════════════════════════════════════════════════════════╗
463
+
464
+ 2022-09-14 13:43:34,051 INFO ___FILE_ONLY___ ╠═ Installing: anthoscli ═╣
465
+
466
+ 2022-09-14 13:43:34,051 INFO ___FILE_ONLY___ ╚
467
+ 2022-09-14 13:43:34,056 INFO ___FILE_ONLY___ ════════════════════════════════════════════════════════════
468
+ 2022-09-14 13:43:34,056 INFO ___FILE_ONLY___ ╝
469
+
470
+ 2022-09-14 13:43:34,061 INFO ___FILE_ONLY___ ╔════════════════════════════════════════════════════════════╗
471
+
472
+ 2022-09-14 13:43:34,061 INFO ___FILE_ONLY___ ╠═ Installing: gcloud cli dependencies ═╣
473
+
474
+ 2022-09-14 13:43:34,061 INFO ___FILE_ONLY___ ╚
475
+ 2022-09-14 13:43:34,066 DEBUG urllib3.connectionpool Starting new HTTPS connection (1): dl.google.com:443
476
+ 2022-09-14 13:43:34,153 DEBUG urllib3.connectionpool https://dl.google.com:443 "GET /dl/cloudsdk/channels/rapid/components/google-cloud-sdk-gcloud-deps-linux-x86_64-20210416153011.tar.gz HTTP/1.1" 200 104
477
+ 2022-09-14 13:43:34,154 INFO ___FILE_ONLY___ ══════════════════════════════
478
+ 2022-09-14 13:43:34,155 INFO ___FILE_ONLY___ ══════════════════════════════
479
+ 2022-09-14 13:43:34,155 INFO ___FILE_ONLY___ ╝
480
+
481
+ 2022-09-14 13:43:34,167 INFO ___FILE_ONLY___ ╔════════════════════════════════════════════════════════════╗
482
+
483
+ 2022-09-14 13:43:34,167 INFO ___FILE_ONLY___ ╠═ Creating backup and activating new installation ═╣
484
+
485
+ 2022-09-14 13:43:34,167 INFO ___FILE_ONLY___ ╚
486
+ 2022-09-14 13:43:34,168 DEBUG root Attempting to move directory [/tools/google-cloud-sdk] to [/tools/google-cloud-sdk.staging/.install/.backup]
487
+ 2022-09-14 13:43:34,168 INFO ___FILE_ONLY___ ══════════════════════════════
488
+ 2022-09-14 13:43:34,168 DEBUG root Attempting to move directory [/tools/google-cloud-sdk.staging] to [/tools/google-cloud-sdk]
489
+ 2022-09-14 13:43:34,168 INFO ___FILE_ONLY___ ══════════════════════════════
490
+ 2022-09-14 13:43:34,168 INFO ___FILE_ONLY___ ╝
491
+
492
+ 2022-09-14 13:43:34,172 DEBUG root Updating notification cache...
493
+ 2022-09-14 13:43:34,173 INFO ___FILE_ONLY___
494
+
495
+ 2022-09-14 13:43:34,177 INFO ___FILE_ONLY___ Performing post processing steps...
496
+ 2022-09-14 13:43:34,178 DEBUG root Executing command: ['python3', '-S', '/tools/google-cloud-sdk/lib/gcloud.py', 'components', 'post-process']
497
+ 2022-09-14 13:43:53,462 DEBUG ___FILE_ONLY___
498
+ 2022-09-14 13:43:53,462 DEBUG ___FILE_ONLY___
499
+ 2022-09-14 13:43:53,708 INFO ___FILE_ONLY___
500
+ Update done!
501
+
502
+
503
+ 2022-09-14 13:43:53,712 INFO root Display format: "none"
.config/logs/2022.09.14/13.43.34.754807.log ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ 2022-09-14 13:43:34,755 DEBUG root Loaded Command Group: ['gcloud', 'components']
2
+ 2022-09-14 13:43:34,759 DEBUG root Loaded Command Group: ['gcloud', 'components', 'post_process']
3
+ 2022-09-14 13:43:34,761 DEBUG root Running [gcloud.components.post-process] with arguments: []
4
+ 2022-09-14 13:43:53,353 INFO root Display format: "none"
.config/logs/2022.09.14/13.43.54.433400.log ADDED
@@ -0,0 +1,165 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 2022-09-14 13:43:54,434 DEBUG root Loaded Command Group: ['gcloud', 'components']
2
+ 2022-09-14 13:43:54,436 DEBUG root Loaded Command Group: ['gcloud', 'components', 'update']
3
+ 2022-09-14 13:43:54,439 DEBUG root Running [gcloud.components.update] with arguments: [--quiet: "True", COMPONENT-IDS:8: "['gcloud', 'core', 'bq', 'gsutil', 'compute', 'preview', 'alpha', 'beta']"]
4
+ 2022-09-14 13:43:54,446 INFO ___FILE_ONLY___ Beginning update. This process may take several minutes.
5
+
6
+ 2022-09-14 13:43:54,454 DEBUG urllib3.connectionpool Starting new HTTPS connection (1): dl.google.com:443
7
+ 2022-09-14 13:43:54,485 DEBUG urllib3.connectionpool https://dl.google.com:443 "GET /dl/cloudsdk/channels/rapid/components-2.json HTTP/1.1" 200 197751
8
+ 2022-09-14 13:43:54,502 WARNING root Component [preview] no longer exists.
9
+ 2022-09-14 13:43:54,502 WARNING root Component [compute] no longer exists.
10
+ 2022-09-14 13:43:54,503 INFO ___FILE_ONLY___
11
+
12
+ 2022-09-14 13:43:54,504 INFO ___FILE_ONLY___
13
+ Your current Google Cloud CLI version is: 402.0.0
14
+
15
+ 2022-09-14 13:43:54,504 INFO ___FILE_ONLY___ Installing components from version: 402.0.0
16
+
17
+ 2022-09-14 13:43:54,504 INFO ___FILE_ONLY___
18
+
19
+ 2022-09-14 13:43:54,507 INFO ___FILE_ONLY___ ┌──────────────────────────────────────────────┐
20
+ 2022-09-14 13:43:54,507 INFO ___FILE_ONLY___
21
+
22
+ 2022-09-14 13:43:54,507 INFO ___FILE_ONLY___ │ These components will be installed. │
23
+ 2022-09-14 13:43:54,507 INFO ___FILE_ONLY___
24
+
25
+ 2022-09-14 13:43:54,507 INFO ___FILE_ONLY___ ├───────────────────────┬────────────┬─────────┤
26
+ 2022-09-14 13:43:54,507 INFO ___FILE_ONLY___
27
+
28
+ 2022-09-14 13:43:54,507 INFO ___FILE_ONLY___ │ Name │ Version │ Size │
29
+ 2022-09-14 13:43:54,508 INFO ___FILE_ONLY___
30
+
31
+ 2022-09-14 13:43:54,508 INFO ___FILE_ONLY___ ├───────────────────────┼────────────┼─────────┤
32
+ 2022-09-14 13:43:54,508 INFO ___FILE_ONLY___
33
+
34
+ 2022-09-14 13:43:54,508 INFO ___FILE_ONLY___ │
35
+ 2022-09-14 13:43:54,508 INFO ___FILE_ONLY___ gcloud Alpha Commands
36
+ 2022-09-14 13:43:54,508 INFO ___FILE_ONLY___
37
+ 2022-09-14 13:43:54,508 INFO ___FILE_ONLY___ │
38
+ 2022-09-14 13:43:54,508 INFO ___FILE_ONLY___ 2022.09.12
39
+ 2022-09-14 13:43:54,508 INFO ___FILE_ONLY___
40
+ 2022-09-14 13:43:54,508 INFO ___FILE_ONLY___ │
41
+ 2022-09-14 13:43:54,508 INFO ___FILE_ONLY___ < 1 MiB
42
+ 2022-09-14 13:43:54,508 INFO ___FILE_ONLY___
43
+ 2022-09-14 13:43:54,508 INFO ___FILE_ONLY___ │
44
+ 2022-09-14 13:43:54,508 INFO ___FILE_ONLY___
45
+
46
+ 2022-09-14 13:43:54,509 INFO ___FILE_ONLY___ │
47
+ 2022-09-14 13:43:54,509 INFO ___FILE_ONLY___ gcloud Beta Commands
48
+ 2022-09-14 13:43:54,509 INFO ___FILE_ONLY___
49
+ 2022-09-14 13:43:54,509 INFO ___FILE_ONLY___ │
50
+ 2022-09-14 13:43:54,509 INFO ___FILE_ONLY___ 2022.09.12
51
+ 2022-09-14 13:43:54,509 INFO ___FILE_ONLY___
52
+ 2022-09-14 13:43:54,509 INFO ___FILE_ONLY___ │
53
+ 2022-09-14 13:43:54,509 INFO ___FILE_ONLY___ < 1 MiB
54
+ 2022-09-14 13:43:54,509 INFO ___FILE_ONLY___
55
+ 2022-09-14 13:43:54,509 INFO ___FILE_ONLY___ │
56
+ 2022-09-14 13:43:54,509 INFO ___FILE_ONLY___
57
+
58
+ 2022-09-14 13:43:54,509 INFO ___FILE_ONLY___ └───────────────────────┴────────────┴─────────┘
59
+ 2022-09-14 13:43:54,509 INFO ___FILE_ONLY___
60
+
61
+ 2022-09-14 13:43:54,509 INFO ___FILE_ONLY___
62
+
63
+ 2022-09-14 13:43:54,512 DEBUG urllib3.connectionpool Starting new HTTPS connection (1): dl.google.com:443
64
+ 2022-09-14 13:43:54,601 DEBUG urllib3.connectionpool https://dl.google.com:443 "GET /dl/cloudsdk/channels/rapid/RELEASE_NOTES HTTP/1.1" 200 867084
65
+ 2022-09-14 13:43:54,691 INFO ___FILE_ONLY___ For the latest full release notes, please visit:
66
+ https://cloud.google.com/sdk/release_notes
67
+
68
+
69
+ 2022-09-14 13:43:54,697 INFO ___FILE_ONLY___ ╔════════════════════════════════════════════════════════════╗
70
+
71
+ 2022-09-14 13:43:54,697 INFO ___FILE_ONLY___ ╠═ Creating update staging area ═╣
72
+
73
+ 2022-09-14 13:43:54,697 INFO ___FILE_ONLY___ ╚
74
+ 2022-09-14 13:43:54,697 INFO ___FILE_ONLY___ ══════
75
+ 2022-09-14 13:43:55,308 INFO ___FILE_ONLY___ ══════
76
+ 2022-09-14 13:43:55,308 INFO ___FILE_ONLY___ ══════
77
+ 2022-09-14 13:43:55,667 INFO ___FILE_ONLY___ ═
78
+ 2022-09-14 13:43:55,759 INFO ___FILE_ONLY___ ═
79
+ 2022-09-14 13:43:55,832 INFO ___FILE_ONLY___ ═
80
+ 2022-09-14 13:43:55,963 INFO ___FILE_ONLY___ ═
81
+ 2022-09-14 13:43:56,035 INFO ___FILE_ONLY___ ═
82
+ 2022-09-14 13:43:56,106 INFO ___FILE_ONLY___ ═
83
+ 2022-09-14 13:43:56,191 INFO ___FILE_ONLY___ ═
84
+ 2022-09-14 13:43:56,298 INFO ___FILE_ONLY___ ═
85
+ 2022-09-14 13:43:56,413 INFO ___FILE_ONLY___ ═
86
+ 2022-09-14 13:43:56,482 INFO ___FILE_ONLY___ ═
87
+ 2022-09-14 13:43:56,603 INFO ___FILE_ONLY___ ═
88
+ 2022-09-14 13:43:56,699 INFO ___FILE_ONLY___ ═
89
+ 2022-09-14 13:43:56,813 INFO ___FILE_ONLY___ ═
90
+ 2022-09-14 13:43:56,907 INFO ___FILE_ONLY___ ═
91
+ 2022-09-14 13:43:57,024 INFO ___FILE_ONLY___ ═
92
+ 2022-09-14 13:43:57,217 INFO ___FILE_ONLY___ ═
93
+ 2022-09-14 13:43:57,338 INFO ___FILE_ONLY___ ═
94
+ 2022-09-14 13:43:57,412 INFO ___FILE_ONLY___ ═
95
+ 2022-09-14 13:43:57,458 INFO ___FILE_ONLY___ ═
96
+ 2022-09-14 13:43:57,526 INFO ___FILE_ONLY___ ═
97
+ 2022-09-14 13:43:57,595 INFO ___FILE_ONLY___ ═
98
+ 2022-09-14 13:43:57,655 INFO ___FILE_ONLY___ ═
99
+ 2022-09-14 13:43:57,706 INFO ___FILE_ONLY___ ═
100
+ 2022-09-14 13:43:57,764 INFO ___FILE_ONLY___ ═
101
+ 2022-09-14 13:43:57,818 INFO ___FILE_ONLY___ ═
102
+ 2022-09-14 13:43:57,882 INFO ___FILE_ONLY___ ═
103
+ 2022-09-14 13:43:57,961 INFO ___FILE_ONLY___ ═
104
+ 2022-09-14 13:43:58,048 INFO ___FILE_ONLY___ ═
105
+ 2022-09-14 13:43:58,131 INFO ___FILE_ONLY___ ═
106
+ 2022-09-14 13:43:58,200 INFO ___FILE_ONLY___ ═
107
+ 2022-09-14 13:43:58,271 INFO ___FILE_ONLY___ ═
108
+ 2022-09-14 13:43:58,325 INFO ___FILE_ONLY___ ═
109
+ 2022-09-14 13:43:58,398 INFO ___FILE_ONLY___ ═
110
+ 2022-09-14 13:43:58,458 INFO ___FILE_ONLY___ ═
111
+ 2022-09-14 13:43:58,514 INFO ___FILE_ONLY___ ═
112
+ 2022-09-14 13:43:58,583 INFO ___FILE_ONLY___ ═
113
+ 2022-09-14 13:43:58,635 INFO ___FILE_ONLY___ ═
114
+ 2022-09-14 13:43:58,702 INFO ___FILE_ONLY___ ═
115
+ 2022-09-14 13:43:58,852 INFO ___FILE_ONLY___ ═
116
+ 2022-09-14 13:43:58,929 INFO ___FILE_ONLY___ ═
117
+ 2022-09-14 13:43:59,310 INFO ___FILE_ONLY___ ═
118
+ 2022-09-14 13:43:59,401 INFO ___FILE_ONLY___ ═
119
+ 2022-09-14 13:43:59,401 INFO ___FILE_ONLY___ ╝
120
+
121
+ 2022-09-14 13:44:01,735 INFO ___FILE_ONLY___ ╔════════════════════════════════════════════════════════════╗
122
+
123
+ 2022-09-14 13:44:01,735 INFO ___FILE_ONLY___ ╠═ Installing: gcloud Alpha Commands ═╣
124
+
125
+ 2022-09-14 13:44:01,735 INFO ___FILE_ONLY___ ╚
126
+ 2022-09-14 13:44:01,740 DEBUG urllib3.connectionpool Starting new HTTPS connection (1): dl.google.com:443
127
+ 2022-09-14 13:44:01,824 DEBUG urllib3.connectionpool https://dl.google.com:443 "GET /dl/cloudsdk/channels/rapid/components/google-cloud-sdk-alpha-20220912133630.tar.gz HTTP/1.1" 200 800
128
+ 2022-09-14 13:44:01,825 INFO ___FILE_ONLY___ ══════════════════════════════
129
+ 2022-09-14 13:44:01,827 INFO ___FILE_ONLY___ ══════════════════════════════
130
+ 2022-09-14 13:44:01,827 INFO ___FILE_ONLY___ ╝
131
+
132
+ 2022-09-14 13:44:01,838 INFO ___FILE_ONLY___ ╔════════════════════════════════════════════════════════════╗
133
+
134
+ 2022-09-14 13:44:01,839 INFO ___FILE_ONLY___ ╠═ Installing: gcloud Beta Commands ═╣
135
+
136
+ 2022-09-14 13:44:01,839 INFO ___FILE_ONLY___ ╚
137
+ 2022-09-14 13:44:01,843 DEBUG urllib3.connectionpool Starting new HTTPS connection (1): dl.google.com:443
138
+ 2022-09-14 13:44:01,872 DEBUG urllib3.connectionpool https://dl.google.com:443 "GET /dl/cloudsdk/channels/rapid/components/google-cloud-sdk-beta-20220912133630.tar.gz HTTP/1.1" 200 797
139
+ 2022-09-14 13:44:01,873 INFO ___FILE_ONLY___ ══════════════════════════════
140
+ 2022-09-14 13:44:01,874 INFO ___FILE_ONLY___ ══════════════════════════════
141
+ 2022-09-14 13:44:01,874 INFO ___FILE_ONLY___ ╝
142
+
143
+ 2022-09-14 13:44:01,885 INFO ___FILE_ONLY___ ╔════════════════════════════════════════════════════════════╗
144
+
145
+ 2022-09-14 13:44:01,885 INFO ___FILE_ONLY___ ╠═ Creating backup and activating new installation ═╣
146
+
147
+ 2022-09-14 13:44:01,885 INFO ___FILE_ONLY___ ╚
148
+ 2022-09-14 13:44:01,886 DEBUG root Attempting to move directory [/tools/google-cloud-sdk] to [/tools/google-cloud-sdk.staging/.install/.backup]
149
+ 2022-09-14 13:44:01,886 INFO ___FILE_ONLY___ ══════════════════════════════
150
+ 2022-09-14 13:44:01,886 DEBUG root Attempting to move directory [/tools/google-cloud-sdk.staging] to [/tools/google-cloud-sdk]
151
+ 2022-09-14 13:44:01,886 INFO ___FILE_ONLY___ ══════════════════════════════
152
+ 2022-09-14 13:44:01,886 INFO ___FILE_ONLY___ ╝
153
+
154
+ 2022-09-14 13:44:01,890 DEBUG root Updating notification cache...
155
+ 2022-09-14 13:44:01,891 INFO ___FILE_ONLY___
156
+
157
+ 2022-09-14 13:44:01,896 INFO ___FILE_ONLY___ Performing post processing steps...
158
+ 2022-09-14 13:44:01,897 DEBUG root Executing command: ['python3', '-S', '/tools/google-cloud-sdk/lib/gcloud.py', 'components', 'post-process']
159
+ 2022-09-14 13:44:21,935 DEBUG ___FILE_ONLY___
160
+ 2022-09-14 13:44:21,936 DEBUG ___FILE_ONLY___
161
+ 2022-09-14 13:44:22,177 INFO ___FILE_ONLY___
162
+ Update done!
163
+
164
+
165
+ 2022-09-14 13:44:22,181 INFO root Display format: "none"
.config/logs/2022.09.14/13.44.02.480346.log ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ 2022-09-14 13:44:02,481 DEBUG root Loaded Command Group: ['gcloud', 'components']
2
+ 2022-09-14 13:44:02,483 DEBUG root Loaded Command Group: ['gcloud', 'components', 'post_process']
3
+ 2022-09-14 13:44:02,485 DEBUG root Running [gcloud.components.post-process] with arguments: []
4
+ 2022-09-14 13:44:21,837 INFO root Display format: "none"
.config/logs/2022.09.14/13.44.22.887116.log ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ 2022-09-14 13:44:22,889 DEBUG root Loaded Command Group: ['gcloud', 'config']
2
+ 2022-09-14 13:44:22,922 DEBUG root Loaded Command Group: ['gcloud', 'config', 'set']
3
+ 2022-09-14 13:44:22,925 DEBUG root Running [gcloud.config.set] with arguments: [SECTION/PROPERTY: "component_manager/disable_update_check", VALUE: "true"]
4
+ 2022-09-14 13:44:22,926 INFO ___FILE_ONLY___ Updated property [component_manager/disable_update_check].
5
+
6
+ 2022-09-14 13:44:22,927 INFO root Display format: "default"
7
+ 2022-09-14 13:44:22,928 DEBUG root SDK update checks are disabled.
.config/logs/2022.09.14/13.44.23.609544.log ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ 2022-09-14 13:44:23,611 DEBUG root Loaded Command Group: ['gcloud', 'config']
2
+ 2022-09-14 13:44:23,638 DEBUG root Loaded Command Group: ['gcloud', 'config', 'set']
3
+ 2022-09-14 13:44:23,641 DEBUG root Running [gcloud.config.set] with arguments: [SECTION/PROPERTY: "compute/gce_metadata_read_timeout_sec", VALUE: "0"]
4
+ 2022-09-14 13:44:23,642 INFO ___FILE_ONLY___ Updated property [compute/gce_metadata_read_timeout_sec].
5
+
6
+ 2022-09-14 13:44:23,644 INFO root Display format: "default"
7
+ 2022-09-14 13:44:23,645 DEBUG root SDK update checks are disabled.
README.md ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: Pixelplanet OCR
3
+ emoji: 🏃
4
+ colorFrom: indigo
5
+ colorTo: red
6
+ sdk: gradio
7
+ sdk_version: 2.8.12
8
+ app_file: app.py
9
+ pinned: false
10
+ license: bsd
11
+ duplicated_from: sneedium/captcha_pixelplanet
12
+ ---
app.py ADDED
@@ -0,0 +1,36 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ import os
3
+ os.system('pip install --upgrade gdown')
4
+ import gdown
5
+ gdown.download(id='1C3XPT8s-ONt88vlNykTkGt8c8frfB9U_', output='workdir.zip')
6
+ os.system('unzip workdir.zip')
7
+
8
+
9
+ import glob
10
+ import gradio as gr
11
+ from demo import get_model, preprocess, postprocess, load
12
+ from utils import Config, Logger, CharsetMapper
13
+
14
+ config = Config('configs/train_iternet.yaml')
15
+ config.model_vision_checkpoint = None
16
+ model = get_model(config)
17
+ model = load(model, 'workdir/train-iternet/best-train-iternet.pth')
18
+ charset = CharsetMapper(filename=config.dataset_charset_path, max_length=config.dataset_max_length + 1)
19
+
20
+ def process_image(image):
21
+ img = image.convert('RGB')
22
+ img = preprocess(img, config.dataset_image_width, config.dataset_image_height)
23
+ res = model(img)
24
+ return postprocess(res, charset, 'alignment')[0][0]
25
+
26
+ title = "Made with IterNet"
27
+ description = "I hate captchas"
28
+
29
+ iface = gr.Interface(fn=process_image,
30
+ inputs=gr.inputs.Image(type="pil"),
31
+ outputs=gr.outputs.Textbox(),
32
+ title=title,
33
+ description=description,
34
+ examples=glob.glob('figs_captchas/*.jpg'))
35
+
36
+ iface.launch(debug=True)
callbacks.py ADDED
@@ -0,0 +1,360 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import logging
2
+ import shutil
3
+ import time
4
+
5
+ import editdistance as ed
6
+ import torchvision.utils as vutils
7
+ from fastai.callbacks.tensorboard import (LearnerTensorboardWriter,
8
+ SummaryWriter, TBWriteRequest,
9
+ asyncTBWriter)
10
+ from fastai.vision import *
11
+ from torch.nn.parallel import DistributedDataParallel
12
+ from torchvision import transforms
13
+
14
+ import dataset
15
+ from utils import CharsetMapper, Timer, blend_mask
16
+
17
+
18
+ class IterationCallback(LearnerTensorboardWriter):
19
+ "A `TrackerCallback` that monitor in each iteration."
20
+ def __init__(self, learn:Learner, name:str='model', checpoint_keep_num=5,
21
+ show_iters:int=50, eval_iters:int=1000, save_iters:int=20000,
22
+ start_iters:int=0, stats_iters=20000):
23
+ #if self.learn.rank is not None: time.sleep(self.learn.rank) # keep all event files
24
+ super().__init__(learn, base_dir='.', name=learn.path, loss_iters=show_iters,
25
+ stats_iters=stats_iters, hist_iters=stats_iters)
26
+ self.name, self.bestname = Path(name).name, f'best-{Path(name).name}'
27
+ self.show_iters = show_iters
28
+ self.eval_iters = eval_iters
29
+ self.save_iters = save_iters
30
+ self.start_iters = start_iters
31
+ self.checpoint_keep_num = checpoint_keep_num
32
+ self.metrics_root = 'metrics/' # rewrite
33
+ self.timer = Timer()
34
+ self.host = self.learn.rank is None or self.learn.rank == 0
35
+
36
+ def _write_metrics(self, iteration:int, names:List[str], last_metrics:MetricsList)->None:
37
+ "Writes training metrics to Tensorboard."
38
+ for i, name in enumerate(names):
39
+ if last_metrics is None or len(last_metrics) < i+1: return
40
+ scalar_value = last_metrics[i]
41
+ self._write_scalar(name=name, scalar_value=scalar_value, iteration=iteration)
42
+
43
+ def _write_sub_loss(self, iteration:int, last_losses:dict)->None:
44
+ "Writes sub loss to Tensorboard."
45
+ for name, loss in last_losses.items():
46
+ scalar_value = to_np(loss)
47
+ tag = self.metrics_root + name
48
+ self.tbwriter.add_scalar(tag=tag, scalar_value=scalar_value, global_step=iteration)
49
+
50
+ def _save(self, name):
51
+ if isinstance(self.learn.model, DistributedDataParallel):
52
+ tmp = self.learn.model
53
+ self.learn.model = self.learn.model.module
54
+ self.learn.save(name)
55
+ self.learn.model = tmp
56
+ else: self.learn.save(name)
57
+
58
+ def _validate(self, dl=None, callbacks=None, metrics=None, keeped_items=False):
59
+ "Validate on `dl` with potential `callbacks` and `metrics`."
60
+ dl = ifnone(dl, self.learn.data.valid_dl)
61
+ metrics = ifnone(metrics, self.learn.metrics)
62
+ cb_handler = CallbackHandler(ifnone(callbacks, []), metrics)
63
+ cb_handler.on_train_begin(1, None, metrics); cb_handler.on_epoch_begin()
64
+ if keeped_items: cb_handler.state_dict.update(dict(keeped_items=[]))
65
+ val_metrics = validate(self.learn.model, dl, self.loss_func, cb_handler)
66
+ cb_handler.on_epoch_end(val_metrics)
67
+ if keeped_items: return cb_handler.state_dict['keeped_items']
68
+ else: return cb_handler.state_dict['last_metrics']
69
+
70
+ def jump_to_epoch_iter(self, epoch:int, iteration:int)->None:
71
+ try:
72
+ self.learn.load(f'{self.name}_{epoch}_{iteration}', purge=False)
73
+ logging.info(f'Loaded {self.name}_{epoch}_{iteration}')
74
+ except: logging.info(f'Model {self.name}_{epoch}_{iteration} not found.')
75
+
76
+ def on_train_begin(self, n_epochs, **kwargs):
77
+ # TODO: can not write graph here
78
+ # super().on_train_begin(**kwargs)
79
+ self.best = -float('inf')
80
+ self.timer.tic()
81
+ if self.host:
82
+ checkpoint_path = self.learn.path/'checkpoint.yaml'
83
+ if checkpoint_path.exists():
84
+ os.remove(checkpoint_path)
85
+ open(checkpoint_path, 'w').close()
86
+ return {'skip_validate': True, 'iteration':self.start_iters} # disable default validate
87
+
88
+ def on_batch_begin(self, **kwargs:Any)->None:
89
+ self.timer.toc_data()
90
+ super().on_batch_begin(**kwargs)
91
+
92
+ def on_batch_end(self, iteration, epoch, last_loss, smooth_loss, train, **kwargs):
93
+ super().on_batch_end(last_loss, iteration, train, **kwargs)
94
+ if iteration == 0: return
95
+
96
+ if iteration % self.loss_iters == 0:
97
+ last_losses = self.learn.loss_func.last_losses
98
+ self._write_sub_loss(iteration=iteration, last_losses=last_losses)
99
+ self.tbwriter.add_scalar(tag=self.metrics_root + 'lr',
100
+ scalar_value=self.opt.lr, global_step=iteration)
101
+
102
+ if iteration % self.show_iters == 0:
103
+ log_str = f'epoch {epoch} iter {iteration}: loss = {last_loss:6.4f}, ' \
104
+ f'smooth loss = {smooth_loss:6.4f}'
105
+ logging.info(log_str)
106
+ # log_str = f'data time = {self.timer.data_diff:.4f}s, runing time = {self.timer.running_diff:.4f}s'
107
+ # logging.info(log_str)
108
+
109
+ if iteration % self.eval_iters == 0:
110
+ # TODO: or remove time to on_epoch_end
111
+ # 1. Record time
112
+ log_str = f'average data time = {self.timer.average_data_time():.4f}s, ' \
113
+ f'average running time = {self.timer.average_running_time():.4f}s'
114
+ logging.info(log_str)
115
+
116
+ # 2. Call validate
117
+ last_metrics = self._validate()
118
+ self.learn.model.train()
119
+ log_str = f'epoch {epoch} iter {iteration}: eval loss = {last_metrics[0]:6.4f}, ' \
120
+ f'ccr = {last_metrics[1]:6.4f}, cwr = {last_metrics[2]:6.4f}, ' \
121
+ f'ted = {last_metrics[3]:6.4f}, ned = {last_metrics[4]:6.4f}, ' \
122
+ f'ted/w = {last_metrics[5]:6.4f}, '
123
+ logging.info(log_str)
124
+ names = ['eval_loss', 'ccr', 'cwr', 'ted', 'ned', 'ted/w']
125
+ self._write_metrics(iteration, names, last_metrics)
126
+
127
+ # 3. Save best model
128
+ current = last_metrics[2]
129
+ if current is not None and current > self.best:
130
+ logging.info(f'Better model found at epoch {epoch}, '\
131
+ f'iter {iteration} with accuracy value: {current:6.4f}.')
132
+ self.best = current
133
+ self._save(f'{self.bestname}')
134
+
135
+ if iteration % self.save_iters == 0 and self.host:
136
+ logging.info(f'Save model {self.name}_{epoch}_{iteration}')
137
+ filename = f'{self.name}_{epoch}_{iteration}'
138
+ self._save(filename)
139
+
140
+ checkpoint_path = self.learn.path/'checkpoint.yaml'
141
+ if not checkpoint_path.exists():
142
+ open(checkpoint_path, 'w').close()
143
+ with open(checkpoint_path, 'r') as file:
144
+ checkpoints = yaml.load(file, Loader=yaml.FullLoader) or dict()
145
+ checkpoints['all_checkpoints'] = (
146
+ checkpoints.get('all_checkpoints') or list())
147
+ checkpoints['all_checkpoints'].insert(0, filename)
148
+ if len(checkpoints['all_checkpoints']) > self.checpoint_keep_num:
149
+ removed_checkpoint = checkpoints['all_checkpoints'].pop()
150
+ removed_checkpoint = self.learn.path/self.learn.model_dir/f'{removed_checkpoint}.pth'
151
+ os.remove(removed_checkpoint)
152
+ checkpoints['current_checkpoint'] = filename
153
+ with open(checkpoint_path, 'w') as file:
154
+ yaml.dump(checkpoints, file)
155
+
156
+
157
+ self.timer.toc_running()
158
+
159
+ def on_train_end(self, **kwargs):
160
+ #self.learn.load(f'{self.bestname}', purge=False)
161
+ pass
162
+
163
+ def on_epoch_end(self, last_metrics:MetricsList, iteration:int, **kwargs)->None:
164
+ self._write_embedding(iteration=iteration)
165
+
166
+
167
+ class TextAccuracy(Callback):
168
+ _names = ['ccr', 'cwr', 'ted', 'ned', 'ted/w']
169
+ def __init__(self, charset_path, max_length, case_sensitive, model_eval):
170
+ self.charset_path = charset_path
171
+ self.max_length = max_length
172
+ self.case_sensitive = case_sensitive
173
+ self.charset = CharsetMapper(charset_path, self.max_length)
174
+ self.names = self._names
175
+
176
+ self.model_eval = model_eval or 'alignment'
177
+ assert self.model_eval in ['vision', 'language', 'alignment']
178
+
179
+ def on_epoch_begin(self, **kwargs):
180
+ self.total_num_char = 0.
181
+ self.total_num_word = 0.
182
+ self.correct_num_char = 0.
183
+ self.correct_num_word = 0.
184
+ self.total_ed = 0.
185
+ self.total_ned = 0.
186
+
187
+ def _get_output(self, last_output):
188
+ if isinstance(last_output, (tuple, list)):
189
+ for res in last_output:
190
+ if res['name'] == self.model_eval: output = res
191
+ else: output = last_output
192
+ return output
193
+
194
+ def _update_output(self, last_output, items):
195
+ if isinstance(last_output, (tuple, list)):
196
+ for res in last_output:
197
+ if res['name'] == self.model_eval: res.update(items)
198
+ else: last_output.update(items)
199
+ return last_output
200
+
201
+ def on_batch_end(self, last_output, last_target, **kwargs):
202
+ output = self._get_output(last_output)
203
+ logits, pt_lengths = output['logits'], output['pt_lengths']
204
+ pt_text, pt_scores, pt_lengths_ = self.decode(logits)
205
+ assert (pt_lengths == pt_lengths_).all(), f'{pt_lengths} != {pt_lengths_} for {pt_text}'
206
+ last_output = self._update_output(last_output, {'pt_text':pt_text, 'pt_scores':pt_scores})
207
+
208
+ pt_text = [self.charset.trim(t) for t in pt_text]
209
+ label = last_target[0]
210
+ if label.dim() == 3: label = label.argmax(dim=-1) # one-hot label
211
+ gt_text = [self.charset.get_text(l, trim=True) for l in label]
212
+
213
+ for i in range(len(gt_text)):
214
+ if not self.case_sensitive:
215
+ gt_text[i], pt_text[i] = gt_text[i].lower(), pt_text[i].lower()
216
+ distance = ed.eval(gt_text[i], pt_text[i])
217
+ self.total_ed += distance
218
+ self.total_ned += float(distance) / max(len(gt_text[i]), 1)
219
+
220
+ if gt_text[i] == pt_text[i]:
221
+ self.correct_num_word += 1
222
+ self.total_num_word += 1
223
+
224
+ for j in range(min(len(gt_text[i]), len(pt_text[i]))):
225
+ if gt_text[i][j] == pt_text[i][j]:
226
+ self.correct_num_char += 1
227
+ self.total_num_char += len(gt_text[i])
228
+
229
+ return {'last_output': last_output}
230
+
231
+ def on_epoch_end(self, last_metrics, **kwargs):
232
+ mets = [self.correct_num_char / self.total_num_char,
233
+ self.correct_num_word / self.total_num_word,
234
+ self.total_ed,
235
+ self.total_ned,
236
+ self.total_ed / self.total_num_word]
237
+ return add_metrics(last_metrics, mets)
238
+
239
+ def decode(self, logit):
240
+ """ Greed decode """
241
+ # TODO: test running time and decode on GPU
242
+ out = F.softmax(logit, dim=2)
243
+ pt_text, pt_scores, pt_lengths = [], [], []
244
+ for o in out:
245
+ text = self.charset.get_text(o.argmax(dim=1), padding=False, trim=False)
246
+ text = text.split(self.charset.null_char)[0] # end at end-token
247
+ pt_text.append(text)
248
+ pt_scores.append(o.max(dim=1)[0])
249
+ pt_lengths.append(min(len(text) + 1, self.max_length)) # one for end-token
250
+ pt_scores = torch.stack(pt_scores)
251
+ pt_lengths = pt_scores.new_tensor(pt_lengths, dtype=torch.long)
252
+ return pt_text, pt_scores, pt_lengths
253
+
254
+
255
+ class TopKTextAccuracy(TextAccuracy):
256
+ _names = ['ccr', 'cwr']
257
+ def __init__(self, k, charset_path, max_length, case_sensitive, model_eval):
258
+ self.k = k
259
+ self.charset_path = charset_path
260
+ self.max_length = max_length
261
+ self.case_sensitive = case_sensitive
262
+ self.charset = CharsetMapper(charset_path, self.max_length)
263
+ self.names = self._names
264
+
265
+ def on_epoch_begin(self, **kwargs):
266
+ self.total_num_char = 0.
267
+ self.total_num_word = 0.
268
+ self.correct_num_char = 0.
269
+ self.correct_num_word = 0.
270
+
271
+ def on_batch_end(self, last_output, last_target, **kwargs):
272
+ logits, pt_lengths = last_output['logits'], last_output['pt_lengths']
273
+ gt_labels, gt_lengths = last_target[:]
274
+
275
+ for logit, pt_length, label, length in zip(logits, pt_lengths, gt_labels, gt_lengths):
276
+ word_flag = True
277
+ for i in range(length):
278
+ char_logit = logit[i].topk(self.k)[1]
279
+ char_label = label[i].argmax(-1)
280
+ if char_label in char_logit: self.correct_num_char += 1
281
+ else: word_flag = False
282
+ self.total_num_char += 1
283
+ if pt_length == length and word_flag:
284
+ self.correct_num_word += 1
285
+ self.total_num_word += 1
286
+
287
+ def on_epoch_end(self, last_metrics, **kwargs):
288
+ mets = [self.correct_num_char / self.total_num_char,
289
+ self.correct_num_word / self.total_num_word,
290
+ 0., 0., 0.]
291
+ return add_metrics(last_metrics, mets)
292
+
293
+
294
+ class DumpPrediction(LearnerCallback):
295
+
296
+ def __init__(self, learn, dataset, charset_path, model_eval, image_only=False, debug=False):
297
+ super().__init__(learn=learn)
298
+ self.debug = debug
299
+ self.model_eval = model_eval or 'alignment'
300
+ self.image_only = image_only
301
+ assert self.model_eval in ['vision', 'language', 'alignment']
302
+
303
+ self.dataset, self.root = dataset, Path(self.learn.path)/f'{dataset}-{self.model_eval}'
304
+ self.attn_root = self.root/'attn'
305
+ self.charset = CharsetMapper(charset_path)
306
+ if self.root.exists(): shutil.rmtree(self.root)
307
+ self.root.mkdir(), self.attn_root.mkdir()
308
+
309
+ self.pil = transforms.ToPILImage()
310
+ self.tensor = transforms.ToTensor()
311
+ size = self.learn.data.img_h, self.learn.data.img_w
312
+ self.resize = transforms.Resize(size=size, interpolation=0)
313
+ self.c = 0
314
+
315
+ def on_batch_end(self, last_input, last_output, last_target, **kwargs):
316
+ if isinstance(last_output, (tuple, list)):
317
+ for res in last_output:
318
+ if res['name'] == self.model_eval: pt_text = res['pt_text']
319
+ if res['name'] == 'vision': attn_scores = res['attn_scores'].detach().cpu()
320
+ if res['name'] == self.model_eval: logits = res['logits']
321
+ else:
322
+ pt_text = last_output['pt_text']
323
+ attn_scores = last_output['attn_scores'].detach().cpu()
324
+ logits = last_output['logits']
325
+
326
+ images = last_input[0] if isinstance(last_input, (tuple, list)) else last_input
327
+ images = images.detach().cpu()
328
+ pt_text = [self.charset.trim(t) for t in pt_text]
329
+ gt_label = last_target[0]
330
+ if gt_label.dim() == 3: gt_label = gt_label.argmax(dim=-1) # one-hot label
331
+ gt_text = [self.charset.get_text(l, trim=True) for l in gt_label]
332
+
333
+ prediction, false_prediction = [], []
334
+ for gt, pt, image, attn, logit in zip(gt_text, pt_text, images, attn_scores, logits):
335
+ prediction.append(f'{gt}\t{pt}\n')
336
+ if gt != pt:
337
+ if self.debug:
338
+ scores = torch.softmax(logit, dim=-1)[:max(len(pt), len(gt)) + 1]
339
+ logging.info(f'{self.c} gt {gt}, pt {pt}, logit {logit.shape}, scores {scores.topk(5, dim=-1)}')
340
+ false_prediction.append(f'{gt}\t{pt}\n')
341
+
342
+ image = self.learn.data.denorm(image)
343
+ if not self.image_only:
344
+ image_np = np.array(self.pil(image))
345
+ attn_pil = [self.pil(a) for a in attn[:, None, :, :]]
346
+ attn = [self.tensor(self.resize(a)).repeat(3, 1, 1) for a in attn_pil]
347
+ attn_sum = np.array([np.array(a) for a in attn_pil[:len(pt)]]).sum(axis=0)
348
+ blended_sum = self.tensor(blend_mask(image_np, attn_sum))
349
+ blended = [self.tensor(blend_mask(image_np, np.array(a))) for a in attn_pil]
350
+ save_image = torch.stack([image] + attn + [blended_sum] + blended)
351
+ save_image = save_image.view(2, -1, *save_image.shape[1:])
352
+ save_image = save_image.permute(1, 0, 2, 3, 4).flatten(0, 1)
353
+ vutils.save_image(save_image, self.attn_root/f'{self.c}_{gt}_{pt}.jpg',
354
+ nrow=2, normalize=True, scale_each=True)
355
+ else:
356
+ self.pil(image).save(self.attn_root/f'{self.c}_{gt}_{pt}.jpg')
357
+ self.c += 1
358
+
359
+ with open(self.root/f'{self.model_eval}.txt', 'a') as f: f.writelines(prediction)
360
+ with open(self.root/f'{self.model_eval}-false.txt', 'a') as f: f.writelines(false_prediction)
configs/.gitattributes ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ftz filter=lfs diff=lfs merge=lfs -text
6
+ *.gz filter=lfs diff=lfs merge=lfs -text
7
+ *.h5 filter=lfs diff=lfs merge=lfs -text
8
+ *.joblib filter=lfs diff=lfs merge=lfs -text
9
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
10
+ *.model filter=lfs diff=lfs merge=lfs -text
11
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
12
+ *.npy filter=lfs diff=lfs merge=lfs -text
13
+ *.npz filter=lfs diff=lfs merge=lfs -text
14
+ *.onnx filter=lfs diff=lfs merge=lfs -text
15
+ *.ot filter=lfs diff=lfs merge=lfs -text
16
+ *.parquet filter=lfs diff=lfs merge=lfs -text
17
+ *.pickle filter=lfs diff=lfs merge=lfs -text
18
+ *.pkl filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pt filter=lfs diff=lfs merge=lfs -text
21
+ *.pth filter=lfs diff=lfs merge=lfs -text
22
+ *.rar filter=lfs diff=lfs merge=lfs -text
23
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
24
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
25
+ *.tflite filter=lfs diff=lfs merge=lfs -text
26
+ *.tgz filter=lfs diff=lfs merge=lfs -text
27
+ *.wasm filter=lfs diff=lfs merge=lfs -text
28
+ *.xz filter=lfs diff=lfs merge=lfs -text
29
+ *.zip filter=lfs diff=lfs merge=lfs -text
30
+ *.zst filter=lfs diff=lfs merge=lfs -text
31
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
configs/template.yaml ADDED
@@ -0,0 +1,67 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ global:
2
+ name: exp
3
+ phase: train
4
+ stage: pretrain-vision
5
+ workdir: /tmp/workdir
6
+ seed: ~
7
+
8
+ dataset:
9
+ train: {
10
+ roots: ['data/training/MJ/MJ_train/',
11
+ 'data/training/MJ/MJ_test/',
12
+ 'data/training/MJ/MJ_valid/',
13
+ 'data/training/ST'],
14
+ batch_size: 128
15
+ }
16
+ test: {
17
+ roots: ['data/evaluation/IIIT5k_3000',
18
+ 'data/evaluation/SVT',
19
+ 'data/evaluation/SVTP',
20
+ 'data/evaluation/IC13_857',
21
+ 'data/evaluation/IC15_1811',
22
+ 'data/evaluation/CUTE80'],
23
+ batch_size: 128
24
+ }
25
+ charset_path: data/charset_36.txt
26
+ num_workers: 4
27
+ max_length: 25 # 30
28
+ image_height: 32
29
+ image_width: 128
30
+ case_sensitive: False
31
+ eval_case_sensitive: False
32
+ data_aug: True
33
+ multiscales: False
34
+ pin_memory: True
35
+ smooth_label: False
36
+ smooth_factor: 0.1
37
+ one_hot_y: True
38
+ use_sm: False
39
+
40
+ training:
41
+ epochs: 6
42
+ show_iters: 50
43
+ eval_iters: 3000
44
+ save_iters: 20000
45
+ start_iters: 0
46
+ stats_iters: 100000
47
+
48
+ optimizer:
49
+ type: Adadelta # Adadelta, Adam
50
+ true_wd: False
51
+ wd: 0. # 0.001
52
+ bn_wd: False
53
+ args: {
54
+ # betas: !!python/tuple [0.9, 0.99], # betas=(0.9,0.99) for AdamW
55
+ # betas: !!python/tuple [0.9, 0.999], # for default Adam
56
+ }
57
+ clip_grad: 20
58
+ lr: [1.0, 1.0, 1.0] # lr: [0.005, 0.005, 0.005]
59
+ scheduler: {
60
+ periods: [3, 2, 1],
61
+ gamma: 0.1,
62
+ }
63
+
64
+ model:
65
+ name: 'modules.model_iternet.IterNet'
66
+ checkpoint: ~
67
+ strict: True
configs/train_iternet.yaml ADDED
@@ -0,0 +1,65 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ global:
2
+ name: train-iternet
3
+ phase: train
4
+ stage: train-super
5
+ workdir: workdir
6
+ seed: ~
7
+
8
+ dataset:
9
+ train: {
10
+ roots: ['output_pixelplanet_dataset/'],
11
+ batch_size: 20
12
+ }
13
+ test: {
14
+ roots: ['output_pixelplanet_dataset/'],
15
+ batch_size: 20
16
+ }
17
+ data_aug: True
18
+ multiscales: False
19
+ num_workers: 8
20
+
21
+ training:
22
+ epochs: 1000
23
+ show_iters: 500
24
+ eval_iters: 500
25
+ # save_iters: 1
26
+
27
+ optimizer:
28
+ type: Adam
29
+ true_wd: False
30
+ wd: 0.0
31
+ bn_wd: False
32
+ clip_grad: 20
33
+ lr: 0.0001
34
+ args: {
35
+ betas: !!python/tuple [0.9, 0.999], # for default Adam
36
+ }
37
+ scheduler: {
38
+ periods: [6, 4],
39
+ gamma: 0.1,
40
+ }
41
+
42
+ model:
43
+ name: 'modules.model_iternet.IterNet'
44
+ iter_size: 3
45
+ ensemble: ''
46
+ use_vision: False
47
+ vision: {
48
+ checkpoint: workdir/train-iternet/best-train-iternet.pth,
49
+ loss_weight: 1.,
50
+ attention: 'position',
51
+ backbone: 'transformer',
52
+ backbone_ln: 3,
53
+ iter_size: 3,
54
+ backbone_alpha_d: 0.5,
55
+ }
56
+ # language: {
57
+ # checkpoint: workdir/pretrain-language-model/pretrain-language-model.pth,
58
+ # num_layers: 4,
59
+ # loss_weight: 1.,
60
+ # detach: True,
61
+ # use_self_attn: False
62
+ # }
63
+ alignment: {
64
+ loss_weight: 1.,
65
+ }
data/charset_36.txt ADDED
@@ -0,0 +1,36 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 0 a
2
+ 1 b
3
+ 2 c
4
+ 3 d
5
+ 4 e
6
+ 5 f
7
+ 6 g
8
+ 7 h
9
+ 8 i
10
+ 9 j
11
+ 10 k
12
+ 11 l
13
+ 12 m
14
+ 13 n
15
+ 14 o
16
+ 15 p
17
+ 16 q
18
+ 17 r
19
+ 18 s
20
+ 19 t
21
+ 20 u
22
+ 21 v
23
+ 22 w
24
+ 23 x
25
+ 24 y
26
+ 25 z
27
+ 26 1
28
+ 27 2
29
+ 28 3
30
+ 29 4
31
+ 30 5
32
+ 31 6
33
+ 32 7
34
+ 33 8
35
+ 34 9
36
+ 35 0
dataset.py ADDED
@@ -0,0 +1,278 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import logging
2
+ import re
3
+
4
+ import cv2
5
+ import lmdb
6
+ import six
7
+ from fastai.vision import *
8
+ from torchvision import transforms
9
+
10
+ from transforms import CVColorJitter, CVDeterioration, CVGeometry
11
+ from utils import CharsetMapper, onehot
12
+
13
+
14
+ class ImageDataset(Dataset):
15
+ "`ImageDataset` read data from LMDB database."
16
+
17
+ def __init__(self,
18
+ path:PathOrStr,
19
+ is_training:bool=True,
20
+ img_h:int=32,
21
+ img_w:int=100,
22
+ max_length:int=25,
23
+ check_length:bool=True,
24
+ case_sensitive:bool=False,
25
+ charset_path:str='data/charset_36.txt',
26
+ convert_mode:str='RGB',
27
+ data_aug:bool=True,
28
+ deteriorate_ratio:float=0.,
29
+ multiscales:bool=True,
30
+ one_hot_y:bool=True,
31
+ return_idx:bool=False,
32
+ return_raw:bool=False,
33
+ **kwargs):
34
+ self.path, self.name = Path(path), Path(path).name
35
+ assert self.path.is_dir() and self.path.exists(), f"{path} is not a valid directory."
36
+ self.convert_mode, self.check_length = convert_mode, check_length
37
+ self.img_h, self.img_w = img_h, img_w
38
+ self.max_length, self.one_hot_y = max_length, one_hot_y
39
+ self.return_idx, self.return_raw = return_idx, return_raw
40
+ self.case_sensitive, self.is_training = case_sensitive, is_training
41
+ self.data_aug, self.multiscales = data_aug, multiscales
42
+ self.charset = CharsetMapper(charset_path, max_length=max_length+1)
43
+ self.c = self.charset.num_classes
44
+
45
+ self.env = lmdb.open(str(path), readonly=True, lock=False, readahead=False, meminit=False)
46
+ assert self.env, f'Cannot open LMDB dataset from {path}.'
47
+ with self.env.begin(write=False) as txn:
48
+ self.length = int(txn.get('num-samples'.encode()))
49
+
50
+ if self.is_training and self.data_aug:
51
+ self.augment_tfs = transforms.Compose([
52
+ CVGeometry(degrees=45, translate=(0.0, 0.0), scale=(0.5, 2.), shear=(45, 15), distortion=0.5, p=0.5),
53
+ CVDeterioration(var=20, degrees=6, factor=4, p=0.25),
54
+ CVColorJitter(brightness=0.5, contrast=0.5, saturation=0.5, hue=0.1, p=0.25)
55
+ ])
56
+ self.totensor = transforms.ToTensor()
57
+
58
+ def __len__(self): return self.length
59
+
60
+ def _next_image(self, index):
61
+ next_index = random.randint(0, len(self) - 1)
62
+ return self.get(next_index)
63
+
64
+ def _check_image(self, x, pixels=6):
65
+ if x.size[0] <= pixels or x.size[1] <= pixels: return False
66
+ else: return True
67
+
68
+ def resize_multiscales(self, img, borderType=cv2.BORDER_CONSTANT):
69
+ def _resize_ratio(img, ratio, fix_h=True):
70
+ if ratio * self.img_w < self.img_h:
71
+ if fix_h: trg_h = self.img_h
72
+ else: trg_h = int(ratio * self.img_w)
73
+ trg_w = self.img_w
74
+ else: trg_h, trg_w = self.img_h, int(self.img_h / ratio)
75
+ img = cv2.resize(img, (trg_w, trg_h))
76
+ pad_h, pad_w = (self.img_h - trg_h) / 2, (self.img_w - trg_w) / 2
77
+ top, bottom = math.ceil(pad_h), math.floor(pad_h)
78
+ left, right = math.ceil(pad_w), math.floor(pad_w)
79
+ img = cv2.copyMakeBorder(img, top, bottom, left, right, borderType)
80
+ return img
81
+
82
+ if self.is_training:
83
+ if random.random() < 0.5:
84
+ base, maxh, maxw = self.img_h, self.img_h, self.img_w
85
+ h, w = random.randint(base, maxh), random.randint(base, maxw)
86
+ return _resize_ratio(img, h/w)
87
+ else: return _resize_ratio(img, img.shape[0] / img.shape[1]) # keep aspect ratio
88
+ else: return _resize_ratio(img, img.shape[0] / img.shape[1]) # keep aspect ratio
89
+
90
+ def resize(self, img):
91
+ if self.multiscales: return self.resize_multiscales(img, cv2.BORDER_REPLICATE)
92
+ else: return cv2.resize(img, (self.img_w, self.img_h))
93
+
94
+ def get(self, idx):
95
+ with self.env.begin(write=False) as txn:
96
+ image_key, label_key = f'image-{idx+1:09d}', f'label-{idx+1:09d}'
97
+ try:
98
+ label = str(txn.get(label_key.encode()), 'utf-8') # label
99
+ label = re.sub('[^0-9a-zA-Z]+', '', label)
100
+ if self.check_length and self.max_length > 0:
101
+ if len(label) > self.max_length or len(label) <= 0:
102
+ #logging.info(f'Long or short text image is found: {self.name}, {idx}, {label}, {len(label)}')
103
+ return self._next_image(idx)
104
+ label = label[:self.max_length]
105
+
106
+ imgbuf = txn.get(image_key.encode()) # image
107
+ buf = six.BytesIO()
108
+ buf.write(imgbuf)
109
+ buf.seek(0)
110
+ with warnings.catch_warnings():
111
+ warnings.simplefilter("ignore", UserWarning) # EXIF warning from TiffPlugin
112
+ image = PIL.Image.open(buf).convert(self.convert_mode)
113
+ if self.is_training and not self._check_image(image):
114
+ #logging.info(f'Invalid image is found: {self.name}, {idx}, {label}, {len(label)}')
115
+ return self._next_image(idx)
116
+ except:
117
+ import traceback
118
+ traceback.print_exc()
119
+ logging.info(f'Corrupted image is found: {self.name}, {idx}, {label}, {len(label)}')
120
+ return self._next_image(idx)
121
+ return image, label, idx
122
+
123
+ def _process_training(self, image):
124
+ if self.data_aug: image = self.augment_tfs(image)
125
+ image = self.resize(np.array(image))
126
+ return image
127
+
128
+ def _process_test(self, image):
129
+ return self.resize(np.array(image)) # TODO:move is_training to here
130
+
131
+ def __getitem__(self, idx):
132
+ image, text, idx_new = self.get(idx)
133
+ if not self.is_training: assert idx == idx_new, f'idx {idx} != idx_new {idx_new} during testing.'
134
+
135
+ if self.is_training: image = self._process_training(image)
136
+ else: image = self._process_test(image)
137
+ if self.return_raw: return image, text
138
+ image = self.totensor(image)
139
+
140
+ length = tensor(len(text) + 1).to(dtype=torch.long) # one for end token
141
+ label = self.charset.get_labels(text, case_sensitive=self.case_sensitive)
142
+ label = tensor(label).to(dtype=torch.long)
143
+ if self.one_hot_y: label = onehot(label, self.charset.num_classes)
144
+
145
+ if self.return_idx: y = [label, length, idx_new]
146
+ else: y = [label, length]
147
+ return image, y
148
+
149
+
150
+ class TextDataset(Dataset):
151
+ def __init__(self,
152
+ path:PathOrStr,
153
+ delimiter:str='\t',
154
+ max_length:int=25,
155
+ charset_path:str='data/charset_36.txt',
156
+ case_sensitive=False,
157
+ one_hot_x=True,
158
+ one_hot_y=True,
159
+ is_training=True,
160
+ smooth_label=False,
161
+ smooth_factor=0.2,
162
+ use_sm=False,
163
+ **kwargs):
164
+ self.path = Path(path)
165
+ self.case_sensitive, self.use_sm = case_sensitive, use_sm
166
+ self.smooth_factor, self.smooth_label = smooth_factor, smooth_label
167
+ self.charset = CharsetMapper(charset_path, max_length=max_length+1)
168
+ self.one_hot_x, self.one_hot_y, self.is_training = one_hot_x, one_hot_y, is_training
169
+ if self.is_training and self.use_sm: self.sm = SpellingMutation(charset=self.charset)
170
+
171
+ dtype = {'inp': str, 'gt': str}
172
+ self.df = pd.read_csv(self.path, dtype=dtype, delimiter=delimiter, na_filter=False)
173
+ self.inp_col, self.gt_col = 0, 1
174
+
175
+ def __len__(self): return len(self.df)
176
+
177
+ def __getitem__(self, idx):
178
+ text_x = self.df.iloc[idx, self.inp_col]
179
+ text_x = re.sub('[^0-9a-zA-Z]+', '', text_x)
180
+ if not self.case_sensitive: text_x = text_x.lower()
181
+ if self.is_training and self.use_sm: text_x = self.sm(text_x)
182
+
183
+ length_x = tensor(len(text_x) + 1).to(dtype=torch.long) # one for end token
184
+ label_x = self.charset.get_labels(text_x, case_sensitive=self.case_sensitive)
185
+ label_x = tensor(label_x)
186
+ if self.one_hot_x:
187
+ label_x = onehot(label_x, self.charset.num_classes)
188
+ if self.is_training and self.smooth_label:
189
+ label_x = torch.stack([self.prob_smooth_label(l) for l in label_x])
190
+ x = [label_x, length_x]
191
+
192
+ text_y = self.df.iloc[idx, self.gt_col]
193
+ text_y = re.sub('[^0-9a-zA-Z]+', '', text_y)
194
+ if not self.case_sensitive: text_y = text_y.lower()
195
+ length_y = tensor(len(text_y) + 1).to(dtype=torch.long) # one for end token
196
+ label_y = self.charset.get_labels(text_y, case_sensitive=self.case_sensitive)
197
+ label_y = tensor(label_y)
198
+ if self.one_hot_y: label_y = onehot(label_y, self.charset.num_classes)
199
+ y = [label_y, length_y]
200
+
201
+ return x, y
202
+
203
+ def prob_smooth_label(self, one_hot):
204
+ one_hot = one_hot.float()
205
+ delta = torch.rand([]) * self.smooth_factor
206
+ num_classes = len(one_hot)
207
+ noise = torch.rand(num_classes)
208
+ noise = noise / noise.sum() * delta
209
+ one_hot = one_hot * (1 - delta) + noise
210
+ return one_hot
211
+
212
+
213
+ class SpellingMutation(object):
214
+ def __init__(self, pn0=0.7, pn1=0.85, pn2=0.95, pt0=0.7, pt1=0.85, charset=None):
215
+ """
216
+ Args:
217
+ pn0: the prob of not modifying characters is (pn0)
218
+ pn1: the prob of modifying one characters is (pn1 - pn0)
219
+ pn2: the prob of modifying two characters is (pn2 - pn1),
220
+ and three (1 - pn2)
221
+ pt0: the prob of replacing operation is pt0.
222
+ pt1: the prob of inserting operation is (pt1 - pt0),
223
+ and deleting operation is (1 - pt1)
224
+ """
225
+ super().__init__()
226
+ self.pn0, self.pn1, self.pn2 = pn0, pn1, pn2
227
+ self.pt0, self.pt1 = pt0, pt1
228
+ self.charset = charset
229
+ logging.info(f'the probs: pn0={self.pn0}, pn1={self.pn1} ' +
230
+ f'pn2={self.pn2}, pt0={self.pt0}, pt1={self.pt1}')
231
+
232
+ def is_digit(self, text, ratio=0.5):
233
+ length = max(len(text), 1)
234
+ digit_num = sum([t in self.charset.digits for t in text])
235
+ if digit_num / length < ratio: return False
236
+ return True
237
+
238
+ def is_unk_char(self, char):
239
+ # return char == self.charset.unk_char
240
+ return (char not in self.charset.digits) and (char not in self.charset.alphabets)
241
+
242
+ def get_num_to_modify(self, length):
243
+ prob = random.random()
244
+ if prob < self.pn0: num_to_modify = 0
245
+ elif prob < self.pn1: num_to_modify = 1
246
+ elif prob < self.pn2: num_to_modify = 2
247
+ else: num_to_modify = 3
248
+
249
+ if length <= 1: num_to_modify = 0
250
+ elif length >= 2 and length <= 4: num_to_modify = min(num_to_modify, 1)
251
+ else: num_to_modify = min(num_to_modify, length // 2) # smaller than length // 2
252
+ return num_to_modify
253
+
254
+ def __call__(self, text, debug=False):
255
+ if self.is_digit(text): return text
256
+ length = len(text)
257
+ num_to_modify = self.get_num_to_modify(length)
258
+ if num_to_modify <= 0: return text
259
+
260
+ chars = []
261
+ index = np.arange(0, length)
262
+ random.shuffle(index)
263
+ index = index[: num_to_modify]
264
+ if debug: self.index = index
265
+ for i, t in enumerate(text):
266
+ if i not in index: chars.append(t)
267
+ elif self.is_unk_char(t): chars.append(t)
268
+ else:
269
+ prob = random.random()
270
+ if prob < self.pt0: # replace
271
+ chars.append(random.choice(self.charset.alphabets))
272
+ elif prob < self.pt1: # insert
273
+ chars.append(random.choice(self.charset.alphabets))
274
+ chars.append(t)
275
+ else: # delete
276
+ continue
277
+ new_text = ''.join(chars[: self.charset.max_length-1])
278
+ return new_text if len(new_text) >= 1 else text
demo.py ADDED
@@ -0,0 +1,109 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import argparse
2
+ import logging
3
+ import os
4
+ import glob
5
+ import tqdm
6
+ import torch
7
+ import PIL
8
+ import cv2
9
+ import numpy as np
10
+ import torch.nn.functional as F
11
+ from torchvision import transforms
12
+ from utils import Config, Logger, CharsetMapper
13
+
14
+ def get_model(config):
15
+ import importlib
16
+ names = config.model_name.split('.')
17
+ module_name, class_name = '.'.join(names[:-1]), names[-1]
18
+ cls = getattr(importlib.import_module(module_name), class_name)
19
+ model = cls(config)
20
+ logging.info(model)
21
+ model = model.eval()
22
+ return model
23
+
24
+ def preprocess(img, width, height):
25
+ img = cv2.resize(np.array(img), (width, height))
26
+ img = transforms.ToTensor()(img).unsqueeze(0)
27
+ mean = torch.tensor([0.485, 0.456, 0.406])
28
+ std = torch.tensor([0.229, 0.224, 0.225])
29
+ return (img-mean[...,None,None]) / std[...,None,None]
30
+
31
+ def postprocess(output, charset, model_eval):
32
+ def _get_output(last_output, model_eval):
33
+ if isinstance(last_output, (tuple, list)):
34
+ for res in last_output:
35
+ if res['name'] == model_eval: output = res
36
+ else: output = last_output
37
+ return output
38
+
39
+ def _decode(logit):
40
+ """ Greed decode """
41
+ out = F.softmax(logit, dim=2)
42
+ pt_text, pt_scores, pt_lengths = [], [], []
43
+ for o in out:
44
+ text = charset.get_text(o.argmax(dim=1), padding=False, trim=False)
45
+ text = text.split(charset.null_char)[0] # end at end-token
46
+ pt_text.append(text)
47
+ pt_scores.append(o.max(dim=1)[0])
48
+ pt_lengths.append(min(len(text) + 1, charset.max_length)) # one for end-token
49
+ return pt_text, pt_scores, pt_lengths
50
+
51
+ output = _get_output(output, model_eval)
52
+ logits, pt_lengths = output['logits'], output['pt_lengths']
53
+ pt_text, pt_scores, pt_lengths_ = _decode(logits)
54
+
55
+ return pt_text, pt_scores, pt_lengths_
56
+
57
+ def load(model, file, device=None, strict=True):
58
+ if device is None: device = 'cpu'
59
+ elif isinstance(device, int): device = torch.device('cuda', device)
60
+ assert os.path.isfile(file)
61
+ state = torch.load(file, map_location=device)
62
+ if set(state.keys()) == {'model', 'opt'}:
63
+ state = state['model']
64
+ model.load_state_dict(state, strict=strict)
65
+ return model
66
+
67
+ def main():
68
+ parser = argparse.ArgumentParser()
69
+ parser.add_argument('--config', type=str, default='configs/train_iternet.yaml',
70
+ help='path to config file')
71
+ parser.add_argument('--input', type=str, default='figures/demo')
72
+ parser.add_argument('--cuda', type=int, default=-1)
73
+ parser.add_argument('--checkpoint', type=str, default='workdir/train-iternet/best-train-iternet.pth')
74
+ parser.add_argument('--model_eval', type=str, default='alignment',
75
+ choices=['alignment', 'vision', 'language'])
76
+ args = parser.parse_args()
77
+ config = Config(args.config)
78
+ if args.checkpoint is not None: config.model_checkpoint = args.checkpoint
79
+ if args.model_eval is not None: config.model_eval = args.model_eval
80
+ config.global_phase = 'test'
81
+ config.model_vision_checkpoint, config.model_language_checkpoint = None, None
82
+ device = 'cpu' if args.cuda < 0 else f'cuda:{args.cuda}'
83
+
84
+ Logger.init(config.global_workdir, config.global_name, config.global_phase)
85
+ Logger.enable_file()
86
+ logging.info(config)
87
+
88
+ logging.info('Construct model.')
89
+ model = get_model(config).to(device)
90
+ model = load(model, config.model_checkpoint, device=device)
91
+ charset = CharsetMapper(filename=config.dataset_charset_path,
92
+ max_length=config.dataset_max_length + 1)
93
+
94
+ if os.path.isdir(args.input):
95
+ paths = [os.path.join(args.input, fname) for fname in os.listdir(args.input)]
96
+ else:
97
+ paths = glob.glob(os.path.expanduser(args.input))
98
+ assert paths, "The input path(s) was not found"
99
+ paths = sorted(paths)
100
+ for path in tqdm.tqdm(paths):
101
+ img = PIL.Image.open(path).convert('RGB')
102
+ img = preprocess(img, config.dataset_image_width, config.dataset_image_height)
103
+ img = img.to(device)
104
+ res = model(img)
105
+ pt_text, _, __ = postprocess(res, charset, config.model_eval)
106
+ logging.info(f'{path}: {pt_text[0]}')
107
+
108
+ if __name__ == '__main__':
109
+ main()
figs_captchas/1fkL.jpg ADDED
figs_captchas/4QS3.jpg ADDED
figs_captchas/4eLO.jpg ADDED
figs_captchas/5UZB.jpg ADDED
figs_captchas/6H3e.jpg ADDED
figs_captchas/8Kgm.jpg ADDED
figs_captchas/8sAX.jpg ADDED
figs_captchas/944m.jpg ADDED
figs_captchas/ArYq.jpg ADDED
figs_captchas/EH65.jpg ADDED
figs_captchas/GIQR.jpg ADDED
figs_captchas/GYyo.jpg ADDED
figs_captchas/H184.jpg ADDED
figs_captchas/HSv8.jpg ADDED
figs_captchas/JRRW.jpg ADDED
figs_captchas/L24g.jpg ADDED
figs_captchas/MIly.jpg ADDED
figs_captchas/NCpP.jpg ADDED
figs_captchas/NshS.jpg ADDED
figs_captchas/R0cv.jpg ADDED
figs_captchas/ReEH.jpg ADDED
figs_captchas/Sp87.jpg ADDED
figs_captchas/TJll.jpg ADDED
figs_captchas/TwO2.jpg ADDED
figs_captchas/Uk9I.jpg ADDED
figs_captchas/VDke.jpg ADDED
figs_captchas/VKft.jpg ADDED
figs_captchas/ZEmP.jpg ADDED