File size: 194,599 Bytes
d1b39b2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GnFaZD50Obo-"
      },
      "source": [
        "# Agent-Environment Interface\n",
        "\n",
        "\n",
        "\n",
        "![image.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAbIAAACvCAYAAACYTFvtAAAgAElEQVR4Ae2dB5gURd6Hl2AWEBUQFVCROwyIARQxcICAohgIJlRETjEgnqKCoN6ZCAomFBVFEBTMAocZQTEgnFlEBTGBoIAKoiCGr77nrZvq65mdme3ZnZnumf3t8/TT01XVVdW/6q23q+pfVSUl+pMCEVdg8eLFZvDgwaZ169amfv36pmrVqqakpERHnjSoVauWady4sTnxxBPNlClTzMaNG9Fef1JACkgBKVCWAuvXrzcXXXSRwJUnYAX9OGjSpImZM2eOYFbWCyx/KSAFKrcCP/zwg2nRokVcq4sKtGfPnqZv37468qzBEUccYbbYYguvPGgVjx8/XjCr3P+menopIAVSKfB///d/pkOHDl6l2bRpUzNz5kxVmqkEy5M7HxcDBgww1apVs2VTvXp188orr6hc8qS/kpECUqCAFJgwYYIHsVatWpm1a9eqsoxQ+TFO5mD2l7/8xfz+++8qnwiVj7IiBaRAyArQGqMFxnhNzZo1zTfffKNKMuQySZY8Y5duTG3y5Mkqo2QiyU0KSIHKqcBHH33kVZD/+Mc/VEFG9DVYuXKl2WyzzWxZde/eXeUU0XJStqSAFAhBgUmTJnkgmz17tirIEMogaJJt27a1ZbXbbrupnIKKpnBSQAoUvwIjR470QLZ06VJVkBEu8rPOOsuWVY0aNcyff/6psopwWSlrUkAK5FGBq666ygPZihUrVDnmUftMkzrvvPO8spLBR6bqKbwUkAJFq4BAVjhFK5AVTlkpp1JACuRRAYEsj2JXMCmBrIIC6nYpIAXCUeDDDz80CxcuzNmYiEAWTrmWJ1WBrDyq6R4pIAVCV+Dyyy+34yL16tWzi8jeeeedBpP5bA32C2ShF3HgDAhkgaVSQCkgBaKkgAOZmwzrzoCtR48e5o477qgQ2ASyKJV2+rwIZOn1ka8UkAIRVSAVyBzQ3Llu3bqGibKAbcGCBYFbbAJZRAs+SbYEsiSiyEkKpFKAuUWDBg3SEQENDjroIM/k2kEryLlOnToWbKNHjzaMs6XqihTIUv0XRM9dIItemShHEVaAhUmDVJYKUzibTQK2bt26mdtvv912RbLGIq9gWCB79913zRlnnGGPYcOG2bxE+F8iElkTyCJRDMpEoSggkBUOoMrzMdGgQQNz6aWXmlWrVpmwQOavlNl3q5BX3P/yyy/NkiVLzB9//JFTIPs104ToQqlNlc/QFBDIigtkDRs2NKeddpoZN26cWbRoUVxlGwbIfvvtN0MLEQi7hXDvv//+uHyF9vKXI2H3LN99911On0EgK0fh6JbKq4AfZIcccohhg798HVS0mJfr+K8Gbn29TFpejRo1suACDosXL05buYYBsmnTplmI7bDDDia24r7p1KlT2nxG+b9RIIty6ShvlVYBP8jatGlTsBVMMRRgEKvFXXbZxZx++ukGcH322WcZlVcYIDvxxBMtyNhja968efY3G0cG2Qtt48aN5pNPPrHPmUlX3rfffmutOb/++uuUhi/J3hdaWR9//LFZtmyZceOKieEEskRFdC0FIqCAQBaBQohlIRnIABeGEuPHj7djMxXJbb5BtmbNGrPllltaeL311lsWDjwPLU4sLFM9C2NCV199tdlmm21sWMLvuOOO5sEHH7Qw3GOPPcyNN95Y6v4XX3zRYPlZpUoV7z62QRk7dmwpMD333HOGeO69916ra7t27eLu23fffc2bb77ppTF48GAb3r97M/evXr3aC5Pqecrjrq7F8qimeyqtAgJZdIoekFHR9+rVy0yYMMF8/vnnWa0k8w0yngEI7bnnnh5ILrvsMuvWsmXLpM9GS+iEE06wYTbffHPTpUsX07NnT7Pzzjtb0Jx66qnWb8CAAXH3s5Ny9erVrV+LFi1sdytwqlq1qnXr06ePlwdK/NFHH7Xu/fv3N4wr7rrrroZWY79+/ew1+a5Vq5ahdUd4rC0PO+wws8kmm9j7WrVqZa/phs/FGySQ5UJVxVm0Cghk0SnaX375JSeVonvCfIMMkACE66+/3nuu+fPnWzdaTVj/uby580MPPWT9AZd/zG/9+vXm+OOPt37E6QcZ3ZS0/GgtPfzww3FxvvLKKxZI3PP44497fg5km266qYXlhg0bPD9aklh7cs9dd93luZNHdS26ktJZCkRIAYEsQoWR46zkE2TLly+3YAFY/pYlLS5aP0DimmuuiYMEjx9rqZknnniilN/KlSsN5vuJIBs4cKB1u+CCC0rdQ5zMp+Oejh07ev4OZLTiMKlPlN4Zptxwww1xfmGAbNasWQYg6ygMDaZPn26mTJniHZo+kfjflYNrgSwHokY0ynyC7Oabb7bwoDsuUQ43Frj33nvH+X3//fe2K5CxMcz2E+/j+phjjrHx+ltk+++/v3Xzj2n57411D7oxN+vlQEYXoT+s+33FFVfYOKMAMiCso3A1+Pnnn5O+Y+5d0zkLCghkWRCxQKLIJ8j2228/W/keddRRZtSoUXHH2Wef7VXMb7/9tvdPPnfuXOseW6orqaoxgHldi7Twtt56a3sf608yhpZ4nHTSSdaf1iHdhkTsQMY9yRISyAoXHFGDvkCW7D8sy24CWZYFjXB0+QIZ8wL9loPpKpaY8YdVbebMmRY47du3TwoXAsWsFT2QrVu3zt5DGgCtRo0aaQ+BTIBK9z7mwk8gy0OlKJDlQeSIJJEvkA0ZMsTCpVmzZgZrwmQHq49QaWDU4RY5di2yWFdhUtXc2JXrWvS3yDJZ+qqQWmSTJk1KqmEyXeWW/H3Lpy70QviBKJAl/VfOrqNAll09oxxbPkAGWFhthH/k2267jXPSP7afcf/sGDEQiDlZmMtjdh9raZW6t23btvY+BzICuDGymEVkqXt++uknM3v2bPPOO+94+SkkkMlYoFSRRtrBfWy591sgy0NxCWR5EDkiSeQDZHPmzLGgwRQey8V0j77XXnvZsOecc44Xzlktstda4r2s8uHmivlBduWVV9p4YnPMEm8rufbaa60/8/Ocp0DmlNA52woIZNlWNEB8AlkAkUIMkkl3WVnZzAfIzj//fAuNI4880oNGqnwNHz7chsWk/ddff7Xhp06dasfXGOviNy087mddTlbbcBOS/SADmLVr17ZxMebm5uPRZTlx4kS7WDFgzWaLbOnSpTZfqZ6tou6aEF1RBcO7XyALQXuBLATRM0iS5amYd5WN1T5yDTJM5rfddlsLlNiqHmmflLUinVEIc29c4BikbDz169c3u+++u52TxpJQznTfDzLue/755705ZkyObtKkibfqPt2VsekALolyWy0SL11GTZs2NYcffriWqPIU1Q+ngEDmlMjjWSDLo9jlSAqQub52dwZs5Vl/MdcgY9HdW265xR6pxrgSJWAdRO6JdUlab1phjzzyiGE3BpaJqlu3rsFknwnRbokrTPoT41q4cKE588wzrQEJ97HifteuXc3LL79cKiwtPNKdMWNGKT/ife2116x/bKFjL6mnn37aQpL4Wf9Ray160uhHTAGBLIRXQSALQfQMkkwGMgc0d3Yr4rOwcLoV8XMNsgweK2VQ8g+QWPk+WSB2vua5/S24ZOEK3U1di4VbggJZCGWXC5CxJBGFOWjQIDNixAgdFdDAWeQ5aAU5YzXotnrxr1dYCCBr3bq1BVVsH7O4/wjGwpgrxjJVuVqsNy7BEC8EshDFr2DSAlkFBSzP7bkA2euvv24royCVrsLkdoIqFT8bdrJAbyGADOMM3gm6E9liBaMKlph68skn7TYq+MXGycrzuhfMPQJZwRRVqYwKZKUkyb1DLkHGauWsWh7Fg72rMMGO+oFBQSawB1wdOnQwrBHIOI+zBuRNKgSQMT7GavnOOtH/7Fge9u3bN+6Zcv8fEk4KAlk4umcjVYEsGypmGEcuQRYbz8gwRwruV6CsMTIHrqFDhxpawqnGloizEEDmnp1uxHHjxhlWCaGLmh2xv/jiC6BeKf4EssItZoEshLITyEIQPYMkE0HG/Cq2IwFcb7zxRlpwJSZTSCBLzHtluxbICrfEBbIQyk4gC0H0DJK88MILTadOnQw7FGcKrsRkBLJERaJ7LZBFt2zKyplAVpZCOfAXyHIgakSjFMgiWjBJsiWQJRGlQJwEshAKSiALQfSQkhTIQhK+HMkKZOUQLSK3CGQhFIRAFoLoISUpkIUkfDmSFcjKIVpEbhHIQigIgSwE0UNKUiALSfhyJCuQlUO0iNwikIVQEAJZCKKHlCQGI25e1ooVKyqNKXtIclcoWQcy9mZzG49WKELdnDcFBLK8Sf2/hASy/2lR7L9YKcOB7K233hLIIlzgbpfhBg0aqJwiXE7JsiaQJVMlx24CWY4FjlD0mO87kN14442qICNUNv6sMKndbYfTvn17lZNfnAL4LZCFUEgCWQiih5Qky1WxiSUwYysYlg4LKStKNo0C99xzjy0jyomtZtIElVcEFRDIQigUgSwE0UNMkiWfXKuMPb5CzIqSTqLAxx9/bPdgo4zY8yxX+50lSVpOWVJAIMuSkJlEI5Blolbhh2XDS3ZcdjDr2bNn0W+JUiilxiafrkuR8hkzZow+NAql8Hz5FMh8YuTrp0CWL6Wjkw4bV7ouRirM2rVrm969exu6tO677z4dedaAnQoOOugg7+OCMmFpMnYCiM5bo5wEVUAgC6pUFsMJZFkUs4Ci+vTTT80+++wTV3m6VprOud0jLp2+bF9z3XXXCWIF9L+UmFWBLFGRPFwLZHkQOaJJ/PHHH3Z7lAMPPNBUrVpVUCsJD2C0kJk7Rms5oq+LshVQAYEsoFDZDCaQZVPNwo1r7dq15r333jP/+c9/dORZg8q0z1rh/ocEz7lAFlyrrIUUyLImpSKSAlJACpQIZCG8BAJZCKIrSSkgBYpWAYEshKIVyEIQXUlKASlQtAoIZCEUrUAWguhKUgpIgaJVQCALoWgFshBEV5JSQAoUrQICWQhFK5CFILqSlAJSoGgVEMhCKFqBLATRlaQUkAJFq4BAFkLRCmSZi87CrkuWLNHE1cyl0x1SoOgVEMhCKGKBLHPRDz30UHPaaacJZJlLpzukQNErIJCFUMQCWeaiC2SZa6Y7pEBlUUAgC6GkowCyP//807Dun//xcfv9998NZ797RX4HjZO8+Fce57c/LwJZRUpB90qB4lZAIAuhfMMC2cMPP2w6depk5s6daxo0aGBYuJbHBxiXXXaZty/TdtttZ6688koPdOyfdf7553twW7lypWnevHlcVx87IR9wwAHm7rvvtuG++eYb06VLF7PFFltwbTbffHPTo0ePuE0LjzrqKDN58mS7fUa1atXMY489Zu+dMmWKady4sb2vZs2aZujQoUYgC+FFVZJSoEAUCBtkfIgnNgwKRLryZzMskN15550GSO22226mf//+Zvr06RYcQGrLLbc0I0eONM8++6y5+uqrDWC5+OKLrf8VV1xhAIorqCeeeMJCZrPNNjPr16+3YV588UXr9uqrr9prwLPjjjuaBx980MY5atQom0a/fv2sP+o1atTIbmsCFNlenoVcn3/+eVOlShVz9NFHm2nTppmHHnrI7LLLLoa0NEZW/ndOd0qBYlYg3yD77bff4nquaCB07drVq9uKWWvv2cIEGa2jcePGeYJ/9dVXdjuRe++913Mjo7SC2Kfpxx9/NG+88QZ+Zv78+TYML03r1q2t28yZM63bkCFDzPbbb+/Brlu3bmbq1KnWzz04LTQK3F0Dsp133tmDIe6HH364be3RSnThSBe4CWROEZ2lgBTwK5BvkLGrOHWky8M111xjRowY4V0795ye+/TpY2bNmpVRogsWLLA7+q5atSqj+5I9SJggYw8stg9x+Zo4caKFxPvvv2++/PJL76BlBrxmz55tvzxoybH5IPfRhXjHHXeYPfbYw1x11VXW7eCDDzZ0Qbp43ZkWG/s9Pf7447Y7MxFk5557rnfPhg0bzKabbmpuvfVWz83Fs+uuuwpkTgydpYAUiFMgbJDFZSZfF3SbjRkzplRlmS59uryo2D///POM7ksWZ5ggq169elz+YzpYgACRxOPpp5+24U8++WTTvn1720IDhm+99ZbtnqQL8eeff7b3Mbblnvepp54y++23n23VbbPNNqZly5aGFlgiyOi2dPd8//33/PbGypw7Z42R+dXQbykgBfwKZAKyJ5980tZDzZo1s0MbNGwS96djN/dTTz3VEAZbAj7i+dCmIXPmmWfaoY7999/fXHrppbb+Ylhm9OjRXl2GkRsf+3zgE0fnzp1LNZ4uuugi8/LLL5vx48fbHi7C0RhYunTpf+NZvXq1oaVBxIzRuBYIFS7dZFTEZIBWiBNj3bp1tgLlHsaSqKid37Jly1wrwTA+RHec8yPORx55xKbFeA7xOL9U5yiB7L777rN60OebKr+4oyfjVDwrYKKg6DrEDWjxcYDuhGXyMtfsvksrz8V7wgknpAUZBiN0Z95+++3ePe5eWn/qWnRq6CwFpIBfgaAgo2eIYYpjjz3Wdg0OHDjQDonstddeXp1DnUUP1D777GOuv/56ayuAsRpgY6iFLkVsCjp27GhZQT4Sx8j69u1raDT8/e9/t+Hbtm1r69lnnnnGS6devXqmTZs25pBDDjFjx441N910k6ldu7Zp1aqVKWG33Vq1atkDIwEygIXeihUrDN2DuPHVT0ahJJlYtGiR2WmnnWxY/PEjzOWXX279ARuJ4saYjusL5T5aGVTm3EdrBgOHd999197nF9r/O0ogY4dinivWlehlEyjTovrll1/sswAp4MRXCGNdBKQFxUtBa+uwww7znvn++++3cTpDEMJiTk8BpWuREY6xNwrWPwXgo48+si+BQOYVj35IASngUyAoyAYMGGA6dOgQN9WHj3nqwDVr1tg6rHfv3rY+p5HikuDjmgYQFtu4JY6R+UEGFwiLgZu7n/qMHq19993Xc4MpGN7560nG2shLCZUsYzguE4sXL7Ym4MOHD/ciSOxapGlZv359Q8uLhLHOO/vssy2gnNFBsq5FKnAq9m+//dbeh8k5Gd1zzz3jKmL3MO4cJZCRJ4C+ww472FYsTeoJEyaYGjVq2K8Jl2fO9kuhpMTcfPPNnpYxE34zbNgwzy1muWjQnJcDEJ1xxhkWeoTfuHGjDctHgL9rkTT4YqIg+fp5++23zUsvvWSaNm1qy1Ag85eGfksBKeAUCAoyF96daWHRxecHGSxg+pELw5lwMMD1uKUDGVOQYIwL6+KhccCHv7OzAGTOMtyFueeee2xeSuiTpLnm7yr75JNPPEhxQyLIaJW88847cRmPzYfyKt1EkFHJ8vCJRiOxaxNr6bj8xZ3DAhndolgFxmWmpKSErlfGwOjW45loYdI0/umnn+LCIjKtpQ8//NBzxzADN7oT/fEOGjTItlCJb+uttzY04WMfE+bII4+0YbFsdPPO/Pfy9UMTm3vJEyb7gwcPNtdee21cGv579FsKSIHKq0BQkFGnMd5FnVWnTh0LHNfbxkd3rAfKftSnUzMdyLDgbtiwYam6at68ebZOcz12pBtrgXlJeSB74IEHbN8kEdHSYmwnZkTgBU4EGR70i9IlRguBibvMm6Iida2HRJBNmjTJ+iMglbY7YhOHzaOPPlrqQVwGwgKZSz/VGaDR/epv6qYKG8SdlwaLRQZJXXgGVd2YpXNLdma8jCa6a+4nCyM3KSAFpAAKBAUZ81MBCEZuNF5oNbFQBHU9dQ31PR/Pd911l1dnET9DI9RjbsgjHcgAZWx4Kq5wXCPHffSnBRl30pUFkLB0o3VBi+Df//63l7FEkNGqYGCOrsILLrjA3HbbbW58LCXIACYPD7iYXJx4JLbw/E8UVZD586jfUkAKSIFCUSAIyPhAp57HsML/XIljZHvvvbc56aST4sK41lRsTm3aMbLY/NlSvXx0V2611VYeU9KCjPEZDDtcRmmNMcjGOItz84OML3+WUaLry/lzdgYLqVpkM2bMIHycBSP30ZdKHvytEH+8/BbIEhXRtRSQAlKg/AoEARnDTRgC0lPnbB9ef/11awxIXe7GrmitwQi6+aj/sYFo166dNexzQ1a0yPxDHX5jD+7BiANjuc8++8wuEoExHQ0q/7zZtCAjMGbffknOOecct3afdfaD7IcffrADcNjyu3vI7DHHHGNBlQpkNEkxQ6cF5xa75UxarHCRrntOIHNK6ywFpIAUqLgCQUBGKnQZ0ipjfAxLdQ7M3jHCwCKd4RCM/QAObjRy4AV1ult+j3gwKMQiHnsMrjEyxM7APQnTtzBmA5C0wjiztqzfAARLd//qINxL69CuT4sVCKaPmFhiPk9rjIzQXegSISCEdfDiN4SlO5J1BplTwBwCEneL57LCBdeAK/a7hHEy0sISj3QxQSctF69LL/EcNsiwrnzuuefsQd6mT5+edBJyYr51LQWkgBSIogJBQUbeGadnPizL69Ejh9trr71m1551Y2C4sfgFtg5+a0X37PT04YdVtXNLPNMgeuWVV+zC6P45y4nhkl5DU6jWvXt3CxbmBMQWtPXCAxoA50wfGeTDKg43qErFDjkx92biHHGSqUsuucQSmBXbXWQ0TZn0BsRosro+VOef7BwmyGg1Umh8fbC8FLPPyU/s6yFZduUmBaSAFIi0ApmALNIPUkiZCxNk6ASYscoE3nxtLF++3M71KiQNlVcpIAWkgFNAIHNK5PEcNshY/QQTU+Y75PGxlZQUkAJSICcKCGQ5kTV9pGGDDGsbBjyXLFkikKUvKvlKASlQAAoIZCEUUtggYzyPiYHJHp21xD744IOkfsnCy00KSAEpELYC5QEZBhtYMbK6EHYD9FC5tRTDfp6CSD9MkDHPjW5F1v1KFAtDFebbxdY7TPTWtRSQAlIgkgpkCjI262W5Prd9C+b2WK67+WWRfMhcZIolrdirCyEy7aILE2TsxcMq/YnLdqERa4CxXUqmz5MLfRWnFJACUiCoApmAjGX4mIzsXzOWtWWxdA+aXlGEmzZtmp07BsyYTOcmxQV9uDBBRp75GkmWVyDnpiQk85ebFJACUiCKCgQFGfPEWIqQhSvcczAliWWp/BtjOr+iPjPXjDlnPCQisEhkJg8cJsjS5ZOvErcjdLpw8pMCUkAKREmBoCB74YUXqKvjPuax4saNLayi9Ew5zwurhLB8iduM0u19FjThqIKMfdRYcNmtJxb0eRROCkgBKRCmAkFBxsaarEzvVvCgIXLccceZunXrem48B3Nt6aEK85lynjYVPUtYIQhddZkmGEWQsVQLS23NnTs34+fJ9PkVXgpIASmQTQWCgqxnz552fUUARvrsYoJb165d7bZRbB3FQr9YdTNm5jZbzmZeIxGXW/SRfbbYIqZXr15xFT9GFG4V5VQZjiLIUuVV7lJACkiBqCsQFGTsT0lvGq0wxsnYIPmII44wzZs3N7GtuUpoqOBGlyMts6g/e8b5Y2uWYcOGeQ92yimnWAFcRIsXLzatW7c23333nRfG+fnPAplfDf2WAlJAClRMgaAgIxWmGWGwx0r3XGPFCNBcDtjxhGlIrtXm3IvmTJOzSZMmdidRHrJjx45xy/CPHDnSds/FdgJN+dwCWUpp5CEFpIAUyFiBTEBWVuTsbnL66ad7YCsrfMH5051I05TZ4KNGjbJbZPupjREI7mU9mEBWlkLylwJSQAoEVyCbIGOFD1b7oOESPAdFFJItX5hUXNYjCWRlKSR/KSAFpEBwBbIJsuHDh9v9K/2NlOA5KfCQLG1CtyPbYpf1KAJZWQrJXwpIASkQXIFsgix4qkUYkgHDxo0bG7a4LuvxBLKyFJK/FJACUiC4AgJZcK3KDIm1S5mBSkpKBLIgKimMFJACUiCYAgJZMJ2yGkogy6qcikwKSIFKroBAFsILIJCFILqSlAJSoGgVEMhCKFqBLATRlaQUkAJFq4BAFkLRCmQhiK4kpYAUKFoFBLIQilYgC0F0JSkFpEDRKiCQhVC0AlkIoitJKSAFilYBgSyEohXIQhBdSUoBKVC0CghkIRStQBaC6EpSCkiBolVAIAuhaAWyEERXklJAChStAgJZCEUrkIUgupKUAlKgaBUQyEIoWoEsBNGVpBSQAkWrgEAWQtEKZCGIriSlgBQoWgUEshCKViALQfRyJLl+/Xq7eepFF11kjj/+eMN+czqkQS7egT59+pjRo0ebpUuXBlp4vByvc1HfIpCFULwCWQiiZ5AkG+rdeuutpm7dulQqOqRB3t6B6tWrm7POOsusXr1aQMvgf1Ygy0CsbAUVyLKlZPbj+eWXX0znzp3jKq4qVarYvebq169vdEiDbL8D7GO41VZbxb1zO++8s3n//fcFs4D/4gJZQKGyGUwgy6aa2YuLlliXLl28CmX77bc3t9xyi1m1apUqlOzJrJiSKPDrr7+aGTNmmBYtWnjvX506dcyyZcv07iXRK9FJIEtUJA/XxQiy66+/3rz55psF/U83duxYrxLZa6+9zPLlywv6efLwKiuJLCvw+++/m3PPPdd7Dzt27Kh3MIDGAlkAkbIdpBhBdtxxx5mpU6cW7D/db7/9ZnbaaSdbgdSsWdN8/fXXBfss2X5fFV9+FaBnoFOnTvZdZIz21Vdf1btYRhEIZGUIlAtvgSy5qnPmzDEXXnihOfHEE82ff/5pBg0aZA488EDz448/5vwfedasWV7FMXTo0Jynl1wBuUqB/yqwcOFCw9gsIOvbt6/exzJeDIGsDIFy4S2QpVYVM+Tu3bubm2++2QAUBsLzYcF13XXXeSBbtGiRKo7URSSfPCnQrFkz+042b95c72MZmgtkZQiUC2+BLLWqLVu2ND169DAzZ87M6z9v7KvXbL755rY1mDqH8pEC+VHg5JNPtiDbdttt8/q/kJ+ny24qAll29QwUm0CWXCZaXtWqVbPzaJKHyJ3r6aefbisNQJa7VBSzFAiuwKmnnmrfyZhpfvAbK2FIgSyEQmjSKe8AABFASURBVC9GkC1ZssSsWbOmQhCYMmWK/cedP39+0ngYBA9aXMzBYbzts88+C3SPQBZUWYXLlwICWXClBbLgWmUtZDGCLBvisKLBvvvua5IBa/Hixeaee+6JgxKwev311+PcXD6Io2nTpgZrROeW7iyQpVNHfmEoIJAFV10gC65V1kIKZMmlxPw9ZnQRF2DevHmmYcOGZsCAAeaHH37wwPTggw9ao5C4wLGL9957z66PmMwvmZtAlkwVuYWpgEAWXH2BLLhWWQspkJWWEktBYPXhhx96oHKhMMX/61//alj9ADcmjXI88MADZuTIkfY31y485xEjRpg777wzzs3vn/hbIEtURNdhKyCQBS8BgSy4VlkLWYwg+/TTT+NaS1kTq6SkZMGCBW79QxttmzZtzEEHHWRN8xs1amR/t2vXLg5aXAcdHyNSgSybJaa4sqGAQBZcRYEsuFZZC1mMIMvlyh6jRo0yt912WxyoKIxUXYtr1641LDGVSYEJZJmopbD5UEAgC66yQBZcq6yFrKwgY2X5ZIYcZQl7zDHHmAkTJphvvvkmDk6pQPbEE09kvBqCQFZWKcg/3woIZMEVF8iCa5W1kJUNZB9//LHp16+fqVWrVmArQr/YEydONG+//XYcxPDH5J9uR39YNiZ89NFHzZNPPmm+/fbbOD9/uMTfAlmiIroOWwGBLHgJCGTBtcpayMoAMszeH3vsMdO+fXtg4h1BzeGzJnbAiASygEIpWN4UEMiCSy2QBdcqayFzCTKs+6644oq8Hmzf4sbI2D/pn//8p2FjQD/A3O9HHnnEPP7445E7MCAhj1rZI2uvuSKqoAICWXABBbLgWmUtZC5B5oCRz/Ndd91lBg4caFtfbNWez7SznZZAlrXXXBFVUAGBLLiAAllwrbIWMhcgY51CDCLCOD755BPgVbJhwwY7twvTeLcFRSJoLrjgArt0FMtHRelgFRDyKpBl7TVXRBVUQCALLqBAFlyrrIXMBciylrksRYRxBktObbnllnEttMoyRvbVV1+Zzz//PNBB2CzJHigaJpiTty+//DKv6QbKnAJ5CghknhRl/hDIypQo+wEqA8icat9//7255ZZbTJMmTSzQKgvI6tatGwfwxJap/7p+/fp5BUpsmS9Tu3btvKbr3gmdgykgkAXTiVACWXCtshayMoHMicb8sZdeeimye31l22rRgYyJ2YceemjaA0MZp1M+zgJZblS+5JJLDO/Rd999l5XyzAXI+D9kxZv77rvPnHbaaaZbt25ZyWtuFA0eq0AWXKushSxGkNHqSjbXK2ui5TiiXIHs6aefjlxFsXHjRsPcvIcffjhyectxMec0+l133RU9DfMbs5FQNkAGuNg54t5777XgatCggc2j6xFgPDsbeQ07DoEshBIoRpA58/sQ5MxKkpUJZFkRTJGUUiAKIANcLMA9duxY07Nnz5TTYASyUsUnh0wVEMgyVSz34aMEsjlz5tjW0vr16w2WoHfffbfp1auXOemkk8xll11mWCnFrwgTz2ldrVq1Ks7dhVm3bp31JwxjlLTI+M3qJy4MZ7p+cWcnAaxgmY94yimnmBUrVsSFmzt3rt1Sp3v37rYr7cYbbyy1fJiLl+XCZsyYYe9npZVrr73WxklrY/jw4Unz/Oyzz9rVWYgDg5Qrr7zSPjsaTJo0yVvmDCOZq666ypAPusluv/12q5dL23+mgn/hhRfsCjOE7927t22lsGyaPxy/ly9fbnX44IMPrB973mFhi/59+vSxfn/88Yd3Hz0R6Oa6k8kH1+ieGHcm10FaZA5c7NUHuNgKyUEqyLlly5YGDQr9YOUg//P+/PPPFdI+k3KqtGEFstRF/9FHHxkqMirlfFrzRQlkxx57rP2nZOPQ/fff3/6uWrWq94+6ySabmKlTp3r/qH/729+s3+jRoz03v8L333+/9afSwj3VGNlhhx1mw6E7E+tdxcAXPvdReZ9zzjlJp1Zgncral/50+b3NNtuY3XbbzXY7b7/99jZO/7Ng6OLid/c2a9bM8IyzZs0yW2+9dal7sIZ9+eWXTY0aNUr50VXmhwxxUqkdddRRNqx7Jnemq+2NN97Az/t75plnbNghQ4aYoUOHGvLLdBL/lBJ6IFw6QM7F5z9nsvuCl7jvRzKQYXHKThOAC/9MweXPXzH/Fsh8L1KufgpkyZVlU03+OalIzzjjDMPgefKQ2XeNIsjYLbtFixbmtddes1/3VPidO3e2lSYVGF/jKEGLjUoptjpJKXHcMmFjxoyx4csCWceOHS1AmPMHHH/66Sd73+DBg206rJnJJHjGXoDtxRdfbCv5atWq2bz6MwDIaKnsuOOOtiVGJUxLhWdylqwnn3yyjd/dB8iAB/fyDrBY9Jo1a2xrzlW+m222mW2BfPHFFzY+NCB9/P2QJ06n2T777GP9AAytrC5dutjwpEM8Ln0HMvQH0Oy8QAsVC9xk6fBMtGZj1qcW6FzTonZxlufsQEYe+BihdYyOTgOd/7f0XaIWAll53rgM7xHIkgtGxcYKIfjSvcMAdfKQ2XfNFcioPNu2bZv2YJsa/xO5Ftm2225rK0+/H5XppptuaiszFkjGz7lRkaObPzwWdKy2QmUIDPArC2S0ghYuXBgXz8qVK+1kcQBDa8ifBr9j5Waf0+8HJKhkDj/8cA+8zj9mCGNbbM6NMyDjnkTAAe7dd9/d+h188MGl4uvRo4f1o/vSxUd3InHtsssu3vM7P+KLWe3ZlqZzdyDjPgd/58eZbkz8YmD3vHI5RkbFPHPmTEMrEStYQE4edMRrQNczXe+0XL2C0Y/cKFCMIGM8IdUYTVAVqSBq1qxpJ+tyT0XHGIKmS7hcgSxIRUN3nT+vDmSMh/nd3W9nefb11197/mx1Q1p33HGH50Z4rElxj1W+NoqyQEYLy6Xlzq57slOnTqX8CMMecMASmPrfAwey6dOnl7qPSdnkDci4dDg7kMUg5PcqcV2EsfzE+V199dU2vn/9619efK5cU+0WHtuR3NSrV88DowMZHxKMJ8YlUlJSQncj+WYM0e+XS5D50+G3Axvjh3QJlxdstCJp9RbLkS2L0US9dZ1EgWIEWZLHzNiJiplJuvvtt1/KQfugkWKwEKuwA93iKrxsLVHlBv6xHmMJr3RH4nYzDmQYDCTLfDKQTZkyxVautHz897Ru3dq609Xl3MsCWWLXHPc58+Zhw4Z58bj43DlZWg5kiXvJcU9ZIKPb0sXtzg5kyaY1JAOZ674kPF2AyQ43ZudarA5kqbpqowAyp4c7AzbKGOMX3oHYe2zLHuimOorF/N7poHMeFRDI4sX+9ddfvX3KYpW3HZvxh0o0CPD7Jf6me61r164m1Vd4YniucwWyZBVusvT9bg5k06ZNK1WREy4ZyBjHwvjB373IuA+VNMYW/q6WskCG1aQ/P/yOdfOZyZMnl/JzYbGao8Kke8e5OZAl+6jIB8i22morm6dUFbnf3bVwHciOPPJI7znc83COIsj8+eM3VogYy5QFNoEsUTldB1ZAIIuXCnNuLBVxZdwCfbBMc9fAANNngOfupCvr6KOP9q6duztjsIAxgrsu61zoIOP5HEic9aKrcDGi8T9/eUDmDA+YSO2Py//bhYkiyNhaiLHIdIczaikGkPnLhd8ObLRaaWm6FptAlqiUrgMrUIwgYx+y8o5psUcZrRBaDfzDNWzY0AMbRga0rvr27Rs3f4puoHbt2iWtVOlmYZzFWfUFKZhiAJmrgDt06GB1weqOFlriNIbygCw2HmT3mkulZ8y83/hbdGG3yFzXIl2KqfKd6O50LOQWWeIzJV5jUTl79my7W0ain66lQCAFihFkFVnZg759vuIxbb7hhhtKWcVRMbuKCKOBm266ydDKwIKN3xzOgo8CoCI699xzA1dc3FMMIGOyM3O1mIMVGxNz86fi3svygIzNUOmGO+CAA5J+IKA/1pGk/eOPP3rahw2yM8880+Y7ZvQSpwMX7733nm2hnH/++V6eKwPISgkhBymQqQICWXDF+HLcY489vMqT+T8YNowbN86wIC+/OfwLtfbv39889dRTXsUUJLViABnPGauQTePGjW0FHjMCiZOgPCCjpewmNGPA4o+QicHOjD1xEdqwQYbJOgCuU6eOXSzXn2/AHzOOsROfnZ9A5pTQWQqkUUAg+684bmWENFLZZZP4qqaL0R8uXdfi3nvvbRhD84cv63euQMZYBAYHZR2A2eWxPMYe7t5Yt56tvLfbbru4cUUXpjwg494HHnjAm/jMEk+0ooGaWxEEaCWaP4cNMvLN5HpgxtQO5n6xAgmGQHS94k7L3v++VBRkjMPRIqaL22lenrMbc4wZrJQnCt0jBXKnQGUHGeNpWFMBnLJUxtwbQw9/1yH3UPHEVmbwomCwnvE2dntOXI/QC5TiR7ZBRuXIChhBj/Hjx3taYCHIfaksHvfcc0/rn6gJj8bHAS1V7k81D42uP/wbNWrkpcm9jAnhTqs3hUx2FRHCAAD/wZJWyXY/YLyT8P7uRhc36yjiF5s35pxLMOPHfcGCBaXywTqJ+CWbY4ZxC36s4ehFVlJSQsvrvPPO81b+8OcbCPtX9eC+F1980caT2Lp0cQIq0rnmmmvi0mEdRn/cuViiyuVBZykQugKVEWQYXtDNQ+XAWAr/8LQYyioMKmbmhJUVrqL+2QZZRfMT5ftpDfPBwLjTrbfealsf+SijimoCOPlgIN8ssfXOO+94XdYVjZv70cDpwnhveY2fXF7UInNK6BxJBfwga968ua3gqeQL9Zg3b55dVTxmYBCnOZUelR3jXP6vVX6zcgJf6mEfmPULZHHFposIKCCQRaAQlIXUCvhBlli5F+I1K3EkPi3dTCy9lMmE1LCefcSIEQJZYgHqOnQFBLLQi0AZSKdAMYPszTffNK1atYrb8iIsQAVNVyBL97bKLywFBLKwlFe6gRQoZpAxqZnV0VmJ3I2FBQVKWOEEskCvrQLlWQGBLM+CK7nMFGA5Jnb1LZYDK69kCrBQLEsDpdr8b4sttjCDBg0K/QC8GiNLVoJyC1MBgSxM9ZV2pVSAVs38+fOTAg3zZ6AdW1LKM/oIYrWYLzEFsnwprXSCKiCQBVVK4aRAlhQIukQVGzb269fPTk4VyLIkvqIpSgUEsqIsVj1UlBUICjL3DExeZoUFdx32WS2ysEtA6ScqIJAlKqJrKZBjBTIFWY6zk3H0AlnGkumGHCsgkOVYYEUvBRIVEMgSFdG1FKiYAgJZxfTT3VIgYwUEsowl0w1SIK0CAllaeeQpBbKvALs8L1++PDJjXpk+oboWM1VM4XOtgECWa4UVvxQoMgUEsiIr0CJ4HIGsCApRjyAF8qlAr1697Pw29g/LZ7pKSwqkUkAgS6WM3KWAFEiqwJAhQyzIqlSpErfbdNLAcpQCeVCgTZs29p2M7RyRhxSVhBSo5AqwFQvboRSqDOwj5dZ6nDhxYsE+R6Hqr3zHK7B69WrDEm68k2wsG++rKykgBXKiQKFbLQJit+UMOyxv3LhRlUdO3hRFGkSBgQMHWogBssmTJ+tdDCKawkiBiipQ6CDj+fv37+9VHmeddZZhJf+K6qL7pUCmCjz11FOmWrVq9l1s3LhxQfd0ZPrsCi8FQlWgGEC2du1a06RJEw9mHTt2NIsWLRLMQn2zKk/i69atM4zVuu2PNtlkE5Ns1/XKo4ieVArkWYFiABmSsaixf9sZjD8OPvhg07dvXx3SIGfvQLdu3byubboTgdmYMWP0EZXnekzJVXIFigVkFOOyZctM586dvZaZMwLRuUSalOReg4YNG5qZM2cKYpW8TtXjh6AAe5GtWLGiqP75Zs2aZXr37m0Yp3DdPYJZ7ivyyqhxvXr1zNFHH23Gjx9vNmzYUFT/RyFURwWf5P8D/bEUDHRibs8AAAAASUVORK5CYII=)\n",
        "\n",
        "## Markov Decision Process (MDP)\n",
        "\n",
        "An MDP is defined by a tuple:  **(S, A, P, R, γ)**\n",
        "\n",
        "* **S**:  **State Space** - The set of all possible states an agent can be in.\n",
        "* **A**:  **Action Space** - The set of all possible actions an agent can take.\n",
        "* **P**:  **Transition Probability** -  `P(s'|s, a)`: The probability of transitioning to state `s'` when taking action `a` in state `s`.\n",
        "* **R**:  **Reward Function** - `R(s, a, s')`: The reward received for transitioning to state `s'` by taking action `a` in state `s`.\n",
        "* **γ**:  **Discount Factor** - A value between 0 and 1 that determines the importance of future rewards.\n",
        "\n",
        "\n",
        "## Other Common Notations\n",
        "\n",
        "* **s**: Current state\n",
        "* **a**: Current action\n",
        "* **s'**: Next state\n",
        "* **r**: Reward\n",
        "* **π**: Policy - A function that maps states to actions (determines the agent's behavior).\n",
        "* **V(s)**: Value function - The expected cumulative reward starting from state `s` and following a policy `π`.\n",
        "* **Q(s, a)**: Action-value function - The expected cumulative reward starting from state `s`, taking action `a`, and then following a policy `π`.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 🔄 The RL Loop\n",
        "\n",
        "No matter if you are playing GridWorld, CartPole, or training a robot to walk, the code structure is *always* the same. This is the skeleton you will see in every RL library:\n",
        "\n",
        "```python\n",
        "# The Universal Skeleton\n",
        "env = gym.make(\"YourEnvironment\")\n",
        "state, _ = env.reset()\n",
        "done = False\n",
        "\n",
        "while not done:\n",
        "    # 1. Agent picks an action\n",
        "    action = agent.choose_action(state)\n",
        "    \n",
        "    # 2. Environment responds\n",
        "    next_state, reward, terminated, truncated, info = env.step(action)\n",
        "    \n",
        "    # 3. Agent learns from the result\n",
        "    agent.learn(state, action, reward, next_state)\n",
        "    \n",
        "    # 4. Move to next state\n",
        "    state = next_state\n",
        "    \n",
        "    if terminated or truncated:\n",
        "        break"
      ],
      "metadata": {
        "id": "ucTVrR_NQvu5"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ICZsiFY2HqMb"
      },
      "source": [
        "# Example\n",
        "\n",
        "A Treasure Hunter agent is trapped in an m x n island from which it cannot escape. Its task is to dig for buried treasures that are spread across different locations on the island. The island also contains several traps which always incur damage to the agent whenever visited. At any location on the island (indicated by a cell), the agent can take possible actions among lef t, right, up, and down, but not all actions are valid in every cell, i.e., if the agent tries to go outside the island, it is considered as an invalid action. On visiting a cell, the agent tries to dig for a possible treasure which causes a small digging cost. If the agent finds the buried treasure, it earns a positive reward, whereas if the cell turns out to be a trap, it receives a negative reward. The cells containing treasure are assumed to have an infinite amount of buried treasure.\n",
        "\n",
        "\n",
        "![image.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAaYAAAGKCAYAAABHH+VOAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAEnISURBVHhe7d15fBXV3T/wz8zcLXsI+5aENewgQhRccKGCgggVFa2i6A8Vt6dF0KItdcGlqFXUVlpt1YqiVVDcEBdAlEXZIUBCSEISQiDEkI0s996Z8/sjYW5mIiEhCZm5+bxfr3k9nJnvncc0N/dzz8yZcyQhhAAREZFFyOYdRERELYnBRERElsJgIiIiS2EwERGRpTCYiIjIUhhMRERkKQwmIiKyFAYTERFZCoOJiIgshcFERESWwmAiIiJLYTAREZGlMJiIiMhSGExERGQpDCYiIrIUBhMREVkKg4mIiCxF4gq2jeP1erF161asW7cOaWlpKCsrQ5cuXTBw4ECMHTsWXbp0gSRJ5pcFFSEEMjIysG7dOmiaBgC47LLLEB8fby61FU3TcOjQIfz8889ISUnB4cOH4fV60aZNG8TGxmL06NEYNGgQXC6X+aWWpqoqsrOzsWXLFqSkpCA3NxelpaWIjo5G9+7dkZiYiHPPPRehoaHmlwaFdevWISUlRW/3798fF154oaHGTlatWoWcnBzz7jp169YNV1xxhXm3dQg6I5qmic8++0wMGzZMSJIkANTawsPDxaxZs0R+fr755UFB0zSRkpIiZs6cKcLDww0/+7Jly8zltlFRUSE++OADMWbMGOHxeGr9Xk9ukiSJIUOGiPfee0/4/X7zaSxF0zRRVFQkXn75ZTFixAjhcrlq/Tw1tx49eohXXnlFVFZWmk9la0lJSSIiIsLws86aNctcZisjRoyo9fs73XbllVeaT2MpDKYz4Pf7xbx5804ZSOatf//+Ii0tzXwa29I0Tezdu1dMnz5duN3uWj8vbBxM27ZtE4mJibV+ntNtM2fOtPSHuN/vF5dcckmt/+7TbVOmTBEVFRXm09lSUVGRGDx4cK2f0c7B5PP5RFRUVK2f6XSb1YOJl/IaSAiB+fPnY8GCBfq+4cOHY8GCBbjkkkugKAp27tyJhQsX4qOPPtJrBg0ahHXr1qFNmzb6PjuqrKzEHXfcgffffx+qqur7IyMjUVxcrLeXLVuG3/72t3rbLt555x1Mnz4dAKAoCsaMGYMJEyZg4MCB6NChA44fP459+/bhww8/xLp161Dzz+ehhx7Cs88+a9lLt88++yzmzZuHiIgIXHnllRg3bhwGDx6Mjh07oqysDLt378YHH3yAjz/+WL8kCwBz587FX//6V8v+XPWhqipmzJiBd955B6j+3Z58/86aNQv/+Mc/TK+wh5ycHMTHx8Pv98PpdOKBBx6A0+k0l9XSt29fzJgxw7zbOsxJRXX79ttvhdPp1L95XHrppaKoqMhcJnw+n5gzZ47hW8oDDzxgLrMdVVXF0KFD9Z9p2LBh4oMPPhCvvfaa4We1a4+pvLxcDBw4UEycOFHs2LFDqKpqLhGiugfy+uuvG94LHo9H7Nq1y1xqGYcOHRKPPPKIyM3NFZqmmQ8LUf37feutt4SiKPrPFRISYvvL0a+99pp+hePKK6809Jzs3GNas2aN/nP07NnzlL9Xu2EwNYDX6zVcz42OjhYZGRnmMl1FRYUYNmyY4Q88PT3dXGY7Dz/8sBg+fLj43//+p1+++s9//hMUwSSEEJmZmcLn85l316JpmnjwwQcNP/eDDz5oLrMdTdPEpEmTDD/XJ598Yi6zjZ9//lm/B+pyuURSUpLo3bu3/rPZOZhq/t2NHz/efNi2OFy8AdauXYutW7fq7TvvvBNxcXGGmprcbjfmzp2rt8vLy/Huu+8aauzooYcewqZNm3DdddfZbkRafcTGxsLhcJh31yJJEu69917Dvg0bNhjadiRJEsaMGWPYV1hYaGjbRUFBAW6++WaUlpYCAB588EEMHDhQb9tdcnKy/u/BgwcbjtkZg6kB3nvvPcM9hdtvv/20190nTZqE8PBwvb18+XLDcTuKiYmp13Xs1qBr167o2rWr3s7KyjIct6ua9w8BoGPHjoa2HWiahrvuugv79+8HqoeFP/LIIwCAiooKU7X9CCFw4MABvd2nTx/DcTtjMNWTqqr45ptv9HavXr3Qt29fQ82vCQ8PR2Jiot7euXMnioqKDDVkX5IkGUI6LCzMcNyOVFXF559/rrfbtGmDCy64wFBjdUIILFq0SB+AFBkZif/+97/6l8SagzvsyhxMAwcONBy3MwZTPWVlZeHIkSN6OzEx8bS9pZOGDx+u/1vTNCQlJRmOk32VlpYaHm7s3bu34bjd+P1+PP7441i3bp2+7/HHHzf0+u3gxx9/xKOPPgoAkGUZixYtwrnnnmsuszW/34/09HQAgMfjqfO2gt0wmOpp//79hssbDfkA6tatm6GdmZlpaJN9rVmzBj6fT29ffvnlhuN2oGkaSktLsWrVKlx11VV48skngeoh1XPmzME999xT7y9hVnDkyBHMmDED5eXlAIB77rkHt9xyi61+hvo4evSofq9M0zRMnz4dQ4YMQb9+/XDBBRfg+uuvx1NPPYVNmzbB6/WaX25pDKZ6Onr0qKHdvXt3Q7suNe9BAMCxY8cMbbInv9+PV155RW+HhYVh6tSphhqr+uMf/4jJkydj4sSJSExMRFxcHMaPH69frh48eDA++eQTLFy4EIqimF9uWX6/H3fddRfS0tIAABdccIHtfob6qjnwwev1YvXq1di9ezdSUlKwYcMGfPjhh/jTn/6EUaNG4dxzz8U777xT696hVTGY6sk8KqkhD8qaL4PUfBCV7Ot///sf1q5dq7fvvPNOxMbGGmqsav369VixYgW++OILbN26FQUFBfqxm2++GR999BEmTJhgu17GwoUL8emnnwIAOnfujPfffx8hISHmsqCQlZWFLl26oH///jjnnHNw7rnnYujQoYiPj681qjQpKQnTp0/H5MmTcfz4ccMxK2Iw1VNZWZmh3ZBRaeZvazUv/ZA97d+/H3/4wx/0do8ePfR7GnbQrVs39OnTB3369EFsbKxhwtYlS5Zg0KBBuPrqq7Fz507D66xs9erVeOyxx4Dqey7vvPNOrcvoweSWW25BWloadu3aha1bt2Lz5s3Ytm0bUlJScOjQIXzwwQcYN24cZDnwMf/5559j2rRplv8MYjDVk7nX05AusflNYD4X2UtBQQGmTZuGvLw8oPp5tTfffBNt27Y1l1rW0qVLsX//fqSkpCA1NRXp6en49NNPcc0110BRFPh8PnzxxRe48MIL8f777xsek7CinJwcTJ8+HT6fD5Ik4cknn8Rll11mLgsqLpcLHo8HDocDkiRBkiTIsgyXy4WOHTvi+uuvxxdffIF3333XMFr066+/xuuvv244l9UwmOopJibG0G7IkG9zrZ0+wMjoxIkTmDp1KrZv3w5Uj/h68cUXcfHFF5tLbUGSJP2D7Oqrr8bHH3+MZcuWISoqCqgedXjrrbdi06ZN5pdaRmVlJW6//XZ9dOQNN9yA3//+97a7DNkcFEXBtGnT8Morrxj+93jllVdqfWG2EgZTPXXo0MHQbsiDlOba9u3bG9pkDyUlJfjtb3+LNWvWANUf6vPnz8fdd98dNB+CkiThmmuuwb/+9S/9Z/J6vZg7dy78fr+5vMUJIbBgwQJ8/fXXQPXzSpMnT8a3336Lr7766pRbzZ8lKyvLcKzmoIJgccstt6B///56Ozk5GYcOHTLUWIp5jiL6dVlZWcLhcOjzUk2fPt1cckp33323Yd6x1NRUc4ntBdNceb8mPz9fXHrppfrPJ0mS+NOf/mT5dZjOlM/nM0x0KkmSOHDggLmsxRUVFYm2bdsa3nuN3R566CHz/5ugcM899xh+zm+//dZcYhnsMdVTly5d0LlzZ739888/1+u6uxDCML9edHS07Vd2bW0OHTqECRMm6D0lWZbxxBNP4LHHHqs1sCVYOBwODBo0SG8LIfSpfaymPn+HVPsWQs1BEVbD9ZgaYPr06fp6LgCQlpaGnj17GmrMjhw5gvj4eFRWVgIApkyZEhTz5Zm9+eabuP322/W2XddjqkkIgZSUFEyaNAmpqalA9TX7F198Effee6+l/7Cbwo033oj3339fb69cuRLjx4831LQ0r9eLJUuW6H9f9TV79mx9vryLLroIN954o35s2LBhGDVqVI3q4GD+fR48eNC6s0WYu1B0ap9//rmhK/znP/+5zvVPNE0TixYtMrzmvffeM5cFhWC7lKdpmvjxxx9Fx44d9Z8pNDRUvP/+++bSoFRRUWFYGkKSpKBahTkyMlL/2ey87EV95eXliZiYGP1n7t+/f72Wdmkpwf2Vr4mNHTvWMFHiq6++WucNxKKiIjz33HN6Oz4+HpMnTzbUkPUIIfDhhx/iN7/5jT7jR4cOHbBy5UrccMMN5vKg9M9//tMwQejIkSNt8/AwGfn9fjzwwAOGh6jvueeeWg/hWgmDqQHcbjcWLFigX8I5fvw4brvttl99krqsrAwzZ87Ug+vksxXB+hR6sNA0DS+++CJuueUWfa61IUOGYN26dbYdEg4Aq1atwvPPP4/c3Nw678l4vV68/PLLhnXEZFnGX/7yF0t/kLU2SUlJ+PLLL+tcvkMIgaysLNxwww2GS3jnnXceZs6caai1Gt5jaiBVVXH//ffjtdde0/cNHjwY8+bNw+jRoyFJErZt24aFCxdi48aNes1tt92GN954I2hvlgfDPSafz4eHH34YL730kv7hLcsyZsyYUevGcV1GjRplqZ6xpmmYNGkSvvjiC4SHh2PUqFG4+OKL0a9fP3Tr1g2SJOHw4cPYvn07li9fjj179uivlSQJ8+bNw4IFC4JmSDwAREVF6VODzZo1C//4xz/MJZY2a9YsLF68GHFxcbjkkkswYsQI9O7dGzExMfD5fEhJScGaNWvw2WefGZ6jTEhIwFdffWX9AVjma3t0euXl5eKWW24x3FM51SZJkpg+fbooKysznyaoBMM9phUrVtT6/Z3Jdu+995pP3aJycnJEWFhYrf/O020RERHi5ZdfFqqqmk9pe3a+x1RcXCw6d+5c6/dV1yZJkrjmmmvE4cOHzaezJF7KOwMejwdvvfUW3nrrrTpH5Q0cOBDvvfce3nzzTV7CswG7LQ1QX507d8b333+Pu+++u15zx3Xt2hVz5szB7t27cf/99wf96EO7CQ0NxRtvvIHJkycjMjLSfNjA7XbjqquuwqpVq7B8+XLDIy9Wxkt5jeT1evHTTz9h27ZtyM7OhtvtRnx8PEaOHIlBgwa1muvy2dnZ2LVrl94+99xz0alTJ0ON1eXl5TXJIo5du3ZFQkKCeXeLE0LA7/fj4MGD2LFjBw4fPoz8/HyoqoqYmBh06dIFQ4cORd++ffX514LVqlWr9Nkf4uLiDM9s2YUQAuXl5dizZw+Sk5ORm5uL48eP61NM9erVC4mJiYiOjrbd75LBRERElsI+OhERWUqr7TFVVFTghx9+MO8mIqLT6NixIwYPHtxslwhbbTBlZ2fzgUEiojMQEhKCN954A1OnToXL5TIfbjQGU/WElcHm5K+1ub7RtDQhRND+bAjyn4/vTfsRQkDTNMPD2W63Gy+88ALuvfdeQ21TaPXB5Ha78f333xtWeAwGBw8ehMfjsd3IuPoQQmDLli0YOnRos3xba2knTpxARkaGLUeK1UdaWhrCwsKC8r0JAFu2bMGwYcOC6guv3+/HiRMncPToUSxcuBA//fQTAOC5557DnDlzzOWN1uqDyePxIDc3F9HR0eYSW0tOTkZoaGhQXq4UQmDt2rUYNWoUPB6P+bDtFRUVISUlBYmJieZDQWHPnj0IDw+37szWjbR27VqMHj06KL80FRQU4Pnnn8czzzwDAJg/fz4ef/xxc1mjcVQeERHVW83JAgoLCw3HmgqDiYiI6i0sLEy/h1ZSUmI+3CQYTEREVG8Oh0OfpkpVVfPhJsFgIiIiS2EwERGRpTCYiIjIUhhMRERkKQwmIiKyFAYTERFZCoOJiIgshcFERESWwmAiIiJLYTAREZGlWDaYhBD6dnIdkFY6EToRUatiuWASQmDDhg2YNWsWhgwZgvDwcDidTsTFxWHChAl4/fXXm23iQCIianmWCqbc3Fxcd911uOiii/DPf/4TSUlJKCsrg6ZpyM7OxsqVK3HnnXdi6NChWL16NXtQRERByDLBlJGRgcsuuwzLli2DpmkAgPj4eFx77bWYOnUqEhIS9KnWMzIyMHHiRKxYsYLhREQUZCwRTKWlpbjuuuuQnJwMAHA6nVi4cCH27t2Ljz76CB9++CF2796NN998E+Hh4QCA8vJyzJgxQ38NEREFB0sE07PPPoutW7fq7YULF2LOnDmGlRKdTiduvfVWvPnmm/paIIWFhZg7d67ewyIiIvtr8WDKzc3FokWL9PZ5552H+++/X79sZzZlyhRcffXVenvVqlXYtm2boYaIiOyrxYNpyZIlKC0t1dsPPvggFEUx1NSkKAruv/9+ve33+/Hee+8ZaoiIyL5aNJg0TcPy5cv1dnR0NMaPH2+o+TUXXngh2rZtq7e/+uorDoIgIgoSLRpMxcXF2LJli94eMWIEIiIiDDW/xuVyYcSIEXo7PT0deXl5hhoiIrKnFg2m3bt3w+/36+2aYVMXSZIwYMAAvV1ZWYnU1FRDDRER2VOLBlNGRoahHRsba2jXpVu3bob20aNHDW0iIrKnFg2mY8eOGdpdu3Y1tOvSoUMHQzs/P9/QJiIie2rRYCouLja0w8LCDO26uN1uQ5vz5xERBYcWDaaa95cA6A/Ongmn02neRURENnTmSdAEoqOjDe3KykpDuy7m2vqM5iMiIutr0WBq166doW2+51QX82AH87mIiMieWjSYzCPrsrOzDe265OTk6P+WJAldunQxHCciIntq0WAaPHiwoZ2UlGRon4oQwjCruMfjQc+ePQ01RERkTy0aTO3bt0efPn309qZNm2oNiPg1Pp/PMGNE//790aZNG0MNERHZU4sGkyzLhrnxMjMzsX37dkPNr/npp58M96OuuuqqU85GTkRE9tKiwSRJEm655RZ9mLimaXj11VfrnJBV0zS89tpretvlcmHatGmGGiIisq8WDSZUz493+eWX6+333nsP69evP2U4rVu3Dh9++KHenjRpkmHePCIisrcWDyZJkvDCCy/oS6b7/X5cf/312LRpU61wWr9+PaZNm6bfh2rbti2effZZXsYjIgoiLR5MADBo0CD8/e9/h8PhAKpXtR0zZgyuvfZaPPPMM3j66acxefJkXHLJJfrzSx6PB2+99RZH4xERBRlLBNPJe03//e9/9QUAfT4fPv74YzzyyCN49NFHsWLFCr2nFBsbi88++wwTJkxgb4mIKMhYIphQHU7Tpk3Dtm3b8Pvf/x6xsbH6RK2yLCM0NBTDhg3DwoULsX37dowdO5ahREQUhCwTTKgOp9jYWLz44otIS0tDamoqtm7dil27diE7Oxvbtm3D3LlzERMTY34pEREFCUsFU00OhwPdu3fH8OHDMXDgQMTExLCHRETUClg2mIiIqHViMBERkaUwmIiIyFIYTEREZCkMJiIishQGExERWQqDiYiILIXBRERElsJgIiIiS2EwERGRpUjCvOhRK5Gdna1PFLtv3z5ERkaaS2wtKysLHrcbHTt2NB+yPSEEtu/YgYEDB8LlcpkP296JEyeQlZWF/v37mw8FhYMHD0KEOhHSsY35UFDI2LkPIwcM1ZfxCSYlJSVYsWIFZs+eDU3TMG3aNCxdutRc1mitPpicTie++eYbfaHCYFFcXAxPehqifjlmPmR7Qggc7h6HqN59oCiK+bDt+Xw+FBcX60vABJuioiIUuVQcjwSCcfZLV14F+kZ1giwH3wUpr9eLTZs24cEHH4QQAhMmTMDnn39uLmu0Vh9MHo8Hubm5iI6ONpfYWnJyMuSd29GxoMB8yPaEENgZ0xbnTZ4Cj8djPmx7RUVFSElJQWJiovlQUNizZw+KXSrKO3oQjNFUsDMDk867JCh78wUFBViyZAlmz54NVVUxffp0vP322+ayRgu+SCciIltjMBERkaUwmIiIyFIYTEQU3ISAEAJCqBBCA4So2siyGExUN9UH4avQN2iqueL0hGY8h+ozVxA1LSFQouVjZ8UX+ObEC/is5BF8UvwQVpT8EV+WPoVN5Utx1J9eFVRkOQwmqpN6MAm+L/6tb/7d3wNoyLdNATV1s+Ec6sEkcxFRk6nQSrG+/L9YVfwYUso/x3FfGirUQvi0MnjVEpT6c5BV+QPWlTyPb048j0L1iPkU1MIYTFQnuXsCIMuA1wd4fdAy9kFUlpnLTk31QUvdqb8esgy5W19zFVGTOK4exqqSJ5FTuQmaqLt3LyBQ6MvAtyVPI8O7Da30yRlLYjBRnSRXKOT4GjMQeH1Q03fVLKmTmnMA4kQgyOQeAyC5Qw01RE2hWMvDmpKFqNSKDftlSUG4ozPauQYg2tkTDtn47JsmfNhy4t/I9O1gOFkEg4lOS+lzLuBy6m0tbReE32uo+VWaCm3/lkDb5YDSa3hQPlRJLcunlWNt6Yvwi0p9nwQJPTyXYVLUc7gqYj4uC7sfV4TPxZTIF3BO6C1QpMADsAIatpT9BwVqjr6PWg6DiU5LCgmH3KNfYEdZObSc/TVLfpWWfwjieGDmCTk2AVJocE39RC1PCIGfK/6HCrVQ3ydLCkaGzcQIz1S4pBBDvSTJ6OMehcsj5sEpB3rvmvBjU/m/oQoOzmlpDCaqBwlK7xGAIzAvnZayBdD8hioDoUFN+TkwTsKhQEkYwd4SNbkC7RAOV/5s2DcwZCrinMMgSad6v0mIVjrhovD/gywF3tcn/EeQ6t1gqKSzj8FE9SKFRUKO7aO3RVEhtKOZhpqatKJjEHm5elvu1gNSWHDNR0gtTwiB3RWfQSAw7LuNszf6uS6qI5QC2srd0cdzpWFfSsWX0EQdX7qo2Vk6mIQQSEpKwtdff42vv/4aP/zwg7mEzhoJSr/zAKX6LSNQ1SP61aHjAtqBbYBa/WEhSVD6nc/eEjW5MlGIfN8+vS1BwrCQqZBq9ILqIkkSBrp/A4ccuNxXqRUj27fHUEdnlyWDSdM0rFu3DhMmTMCQIUMwbtw4jBs3Drfeequ5lM4iKTwacreeelscy4NWmGeoAQBRXgItO01vS91iIUUG5xIO1LIyvFsMvZtwRxe0k2MNNafjkFyIc11k2HfA9z1nh2hBlgomVVXx7bffYuzYsbj00kuxcuVKDt+0FAlyv8Sq55pQ9XS9lly716Sm7QB81R8WsgSl3yj2lqhZHPJtNbTjXRfW6xKeWW/3aEg1Pg4LfelQwUEQLcUywbR161ZceumlGDduHNasWQNN06AoSlAuBGdnclR7SF266W3t8EGIEzWeG1F90DL26k2pQ2fIbYJvFV1qeX5RiRNq4D6mBAlxzmGGmvqKlNrDrQTugaqiEr+ohww1dPZYJpgOHz6MH374AZqmQZZljB8/HqtXr0bfvpwlwFqq7xfJ1d9K/SrUA1v0XpOauQcor6gulaD0S2RviZrFcTUXmgg8T+eUwxEiRRlq6kuSZLRxxBv25foD967o7LJMMI0aNQpRUVG46qqrsH79enz55Ze4+OKLg3J5YruTYzpDah/oBWmZKRCV5VUP1KZu1/dLbdtBbt+w6/1E9ZWvHjRcRA53dGrUV6B2Sm9DO98fuE9KZ5dlPvXbtm2LzZs34/PPP8f5559/RteJ6WyRoAy4ADj5O6rwQju4G9qRdIiiouoSCUr/8wHJMm8xCjKFplkawqVOgffkGYhSOhnaZVperfundHZY5lNDkiT06dOHgWQTcruukNp10Nta2k6oyYGHHKXoaMgdjZdGiJpSmThmaEeagqWhwqW2kGr0uXxaKfx8nqlFWCaYyGYkGcqAUXpTlJZB5Ac+KKpG73HgCjUf82StEXLgi9KZcEvhkCWH3laFHyrqMSckNTkGE50xuUMspOjaszlIERFQuhiv1xM1Nb8oN7TdUpih3VCKpJgezNU4b14LYTDRmZNkyDV6TSfJCSMAJfDNk6g5mKcNctaYLfxMSJAhIRBMAgJ+9phaBIOJGsdXPTS8Ji5XTWeBQGAhQAkSFCmwNMuZkCAbJnRF9eU8OvsYTHTmNLVqlnHz7tRtgMpLINTcjAOlRLOMoONgrJbAYKIzph4+AFFcYt4NUVwC9dDp12siaoyaUwgJiEb3bjRotZZjd9QYDEFnD4OJzoymQkv+SW9KkRGQwmssupayue71mogaSTbdU/KJX7ms3AACau3Lg2jcfSs6MwwmOiPakXTj6rT9EiH3Pkdvi8IiqIf55Dw1H6cU+CIEABVa7d57Q2jCbxhQIUFu9H0rOjMMJmo4oUHd91PgofjQEMjdEiDHDwY8gW+YWsrPgOnSCFFT8cjGefFKtNpLsDREuSiBqPF+lWUXnOwxtQgGEzWYln8IouAXvS33HAzJ4YLkckPuMUDfLwoKoB3J0NtETSlcNs5aX6QFZho/EyXaL4YBFG45wvDALZ09DCZqIAF178bAImouJ5SeQ6uPSVB6nws4qofcCgE1+SfON0bNoo3c3dAu1XIbtbjfcTXb0A6TGzfFEZ05BhM1iJafA5F3RG/LcX0heQLX+qWQCMjdeuhtkZ8P7RjXtaGm194Rb5jb7oT/CLQagxcaKs+fYmi3VXoZ2nT2MJioAQTUlJ8D30oVGUrf2ustyX0TAaX6rSUE1L0b2GuiJhcht4dDDnwp8otKFGvGiV3rSxMqSkwLA3ZxBi5L09nFYKJ604ryIXIDf7xS5+6QwiINNQAgR7eH1KGz3hZ5udAKGnf9n8hMlhREOeIM+zJ8mw3t+vpFy4ZfK9PbTjkEbXgpr8UwmKieBLR9GwGterohSYKSULu3VH0QSt+RgUMCVfelwKmKqGnFO883tLO9G2o9JFsf+yvXGgY+xDgSINeYN4/OLgYT1YsoKYCWc1BvS+07QI4J9IrM5PbdIbWJ0dviSA60wnxDDVFjdXMOhEPy6O0KtQi5asNmHfGJChzxBlZeBoC+7ksateggNQ6Dieqh+t6SGujxKP3Oq3t1WlmpmmX8JE2DmryR95qoSbmkEHR3jzbs21n+Ub2XqxBCYGfFl1BFYBbxEKUdOil9DHV0dtXxyUJURZwogpYdmMVBioqC3CHWUPNr5M69IEWE621xOAuiJDBbBFHjSRjo/g2UGtMTlfoPY3flVxD1GDqep2Ygo+I7w75Bnmsg1fWli5qd5f/Xnz9/PhYvXozFixdjwYIF5sPU7ATU/ZsBX2CqFrnvcEA+/YOHksMFuffJZ5wA+FWoKZvZa6ImFSpHIcEzwbAvtXwlUrw/1BFOAr+oh7DhxKsQNe59RjnjEecMTK1FLcPywXT99dfjrrvuwl133YWbbrrJfJiamagohZZZ45p9aAjk7v1rltRBghw3CHDXmKYo6wDEiSJDFVHjSBjgvgxtnIHLbwICu8vex7ryf+GEVgAhhL75hRe7K77G2pKF8GmBVXAdkhujQ2+vtSYTnX2WDyZqWeqBbYA3cL3+5PRD9SW5QiDH1wgyv7+qB0bUhGTJgTGhsxCitNP3CQgcrdyBL4v+hM9K5mHViWfwZelj+KToQewr/8RwH0qWFJwfPgsRcnt9H7UcBhOdkvCWQ0vbE9jhckLpNaxmSb0ofUcGpikCoGWkQFSUGmqIGsslh+CK8D8iytHTsF9AoEItQrEvGyf8ebWWZHfIIbgw/PforPQ17KeWw2CiU9JyUgFZBkI8QIgHcs+BkNwh5rLTkkLCIffop58HTgc0LiRIzcAth2Fs+B/QP2QyHHLd71UJMjq4BmNcxJ/RydELEoeHWwaDiU5J6TEIrol36ptjyJhTPFB7eo5zxhrOpfRueM+LqD4UyYHBnnGYGLkAI8JmoJNrOMIdneFR2iDU0R4xzr5ICJmIcZHzMSZ0FsLkNmf8vqbmwWCiOshVzyqd3Br1xyuZzsW3HjUvlxSKnq5EXBw2E1dFzMekyKcxMeIJjA3/A4Z6JiBS6chekkXx04GIiCyFwURERJbCYCIiIkthMBERkaUwmIiIyFIYTEREZCkMJiIishQGUzA71cTKRC0sTHGhm6sNurmig24L0fix2liSOPW88EEtOzsbsbGxcLlc2LRpEyIiIswltnbs2DEoqoqokMDqnsFCAMg8moe4nj3hcJx++Q27qaiowLFjx9C9e3fzoaBw9OhRuFwutGnTxnwoKKSnpyMuLg6KEnyzlJeXl+Pbb7/FnDlzoGkapkyZguXLl5vLGo3B5HJhw4YNQRdM+fn5cDqdiIqKMh8KChkZGejevXtQBlNlZSWOHTuGbt26mQ8Fhby8PLhcLkRHR5sPBYWMjAzExsYGbTCtXr1aD6bJkyfj448/Npc1WqsPJo/Hg9zc3KD7I0lOTkZoaChiY0+/0qzdCCGwdu1ajBo1Ch5P8PUIi4qKkJKSgsTERPOhoLBnzx6Eh4cjLi7OfCgorF27FqNHj4bLVf/lYeyioKAAS5YswezZs6GqKqZPn463337bXNZovBhKRESWwmAiIiJLYTAREZGlMJiIiMhSGExERGQpDCYiIrIUBhMREVkKg4mIiCyFwURERJZiyWASQsDr9aKkpASFhYUoKSmBz+dDK52kgoioVbFMMAkhkJ2djRdffBGTJk3CgAED0L17d3Ts2BGxsbHo378/rrvuOrz77ruoqKgwv5yIiIKEJYJp//79+N3vfoeePXti9uzZ+Pzzz5GWloaioiJ4vV4UFhYiLS0Ny5Ytw80334wBAwbgu+++Yw+KiCgItXgwbd68Geeeey6WLl0Kv98PAHC73ejZsydGjRqFYcOGoXPnzobXZGRkYMKECVixYoVhPxER2V+LB1P//v31dVkGDRqEf/zjHzhw4ACSk5OxYcMGbNu2DQcOHMDatWsxZswY/XWVlZW44447kJmZWeNsRERkdy0eTOHh4fjzn/+Ml156CZs3b8asWbPQrVs3OJ1OAIAkSQgNDcWYMWOwcuVKTJw4UX9tQUEBXnrppRpnIyIiu2vxYAKAmTNn4v/+7/9Ou7ZOSEgIXnrpJUPdJ598Aq/Xa6gjIiL7skQwNUSPHj0wdOhQvX306FHk5+cbaoiIyL5sF0yyLCM+Pl5vV1ZWory83FBDRET2ZbtgEkKgpKREb4eFhSEyMtJQQ0RE9mW7YKqsrERSUpLe7t69O2JiYgw1RERkX7YLph9++AFZWVl6e9KkSVAUxVBDRET2Zatg8vl8WLBggd4ODQ3FzJkzDTVERGRvtgqmN954Az/88IPevu+++9CjRw9DDRER2Zttgumnn37CQw89pM+PN2jQIPzpT3+CJEnmUiIisjFbBFN6ejquvfZalJaWAgAiIyOxdOlSREREmEuJiMjmLB9MmZmZGD9+PHJycoDq2R8++OADDBo0yFxKRERBwNLBlJaWhnHjxiE1NRUA4PF48J///Afjxo0zlxIRUZCwbDAlJSXhiiuuQEpKClDdU/rPf/6DG264gfeViIiCmCWDaePGjfjNb36D9PR0oHpY+Pvvv49p06YxlIiIgpylgkkIgU8//RRjx47FkSNHAABt27bFl19+iauvvpqhRETUClgmmDRNw6uvvorrrrsOZWVlAIBevXphzZo1GDNmDEOJiKiVsEQweb1ePPLII/j973+vr600atQorF69GoMHDzaXExFREGvxYCorK8Ntt92Gv/71r9A0DQAwZcoUrFixAh06dEBFRcVpt8rKSvNpiYjIplo0mMrLyzFx4kQsXbrUsP+7775Dv3790LVr13ptHD5ORBQ8WjSYjh8/jjVr1ph3o7i4GAUFBfXeiouLzacgIiKbatFgUhQFAwcObPTWs2dP86mJiMimWjSYOnbsiKSkpEZvH330kfnURERkUy0aTERERGYMJiIishQGExERWQqDiYiILIXBRERElsJgIiIiS2EwERGRpTCYiIjIUhhMRERkKQwmIiKyFAYTERFZCoOJiIgshcFERESWwmAiIiJLYTAREZGlMJiIiMhSJCGEMO9sDbKzsxEbGwun04lvv/0W4eHh5hJbKyoqgsPhQFhYmPmQ7QkhkJeXh3Zt20JRFPNh2/N6vSgpLUXbtm3Nh4JCYWEhnE5nUL43AVS9N9u1gywH3/f+yspKbNy4EXPmzIEQAhMnTsRnn31mLmu0Vh9Mbrcb+/btQ1RUlLnE1jIzM+HxeNCxY0fzIdsTQmD7jh3oc6IUjiOHzYdtz+v14Wiv3uh73vnmQ0EhIyMDISEh6NSpk/lQUNi+fTsGDRoEp9NpPmR7xcXF+OSTT/Dggw9C0zTccMMNeP/9981ljdbqg8nj8SA3NxfR0dHmEltLTk5GaGgoYmNjzYdsTwiBtd9/jyEnSuDIyjIftr1KbyUO9RuA4ePGmw8FhT179iA8PBxxcXHmQ0Fh7dq1GD16NFwul/mQ7RUUFGDJkiWYPXs2VFXF9OnT8fbbb5vLGi34+ppERGRrDCYiIrIUBhMREVkKg4mIiCyFwUStknY8F/5Nnwa2vesBNGwckCgvgf/nLwLn2LkaEJq5jIgaiMFErZIUHgMtNwtaVnrVtncLtKJj5rI6qQe2QTuYqp8DQgMkyVxGRA3EYKJWSXK6IfceHNihCWj7N9e71yR85dAy9gZ2OBxQ+iYCYDARNZalgsnv9yMzMxMfffQRnnvuOcyZMwf33XcfHnvsMbz55ptISUmB3+83v4zojCh9RwBOh97WstMhyksNNaeiZe4DKir1thyfACk0wlBDRGemxYNJCIFffvkFCxcuRGJiInr16oXrrrsODz30EF544QX8/e9/x+OPP47bb78dAwYMwMUXX4xVq1ahlT4XTE1IcodB7tEvsMOvQk3bcfpek9CgHdgeaMsylISR7C0RNZEWD6aysjIMGTIEDz/8MLZv3w5VVc0lOk3TsHHjRlx55ZV46qmnoGm80UyNo/QZCTgC8+1p6UkQPq+hxkzLTYMoLtHbUrc4SGHBNXMIUUtq8WAKDQ3F+edXzQkWExODm266Cf/+97+xceNG7N27Fxs2bMArr7yCESNG6K8RQmD+/PlYvnx5jTMRNZwUFgm5W8/AjopKaJk17h39CjX550BDlqD0C8457YhaSosHkyRJuO+++/D0008jJSUFS5Yswe23347zzz8f/fv3x6hRo3Dffffhxx9/xAMPPKC/TgiBp556Cj6fz3A+ooaRoPRLBJTAn4J2YBsgfr3nLorzIfIDo/ekTl0gR3cw1BBR47R4MAHApZdeinnz5qFdu3aQTjHc1u124+mnn0bPnoFvt7t27cLhw8E3uzSdXVJkO0idu+ttUVwC7UiGoab6CNR9mwJNSYKSwJF4RE3NEsFUX2FhYRg9erTe1jQNhYWFhhqihpOg9D/f8AySuu+nWg/LirISaDmBwJLatYPcPhBoRNQ0bBVMqL6Ed5KiKGjTpo3hONGZkNt0gtSpi94Wv+RD+8XYG1dTNwP+6kt8EqrCzH5/QkSWZ6u/quLiYvzwww96e/jw4ejSJfBhQnTmJCj9RwV6TUJATf5JHzouvGXQDqYEqtvEQO4Yr7eJqOnYJpjKy8vxwAMPIKt6YThFUfDEE0/A4Qg8IEnUGHK7rpDaBwYyiCM5ECUFAAAtfRdQGRhGLvdLBKTgW9adyAosHUxCCJSXl2PFihW4+OKL9ZUSnU4nFi1ahHHjxplfQtQIJ+81VTc1DWrKZgi/ryqYTlZFRkDp3FtvE1HTslQweb1e3HXXXbjpppswdepUjBo1Cp06dcLkyZOxZcsWAMDo0aOxZs0a3HvvvaccwUd0puQOcZDattfbWnYqtPTtEKVlgZr+5wMKe+pEzcVSwaSqKj7++GMsXboUy5Ytw08//YTi4mIAgMvlwrx587B8+XJccMEF5pcSNQ1JhjJgVKDtU6HuDgwRl8JCocTWmMaIiJqcpYIJADp16oSuXbuic+fOaNeunX4Pyev14plnnkG/fv1w//3349ixhi1RQFRfcsc4SDVHe6qBYeNyv5G8t0TUzCwVTCEhIdi1axcOHTqEQ4cO4eDBg0hKSsLixYsxdOhQAEBhYSFeffVVJCYmYufOneZTEDWepFRdrjML8UCJH2jeS0RNzFLBVJMsywgLC0NCQgLuuusubN68GQsWLICiVH1bPXjwIK6++mocOXLE/FKiRpM7xBomdwUAuVMsoLgM+4io6Vk2mMycTiceeeQRzJ49W9+XnZ2Np556ylBH1BS0zKTAw7Qn9+VkQHjLDfuIqOnZJphQPeHrgw8+iPDwcH3f8uXLUV7ODwtqOsLvhba/xnpLJ3l91es1EVFzslUwAUC7du0QGxurt/Pz85GXl2eoIWoMLTsZ4kRgeLhh5vHU7YDKGe2JmpPtgslMCMHVbKnpaH5oKZsD7VAP5P7DA+0KL9T0XfpURUTU9GwXTEePHsXBgwf1dkxMDDp04Ho41DTU7GTD6rRyz8FQeg0H3IFBD2rKFgh/3avcEtGZs1UwaZqGp556CmVlgcss48ePR0hIiKGO6IwIFVpyjd6Sywml1zmQ3CGQ42s8VFtWDu1gUqBNRE2qRYNJCIG3334by5YtQ2lpqfmwQWlpKR599FEsXrxY3xceHo65c+dyaiJqEmrOAYiiIr0tx/aG5A6tmkOv93DD8HEtdTt7TUTNpEWDye/348knn8TUqVORkJCAW265BYsXL8bq1auRlJSEnTt34tNPP8Wjjz6KYcOG4dlnn4WmVT2FrygKFi1ahAEDBphPS9RwQoO276dA2yFD6ZuoN6WwKMhd4vS2KCmFlp2st4mo6bRoMK1fvx5paWkAgMOHD2PJkiWYNWsWLr/8cgwePBjDhg3DNddcg6efflqvA4C2bdvigw8+wIwZM9hboiahHcmAOF61xAUASJ3jIIVH16iQIPcfBciB95uWsgXQ/DVqiKgptGgwXXjhhVi+fDmuvvpqREVFmQ/X0qtXL/z5z3/Gvn37cO211zKUqGkIrXpRwGqyDGXA6KplamuQo9pB6tRVb4viYqg5qYYaImq8Fg0mh8OBKVOmYMWKFcjMzMT69evxz3/+E4899hgeeOAB/OEPf8BTTz2Ft956C0lJSdizZw+eeOIJtG8fWJaAqLG0/ByI/MCkwFK7DpCj2hlqqo9UrXJbg5b8M4DAJK9E1HgtGkwnSZKEqKgojB49GnfeeSf+8pe/YNGiRfjb3/6GRx55BLfeeisGDhwIt9ttfilRIwmoyZuAGs/CKf0Sa/WWTpLbdoEUHbjEJ44fh3Ykw1BDRI1jiWAiailaYR7EkcN6W4qKqprA9ZQkyAkjDHvUvZv4wC1RE2IwUStWu7ck9x0OyHWvTqt0T4AUFnh2TuQfg3bskKGGiM4cg4laLVF6HCInM7AjxAMlth6PH8hOyL2q1gc7Sd23kb0moibCYKJWS03eZFydttdgQHEaak5Fjh8MuAO14mgutONcG4yoKTCYqFUS5SXQDh4I7HA5ofQaVrOkTpInzDhNkRBQ92xgr4moCTCYqBUSUPduBKpnEQEAOa4vJHeYoep0lN7nGqYpEkdzoBUFhp0T0ZlhMFGrI8qKoWWmBHYoStVceA1UNU1RfGCHqkHdy14TUWMxmKj1EQLKkAugDL+4ajvvCkgRMeaqepCgDL4ocJ7hF0Pu2tswyo+IGo7BRK2OFBYNpffwwNYt4ZQP1J5OrXPFDgIk/lkRNQb/goiIyFIYTEREZCkMJiIishQGExERWQqDiYiILIXBRERElsJgIiIiS2EwERGRpbT6YBJCQFXVoNs0TYOmabX2B8smhIAmBLTqhc2DahPB+75UW8l707wvWDZN0yCEgKie3eTk/21qkmiuM1tcdnY2YmNj4XQ6sXz5coSFNWwCT6urrKyELMtwOuu3jIOdCCFQXl6OEE2DpKnmw7anaRq8TheU8FAUO/zmw7bn9XrhgYw2UmCxxWBSVlaGkJAQSNKZzSZiZaqqYufOnZg7dy6EELjiiiuwatUqc1mjtfpgcrvdyM7ORlRUlLnE1lJTUxEaGoquXbuaD9meEALr16/HyJEj4Xa7zYdtr7i4GAcOHEDPof2xozzHfNj2juQeQZewGFzcpR6LMtrQ+vXrkZiYGJRfCgsLC7F06VLMnj0bmqbh5ptvxjvvvGMua7RWH0wejwe5ubmIjo42l9hacnIyQkNDERsbaz5ke0IIrF27FqNGjYLH4zEftr2ioiKkpKSgz/BB2F4WfMF0OCcHXcPb4tLug82HgsLatWsxevRouFwu8yHbKygowJIlSzB79myoqorp06fj7bffNpc1Wqu/x0RERNbCYCIiIkthMBERkaUwmIhaqQpRigOVm7Ch7G18U/o3fFXyDL4rXYTN5R8ix78Pqgi+EYFkDwwmolamQivBhrJ38HnRPGwrexuHKjbiuC8Vxf4s/OJLRkb5d1hf/DI+K56H/ZU/QgjNfAqiZsVgImpFjvhT8VXJEzhUuQHayR6R+XkbSQIkwKuVYkfZu1h94hV4tXJjDVEzYjARtQZC4Ig/FT+WvgyvVmo4JEGCUw6DR4mGQ649/P4XXzK+O/EivILhRGcHg4moFSjW8rC+9NVALwmABBk9PJdhfNQTmBy5EJMin8E1kQtxScRcRDt7Gl5f4s/Gj2X/hhDBN9MGWQ+DiSjIacKP9WWvQxVefZ8iuXBRxP9hhGcqIuR2kKSqjwJFcqKDoyd+EzYbPT1jUfMiX753D1K8P9bYQ9Q8GExEQS7N+zNK/IEZJCTIGBU+C50cfU85n5skKTjXMwVd3OcZ9u8p/xgVWolhH1FTs1UwnZzVtuZGRKemCT/2Vn5q2BfnGYPOSoJh36+RJBmJIdPgVgLzSKpaBfZUfmuoI2pqtgmm3bt3Y/To0UhMTNS3nJzgm0eMqCll+3ajUi3S2w7ZgyHuq07ZUzJzSh4M9dwQ2CFJyPL+CH+Ny4JETc0WwVRcXIwbb7wRmzZtwpYtW/TN6+UfB1Fd0n0bDO3urlFwSw1b4iXWOQRuJVJv+7Qy5PpSDDVETcnywaSqKu6//37s2bPHfIiI6uAT5Sjw7dfbEiT0cV1c797SSbKkoKsz0bAv3beBl9Kp2Vg6mIQQeOONN/T1Pnr2NA5hJaJTO6ZmGUbiOeUIRMkdDDX11cM1AlKNMXoF/lQIcOg4NQ9LB9PWrVsxZ84cCCHQpk0bPPLII+YSIjqFo/5kQzvG0csQLg0RJXeGUuPhW79WhnJRbKghaiqWDaZffvkFN998M0pLq55Snz9/PiIiIsxlRHQKBWqGod3e0bf29EP15JBcCFM66W0BgXx/tqGGqKlYMpj8fj/uvvtupKRU3WAdOXIk7rnnHlRUVJhLiehXCZxQ8wx7opXOhnZDRciBYAKAAi3T0CZqKpYMpldeeQUfffQRACAsLAyLFy+Gy+ViMBHVk1/44NNO6G0JEiLl9oaahgqT2wI1BjwUq0cAcAAENT3LBdOmTZvwxz/+EQAgyzJeeuklDB8+3FxGRHVQhQ+a8OltWXLAJYUYahoqwhRsFeK4oU3UVCwVTHl5ebjxxhv155PuvPNO3H777eYyIjoNL8oNvRlJckCG01DTUC453HCPyifKanagiJqMZYLJ6/XijjvuwMGDBwEAF1xwAZ5//nnIsmX+E4lswycqDRfZJMiQG/nn7oTL0NaEDwJcRJCaXuPeqU1ECIHnn38en3/+OQCgU6dOWLJkCcLCGvaEOhFV0WBcFl2WlAY/WGvmkDyGweYCKgTvMVEzsEQwfffdd3jyyScBAA6HA0uWLEF8fLy5jIjOWONCCdUDKIjOhhYPppycHMyYMUMfcbdgwQJcfvnl5jIiagAZiqEthNroKYT8qH15kGFFzaFFg8nn8+HWW2/FoUOHAADXXXcd5syZYy4jogZymO4HVV12a9z9IJ+oNLRlycFgombRYsEkhMATTzyB7777DgAwaNAgLF68GIpi/KZHRA3nkkINoaEJf637Tg3l1coMzzE5pNBG37ci+jUtFkxr1qzBX//6V70tyzLuuOMOTJky5ZTba6+9ZjjH3XffbTheVBRYd4aoNXNILsiSQ29rwg+vqcfTUKVavqHtkQJLYRA1pRYLpt27d8PnCzwAuGvXLnzyySd1bjt27DCc45tvvjEc5/pMRFUUOOGQQ/W2gECJKVga6oQ4ZniOKVzu2CSDKojMWiyYiKj5SJKEULmdYV+hmmtoN1SJae69GCXW0CZqKi0WTA888ABUVW3QtnjxYsM5UlNTDcfbtTP+IRK1Zm0U4/plR9V9hntEDeEXXpSqVYOUUN1PaqvEGWqImkqLBZMkSZBluUGb+Ubr6Y4TtWadnAmGdqE//YwfiC3RjsGvBSZRVmQPwuQ2hhqiptJiwUREzau93AOKFJgfr1ItwgntzCZezfbtMoRapBILpZFz7xGdCoOJKEi55RBEOXrobQGBVN+PDV6qQggNmd71hn2xzpG8QkHNhsFEFLQk9HCONuzJ8m6Av8ZyGPVRoB1ChfqL3pYlB+Kc5xhqiJoSg4koiMW5zoFTDkyGXKkW44B3o6GmLkJo2Fb+v0AfSwh0cg2Hu8ZQdKKmZqtgCgsLQ7du3fSNs0QQ1c0hudDXPc6wb2/FCpRqBYZ9p3LA+xMKfWl6W5IUDHJfyeeXqFnZKph+97vfITs7W9/i4jhcleh0+rkvQYjSVm/7tXJ8f2IRyrUSQ11NQggc9qdgZ/l7hjtSsZ4LEK10rLGHqOnZKpiIqOEUyYnzQ/8fZClwheGEPw9flz6FHP8+CGGc3FUVPuyp/AYbSl+FJgLz64UobTHc81v2lqjZMZiIWoF2ShzOCb3FMLFrpVqE9SUv47OSP+H7E//ExvJ3sPrEK1hR/DD2ln9sCCWH5MFFYffBKXn0fUTNhcFE1ApIkoSezkQMC73ZMLkrAFSox3HUuwPZFRuQ790Lv1ZuOO6Uw3BJxIO8hEdnDYOJqJWQJAl93KMwJmIOwh2d674gJwQkSOjgGoLxEX9BjNKNl/DorGEwEbUqEtorcbgy/M+4MOL36OgaBrcSBUVyQpYUKJILoY72iPOMwdjIP+OS0LsRIkeYT0LUrBhMRK2QJEno7EjAmLC7MCniGUyO+lv19gImRjyO80JvRBuls2GZC6KzhcFE1MpJkgRFcsAhuaBIDl6yoxbHYCIiIkthMBERkaUwmIiIyFIYTEREZCkMJiIishQGExERWQqDicjSRNBtJ/9FdCqSEKJVvkeys7MRGxsLt9uNjIwMREVFmUtsLS0tDSEhIejSpYv5kO0JIbBp0yYMHz4cbrfbfNj2SkpKkJ6ejoTBA3DUd+qlKewqKzsbbUIjMKhLT/OhoHDyvelyucyHbK+wsBAffvghZs+eDU3TcNNNN+Hdd981lzVaqw8mp9OJzz77DOHh4eYSWysvL4csy0H5wS2EQHFxMSIiIiDLwdfp9/v9KC8vR0REcE4FVFZWBlmW4fEE50zlxQUFCC8qghyEs2b4ZQlbD+dizty5EEJg/PjxWLlypbms0Vp9MLndbuTm5iI6OtpcYmvJyckIDQ1FbGys+ZDtCSHw/fff4/zzzw/KD7fi4mIkJycjMTHRfCgo7NmzB+Hh4UG70Oe6r1ZiYEYaHEE4g0ap04nl5ZWYPXs2VFXF9OnT8fbbb5vLGi34vm42kCRJ3Gy4BfPvLph/ttawAbX3BcsGqeaKXs2n1QcTERFZC4OJiIgshcFERESWwmAiIiJLYTAREVmQqDwBreCwvomyInPJ6QnVeI7ifHOFJTGYiIgsSFScgH/NR/B/97+q7YflgNDMZXVSM/fC/23167/7H9SMneYSS2IwERFZkBzVHlLHzvpsTqKoCNqRdHPZqWl+aPu3Vv1bAHA6oPQ+11xlSQwmIiJLkqAknA/IgSeH1OSf691r0vKyIAoL9bbcvTekMHtMvcZgIiKyKLldV0jtOuptkX8MWmGeoebXaVBTq3tLAKDIUBISgbPyeGzjMZiIiKxKkqH0SwzkiRBQU3467fzs2vE8iKO5elvqGgcpvI2hxsoYTEREFiZ3iIMU01Zvi8NZEKXHDTVGAlrqFkCrvuQnSVD6jbJNbwkMJiIii5MVKP3PD7T9qvEynYk4UQztUIbeljp3gxzd3lBjdQwmIiKLkzv1hNQmcClOy9wPUXnCUHOSmr4D8KtVDQnVoWaf3hIYTERENiArkBNGBNpeH7SM3TUrAADCXwktY6/eltp1gNzWfouFWiaYhBDYunUrfvzxxwZt6ekNGNdPRGRTSvf+kCICC5qqB3YCqs9Qox1MAioqqxoSoAyw172lkywTTOXl5Zg0aRIuuuiiBm2vvPKK+VRERMFHkiH3q7F4ZFk51KxA7wiaH9qBHXpTimkLuYM9F2O0TDAVFBTg2LFj5t1ERFRNie0PKSxEb2up2wBRdT9JzU2HKC6pOiABcsJIQLLMR3yDWOa/OicnBz5fVbc0IiICkydPxpQpU067DR482HwqIqLgpDgh9xmuN0VhEbS8LEBo0PZv1vdLUVFQOvfS23ZjmWDat2+f/u8RI0Zg+fLl9dpuv/12w3mIiIKZHD8ICPXobTXlZ2gFuRD5gZnD5b4jAMWpt+3GMsFUcxBDv379qtaXJyIiA8nlgdJ7mN4WeUegbv0GEFWzQUgR4VC696vxCvuxTDDV7DENGjTIcIyIiE6SIPcYAnhcVU1NGCdr7XOOrXtLsEowCSEMPaY+ffoYjhMRUYDkDoHc81e+wId4IMfb/767JYKpoqICmZmZeps9JiKiukhQeg0DFMWwV+45EJKjuidlY5YIpiNHjqCkpGqYY1RUFDp2DEzzTkREtYnCPECtnnro5L7jeaededwOLBFMWVlZ8Hq9AACXy4V7770Xv/3tbzFlyhTceeedePzxx/HFF18gPz8fovoGHxFRqyVUqHs3mfdCHDkErag+6zVZmyWCaf/+/fq/jx07hsWLF+Pjjz/GJ598gtdffx2PPfYYJk6ciN69e+PWW29FUlISA6oOmqZBVVXIsiV+vU3O7/fD4XCYdwcNr9cLh8MRtO/xYP79qaYeTHPR8g9BFASGh+uzDmkC6r5NzdJrEkJA0zQ4nM5mf29a4pMrIyMDbrcbYWFhiIiIQGRkJMLDw+F2uw11RUVFeOedd3Deeedh0aJFZ+1NYEeqqgbtH7/P54PT6QzaRwoqKytrvfeDid/vh8tl//sgv8bv90OR5eZ9bwqtqrdUnQ1SWCik7j0Chw9nQZQUBOqbkKZp8JyF96Ylgunee+/FwYMHkZeXh+PHj+P48eM4duwYMjIy8N133+Ghhx4y3HcqKyvDH/7wBzz//PPNntxERFai/XIY4tgRvS33GgKl/2hAqf4496v1WuXWyiwRTF27dkWnTp0QGhoKRVEgyzI8Hg86d+6Myy67DM8++yxSU1Nx1113Gb6JzJ8/H1u2bDGci4goeAmo+zYCWnXouJyQew6FHNUOUsfA8hZaVhrEicCzTXZjiWA6HUmSEBERgVdffRUzZszQ93u9Xvztb38z1BIRBSut8BjE0Vy9Lcf1heTyVA0fTzgPOPnF3a9CTdls216TLYLpJIfDgSeeeALh4YE1SdasWYOysjJDXUPwUiAR2YWavBHQtKqGQ4HSd6Q+8kFu1wVSTIxeq2WlQpRXzzZuM7YKJgDo0qULzjnnHL198n4UEVEwE6XHIXICExFIXWIhhUUFCiSlavLWk7w+qAe2B9o2YrtgkiQJXboErqWqqsrReUQU5ATU5E2AWt1bkiUofRNrjBOvonTtAyk8TG9raUkQ3jO/otRSbBdMmqYhJydHb0dGRiKmRveViCjYiBPF0LIO6G2pXQfIMZ0MNQAA2QG5b2C9pqpe0w7b3WuyXTBlZmZi69atenvo0KGIiqrRnSUiCioC6v6fAX/1lSEJUBIC95bM5Nj+gZnHAWgHdkL4Kg01VmerYPJ6vZg7dy7Ky8v1fTNnzmzeh9mIiFqQKCuGlhmYHUeKiobcMd5QU5PkCoUcPyCwo6ISWvpOW/WaWjyY9u3bh6NHj5p311JQUIDbbrsNy5Yt0/ddcsklmDp1qqGOiCh4CKhpOwCvT98jJ4wE5LpndVF6nQM4AzVq6g4IX9V8pHbQosEkhMC8efMQGxuLq666Ci+++CLWrFmD9PR0HD16FJmZmfrMDwkJCVi6dKn+2t69e2PJkiVBO7UJEZGoOAEtY09gR2gIlG6nX69OCouE3L1nYEdZObSsvTVLLK1Fg+nYsWNYtWoVvF4vVq5cidmzZ+Oyyy5Dv379EB8fj759+2Ls2LF47rnnkF9jPfuxY8fi+++/R9euXQ3nIyIKJmr6TqAi0NNReg8FlPp8GZcg9z43ME0RAC11K4TfHveaWjSYPB4P5s6di549e0KpseCVz+dDRUWFvhQGqh+uHTlyJJYuXYovvvjCMGSciCjYCF8ltLTdgR3V0w/VlxzVHlKHznpbFJdCO5RiqLGqFg2myMhIPPHEE9i3bx+2bt2KN954Aw8//DBuvfVWTJkyBTfddBNmz56Nf/3rX9i5cyc2btyIadOm8fIdEQU9LXM3UF6ht+X4BEiuEENNnSQZSr/zDbu05M2AsP5zny0aTCe5XC4MHToUd9xxB5599lm89dZbWL58Od5991288MILmDlzJgYMGGDoVRERBS3NDzU58FgMFAVKnxqzOtST3L4rpDaB5zxFcQnUnFRDjRVZIpiIiChAK8qHFN0WUpeukLp0hdz/HOP0Q/UmQxl0gX4eqUtXiOJfzEWWw2AiIrIYuU0nOC+cCueF18F54XVwDLjwlA/Uno7cuZd+nqpzXWAusRwGExERWQqDiYiILIXBRERElsJgIiIiS2EwcRVbIqJ6Oxufl60+mFRVhc8XmCCRiIhOrbyiAlr18u5ut9t8uEkwmFTVsIwGERGd2onSUv3fzbUWXqsPJiEEfvnF+g+cERFZQWFRkf7v5gomSZyNC4YWlJ2djdjYWADA4MGDcc455yA8PBzh4eFwOOpe68TqToZtaGgoQkNDzYdtz+/3Iz8/H+3btw/KaapKS0tRWVmJmJiYoFwEMy8vDxEREQgJacC8bzahqiqO5eSgT2lxUH3rV/1+eL1e/FJZiff3JuPYsWMAgMcffxzz5883lzdaqw2mo0ePIiEhAUU10p+IiOqnX79+eOutt3DeeeeZDzVaMIV6g0RGRuLFF19Eu3btzIeIiKgOAwcOxMqVK5sllNCae0xERGRNrbbHRERE1sRgIiIiS2EwERGRpTCYiIjIUhhMRERkKQwmIiKyFAYTERFZyv8HQeeha9iNrXgAAAAASUVORK5CYII=)\n",
        "\n",
        "Above figure is a sample island. The X-marked red cells indicate traps, and 0 marked green cells indicate buried treasure.\n",
        "\n",
        "- For the above island, indexing starts from 0 (starting from left), At cell (0,5), the agent has only two valid actions: lef t and down. If the agent decides to move down, he goes to (1,5) and will get a reward of -1 (digging cost).\n",
        "- At state (1,5), valid actions are up, down, and left. if the agent decides to move left, it goes to (1,4), getting a reward of +5 (treasure reward).\n",
        "- At (1,2) agent can take all four actions. On taking the action left, it goes to the state (1,1) and receives a reward -2 (trap)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "EQKUUJBu0evy"
      },
      "outputs": [],
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import matplotlib.patches as patches\n",
        "\n",
        "def create_grid(rows, columns, cell_size=1, line_width=1, border_width=3,\n",
        "                error_states=None, goal_states=None, arrows=None, Q_values=None,\n",
        "                save_path=None):\n",
        "    \"\"\"\n",
        "    Create and display a grid with error and goal states, arrows, and optionally a second grid for Q-values.\n",
        "\n",
        "    Args:\n",
        "        rows (int): Number of rows in the grid.\n",
        "        columns (int): Number of columns in the grid.\n",
        "        cell_size (int, optional): Size of each cell. Default is 1.\n",
        "        line_width (int, optional): Width of the grid lines. Default is 1.\n",
        "        border_width (int, optional): Width of the outer border. Default is 3.\n",
        "        error_states (list of tuples, optional): List of (row, col) indices for error states ('X').\n",
        "        goal_states (list of tuples, optional): List of (row, col) indices for goal states ('O').\n",
        "        arrows (dict of tuples to str, optional): Dictionary mapping (row, col) to arrow directions ('up', 'down', 'left', 'right').\n",
        "        Q_values (dict of tuples to float, optional): Dictionary mapping (row, col) to Q-values.\n",
        "        save_path (str, optional): Path to save the image (e.g., 'grid.png'). If None, the image won't be saved.\n",
        "    \"\"\"\n",
        "    # Calculate the size of the grid\n",
        "    width = columns * cell_size\n",
        "    height = rows * cell_size\n",
        "\n",
        "    # Create a figure and axis for both grids\n",
        "    if Q_values is None:\n",
        "        fig, ax2 = plt.subplots(figsize=(columns, rows))\n",
        "        ax2.set_xlim(0, width)\n",
        "        ax2.set_ylim(0, height)\n",
        "    else:\n",
        "        fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(columns * 2, rows))\n",
        "        ax1.set_xlim(0, width)\n",
        "        ax1.set_ylim(0, height)\n",
        "        ax2.set_xlim(0, width)\n",
        "        ax2.set_ylim(0, height)\n",
        "\n",
        "    # Create a set of arrow locations for easy checking\n",
        "    arrow_locations = set(arrows.keys()) if arrows else set()\n",
        "\n",
        "    # Draw the colored cells for error and goal states (on ax2)\n",
        "    if error_states:\n",
        "        for state in error_states:\n",
        "            x_start = state[1] * cell_size\n",
        "            y_start = height - (state[0] + 1) * cell_size  # Flip the y-axis to match grid indexing\n",
        "            rect = patches.Rectangle((x_start, y_start), cell_size, cell_size, color='red', alpha=0.3)\n",
        "            ax2.add_patch(rect)\n",
        "            if Q_values:  # Add the red cell to ax1 for Q-values grid\n",
        "                rect_q = patches.Rectangle((x_start, y_start), cell_size, cell_size, color='red', alpha=0.3)\n",
        "                ax1.add_patch(rect_q)\n",
        "            if state not in arrow_locations:  # Skip if an arrow is already defined for this cell\n",
        "                # Add 'X' label\n",
        "                x_pos = state[1] * cell_size + cell_size / 2\n",
        "                y_pos = height - (state[0] * cell_size + cell_size / 2)\n",
        "                ax2.text(x_pos, y_pos, 'X', ha='center', va='center', fontsize=24, color='blue')\n",
        "\n",
        "    if goal_states:\n",
        "        for state in goal_states:\n",
        "            x_start = state[1] * cell_size\n",
        "            y_start = height - (state[0] + 1) * cell_size\n",
        "            rect = patches.Rectangle((x_start, y_start), cell_size, cell_size, color='green', alpha=0.3)\n",
        "            ax2.add_patch(rect)\n",
        "            if Q_values:  # Add the red cell to ax1 for Q-values grid\n",
        "                rect_q = patches.Rectangle((x_start, y_start), cell_size, cell_size, color='green', alpha=0.3)\n",
        "                ax1.add_patch(rect_q)\n",
        "            if state not in arrow_locations:  # Skip if an arrow is already defined for this cell\n",
        "                # Add 'O' label\n",
        "                x_pos = state[1] * cell_size + cell_size / 2\n",
        "                y_pos = height - (state[0] * cell_size + cell_size / 2)\n",
        "                ax2.text(x_pos, y_pos, 'O', ha='center', va='center', fontsize=24, color='blue')\n",
        "\n",
        "\n",
        "    if arrows:\n",
        "      for (row, col), directions in arrows.items():\n",
        "          # Center of the cell\n",
        "          x_center = col * cell_size + cell_size / 2  # Center x\n",
        "          y_center = height - (row * cell_size + cell_size / 2)  # Center y\n",
        "\n",
        "          # Arrow size scaling factor\n",
        "          arrow_length = cell_size * 0.3\n",
        "\n",
        "          for direction in directions:\n",
        "              dx, dy = 0, 0\n",
        "\n",
        "              # Define arrow directions\n",
        "              if direction == \"up\":\n",
        "                  dx, dy = 0, arrow_length\n",
        "              elif direction == \"down\":\n",
        "                  dx, dy = 0, -arrow_length\n",
        "              elif direction == \"left\":\n",
        "                  dx, dy = -arrow_length, 0\n",
        "              elif direction == \"right\":\n",
        "                  dx, dy = arrow_length, 0\n",
        "\n",
        "\n",
        "              # Draw the arrow starting from the center\n",
        "              ax2.annotate(\"\", xy=(x_center + dx, y_center + dy),  # Arrow tip\n",
        "                           xytext=(x_center, y_center),       # Arrow base\n",
        "                           arrowprops=dict(facecolor='black',      # Arrow color\n",
        "                                           edgecolor='black',      # Outline color\n",
        "                                           shrinkA=0,              # No shrinking at the arrow base\n",
        "                                           shrinkB=0,              # No shrinking at the arrow tip\n",
        "                                           width=1,                # Narrower tail width\n",
        "                                           headwidth=5,            # Smaller head width\n",
        "                                           headlength=5            # Smaller head length\n",
        "                                           )\n",
        "                            )\n",
        "\n",
        "    # Display Q-values (on ax1)\n",
        "    if Q_values:\n",
        "        for (row, col), q_value in Q_values.items():\n",
        "            x_pos = col * cell_size + cell_size / 2\n",
        "            y_pos = height - (row * cell_size + cell_size / 2)\n",
        "            ax1.text(x_pos, y_pos, f'{q_value:.2f}', ha='center', va='center', fontsize=12, color='black')\n",
        "\n",
        "        # Draw the grid lines for the Q-value grid (on ax1)\n",
        "        for x in range(0, width + 1, cell_size):\n",
        "            ax1.plot([x, x], [0, height], color='black', linewidth=line_width)\n",
        "        for y in range(0, height + 1, cell_size):\n",
        "            ax1.plot([0, width], [y, y], color='black', linewidth=line_width)\n",
        "\n",
        "        # Draw the thicker outer border (on ax1)\n",
        "        ax1.plot([0, width], [0, 0], color='black', linewidth=border_width)  # Bottom border\n",
        "        ax1.plot([0, width], [height, height], color='black', linewidth=border_width)  # Top border\n",
        "        ax1.plot([0, 0], [0, height], color='black', linewidth=border_width)  # Left border\n",
        "        ax1.plot([width, width], [0, height], color='black', linewidth=border_width)  # Right border\n",
        "\n",
        "    # Draw the inner grid lines (on ax2)\n",
        "    for x in range(0, width + 1, cell_size):\n",
        "        ax2.plot([x, x], [0, height], color='black', linewidth=line_width)\n",
        "    for y in range(0, height + 1, cell_size):\n",
        "        ax2.plot([0, width], [y, y], color='black', linewidth=line_width)\n",
        "\n",
        "    # Draw the thicker outer border (on ax2)\n",
        "    ax2.plot([0, width], [0, 0], color='black', linewidth=border_width)  # Bottom border\n",
        "    ax2.plot([0, width], [height, height], color='black', linewidth=border_width)  # Top border\n",
        "    ax2.plot([0, 0], [0, height], color='black', linewidth=border_width)  # Left border\n",
        "    ax2.plot([width, width], [0, height], color='black', linewidth=border_width)  # Right border\n",
        "\n",
        "    # Remove axes for both grids for a cleaner look\n",
        "    if Q_values is not None:\n",
        "        ax1.axis('off')\n",
        "    ax2.axis('off')\n",
        "\n",
        "    # Save or display the grids\n",
        "    if save_path:\n",
        "        plt.savefig(save_path, bbox_inches='tight', dpi=300)\n",
        "    plt.show()\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 422
        },
        "id": "hrM0eyVCNqD2",
        "outputId": "d2040c13-3332-4d93-be98-7d02932afb45"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 500x500 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZcAAAGVCAYAAAAyrrwGAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAKLlJREFUeJzt3Xt0VOW9//HPJCFhciOTQAiXA6EkwTSQCHGpQOsN2mrPcgFWV3s4eAsuUDla0C6X2p9WdNUlarVyU0LBY6WipVjaLrwdONriUSliIJEMBCgh3EISDCH3TCb5/TG4Q7iESXiSPZO8X/9knvH5zvoyTvLJfvZ+dhytra2tAgDAoBC7GwAA9D6ECwDAOMIFAGAc4QIAMI5wAQAYR7gAAIwjXAAAxhEuAADjCBcAgHFh/k50OBzd2QcAIEj4c2MXjlwAAMb5feTyrcJlyzQwNrY7eul18t1uTX32WW3asEGZkybZ3U5QyM/P19SpU7Vp0yZlZmba3U7Q4H3rvM+2fabp/z5dj+c+rvSx6Xa3E/BOVZ7SvH+f5/f8TofLwNhYDRowoLNlfZIrOtr3NS5OgwYNsrmb4OByuayvvGf+433rvDhXnCQpOjZaA+L5mWYay2IAAOMIFwCAcYQLAMA4wgUAYBzhAgAwjnABABhHuAAAjCNcAADGES4AAOMIFwCAcYQLAMA4wgUAYBzhAgAwjnABABhHuAAAjCNcAADGES4AAOMIFwCAcYQLAMC4MLsb6MjPV2Zo8d++I0kaEt+gwqUfKy662a/a+Ssz9Mrp2iRXgwqXfSJXtKfbegWAS1F2xKmtm5O0fUuiSksidfJEhJo9IYpLaFT84AZlTazQldeXKnVcld2t+iWgw+XZ23frr1uTVFwWqWPf9NfDqzO06sGdF637fLdLSzaOssbL7y0gWAAEpFOV4XprSZo+/ONIeZvPXUwqOxqpsqOR2p0Xr3eWp2n898qU80ihRqZV29Ct/wJ6WSyqv1e589rCZPWmEdq0Y2CHNY2eEOUszlJLi0OSdOuko5oxsbRb+wSArijZF635t1yj994aZQVLaFiLvpt9QtfefFhTf1KiCd8vU+QZvxznfZqoh279vj77cIhdbfsloI9cJOkH4yt095QSvb55hCRpzrJMFSz5u6L6e887f+HaNO0+HCNJio9p0tK5BT3WKwD4q2RftB79z8mqqQqXJIX1a9H0u/dr+t37Fetqv9LiaQrRlveG6vXnv6uqbyLkaQrV8wuyNX9Rnq67+Ygd7V9UQB+5fOul2bs0JL5BknTgeJQef/Oy887L2x+rF/482hr/9p5dGuxq6pEeAcBfTY0hemFBthUsEc5mPb3qC93x0O5zgkWS+oW36Ibph/XKhr9r6MgaSVJLi0PLf5WpIweierR3fwVFuMRFN2vZGUcgSzeO0ue7Xe3mNHsdyll8uZq9vn/STdnHdfv1h3u0TwDwx7oVqTq4N9YaL1iUp7FXnrhoXXxio55e/YWcUb4AaqgL09Insrqtz0sRFOEiSTMmluq2yUcl+RJ79pIsNXra2n/uTynacWCAJCnG6dFr97McBiDwNNaH6r0/JFvjq6ce06Qf+n9eOHFYvWY+UGSNd32ZoKL8OIMdmhE04SJJS+cWKD7Gt8zlPhSjZ95JlSQVlkRbjyVp0Z1ujRhUb0uPANCRTz8YourTy2GSNO2uf3X6NX5420E5I9u2Zby/dqSR3kwKqnBJjGvSy7N3WeNF61P01f4Byll8uZqaQyVJ12Sc0L03HbSrRQDoUMHWtiteBw6pV8YV33T6NZxRXl05pe1op+CfHV9Fa4egChdJuuOGw7pxQpkkqdkbout/OVFbi3znX5zhXv3ugZ1yOOzsEAAurPCreOtxWmZll19nzBm1ZUcideJ4/0vqy7SgCxdJWjEvXzFO3wmtU3X9rOcXztyj1KG1drUFABdVftRpPR6Z2vWNkCPOqj1+xHmBmfYIynAZMahev5ixv91zV6Sc1EPT9l+gAgDsV1cT1m4XfnRs1+8cEj2gfW3NyfALzLRHUIZLk8eht7cMa/ecw9FqUzcA4J/62vb71iMi/btX4vn0P6u2rjaw9sQHZbg8806a3Idi2j23ba/LulElAAQiZ1T7QGis63ogNJxVGxnV9aDqDkEXLjsPxGrRuynWeEpWufX4ybfGqPh4YK07AsC3IqObFRrWYo2rq/p1MLtjNWfVRscF1t1IgipcvF5p9pIseU6vWd58Zak+eGqrskb5bkFd2xCmucsz7WwRADo0aGjbHrySvTEdzOzY2bWDhwXW3r6gCpcXN4zW9n1xkqTYSI9eva9AYaGtWvXAToWG+H4b+CgvUW9sHm5jlwBwYenj2/a1FOW7OpjZsT1n1CYOq1PC4IZL6su0oAmXoiNRemrtGGv8/F1uDUvwvZnZKVVaMK1tl+vDqzNUXhVYV04AgCRlXlVhPa4oderrbfEdzD6/+tpQ/XNzkjUed8ZrBoqgCJfWVt9yWEOTbxf+deMqNOdH7XfhPz1zj0Yn+fa4nKgO14O5Y3u8TwC4mMk3HlP0gLbzI3/579EdzD6/j9aNVP0ZJ/Rv+lng3ZUkKMJl+XvJ+rQwQZJvF/7KeefuwndGtGjFvHxr/PaWYdq4LbEn2wSAi+of6dWP/6PYGm/dnKTP/yfpwgVnKTvi1FuL21ZxMq44obTMkwY7NCPgw+VgmVOP/j7dGi+cuUcpQ+vOO3dKVoVyppZY4/tezVR1XWi39wgAnXHb3H36t5S2HfYvPTLer+WxyvII/Wr21dZRS//IZv3XMxf/0+92CPhwmbssUzX1vjcy249d+C/mFCrJ5TsXc6jCqcfOCCYACAQRTq8eeWm7omJ8u+wb68P0ZM5EvfnyZTpVee7lyZ4mh/53w3A9OP1aHSmOliSFhLTq/oX5GjYqMG95FVhbOs/yxubh+jDPt7TVL6zFd1XYRQ5EXNEeLZnztW5bdIUkafn7yZp57RFNSu/6DeIAwLSRadV67g//p4VzrlJFqVPNnhCtW5Gqd1eN1pjLK5U4tF5h/VpUWR6h3Xnxqq1uC51+4V4teD5P37vxmI3/go4F7JFLaWWEFqzKsMaP3LJPWaNO+VV76+Rjmn61701vbXXonqVZavJwq2QAgWVkWrVeWv8P3fizYmtzpbc5RIVfJuiTvw7XpvUjtP0fg9sFy/jJZfrNui0BHSxSAB+5zHttnCprfJcTXza8Wk/8dG+n6pfNLdDHBQNVVdtP7kMx+vW6VC2cWXTxQgDoQXEJTbr/qQL95J59+mJzkr7akqhjJVGqOhEub3OIYuObNHBwvTKvrtBVU0qVOq7K7pb9ErDhsv6xLy+pfmhCo06u/cBQNwDQvQYPr9e0Ow9o2p0H7G7FiIBdFgMABC/CBQBgHOECADCOcAEAGEe4AACMI1wAAMYRLgAA4wgXAIBxhAsAwDjCBQBgHOECADCOcAEAGEe4AACMI1wAAMYRLgAA4wgXAIBxhAsAwDjCBQBgHOECADCOcAEAGEe4AACMI1wAAMYRLgAA4wgXAIBxhAsAwDjCBQBgHOECADAurLMF+W63XNHR3dFLr+MuLvZ93bHD1j6CibuoyPfV7ba5k+Dy7fvF++a/oj2+z9qePXvU0NxgczeBr/ZUbafmO1pbW1v9muhwdKkhAEDv4k9sdPrIZdOGDXLFxXWlnz7HvWOHZs2frzVz5ig9OdnudoKCu7hYs3JztSY3V+nZ2Xa3EzTcbrdmzZqlNWvWKD093e52gsL2/O2ac/cc3fn0nRo5eqTd7QS82lO1+s19v/F7fqfDJXPSJA0aNKizZX1aenKyJowda3cbQSU9LU0TJkywu42gk56ezvvmp+rGaknSmLQxGns5358XU/VNVafmc0IfAGAc4QIAMI5wAQAYR7gAAIwjXAAAxhEuAADjCBcAgHGECwDAOMIFAGAc4QIAMI5wAQAYR7gAAIwjXAAAxhEuAADjCBcAgHGECwDAOMIFAGAc4QIAMI5wAQAYR7gAAIwLs7sBmPfzlRla/LfvSJKGxDeocOnHiotu9qt2/soMvXK6NsnVoMJln8gV7em2XgH4lB1xauvmJG3fkqjSkkidPBGhZk+I4hIaFT+4QVkTK3Tl9aVKHVdld6t+IVx6oWdv362/bk1ScVmkjn3TXw+vztCqB3detO7z3S4t2TjKGi+/t4BgAbrZqcpwvbUkTR/+caS8zecuJpUdjVTZ0UjtzovXO8vTNP57Zcp5pFAj06pt6NZ/LIv1QlH9vcqd1xYmqzeN0KYdAzusafSEKGdxllpaHJKkWycd1YyJpd3aJ9DXleyL1vxbrtF7b42ygiU0rEXfzT6ha28+rKk/KdGE75cp8oxf8vI+TdRDt35fn304xK62/cKRSy/1g/EVuntKiV7fPEKSNGdZpgqW/F1R/b3nnb9wbZp2H46RJMXHNGnp3IIe6xXoi0r2RevR/5ysmqpwSVJYvxZNv3u/pt+9X7Gu9isGnqYQbXlvqF5//ruq+iZCnqZQPb8gW/MX5em6m4/Y0f5FceTSi700e5eGxDdIkg4cj9Ljb1523nl5+2P1wp9HW+Pf3rNLg11NPdIj0Bc1NYbohQXZVrBEOJv19KovdMdDu88JFknqF96iG6Yf1isb/q6hI2skSS0tDi3/VaaOHIjq0d79Rbj0YnHRzVp2xhHI0o2j9PluV7s5zV6HchZfrmav76NwU/Zx3X794R7tE+hr1q1I1cG9sdZ4waI8jb3yxEXr4hMb9fTqL+SM8gVQQ12Ylj6R1W19XgrCpZebMbFUt00+Ksn3m87sJVlq9LT9b3/uTynacWCAJCnG6dFr97McBnSnxvpQvfeHZGt89dRjmvRD/89vJg6r18wHiqzxri8TVJQfZ7BDMwiXPmDp3ALFx/iWudyHYvTMO6mSpMKSaOuxJC26060Rg+pt6RHoKz79YIiqTy+HSdK0u/7V6df44W0H5Yxs217w/tqRRnoziXDpAxLjmvTy7F3WeNH6FH21f4ByFl+upuZQSdI1GSd0700H7WoR6DMKtrZduTlwSL0yrvim06/hjPLqyiltRzsF/+z4alA7EC59xB03HNaNE8okSc3eEF3/y4naWuQ7/+IM9+p3D+yUw2Fnh0DfUPhVvPU4LbOyy68z5ozasiOROnG8/yX1ZRrh0oesmJevGKfvROCpun7W8wtn7lHq0Fq72gL6lPKjTuvxyNSub4QccVbt8SPOC8y0B+HSh4wYVK9fzNjf7rkrUk7qoWn7L1ABwKS6mrB2u/CjY7t+B4zoAe1ra06GX2CmPQiXPqTJ49DbW4a1e87haLWpG6Dvqa9tv289ItK/e/6dT/+zautqA2tPPOHShzzzTprch2LaPbdtr8u6USWA7uWMah8IjXVdD4SGs2ojo7oeVN2BcOkjdh6I1aJ3U6zxlKxy6/GTb41R8fHAWq8FeqPI6GaFhrVY4+qqfh3M7ljNWbXRcYF1Vw3CpQ/weqXZS7LkOb3We/OVpfrgqa3KGuW7dXdtQ5jmLs+0s0Wgzxg0tG0vWcnemA5mduzs2sHDAmuPGuHSB7y4YbS274uTJMVGevTqfQUKC23Vqgd2KjTE91vUR3mJemPzcBu7BPqG9PFt+1qK8l0dzOzYnjNqE4fVKWFwwyX1ZRrh0ssVHYnSU2vHWOPn73JrWILvQ5idUqUF09p2Bz+8OkPlVYF1xQnQ22ReVWE9rih16utt8R3MPr/62lD9c3OSNR53xmsGCsKlF2tt9S2HNTT5duFfN65Cc37Ufhf+0zP3aHSSb4/LiepwPZg7tsf7BPqSyTceU/SAtvMjf/nv0R3MPr+P1o1U/Rkn9G/6WeDdXYNw6cWWv5esTwsTJPl24a+cd+4ufGdEi1bMy7fGb28Zpo3bEnuyTaBP6R/p1Y//o9gab92cpM//J+nCBWcpO+LUW4vbViMyrjihtMyTBjs0g3DppQ6WOfXo79Ot8cKZe5QytO68c6dkVShnaok1vu/VTFXXhXZ7j0Bfddvcffq3lLYd9i89Mt6v5bHK8gj9avbV1lFL/8hm/dczF/8T5nYgXHqpucsyVVPv+wBm+7EL/8WcQiW5fOdiDlU49dgZwQTArAinV4+8tF1RMb5d9o31YXoyZ6LefPkynao89/JkT5ND/7thuB6cfq2OFEdLkkJCWnX/wnwNGxWYt24KrC2dMOKNzcP1YZ5vaatfWIvvqrCLHIi4oj1aMudr3bboCknS8veTNfPaI5qU3vUb6wG4sJFp1XruD/+nhXOuUkWpU82eEK1bkap3V43WmMsrlTi0XmH9WlRZHqHdefGqrW4LnX7hXi14Pk/fu/GYjf+CjnHk0suUVkZowaoMa/zILfuUNeqUX7W3Tj6m6Vf7PqytrQ7dszRLTR5ulQx0l5Fp1Xpp/T9048+Krc2V3uYQFX6ZoE/+Olyb1o/Q9n8Mbhcs4yeX6TfrtgR0sEgcufQ6814bp8oa3+XElw2v1hM/3dup+mVzC/RxwUBV1faT+1CMfr0uVQtnFl28EECXxCU06f6nCvSTe/bpi81J+mpLoo6VRKnqRLi8zSGKjW/SwMH1yry6QldNKVXquCq7W/YL4dLLrH/sy0uqH5rQqJNrPzDUDQB/DR5er2l3HtC0Ow/Y3YoRLIsBAIwjXAAAxhEuAADjCBcAgHGECwDAOMIFAGAc4QIAMI5wAQAYR7gAAIwjXAAAxhEuAADjCBcAgHGECwDAOMIFAGAc4QIAMI5wAQAYR7gAAIwjXAAAxhEuAADjCBcAgHGECwDAOMIFAGAc4QIAMI5wAQAYR7gAAIwjXAAAxhEuAADjwjpbkJ+fL5fL1R299DruoiLf1+JiexsJIt++V+4dO2ztI9hYnzW32+ZOgkfRHt97Vryv2N5GgkTNqZpOzXe0tra2+jXR4ehSQwCA3sWf2Oj0kcumTZs4cvGT2+3WrFmztCY3V+lpaXa3ExTcO3Zo1vz5WjNnjtKTk+1uJ2i4i4s1KzfX91nLzra7naDw7fdn7uu5ShvD9+fFnKw8qen/Pt3v+Z0Ol8zMTA0aNKizZX1aena2JkyYYHcbQSU9OVkTxo61u42gk56Wxmetk7Iz+f70R3l5eafmc0IfAGAc4QIAMI5wAQAYR7gAAIwjXAAAxhEuAADjCBcAgHGECwDAOMIFAGAc4QIAMI5wAQAYR7gAAIwjXAAAxhEuAADjCBcAgHGECwDAOMIFAGAc4QIAMI5wAQAYR7gAAIwLs7sBIBD8fGWGFv/tO5KkIfENKlz6seKim/2qnb8yQ6+crk1yNahw2SdyRXu6rVcgGHDkAkh69vbdSk6skyQd+6a/Hl6d4Vfd57tdWrJxlDVefm8BwQKIcAEkSVH9vcqdt9Mar940Qpt2DOywptETopzFWWppcUiSbp10VDMmlnZrn0CwIFyA034wvkJ3TymxxnOWZaq2IfSC8xeuTdPuwzGSpPiYJi2dW9DtPQLBgnABzvDS7F0aEt8gSTpwPEqPv3nZeefl7Y/VC38ebY1/e88uDXY19UiPQDAgXIAzxEU3a9kZRyBLN47S57td7eY0ex3KWXy5mr2+b5+bso/r9usP92ifQKAjXICzzJhYqtsmH5UktbQ4NHtJlho9bd8qz/0pRTsODJAkxTg9eu1+lsOAsxEuwHksnVug+BjfMpf7UIyeeSdVklRYEm09lqRFd7o1YlC9LT0CgYxwAc4jMa5JL8/eZY0XrU/RV/sHKGfx5Wpq9p3kvybjhO696aBdLQIBjXABLuCOGw7rxgllkqRmb4iu/+VEbS3ynX9xhnv1uwd2yuGws0MgcBEuQAdWzMtXjNO3KfJUXT/r+YUz9yh1aK1dbQEBj3ABOjBiUL1+MWN/u+euSDmph6btv0AFAIlwATrU5HHo7S3D2j3ncLTa1A0QPAgXoAPPvJMm96GYds9t2+uyblQJ4PwIF+ACdh6I1aJ3U6zxlKxy6/GTb41R8XGnHW0BQYFwAc7D65VmL8mSp9n3LXLzlaX64KmtyhpVJUmqbQjT3OWZdrYIBDTCBTiPFzeM1vZ9cZKk2EiPXr2vQGGhrVr1wE6FhrRIkj7KS9Qbm4fb2CUQuAgX4CxFR6L01Nox1vj5u9waluC7mWV2SpUWTPuX9d8eXp2h8qrwHu8RCHSEC3CG1lbfclhDk28X/nXjKjTnR+134T89c49GJ/n2uJyoDteDuWN7vE8g0BEuwBmWv5esTwsTJPl24a+cd+4ufGdEi1bMy7fGb28Zpo3bEnuyTSDgES7AaQfLnHr09+nWeOHMPUoZWnfeuVOyKpQzte0Pi933aqaq6y78h8WAvoZwAU6buyxTNfVhkqRsP3bhv5hTqCSX71zMoQqnHjsjmIC+jnABJL2xebg+zPMtbfULa/FdFXaRAxFXtEdL5nxtjZe/n6zP3K4OKoC+g3BBn1daGaEFqzKs8SO37FPWqFN+1d46+ZimX31MktTa6tA9S7PU5OFWyQDhgj5v3mvjVFnju5z4suHVeuKneztVv2xugQZE+e6c7D4Uo1+vS71IBdD7hdndAGC39Y99eUn1QxMadXLtB4a6AXoHjlwAAMYRLgAA4wgXAIBxhAsAwDjCBQBgHOECADCOcAEAGEe4AACMI1wAAMYRLgAA4wgXAIBxhAsAwDjCBQBgHOECADCOcAEAGEe4AACMI1wAAMYRLgAA4wgXAIBxhAsAwDjCBQBgHOECADCOcAEAGEe4AACMI1wAAMYRLgAA4wgXAIBxYZ0tyM/Pl8vl6o5eeh23293uKy7OXVTk+1pcbG8jQebb98u9Y4etfQQT67PG96dfKisrOzXf0dra2urXRIejSw0BAHoXf2Kj00cumzZt4sjFT263W7NmzdKaNWuUnp5udztBwXrPcnOVnpZmdztBw71jh2bNn681c+YoPTnZ7naCgru4WLNyc32ftexsu9sJeJWVlZo6darf8zsdLpmZmRo0aFBny/q09PR0TZgwwe42gkp6djbvWRekJydrwtixdrcRVNLT0vis+aG8vLxT8zmhDwAwjnABABhHuAAAjCNcAADGES4AAOMIFwCAcYQLAMA4wgUAYBzhAgAwjnABABhHuAAAjCNcAADGES4AAOMIFwCAcYQLAMA4wgUAYBzhAgAwjnABABhHuAAAjCNcAADGhdndAIDg9fOVGVr8t+9IkobEN6hw6ceKi272q3b+ygy9cro2ydWgwmWfyBXt6bZe0bM4cgHQZc/evlvJiXWSpGPf9NfDqzP8qvt8t0tLNo6yxsvvLSBYehnCBUCXRfX3KnfeTmu8etMIbdoxsMOaRk+IchZnqaXFIUm6ddJRzZhY2q19oucRLgAuyQ/GV+juKSXWeM6yTNU2hF5w/sK1adp9OEaSFB/TpKVzC7q9R/Q8wgXAJXtp9i4NiW+QJB04HqXH37zsvPPy9sfqhT+Ptsa/vWeXBruaeqRH9CzCBcAli4tu1rIzjkCWbhylz3e72s1p9jqUs/hyNXt9P3Zuyj6u268/3KN9oucQLgCMmDGxVLdNPipJamlxaPaSLDV62n7EPPenFO04MECSFOP06LX7WQ7rzQgXAMYsnVug+BjfMpf7UIyeeSdVklRYEm09lqRFd7o1YlC9LT2iZxAuAIxJjGvSy7N3WeNF61P01f4Byll8uZqafSf5r8k4oXtvOmhXi+ghhAsAo+644bBunFAmSWr2huj6X07U1iLf+RdnuFe/e2CnHA47O0RPIFwAGLdiXr5inL5Nkafq+lnPL5y5R6lDa+1qCz2IcAFg3IhB9frFjP3tnrsi5aQemrb/AhXobQgXAMY1eRx6e8uwds85HK02dQM7EC4AjHvmnTS5D8W0e27bXpd1o0r0foQLAKN2HojVondTrPGUrHLr8ZNvjVHxcacdbaGHES4AjPF6pdlLsuRp9v1oufnKUn3w1FZljaqSJNU2hGnu8kw7W0QPIVwAGPPihtHavi9OkhQb6dGr9xUoLLRVqx7YqdCQFknSR3mJemPzcBu7RE8gXAAYUXQkSk+tHWONn7/LrWEJvptZZqdUacG0f1n/7eHVGSqvCu/xHtFzCBcAl6y11bcc1tDk24V/3bgKzflR+134T8/co9FJvj0uJ6rD9WDu2B7vEz2HcAFwyZa/l6xPCxMk+Xbhr5x37i58Z0SLVszLt8ZvbxmmjdsSe7JN9CDCBcAlOVjm1KO/T7fGC2fuUcrQuvPOnZJVoZypbX9Y7L5XM1Vdd+E/LIbgRbgAuCRzl2Wqpj5MkpTtxy78F3MKleTynYs5VOHUY2cEE3oPwgVAl72xebg+zPMtbfULa/FdFXaRAxFXtEdL5nxtjZe/n6zP3K4OKhCMCBcAXVJaGaEFqzKs8SO37FPWqFN+1d46+ZimX31MktTa6tA9S7PU5OFWyb0J4QKgS+a9Nk6VNb7LiS8bXq0nfrq3U/XL5hZoQJTvzsnuQzH69brUi1QgmITZ3QCA4LT+sS8vqX5oQqNOrv3AUDcINBy5AACMI1wAAMYRLgAA4wgXAIBxhAsAwDjCBQBgHOECADCOcAEAGEe4AACMI1wAAMYRLgAA4wgXAIBxhAsAwDjCBQBgHOECADCOcAEAGEe4AACMI1wAAMYRLgAA4wgXAIBxhAsAwDjCBQBgHOECADCOcAEAGEe4AACMI1wAAMYRLgAA48I6W/DZts8U54rrhlZ6n6I9RZIkt9ttcyfB49v3ivesc9xFpz9rxcX2NhJEvn2v3Dt22NpHsKg8ebJT8x2tra2tfk10OLrSDwCgl/EnNjp95PJ47uOKjo3uUkN9TfG+YuX+v1zlvp6r7Mxsu9sJCm63W7NmzdKaNWuUnp5udztBw3rfcnOVnpZmdztBwb1jh2bNn681c+YoPTnZ7nYCXmVNjaY++6zf8zsdLulj0zUgfkBny/q0tDFpmjBhgt1tBJX09HTesy5Iz87mfeuk9ORkTRg71u42Al55VVWn5nNCHwBgHOECADCOcAEAGEe4AACMI1wAAMYRLgAA4wgXAIBxhAsAwDjCBQBgHOECADCOcAEAGEe4AACMI1wAAMYRLgAA4wgXAIBxhAsAwDjCBQBgHOECADCOcAEAGEe4AACMC7O7ga4oO+LU1s1J2r4lUaUlkTp5IkLNnhDFJTQqfnCDsiZW6MrrS5U6rsruVgGgnZ+vzNDiv31HkjQkvkGFSz9WXHSzX7XzV2boldO1Sa4GFS77RK5oT7f1eimCKlxOVYbrrSVp+vCPI+VtPvegq+xopMqORmp3XrzeWZ6m8d8rU84jhRqZVm1DtwBwrmdv362/bk1ScVmkjn3TXw+vztCqB3detO7z3S4t2TjKGi+/tyBgg0UKomWxkn3Rmn/LNXrvrVFWsISGtei72Sd07c2HNfUnJZrw/TJFnvFm532aqIdu/b4++3CIXW0DQDtR/b3KndcWJqs3jdCmHQM7rGn0hChncZZaWhySpFsnHdWMiaXd2uelCoojl5J90Xr0PyerpipckhTWr0XT796v6XfvV6yrfXJ7mkK05b2hev3576rqmwh5mkL1/IJszV+Up+tuPmJH+wDQzg/GV+juKSV6ffMISdKcZZkqWPJ3RfX3nnf+wrVp2n04RpIUH9OkpXMLeqzXrgr4I5emxhC9sCDbCpYIZ7OeXvWF7nho9znBIkn9wlt0w/TDemXD3zV0ZI0kqaXFoeW/ytSRA1E92jsAXMhLs3dpSHyDJOnA8Sg9/uZl552Xtz9WL/x5tDX+7T27NNjV1CM9XoqAD5d1K1J1cG+sNV6wKE9jrzxx0br4xEY9vfoLOaN8AdRQF6alT2R1W58A0Blx0c1adsYRyNKNo/T5ble7Oc1eh3IWX65mr+9H9U3Zx3X79Yd7tM+uCuhwaawP1Xt/SLbGV089pkk/9H+dMXFYvWY+UGSNd32ZoKL8OIMdAkDXzZhYqtsmH5XkW2GZvSRLjZ62H8vP/SlFOw4MkCTFOD167f7AXw77VkCHy6cfDFH16eUwSZp21786/Ro/vO2gnJFtl/m9v3akkd4AwISlcwsUH+Nb5nIfitEz76RKkgpLoq3HkrToTrdGDKq3pceuCOhwKdjadgXFwCH1yrjim06/hjPKqyuntB3tFPyz46syAKAnJcY16eXZu6zxovUp+mr/AOUsvlxNzaGSpGsyTujemw7a1WKXBHS4FH4Vbz1Oy6zs8uuMOaO27EikThzvf0l9AYBJd9xwWDdOKJMkNXtDdP0vJ2prke/8izPcq989sFMOh50ddl5Ah0v5Uaf1eGRq1zdCjjir9vgR5wVmAoA9VszLV4zTdwHSqbp+1vMLZ+5R6tBau9rqsoANl7qasHa78KNju74TNXpA+9qak+EXmAkA9hgxqF6/mLG/3XNXpJzUQ9P2X6AisAVsuNTXtt/fGRHp3713zqf/WbV1tUGxdxRAH9LkcejtLcPaPedwtNrUzaUL2HBxRrUPhMa6rgdCw1m1kVFdDyoA6A7PvJMm96GYds9t2+uyblQZbAI2XCKjmxUa1mKNq6v6dTC7YzVn1UbHBf7uVgB9x84DsVr0boo1npJVbj1+8q0xKj4efOeJAzZcJGnQ0LZrukv2xnQws2Nn1w4eFjzXigPo3bxeafaSLHlOn2O++cpSffDUVmWN8v3JkNqGMM1dnmlni10S0OGSPr5tX0tRvquDmR3bc0Zt4rA6JQxuuKS+AMCUFzeM1vZ9cZKk2EiPXr2vQGGhrVr1wE6FhvhWbz7KS9Qbm4fb2GXnBXS4ZF5VYT2uKHXq623xHcw+v/raUP1zc5I1HnfGawKAnYqOROmptWOs8fN3uTUswffLb3ZKlRZMa7srycOrM1ReFTxXugZ0uEy+8ZiiB7SdH/nLf4/uYPb5fbRupOrPOKF/08+Ca5crgN6ptdW3HNbQ5NuFf924Cs35UfufT0/P3KPRSb49Lieqw/Vg7tge77OrAjpc+kd69eP/KLbGWzcn6fP/SbpwwVnKjjj11uK23woyrjihtMyTBjsEgK5Z/l6yPi1MkOTbhb9y3rm78J0RLVoxL98av71lmDZuS+zJNrssoMNFkm6bu0//ltK2w/6lR8b7tTxWWR6hX82+2jpq6R/ZrP965uJ/ShQAutvBMqce/X26NV44c49Shtadd+6UrArlTC2xxve9mqnqutBu7/FSBXy4RDi9euSl7YqK8e2yb6wP05M5E/Xmy5fpVOW5lyd7mhz63w3D9eD0a3WkOFqSFBLSqvsX5mvYqOC7hQKA3mfuskzV1Pt+8c32Yxf+izmFSnL5zsUcqnDqsTOCKVAFxVb1kWnVeu4P/6eFc65SRalTzZ4QrVuRqndXjdaYyyuVOLReYf1aVFkeod158aqtbgudfuFeLXg+T9+78ZiN/wIA8Hlj83B9mOdb2uoX1uK7KuwiByKuaI+WzPlaty26QpK0/P1kzbz2iCald/2Gvt0t4I9cvjUyrVovrf+HbvxZsbW50tscosIvE/TJX4dr0/oR2v6Pwe2CZfzkMv1m3RaCBUBAKK2M0IJVGdb4kVv2KWvUKb9qb518TNOv9v0sa2116J6lWWryBO6tkoPiyOVbcQlNuv+pAv3knn36YnOSvtqSqGMlUao6ES5vc4hi45s0cHC9Mq+u0FVTSpU6rsrulgHAMu+1caqs8V1OfNnwaj3x072dql82t0AfFwxUVW0/uQ/F6NfrUrVwZtHFC20QVOHyrcHD6zXtzgOaducBu1sBAL+tf+zLS6ofmtCok2s/MNRN9wqaZTEAQPAgXAAAxhEuAADjCBcAgHGECwDAOMIFAGAc4QIAMI5wAQAYR7gAAIwjXAAAxhEuAADjCBcAgHGECwDAOMIFAGAc4QIAMI5wAQAYR7gAAIwjXAAAxhEuAADjCBcAgHGECwDAOMIFAGAc4QIAMI5wAQAYR7gAAIwL62zBqcpT3dFHr1RzqkaSdLLypMrLy23uJjhUVlZaX3nP/Mf71nmVJ0/6vtbUqLyqyt5mgkDFqc797He0tra2+jXR4ehSQwCA3sWf2GBZDABgnN/LYn4e4AAAwJELAMA8wgUAYBzhAgAwjnABABhHuAAAjCNcAADGES4AAOMIFwCAcYQLAMC4/w/y0iM4tnyFAwAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ],
      "source": [
        "import numpy as np\n",
        "np.random.seed(100)\n",
        "\n",
        "# RL environment\n",
        "class gridworld:\n",
        "    def __init__(self, dims, goal_states, error_states, finite_horizon=False):\n",
        "        # for actions\n",
        "        self.actions = [\"up\", \"down\", \"right\", \"left\"]\n",
        "        self.action_map = {\"up\": 0, \"down\": 1, \"right\": 2, \"left\": 3}\n",
        "        self.movement = [[-1, 0], [1, 0], [0, 1], [0, -1]]\n",
        "\n",
        "        # for states\n",
        "        self.rows = dims[0]\n",
        "        self.columns = dims[1]\n",
        "        # self.current_state = None\n",
        "        self.error_states = error_states\n",
        "        self.goal_states = goal_states\n",
        "        self.empty_states = [(i, j) for i in range(self.rows) for j in range(self.columns) if (i, j) not in (self.goal_states + self.error_states)]\n",
        "\n",
        "        # horizon type\n",
        "        self.finite_horizon = finite_horizon\n",
        "\n",
        "        # visual representation of gridworld\n",
        "        self.grid = np.array([[\".\" for i in range(self.columns)] for j in range(self.rows)])\n",
        "        for g in self.goal_states:\n",
        "            self.grid[g] = \"O\"\n",
        "        for e in self.error_states:\n",
        "            self.grid[e] = \"X\"\n",
        "\n",
        "        # track terminal states\n",
        "        self.terminal_states = set(goal_states)\n",
        "\n",
        "    # print gridworld\n",
        "    def print_grid(self):\n",
        "        print(self.grid)\n",
        "\n",
        "    # set current state to start state\n",
        "    def reset(self, start_state=None):\n",
        "        if start_state is not None:\n",
        "            self.current_state = start_state\n",
        "        else:\n",
        "            self.current_state = self.empty_states[np.random.choice(len(self.empty_states))]\n",
        "        return self.current_state\n",
        "\n",
        "    def step(self, action):\n",
        "        # If finite horizon and the agent is in a goal state, it stays there\n",
        "        if self.finite_horizon and self.current_state in self.goal_states:\n",
        "            return self.current_state, 0  # No reward, as it's terminal\n",
        "\n",
        "        action_index = action\n",
        "        action_taken = self.actions[action]\n",
        "\n",
        "        new_state, reward = (-1, -1), 0\n",
        "\n",
        "        # Handle invalid actions explicitly\n",
        "        if (action_taken == \"up\" and self.current_state[0] == 0) or (action_taken == \"down\" and self.current_state[0] == self.rows - 1):\n",
        "            # return self.current_state, -5\n",
        "            return self.current_state, -2\n",
        "        elif (action_taken == \"left\" and self.current_state[1] == 0) or (action_taken == \"right\" and self.current_state[1] == self.columns - 1):\n",
        "            # return self.current_state, -5\n",
        "            return self.current_state, -2\n",
        "\n",
        "        # Inside the board\n",
        "        new_state = (self.current_state[0] + self.movement[action_index][0], self.current_state[1] + self.movement[action_index][1])\n",
        "        if new_state in self.empty_states:  # no reward in empty states\n",
        "            reward = -1\n",
        "        elif new_state in self.goal_states:  # good state\n",
        "            reward = 5 # 10\n",
        "        elif new_state in self.error_states:  # bad state\n",
        "            reward = -2 # -5\n",
        "\n",
        "        # Update current state after action\n",
        "        self.current_state = new_state\n",
        "\n",
        "        return self.current_state, reward\n",
        "\n",
        "size = 5\n",
        "goal_states = [(0, size - 1), (1, size - 1), (size - 1, 0)]\n",
        "error_states = [(o, o) for o in range(size)]\n",
        "\n",
        "grid = gridworld((size, size), goal_states, error_states, finite_horizon=False)\n",
        "create_grid(rows=5, columns=5, cell_size=20, line_width=1, border_width=3,\n",
        "            error_states=error_states, goal_states=goal_states, arrows=None,\n",
        "            Q_values=None, save_path='grid_with_arrows_colored.png')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V0oXXXgNGIcm"
      },
      "source": [
        "# Q-Learning\n",
        "\n",
        "We'll implement the Q-Learning algorithm for the above problem.\n",
        "\n",
        "**Recall Key Components:**\n",
        "\n",
        "* **Q-table:** A table storing the estimated value (Q-value) for each state-action pair.\n",
        "* **Learning Rate (α):** Controls how much the Q-values are updated with each experience. In this problem we will implement step size α as follows: At the nth iteration, `visit_n(s, a)` is the number of times the state action pair `(s,a)` is seen. So the step size `α = 1/visit_n(s, a)`.\n",
        "\n",
        "\n",
        "\n",
        "**Recall Algorithm Steps:**\n",
        "\n",
        "1. **Initialization:** Create a Q-table and initialize all Q-values to 0.\n",
        "2. **Repeat for each episode:**\n",
        "    * Initialize the starting state `s`.\n",
        "    * **Repeat for each step of the episode:**\n",
        "        * Choose an action `a` using an exploration-exploitation strategy (e.g., epsilon-greedy).\n",
        "        * Take action `a`, observe reward `r` and next state `s'`.\n",
        "        * Update the Q-value for the current state-action pair using the following update rule:\n",
        "\n",
        "            `Q(s, a) = Q(s, a) + α * [r + γ * max(Q(s', a')) - Q(s, a)]`\n",
        "\n",
        "         where `a'` represents all possible actions in the next state `s'`.\n",
        "        * Update the current state to `s'`.\n",
        "    * Until the episode terminates.\n",
        "3. **Until convergence or a predefined number of episodes is reached.**"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "grid = gridworld((size, size), goal_states, error_states, finite_horizon=True)\n",
        "q_table = np.zeros((grid.rows, grid.columns, len(grid.actions)))\n",
        "\n",
        "import pandas as pd\n",
        "\n",
        "# Create a readable DataFrame\n",
        "actions = ['Up', 'Down', 'Right', 'Left']\n",
        "# Reshape to (N_States, N_Actions)\n",
        "df = pd.DataFrame(q_table.reshape(-1, len(actions)), columns=actions)\n",
        "\n",
        "# Add (Row, Col) as the index\n",
        "rows, cols = q_table.shape[:2]\n",
        "df.index = [f\"Cell ({r}, {c})\" for r in range(rows) for c in range(cols)]\n",
        "\n",
        "# Display top 10 rows\n",
        "display(df.head(20))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 669
        },
        "id": "ZS2yHuMqUqVT",
        "outputId": "9612db3f-724c-4cae-a949-19c24a17f689"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "              Up  Down  Right  Left\n",
              "Cell (0, 0)  0.0   0.0    0.0   0.0\n",
              "Cell (0, 1)  0.0   0.0    0.0   0.0\n",
              "Cell (0, 2)  0.0   0.0    0.0   0.0\n",
              "Cell (0, 3)  0.0   0.0    0.0   0.0\n",
              "Cell (0, 4)  0.0   0.0    0.0   0.0\n",
              "Cell (1, 0)  0.0   0.0    0.0   0.0\n",
              "Cell (1, 1)  0.0   0.0    0.0   0.0\n",
              "Cell (1, 2)  0.0   0.0    0.0   0.0\n",
              "Cell (1, 3)  0.0   0.0    0.0   0.0\n",
              "Cell (1, 4)  0.0   0.0    0.0   0.0\n",
              "Cell (2, 0)  0.0   0.0    0.0   0.0\n",
              "Cell (2, 1)  0.0   0.0    0.0   0.0\n",
              "Cell (2, 2)  0.0   0.0    0.0   0.0\n",
              "Cell (2, 3)  0.0   0.0    0.0   0.0\n",
              "Cell (2, 4)  0.0   0.0    0.0   0.0\n",
              "Cell (3, 0)  0.0   0.0    0.0   0.0\n",
              "Cell (3, 1)  0.0   0.0    0.0   0.0\n",
              "Cell (3, 2)  0.0   0.0    0.0   0.0\n",
              "Cell (3, 3)  0.0   0.0    0.0   0.0\n",
              "Cell (3, 4)  0.0   0.0    0.0   0.0"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-6e375419-876a-409c-bda6-42a3b7cf9e14\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Up</th>\n",
              "      <th>Down</th>\n",
              "      <th>Right</th>\n",
              "      <th>Left</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>Cell (0, 0)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cell (0, 1)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cell (0, 2)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cell (0, 3)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cell (0, 4)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cell (1, 0)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cell (1, 1)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cell (1, 2)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cell (1, 3)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cell (1, 4)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cell (2, 0)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cell (2, 1)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cell (2, 2)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cell (2, 3)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cell (2, 4)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cell (3, 0)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cell (3, 1)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cell (3, 2)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cell (3, 3)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cell (3, 4)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-6e375419-876a-409c-bda6-42a3b7cf9e14')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-6e375419-876a-409c-bda6-42a3b7cf9e14 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-6e375419-876a-409c-bda6-42a3b7cf9e14');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "summary": "{\n  \"name\": \"display(df\",\n  \"rows\": 20,\n  \"fields\": [\n    {\n      \"column\": \"Up\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.0,\n        \"min\": 0.0,\n        \"max\": 0.0,\n        \"num_unique_values\": 1,\n        \"samples\": [\n          0.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"Down\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.0,\n        \"min\": 0.0,\n        \"max\": 0.0,\n        \"num_unique_values\": 1,\n        \"samples\": [\n          0.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"Right\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.0,\n        \"min\": 0.0,\n        \"max\": 0.0,\n        \"num_unique_values\": 1,\n        \"samples\": [\n          0.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"Left\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.0,\n        \"min\": 0.0,\n        \"max\": 0.0,\n        \"num_unique_values\": 1,\n        \"samples\": [\n          0.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xa82TetkGLa5"
      },
      "outputs": [],
      "source": [
        "class Solution:\n",
        "    def __init__(self, env, gamma=0.9, epsilon=0.1, maxIter=5000, maxTimesteps=500):\n",
        "        self.q_table = {}\n",
        "        self.env = env\n",
        "        self.gamma, self.epsilon, self.maxIter, self.maxTimesteps = gamma, epsilon, maxIter, maxTimesteps\n",
        "\n",
        "    def q_learning(self):\n",
        "        q_table = np.zeros((self.env.rows, self.env.columns, len(self.env.actions)))\n",
        "\n",
        "        for iter in range(self.maxIter):\n",
        "            state = self.env.reset()\n",
        "            visit_n = {}\n",
        "\n",
        "            for _ in range(self.maxTimesteps):\n",
        "                if np.random.uniform(0, 1) < self.epsilon:\n",
        "                    action = np.random.choice(self.env.actions)\n",
        "                else:\n",
        "                    action = self.env.actions[np.argmax(q_table[state])]\n",
        "                a = self.env.action_map[action]\n",
        "\n",
        "                state_dash, reward = self.env.step(a)\n",
        "\n",
        "                # Update rule for Q-learning\n",
        "                if state_dash != state:  # Ensure valid transitions\n",
        "                    best_next_action = np.argmax(q_table[state_dash])\n",
        "                    visit_n[(state, a)] = visit_n.get((state, a), 0) + 1\n",
        "                    alpha = 1 / visit_n[(state, a)]\n",
        "                    q_table[state][a] = (1 - alpha) * q_table[state][a] + alpha * (\n",
        "                        reward + self.gamma * q_table[state_dash][best_next_action]\n",
        "                    )\n",
        "                state = state_dash\n",
        "\n",
        "        return q_table"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SCK8GyVYI80s"
      },
      "source": [
        "# Finite Horizon Problem  \n",
        "In a **finite horizon** problem, the agent interacts with the environment for a finite number of time steps `T`.  \n",
        "\n",
        "### 🎯 Mathematical Objective:  \n",
        "$$\n",
        "J = \\mathbb{E} \\left[ \\sum_{t=0}^{T} \\gamma^t r_t \\right]\n",
        "$$\n",
        "\n",
        "where:  \n",
        "- $\\gamma = 1.0$ (no discounting)\n",
        "\n",
        "<!-- - **t** = Current time step  \n",
        "- **T** = Finite horizon (end of episode)  \n",
        "- **r_t** = Reward at time step t   -->\n",
        "\n",
        "### Key Characteristics:\n",
        "\n",
        "<!-- - The agent knows the task will end after **T** steps.   -->\n",
        "- Terminal states are critical since they define the end of the task.  \n",
        "- The agent tends to **act greedily** toward immediate high-reward states.  \n",
        "- The policy can change depending on the current time step (**non-stationary**).  "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nE3qG76mA184",
        "outputId": "6ad8ba47-dccd-4189-8685-96c0819e2e5d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 439
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Optimal Q function and optimal policy in the finite horizon case: \n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 1000x500 with 2 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxoAAAGVCAYAAAB5OYd2AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAfG1JREFUeJzt3XtYVNX6B/AvDLcZFBEBA0XxAkIiGagRpqF2MCXxhlon00zFDuYdj2klXpCjmWLpTxLT1NI6gppa2tVLpidMNBWR8K7cQu4IAwMz/P6A2TLMDAIOzCDfz/P4wKxZa3j3cmbWfvdae2+jioqKChAREREREemQsb4DICIiIiKiJw8TDSIiIiIi0jkmGkREREREpHNMNIiIiIiISOeYaBARERERkc4x0SAiIiIiIp1jokFERERERDrHRIOIiIiIiHSOiQYREREREemcSV0rGhkZNWYcRESkRUVFhb5DMFgcm4iI9KMuYxNnNIiIiIiISOfqPKOhlPh//wdbK6vGiKXZuXT1Kl6KiMDP33wDT19ffYdjUC5duoSXXnoJP//8Mzw9PfUdjsFgv2h35o8zGBUwCkuil8Ddw13f4ehdQW4BZgbM1HcYzQbHpoc4NmnH72DN2C/acWxSVd+xqd6Jhq2VFezatKlvsydS21atKn9aW8POzk7P0RiWtm3bCj/ZNw+xX7SzbmsNAGhl1QptbPgdQ/XDsekhjk3a8TtYM/aLdhybHg+XThERERERkc4x0SAiIiIiIp1jokFERERERDrHRIOIiIiIiHSOiQYREREREekcEw0iIiIiItI5JhpERERERKRzTDSIiIiIiEjnmGgQEREREZHOMdEgIiIiIiKdY6JBREREREQ6x0SDiIiIiIh0jokGERERERHpHBMNIiIiIiLSOSYaRERERESkc0w0iIiIiIhI55hoEBERERGRzjHRICIiIiIinWOiQUREREREOmei7wCIiIiIiEhdZqoYcb88hfhT9si4K0FetjnKy4xh3a4UNu1L8MzzWeg3KAMuvfL1HapGTDSIiIiIiAxIQa4Z9mx0xQ97O0Nerr4AKTNNgsw0CZIu2OC/m13x7AuZeOvfiejsWqiHaLVjokFEREREZCDuXm+FZdN9kJUuFspEJgr0eCYXdo5SmJopkJNpgaQLbVH8wBQAcOE3e8w/2w4L1l6A79B0fYWuhokGEREREZEBuHu9Fd59vT8e5JsBAExMFRg15QZGTbkBq7ZlKnXLZMY4dcQRn3/4NPJzzFEmE+HDed6Yu+YC/Eak6iN8NTwZnIiIiIhIz2Slxlg7z1tIMszF5Vix7XdMmp+klmQAgKmZAoNHpeDjb07CsfMDAIBCYYTNYZ5IvWXZpLFrw0SDiIiIiEjPYra44M41K+HxvDUX4NEv+5HtbOxLsWL77xBbViYjJcUm2PTBM40WZ30w0SAiIiIi0qNSqQhHdjsLj31eSoevf0ad29t3kOKfs5KFx1fOtUPyJWsdRtgwTDSIiIiIiPTot+8dUFi1ZAoARr55s96v4T/uDsSScuHx0a866yS2x9HkicYf167hnU8/Rc+ZM2E5bhw6vfUWxq9Zg+TUup20kvfgAYI3bYLdxImwHDcOg957D+dv3NBY91BcHLzmzoXF2LHo9NZbCNuzB+VyuS43R6euXLmCcePGoWvXrpBIJLC1tcXAgQNx+PDhOrXPy8tDcHAw7OzsYGlpiUGDBuH8+fMa6x46dAheXl6wsLBAp06dEBYWhvLyco11DdGqVatgZGQEDw+POtVPTU3F+PHjYW1tDSsrK4wcORI3b2r+EG/btg3u7u6wsLCAi4sLNm7cqMvQdebEiRMwMjLS+O/3339/ZPsnsU+qK5OVYcdHO/DmgDcR9EwQQseH4sLpC3Vqm/13NtbMXYPX+r6GCd4TEB4Sjox7mo8s/Rj7I0KGh2Cs51jMGDoD337xrS43g4iIWoDLcbbC77YOUvTsk1Pv1xBbytFvyMOx6vJZ21pqN40mv+rUmn37cPrqVYzr3x+ezs7IyM3Fpu++g9e8efh97Vp4dNaefSkUCgSsWIGLt29j4ejRsLWywuYjR+C3ZAniIyPh4ugo1D0aH49RERHw8/DAxuBgXL5zB+F79yIzLw9RISFNsan1dufOHRQWFmLy5MlwdHREcXEx9u3bh8DAQGzZsgXBwcFa2yoUCgQEBODixYtYuHAhbG1tsXnzZvj5+SE+Ph4uLi5C3aNHj2LUqFHw8/PDxo0bcfnyZYSHhyMzMxNRUVFNsamPJSUlBREREbC0rNuJTg8ePMCgQYOQn5+PJUuWwNTUFJGRkXjxxRfx559/ol27dkLdLVu24O2338bYsWMxf/58nDp1CrNnz0ZxcTEWLVrUWJv0WGbPno2+ffuqlHXv3r3WNk96nwDAhnc34MyPZxA4KRAOnR1w7MAxrJixAqt2rsLT3k9rbSctkuK9Se+h6EERgmYEwcTEBAd3HsSSN5Zgw4ENsGr7cP3s919/j83LNsPX3xcj3xyJxPhERK+KRmlJKcZOH9sUm0lERE+AxPM2wu+unrkNfp0enrk4ebgjACAzVYLsvy3Qrn3JY8fXUE2eaMwfORJ7FiyAmampUDZhwAD0mjULq2Nj8eWCBVrbxp45gzNJSYhZtAhB/fsDAMa/8AJc334bYXv2YE9oqFA3dPt2eDo748cVK2AiEgEArCQSRMTEYE5gINw6dmykLWy44cOHY/jw4Spl77zzDry9vbF+/fpaE43Y2FicOXMGMTExCAoKAgCMHz8erq6uCAsLw549e4S6oaGh8PT0xI8//ggTk8q3gJWVFSIiIjBnzhy4ubk1wtbpTmhoKHx8fCCXy5GVlfXI+ps3b8a1a9dw9uxZYYd82LBh8PDwwLp16xAREQEAkEqleO+99xAQEIDY2FgAwPTp06FQKLBy5UoEBwejbdu2jbdhDTRgwADh/7yunvQ+Sb6UjFNHTmHKwikYPXU0AGDwqMF4Z8Q72LF2Bz78+kOtbY98dQRpd9KwLmYdXHpVJujeA73xzoh38M3n32DS/EkAgNKSUnyx4Qv0ebEP3v3kXQDA0PFDoVAo8N+o/2Lo+KFo1aZVI28pERE9Ce6nPbxnRmeXht90r1ONtn+nivWaaDT50ilfd3eVJAMAXBwd0bNTJ1xNSam1bezp02hvbY0xzz8vlNm1aYPxL7yAg3FxKC2rPNs+8e5dJN67h+ChQ4UkAwBChg1DRUUFYk+f1uEWNS6RSAQnJyfk5eXVWi82Nhbt27fHmDFjhDI7OzuMHz8eBw8eRGlpKQAgMTERiYmJCA4OFpIMAAgJCansm6qdSUP166+/IjY2Fhs2bKhzm9jYWPTt21flqL+bmxuGDBmCvXv3CmXHjx9HdnY2QmrMeM2cORNFRUX47rvvHjv+xlJYWFivpW9Pep+c/uE0jEXGGDphqFBmZm6Gf4z9B5L+TML99Pta25754QxcerkISQYAdOzaEc/4PIPfvv9NKLscdxmFeYUY/k/VgwMB/wxASXEJ/jj5hw63iJ5UcX/9hfe++AKp2Y++skxLolAosHjxYhw+fBgVFRX6Dseg/Pzzz1ixYkWdDrS1JKWlpZg/fz6OHz/e7N4zxQ9MVO7+3cpK/VK2ddWqjWrbB3lmWmo2DYM4GbyiogJ/5+XB1sqq1noXbt6EV7duMDZWDbufiwuKS0uF8zwuVK0z71Nj+Yhju3boaGsrPG+oioqKkJWVhRs3biAyMhJHjx7FkCFDam1z4cIFeHl5qfdNv34oLi5GcnKyUA8A+vTpo1LP0dERHTt2FJ43RHK5HLNmzcK0adPQq1evOrVRKBS4dOmS2vYClX1z48YNFBZWZv/a+sbb2xvGxsYG2zdTpkyBlZUVLCwsMGjQIJw7d67W+i2hT25evYkOzh0gaSVRKXf1dAUA3Lp6S2M7hUKB23/dRncP9aVnLp4uyLibgeIHxcLfAKBWt1vPyu+om4mG/T1DhiHm9GlExMSgy7RpmLVlCxOOKpmZmVi9ejUCAwPRu3dvJhzV7Ny5E2FhYXBycsLixYuZcFRJSkpCZGQkBg8ejAEDBjSrhENapLrAyFzS8HNmLWq0LS7S7725DSLR2H3iBFKzszHhhRdqrZeemwsHDcs0HGwq17Wl5eQI9QBortu2rVDPUC1YsAB2dnbo3r07QkNDMXr0aGzatKnWNunp6XBwcFArV5alpaUJ9aqX16yrrGeIPv30U9y5cwcrV66sc5ucnByUlpbWuW9EIhHs7e1V6pmZmaFdu3YG1zdmZmYYO3YsPv74Yxw8eBDh4eG4fPkyBgwYUGsC8CT3iVLu/Vy0tVP//CvLcjI1fwc8yH+AMlmZxrY2djYqbXMyc2AsMoZ1O2uVeqZmpmht3Vrr3yCqyUQkQplcjqijR5lwaJCQkIDAwEA8++yz+Pbbb5vNzmNjMjY2RklJCT788EMmHBr8/vvvzSrhEFuqJgelxQ1PDkpqtJVY6vdCP/pNcwAkpaRg5pYteN7NDZMHD661rlQmg3mNZVcAYFFVJpXJKn9WLRPSVrdAKn3csBvV3LlzERQUhLS0NOzduxdyuRyyqm3TRiqVwtzcXK3cwsJCeL76T211CwoKHjf8RpGdnY2lS5figw8+gJ2dXZ3bPWp7q9eRSqUwM9M8xWhhYSHUMxS+vr7w9fUVHgcGBiIoKAienp5YvHgxvv/+e43tnuQ+UZKVyGBqpv75NzOv3BblUsKaSksqyzW1NTWvLJOVyoSfJqaav0JNzU2FetQyLNm1C9t++qne7QqrfYbkCgXkAKKOHsWW779H8Msv471x44SDac1RcnIyhg4diuLi4nq1k1e7QqRCoQAAXL58GSNGjMAzzzyDVatWISAgQKexNrWpU6fi22/rf5W6/Px84XeFQiEkHBs2bMDcuXOxcOFC2DTj90xcXByCgoIeud9TU1nZwyVDyvePMuHo378/Vq1ahRdffFGnseqKpFU5RCYKYflUYb76GFRXD2q0bWWt37FIr4lGRm4uAlasQBuJBLGLFkFU7XwKTcRmZsJ5GNWVVJWJq3aIxFU7UNrqirXsOBkKNzc34YTsSZMmwd/fHyNGjEBcXByMjIw0thGLxRp3nkpKSoTnq//UVlf5vKF5//33YWNjg1mzZtWr3aO2t3odsVis9YvNkPumuu7du2PkyJHYv38/5HK5xs9US+gTMwszlMnUP//KnX9NSRYAmFtUlmtqW1ZaWaZMVszMzVBepvlIUVlpmVCPWoYzSUnIrLYD+DiUCcf/ffcd8ouK8MX8+Tp5XX24ceMGbt++rZPXUiYcFy9exIgRI5CWloannnpKJ6+tD7/++isyMzN18lrKhGP16tUoLy/H2rVrdfK6+nD16lWkPOKc3bpSJhynT5/Gyy+/jOzsbEgkkke00g87Ryky7lZeTfPutdYNfp2abdt30O8BQb0lGvlFRRi2bBnyiopw6j//gWO1y2lq49C2rbAsqrr0qqVQjlUZvHLJVHpuLpxqHP1Oz81Fv2qXem0OgoKCMGPGDCQnJ6NHjx4a6zg4OAjLoqpTljlWXfpXuTQmPT0dTk5OanX79euny9B14tq1a4iOjsaGDRtUluqUlJSgrKwMt2/fhpWVlcYjODY2NjA3N69z38jlcmRmZqosFZLJZMjOzhbqGTonJyfIZDIUFRXBSsN5Ty2hT9ratUXO3+pLl3LvV35/2NhrPtrXqk0rmJqZCvWqy7mfo9LWxt4GCrkCedl5KsunymRlKMwr1Po36Mnk6+aGq/fu1btdoVSKMrlc5R5PImNjGBsZYcbLL+O98eN1GWaT69atG5ydnRs0o5FdY/mYsbExFAqFMKPRnJMMABg4cGCDVhHk5+ejrKxMSLyAyr4xNzcXZjSaM3d3d3Ts2LFBMxq5NfYRRSIR5HK5MKNhqEkGALg/myMkGsmXGn41x7+qtbXvUKzXK04Beko0SmQyjFi5Eslpafh55Uo83alTndr17tIFpxIToVAoVE56jktOhsTcHK4dOlTW69oVAHDu+nX0c3UV6qVlZyMlKwvB/v463JrGp1yekl/L0bLevXvj1KlT6n0TFweJRALXqn7o3bs3AODcuXMqSUVaWhpSUlJqvYSuvqSmpkKhUGD27NmYPXu22vNdunTBnDlzNF6JytjYGL169dJ4gnRcXBy6du2K1q0rs//qfVP9MsPnzp2DQqEQnjd0N2/ehIWFBVq10nxp1ZbQJ13duuJy3GUUPyhWOSH8r4t/AQC6uHfR2M7Y2BidXTvjesJ1teeSLybjKaenhNfr4lb5GtcTrqPPiw9Plr+ecB0KhQJd3bvqbHvI8EVMmoSISZPq3S50+3Z8XHVT1uoJxrtBQehQhwNwhs7V1RW3bmm++EJtMjIyhANjygSjV69eCA8PR0BAgNbZ/eZk27ZtDWr3xhtvCJesNzY2hpmZGebNm4f58+fD1lb/N2h7XM899xzuNSBpv3jxojAmKRMMHx8fhIeHw8/PT7dBNgLP57Jw/GDlAeCsDDES/rCBR9/6nesnLRLh7C8PE/Bez+n/vJ0mPxlcLpdjwocf4n9//YWYRYvwvJZ7NqTn5CApJQVl1S7ZGdS/P/7Oy8P+//1PKMsqKEDM6dMY0a+fcE5Gz06d4NaxI6J/+EFlnWfU0aMwMjIS7sFhaDRNoZaVlWHXrl0Qi8V4+unKm4ylp6cjKSlJZT1iUFAQ/v77b+zfv18oy8rKQkxMDEaMGCEsFenZsyfc3NwQHR2t2jdRUZV9U8/7MTQFDw8PHDhwQO1fz5490alTJxw4cABTp04FANy9exdJSUkq7YOCgvDHH3+o7Fj/9ddfOHbsGMaNGyeUDR48GDY2Nmo3LYyKioJEIjG49cD376tfovXixYs4dOgQ/P39hYSzJfWJku9QXyjkCvzw3x+EsjJZGX458Atcn3GFnUPlTOf9tPtIuak6Rd9/aH9cu3wN1y5fE8pSbqbgUtwl9H/54XeHp48nWrdpjaNfHVVpf/TrozAXm6skH0S1KZfLYSoS4V/DhuHWZ59h44wZT0SSoSu9evXC4cOHceHCBbzyyitPRJLxuBQKBcRiMRYtWoR79+4hIiLiiUgydMXHxwfHjx/Hb7/91iySDADo/3I6WrV5OItzcEe3er/GjzGdIa12MviwV+/oJLbH0eQzGgu2b8ehs2cxol8/5BQW4svjx1WenzhoEABg8a5d2HnsGG5t3Qrn9u0BAEG+vvDp0QNTPvkEiffuCXcGlysUWP7aayqvs3bKFASGh8M/LAyvDhiAhLt3sem77zDtH/+Ae40lQ4ZixowZKCgowMCBA9GhQwdkZGRg9+7dSEpKwrp164Qj1IsXL8bOnTtx69YtODs7A6jccfTx8cGUKVOQmJgo3BlcLpdj+fLlKn9n7dq1CAwMhL+/P1599VUkJCRg06ZNmDZtGtzd3Zt6sx/J1tYWo0aNUitXzmBUf27SpEk4efKkyhUmQkJCsHXrVgQEBCA0NBSmpqZYv3492rdvjwXVbhApFouxcuVKzJw5E+PGjcPQoUNx6tQpfPnll1i1apXBnVw3YcIEiMVi+Pr6wt7eHomJiYiOjoZEIsHq1auFei2pT5R6PNMD/V/uj12Ru5CXkweHTg449s0xZKZmYlb4w/N8IhdFIuGPBBxKOiSUDXttGH6I+QEr3l6B0W+NhshEhIM7DsK6nTVGTRkl1DO3MMfrc17Hpys+xeo5q+H1gheunLuCE4dOYOLciWht3fA1ttRyjOvfHxZmZvjXsGFMLqqxt7fHu+++i/79+z8xMxi6MnnyZLi4uCAkJITJRTVubm6YN28eAgMDm01yUZ2FRI7hr93G3k8rV6DE/fIU/vfTU3j+Hxl1ap+ZKsaeTx4ur+/ZJxuunnmNEWq9NHmi8WfVFOrhs2dx+OxZteeViYYmIpEIR8LCsPDzz/HJ4cOQymTo6+KCHXPmoEeNO32/0rcv9i9ejOVffYVZ0dGwa9MGS4KCsPTVV3W7QTo0YcIEbNu2DVFRUcjOzkbr1q3h7e2NNWvWIDAwsNa2IpEIR44cwcKFC/HJJ59AKpWib9++2LFjh9p5Ha+88gr279+P5cuXY9asWbCzs8OSJUuwdOnSxtw8vWndujVOnDiBefPmITw8HAqFAn5+foiMjFS7glVISAhMTU2xbt06HDp0CE5OToiMjMScOXP0FL12o0aNwu7du7F+/XoUFBTAzs4OY8aMQVhYGLp3V78PRHVPap9UN2/NPOz+eDdOHDqBB/kP4NzDGR9EfQCPvh61tpO0kiBiVwQ++89n2Bu1FxWKCnj088DUxVPRxqaNSt3h/xwOkYkI33z+Dc4eOwtbB1tMXTwVgZNq/7wSKT3Xowee03LuXUtmbGyM//znP/oOwyC99NJLeOmll/QdhsExNzfH+vXr9R3GYxk34zr+97MD7l2vPFC1/t/PIiw67pFLqHLvmyNsqo8wm2EhKcc7Ky82erx1YVRRx4sLK48mZH7xBezatHlE7ZbhfEICvJcsQfyJE/Ay0Eum6cv58+fh7e2N+Ph4eHl56Tscg8F+0e7k/07Cz9cPEV9HwKN37clAS5Cfk483fN8AAIO/Brw+cWxSx7FJO34Ha8Z+0a6px6Y7ya3x7uv9UVRYeTqAiakCo9+6gZFv3oBVW9UrIpbJjHDqSAd8vvZp5GdXLpE3Nq7A3DUX4DcitVHiq+/YpPf7aBAREREREdDZtRCrd5/G8uDnkJUhRnmZMWK2uGD/tm7o0TsX9o5SmJgqkHvfHEkXbISEBABMzeSY9+EFvPCy+lUl9YWJBhERERGRgejsWoj1+37Fno098FNsJ8jLjSEvN0biuXZI1NLm2f6ZmPLvRDj3KGzSWB+FiQYRERERkQGxbidDyLLLGDvtOn7/5SmcP2WP9LuWyM82g7zcGFY2Mti2l8LTJwvPDcmASy/d3DBU15hoEBEREREZoPYdpRg5+RZGTq7//WgMQZPfR4OIiIiIiJ58TDSIiIiIiEjnmGgQEREREZHOMdEgIiIiIiKdY6JBREREREQ6x0SDiIiIiIh0jokGERERERHpHBMNIiIiIiLSOSYaRERERESkc0w0iIiIiIhI55hoEBERERGRzjHRICIiIiIinWOiQUREREREOsdEg4iIiIiIdI6JBhERERER6RwTDSIiIiIi0jkmGkREREREpHNMNIiIiIiISOeYaBARERERkc4x0SAiIiIiIp1jokFERERERDrHRIOIiIiIiHSOiQYREREREekcEw0iIiIiItI5JhpERERERKRzTDSIiIiIiEjnmGgQEREREZHOmdS3waWrV9G2VavGiKXZuXr7duXPP//UaxyG6GpycuXPq1f1HIlhUfYH+0Vd8l+V75m//voLJeUleo5G/4oKivQdQrPCsekhjk3acWzSjGOTdhybVNV3bDKqqKioqFNFI6MGBURERI+njl/TLRLHJiIi/ajL2FTvGY2fv/kGba2tGxLPE+fqn39i4ty5+DI4GO7OzvoOx6BcvX0bE6Oj8WV0NNy9vfUdjsG4evUqJk6ciC+//BLu7u76DsegxF+KR/CUYExeMRmdu3XWdzh6V1RQhHX/WqfvMJoNjk0PcWzSjmOTZhybtOPYpKq+Y1O9Ew1PX1/Y2dnVt9kTzd3ZGV4eHvoOwyC5u7rCy8tL32EYHHd3d/ZLDYWlhQCAHq494NGbn6f8nHx9h9CscGxSx7FJO45NmnFsUsexSVV9xyaeDE5ERERERDrHRIOIiIiIiHSOiQYREREREekcEw0iIiIiItI5JhpERERERKRzTDSIiIiIiEjnmGgQEREREZHOMdEgIiIiIiKdY6JBREREREQ6x0SDiIiIiIh0jokGERERERHpHBMNIiIiIiLSOSYaRERERESkc0w0iIiIiIhI55hoEBERERGRzjHRICIiIiIinWOiQUREREREOsdEg4iIiIiIdI6JBhERERER6RwTDSIiIiIi0jkTfQdARERERETqMlPFiPvlKcSfskfGXQnyss1RXmYM63alsGlfgmeez0K/QRlw6ZWv71A1YqJBRERERGRACnLNsGejK37Y2xnycvUFSJlpEmSmSZB0wQb/3eyKZ1/IxFv/TkRn10I9RKsdEw0iIiIiIgNx93orLJvug6x0sVAmMlGgxzO5sHOUwtRMgZxMCyRdaIviB6YAgAu/2WP+2XZYsPYCfIem6yt0NUw0iIiIiIgMwN3rrfDu6/3xIN8MAGBiqsCoKTcwasoNWLUtU6lbJjPGqSOO+PzDp5GfY44ymQgfzvPG3DUX4DciVR/hq+HJ4EREREREeiYrNcbaed5CkmEuLseKbb9j0vwktSQDAEzNFBg8KgUff3MSjp0fAAAUCiNsDvNE6i3LJo1dGyYaRERERER6FrPFBXeuWQmP5625AI9+2Y9sZ2NfihXbf4fYsjIZKSk2waYPnmm0OOuDiQYRERERkR6VSkU4sttZeOzzUjp8/TPq3N6+gxT/nJUsPL5yrh2SL1nrMMKGafJE48qVKxg3bhy6du0KiUQCW1tbDBw4EIcPH65T+7y8PAQHB8POzg6WlpYYNGgQzp8/r7HuoUOH4OXlBQsLC3Tq1AlhYWEoLy/X5ebo1JW7dzFu9Wp0nT4dkqAg2L7+Oga++y4Onz1bp/Z5Dx4geNMm2E2cCMtx4zDovfdw/sYNjXUPxcXBa+5cWIwdi05vvYWwPXtQLpfrcnMa1apVq2BkZAQPD4861U9NTcX48eNhbW0NKysrjBw5Ejdv3tRYd9u2bXB3d4eFhQVcXFywceNGXYauMydOnICRkZHGf7///vsj2z+JfVJdmawMOz7agTcHvImgZ4IQOj4UF05fqFPb7L+zsWbuGrzW9zVM8J6A8JBwZNzT/IX/Y+yPCBkegrGeYzFj6Ax8+8W3utwMIiJqAX773gGFVUumAGDkm5rH49r4j7sDseThfu7RrzrrJLbH0eQng9+5cweFhYWYPHkyHB0dUVxcjH379iEwMBBbtmxBcHCw1rYKhQIBAQG4ePEiFi5cCFtbW2zevBl+fn6Ij4+Hi4uLUPfo0aMYNWoU/Pz8sHHjRly+fBnh4eHIzMxEVFRUU2xqvd3JzEShVIrJgwfD0cYGxaWl2HfmDALDw7ElJATBL7+sta1CoUDAihW4ePs2Fo4eDVsrK2w+cgR+S5YgPjISLo6OQt2j8fEYFREBPw8PbAwOxuU7dxC+dy8y8/IQFRLSFJv6WFJSUhAREQFLy7qtP3zw4AEGDRqE/Px8LFmyBKampoiMjMSLL76IP//8E+3atRPqbtmyBW+//TbGjh2L+fPn49SpU5g9ezaKi4uxaNGixtqkxzJ79mz07dtXpax79+61tnnS+wQANry7AWd+PIPASYFw6OyAYweOYcWMFVi1cxWe9n5aaztpkRTvTXoPRQ+KEDQjCCYmJji48yCWvLEEGw5sgFXbh9Pa33/9PTYv2wxff1+MfHMkEuMTEb0qGqUlpRg7fWxTbCYRET0BLsfZCr/bOkjRs09OvV9DbClHvyEZOHm4Y+VrnrV9RIvG1+SJxvDhwzF8+HCVsnfeeQfe3t5Yv359rYlGbGwszpw5g5iYGAQFBQEAxo8fD1dXV4SFhWHPnj1C3dDQUHh6euLHH3+EiUnlZlpZWSEiIgJz5syBm5tbI2zd4xnepw+G9+mjUvZOQAC858/H+oMHa000Ys+cwZmkJMQsWoSg/v0BAONfeAGub7+NsD17sCc0VKgbun07PJ2d8eOKFTARiQAAVhIJImJiMCcwEG4dOzbC1ulOaGgofHx8IJfLkZWV9cj6mzdvxrVr13D27Flhh3zYsGHw8PDAunXrEBERAQCQSqV47733EBAQgNjYWADA9OnToVAosHLlSgQHB6Nt27aNt2ENNGDAAOHzUFdPep8kX0rGqSOnMGXhFIyeOhoAMHjUYLwz4h3sWLsDH379oda2R746grQ7aVgXsw4uvSoPXngP9MY7I97BN59/g0nzJwEASktK8cWGL9DnxT5495N3AQBDxw+FQqHAf6P+i6Hjh6JVm1aNvKVERPQkSDxvI/zu6pnb4Nfp4ZkrJBqZqRJk/22Bdu1LHju+hjKIczREIhGcnJyQl5dXa73Y2Fi0b98eY8aMEcrs7Owwfvx4HDx4EKWlpQCAxMREJCYmIjg4WEgyACAkJAQVFRXCDlNzIBKJ4GRri7yiolrrxZ4+jfbW1hjz/PNCmV2bNhj/wgs4GBeH0rLKE4QS795F4r17CB46VEgyACBk2LDKvjl9unE2REd+/fVXxMbGYsOGDXVuExsbi759+6oc9Xdzc8OQIUOwd+9eoez48ePIzs5GSI1ZnZkzZ6KoqAjffffdY8ffWAoLC+u1LPBJ75PTP5yGscgYQycMFcrMzM3wj7H/QNKfSbiffl9r2zM/nIFLLxchyQCAjl074hmfZ/Db978JZZfjLqMwrxDD/6l64CTgnwEoKS7BHyf/0OEWERHRk+x+2sN7ZnR2afhN9zrVaPt3qlhLzaaht0SjqKgIWVlZuHHjBiIjI3H06FEMGTKk1jYXLlyAl5cXjI1Vw+7Xrx+Ki4uRnJws1AOAPjVmBxwdHdGxY0fheUNVVFKCrIIC3EhPR+TBgzgaH48hz9R+9YALN2/Cq1s39b5xcUFxaSmSU1OFegDQp8bSGsd27dDR1lZ43hDJ5XLMmjUL06ZNQ69everURqFQ4NKlS2rvBaDyfXPjxg0UFlZ+KLW9b7y9vWFsbGyw75spU6bAysoKFhYWGDRoEM6dO1dr/ZbQJzev3kQH5w6QtJKolLt6ugIAbl29pbGdQqHA7b9uo7uH+tIzF08XZNzNQPGDYuFvAFCr261n5efwZqLhfpaIDJ1CocDinTtx+OxZVFRU6Dscg/Lzzz9jxYoVdZrRb0lKS0sxf/58HD9+vNm9Z4ofmKjc/buVlfqlbOuqVRvVtg/yzLTUbBp6SzQWLFgAOzs7dO/eHaGhoRg9ejQ2bdpUa5v09HQ4ODiolSvL0tLShHrVy2vWVdYzVAu2b4fdxInoPmMGQj//HKN9fLBpxoxa26Tn5sJBwxIWB5vKqbi0nByhHgDNddu2FeoZok8//RR37tzBypUr69wmJycHpaWldX7fiEQi2Nvbq9QzMzNDu3btDO59Y2ZmhrFjx+Ljjz/GwYMHER4ejsuXL2PAgAG1JgBPcp8o5d7PRVs79fe4siwnU/P7/EH+A5TJyjS2tbGzUWmbk5kDY5ExrNtZq9QzNTNFa+vWWv8GET1aZn4+Vu/bh8DwcPSeM4cJRzU7d+5EWFgYnJycsHjxYiYcVZKSkhAZGYnBgwdjwIABzSrhkBapnslgLmn4hYssarQtLtLvvbn1lmjMnTsXP/30E3bu3Ilhw4ZBLpdDJpPV2kYqlcLc3Fyt3MLCQni++k9tdZXPG6q5gYH4acUK7Jw7F8O8vCBXKCArqz27lcpkMDc1VSu3qCqTVvWttGp5mba60kf8H+hLdnY2li5dig8++AB2dnZ1bveo90L1OlKpFGZmmjN/Q3zf+Pr6IjY2Fm+99RYCAwPx7rvv4vfff4eRkREWL16std2T3CdKshIZTM3U3+Nm5pXbolxmWVNpSWW5pram5pVlslKZ8NPEVPMXuKm5qVCPiB5Pwt27CAwPx7Nz5+LbP/5oNjuPjcnY2BglJSX48MMPmXBo8PvvvzerhENsqZoclBY3PDkoqdFWYqnfq63qLc1xc3MTTsieNGkS/P39MWLECMTFxcHIyEhjG7FYrHEHoaSkRHi++k9tdZXPGyq3jh2FE7InDR4M/6VLMSI8HHEffaS9b8zMhPMwqiupKhNX7SyKq3YutdUVa9mp1Lf3338fNjY2mDVrVr3aPeq9UL2OWCzWmuw2h/cNUHm1qZEjR2L//v2Qy+UQVTsPR6kl9ImZhRnKZOrvceXOv6YkCwDMLSrLNbUtK60sUyYrZuZmKC/T/AVeVlom1KMnn1wuR0JCAkQiEXr27Kn1e7olSk5NxdCwMBRrSe61kSsUwu+Kqt8v37mDEStX4pkuXbBq4kQE1LjaXnMzdepUfPtt/S+HnZ+fL/yuUCiEhGPDhg2YO3cuFi5cCBsbm1pewbDFxcUhKCjokQefayqrtl8jr7pcvzLh6N+/P1atWoUXX3xRp7HqiqRVOUQmCmH5VGG++sGuunpQo20ra/0e9NLvfEo1QUFBmDFjBpKTk9GjRw+NdRwcHIRlUdUpyxyrLuGqXP6Rnp4OJycntbr9+vXTZeiNLsjXFzM2b0Zyaip6aLkilEPbtsKyqOrSq5ZCOVZ96SiXTKXn5sKpxsxAem4u+lW7RLChuHbtGqKjo7FhwwaVpTolJSUoKyvD7du3YWVlpfGL1cbGBubm5nV+38jlcmRmZqosFZLJZMjOzhbqGTonJyfIZDIUFRXByspK7fmW0Cdt7doi52/1pUu59ys/Izb2mgfhVm1awdTMVKhXXc79HJW2NvY2UMgVyMvOU1k+VSYrQ2Feoda/QU+enTt3YurUqQCAb7/9FgEBAXqOyHDcyMjA7cxMnbyWMuG4eOsWRoSHI23HDjxlgFe9q6tff/0VmTrsm5KSEqxevRrl5eVYu3atTl5XH65evYqUlBSdvJYy4Th9+jRefvllZGdnQyKRPKKVftg5SpFxt/Ky/XevtW7w69Rs276DflceGEyioVyCUT1Tr6l37944deoUFAqFyknPcXFxkEgkcHV1FeoBwLlz51SSirS0NKSkpNR6CV1DpFzOlF9crLVO7y5dcCoxUb1vkpMhMTeHa4cOlfW6dgUAnLt+Hf2q+gsA0rKzkZKVhWB//8bYhMeSmpoKhUKB2bNnY/bs2WrPd+nSBXPmzNF4JSpjY2P06tVL4wnScXFx6Nq1K1q3rvxQVn/fVL8E87lz56BQKITnDd3NmzdhYWGBVq00X1q1JfRJV7euuBx3GcUPilVOCP/r4l8AgC7uXTS2MzY2RmfXzriecF3tueSLyXjK6Snh9bq4Vb7G9YTr6PPiw5Plrydch0KhQFf3rjrbHjJsyosn1PydgG5PPQVne/sGzWhk1+hLY2NjKBQKYUajOScZADBw4EAUFBTUu11+fj7KysqExAuo7Btzc3NhRqM5c3d3R8eOHRs0o5Fb44CrSCSCXC4XZjQMNckAAPdnc4REI/lSw9/bf1Vra9+hWK+XtgX0kGjUPDIKVL45du3aBbFYjKefrryRVnp6OvLz89GtWzeYVp1PEBQUhNjYWOzfv1+4b0BWVhZiYmIwYsQIYTlEz5494ebmhujoaMyYMUNYPhIVFQUjI6N633OgqWTm5cHe2lqlrKy8HLuOH4fYzAxPV83OpOfkIL+4GN2eegqmVZfvDerfH7FnzmD///4n3Ecjq6AAMadPY0S/fsI5GT07dYJbx46I/uEHzBg69GHfHD1a2TdVbQ2Jh4cHDhw4oFb+/vvvo7CwEB9//DG6desGALh79y6Ki4tV7pMSFBSEd999F+fOnROunvTXX3/h2LFjCK12f5HBgwfDxsYGUVFRKjvVUVFRkEgkBneU8v79+2rnq1y8eBGHDh3CsGHDhISzJfWJku9QXxzYfgA//PcH4T4aZbIy/HLgF7g+4wo7h8p+u592H6UlpejY9eFMYf+h/bFz3U5cu3xNuMRtys0UXIq7hNFvjRbqefp4onWb1jj61VGVROPo10dhLjZXKSNqqVw7dMCtzz6rd7uM3Fw4TJ4M4GGC0atzZ4RPnIiAPn2eiOVp27Zta1C7N954Q7hvmLGxMczMzDBv3jzMnz8ftrb6v0Hb43ruuedw7969ere7ePGicPBLmWD4+PggPDwcfn5+ug2yEXg+l4XjByv387IyxEj4wwYefet3URFpkQhnf3lKeNzrOf2ft9PkicaMGTNQUFCAgQMHokOHDsjIyMDu3buRlJSEdevWCUdhFy9ejJ07d+LWrVtwdnYGULlz5OPjgylTpiAxMVG4M7hcLsfy5ctV/s7atWsRGBgIf39/vPrqq0hISMCmTZswbdo0uLu7N/Vm18mMzZtRUFyMgT17okO7dsjIzcXukyeRlJKCdW+9hVZV6+EX79qFnceO4dbWrXBu3x5A5fIqnx49MOWTT5B4755wZ3C5QoHlr72m8nfWTpmCwPBw+IeF4dUBA5Bw9y42ffcdpv3jH3CvsdTMENja2mLUqFFq5coZjOrPTZo0CSdPnlQ58SskJARbt25FQEAAQkNDYWpqivXr16N9+/ZYsGCBUE8sFmPlypWYOXMmxo0bh6FDh+LUqVP48ssvsWrVKoNb8zphwgSIxWL4+vrC3t4eiYmJiI6OhkQiwerVq4V6LalPlHo80wP9X+6PXZG7kJeTB4dODjj2zTFkpmZiVvjD83wiF0Ui4Y8EHEo6JJQNe20Yfoj5ASveXoHRb42GyESEgzsOwrqdNUZNGSXUM7cwx+tzXsenKz7F6jmr4fWCF66cu4ITh05g4tyJaG3d8KlvInroSUswdEGhUEAsFmPu3LlPTIKhS80pwVDq/3I6tq3piQf5lef3HdzRrd6Jxo8xnSGtdjL4sFfv6DTGhmjyRGPChAnYtm0boqKikJ2djdatW8Pb2xtr1qxBYGBgrW1FIhGOHDmChQsX4pNPPoFUKkXfvn2xY8cOtfM6XnnlFezfvx/Lly/HrFmzYGdnhyVLlmDp0qWNuXmPZcILL2DbTz8h6uhRZBcWorVYDO9u3bBm8mQEPvdcrW1FIhGOhIVh4eef45PDhyGVydDXxQU75sxRO6/jlb59sX/xYiz/6ivMio6GXZs2WBIUhKWvvtqYm6c3rVu3xokTJzBv3jyEh4dDoVDAz88PkZGRajMCISEhMDU1xbp163Do0CE4OTkhMjISc+bM0VP02o0aNQq7d+/G+vXrUVBQADs7O4wZMwZhYWHo3l39PhDVPal9Ut28NfOw++PdOHHoBB7kP4BzD2d8EPUBPPp61NpO0kqCiF0R+Ow/n2Fv1F5UKCrg0c8DUxdPRRubNip1h/9zOEQmInzz+Tc4e+wsbB1sMXXxVAROqv27jIhqZ9+mDd4dOxb9n36aCUYNkydPhouLC0JCQphgVOPm5oZ58+YhMDCwWSUYShYSOYa/dht7P61c1h73y1P4309P4fl/ZNSpfWaqGHs+ebgv3LNPNlw98xoj1HoxqqjjNb+UH/LMzMx6XV70SXb+5El4+/khPiICXh6177y0NOcTEuC9ZAniT5yAl4Fe5UEfzp8/D29vb8THx8PLy0vf4RiUk/87CT9fP0R8HQGP3vw85efk4w3fNwDA4C/NqE+GMDZ9/PHHmDt3LgDgq6++wqt6PmjDsUk7jk2acWzSrinHplKpCPPHDcC965Uz4ubicoRFxz1yZiP3vjmWvOGL1NuVq4IsJOWI3PcrOnQp0nmM9R2b9HYfDSIiIiIiqmQuluPf6+Nh2bryUr2lUhMsfet5fBHphoJc9UvelsmMcOybjpg96kUhyTA2rkDI8kuNkmQ0hMFcdYqIiIiIqCXr7FqI1btPY3nwc8jKEKO8zBgxW1ywf1s39OidC3tHKUxMFci9b46kCzYoKnyYgJiayTHvwwt44WX1y9frCxMNIiIiIiID0dm1EOv3/Yo9G3vgp9hOkJcbQ15ujMRz7ZCopc2z/TMx5d+JcO5hWJfYZqJBRERERGRArNvJELLsMsZOu47ff3kK50/ZI/2uJfKzzSAvN4aVjQy27aXw9MnCc0My4NJL+33o9ImJBhERERGRAWrfUYqRk29h5ORb+g6lQXgyOBERERER6RwTDSIiIiIi0jkmGkREREREpHNMNIiIiIiISOd4MjgREVEDyGQylJeXQyaTCWWlpaUoLi6GqakpTE3Vb7BFRNSScEaDiIionjIyMmBtbQ1LS0v8+9//hpGREQDgzTffhKWlJezs7FQSECKiloiJBhERUT3Z2trC3t5eeFxRUSH8bmRkBBcXF85oEFGLx0SDiIionkxMTLBs2TKNz1VUVCA8PFyY5SAiaqmYaBARETXAxIkT0blzZ5WEQiQSoU+fPvD399djZEREhoGJBhERUQMoZzWqL5uSy+WczSAiqsJEg4iIqIGqz2qIRCJ4e3tzNoOIqAoTDSIiogaqPqshl8uxatUqzmYQEVVhokFERPQYJk6cCGdnZ/Tt25ezGURE1fCGfURERI/BxMQEcXFxMDc352wGEVE1TDSIiIgeU/V7ahARUSUunSIiIiIiIp1jokFERERERDrHRIOIiIiIiHSOiQYREREREekcEw0iIiIiItI5JhpERERERKRzTDSIiIiIiEjnmGgQEREREZHOMdEgIiIiIiKdY6JBREREREQ6x0SDiIiIiIh0jokGERERERHpnEl9G1y6dAlt27ZtjFianavJyZU/b9/WbyAGSNknV//8U69xGBrhPXP1qp4jMTzJf1X2ze3rt/UbiIF4UPBA3yE0KxybHuLYpB3HJs04NmnHsUlVfccmo4qKioo6VTQyalBARET0eOr4Nd0icWwiItKPuoxN9Z7R+Pnnn3nUqMrVq1cxceJEfBkdDXdXV32HY1Cu/vknJs6diy+Dg+Hu7KzvcAzG1du3MTE6uvI94+2t73AMivLzFP15NFx78POUl5uHUQGj9B1Gs8Gx6SGOTdpxbNKMY5N2HJtU1Xdsqnei4enpCTs7u/o2e6K5e3vDy8tL32EYJHdnZ3h5eOg7DIPj7urK94wW3p78PAHA/fv39R1Cs8KxSR3HJu04NmnGsUk7jk2V6js28WRwIiIiIiLSOSYaRERERESkc0w0iIiIiIhI55hoEBERERGRzjHRICIiIiIinWOiQUREREREOsdEg4iIiIiIdI6JBhERERER6RwTDSIiIiIi0jkmGkREREREpHNMNIiIiIiISOeYaBARERE1sdTsbBSXluo7DIMjl8tx8+ZNfYdhkO7cuYOysjJ9h1EvTDSIiIiImtADqRSes2bh1Q8/1HcoBmf58uVwdXVFUlKSvkMxKBkZGXBzc8PMmTP1HUq9MNEgIiIiakL/d+QIch48wOE//sCFGzf0HY7ByMnJwbp16yCXy7FixQp9h2NQ1qxZg5KSEmzfvh137tzRdzh1xkSDiIiIqIk8kEqxOjYWACAyNkbYnj16jshwREZGoqSkBADw9ddfc1ajSkZGBjZv3iw8XrVqlR6jqR8mGkRERERN5P+OHEF+cTEAQK5Q4PAff+A8ZzWQk5OD9evXQ6FQAABEIhGWL1+u56gMw5o1ayCXywFUnsPSnGY1mGgQERERNQHlbEZFRYVQJjI2xjLOamDDhg3CbAYAlJeX47///W+Ln9VQzmYoEw2l5jKrwUSDiIiIqAlUn81QUs5qtORzNWrOZiiJRKIWf65G9dkMpeY0q8FEg4iIiKiRaZrNUDICWvS5Ghs2bEBRUZFaeXl5eYs+V0PbbAZQmWw0h1kNJhpEREREjezu/fsoqrY0yKjacxUALjWDo9ON5eLFi1qfq6iowJUrV5owGsORnJyM8vJyrc/X1m+GgokGERERUSN7ulMnFO7di6KYGISOHg3lvMaRsDAUxcTg+pYteo1Pnw4cOICioiLcu3dPKPP390dRURGKi4sxduxYPUanPwMHDkRRURGKiorwxhtvCOXx8fEoKirCmTNn9Bhd3ZjoOwAiIiKilsDc1BQAYCoSCWUWpqaQmJvrKySDYGxsDIlEonIyuEgkgkQi0WNUhsHCwgIAYGLycJddLBY3m77hjAYREREREekcEw0iIiIiItI5JhpERERERKRzTDSIiIiIiEjnmGgQEREREZHOMdEgIiIiIiKdM4hEY9WqVTAyMoKHh0ed6qempmL8+PGwtraGlZUVRo4ciZs3b2qsu23bNri7u8PCwgIuLi7YuHGjLkPXqStXrmDcuHHo2rUrJBIJbG1tMXDgQBw+fLhO7fPy8hAcHAw7OztYWlpi0KBBOH/+vMa6hw4dgpeXFywsLNCpUyeEhYXVelMYfbpy9y7GrV6NrtOnQxIUBNvXX8fAd9/F4bNn69Q+78EDBG/aBLuJE2E5bhwGvfcezt+4obHuobg4eM2dC4uxY9HprbcQtmcPyjXckdNQ8bNU6cSJEzAyMtL47/fff39k+ye1X4iIiJqS3u+jkZKSgoiICFhaWtap/oMHDzBo0CDk5+djyZIlMDU1RWRkJF588UX8+eefaNeunVB3y5YtePvttzF27FjMnz8fp06dwuzZs1FcXIxFixY11iY12J07d1BYWIjJkyfD0dERxcXF2LdvHwIDA7FlyxYEBwdrbatQKBAQEICLFy9i4cKFsLW1xebNm+Hn54f4+Hi4uLgIdY8ePYpRo0bBz88PGzduxOXLlxEeHo7MzExERUU1xabWy53MTBRKpZg8eDAcbWxQXFqKfWfOIDA8HFtCQhD88sta2yoUCgSsWIGLt29j4ejRsLWywuYjR+C3ZAniIyPh4ugo1D0aH49RERHw8/DAxuBgXL5zB+F79yIzLw9RISFNsamPhZ8ldbNnz0bfvn1Vyrp3715rm5bQL0RERE1B74lGaGgofHx8IJfLkZWV9cj6mzdvxrVr13D27FlhB2LYsGHw8PDAunXrEBERAQCQSqV47733EBAQgNjYWADA9OnToVAosHLlSgQHB6Nt27aNt2ENMHz4cAwfPlyl7J133oG3tzfWr19fa6IRGxuLM2fOICYmBkFBQQCA8ePHw9XVFWFhYdizZ49QNzQ0FJ6envjxxx+FG8BYWVkhIiICc+bMgZubWyNsXcMN79MHw/v0USl7JyAA3vPnY/3Bg7UmGrFnzuBMUhJiFi1CUP/+AIDxL7wA17ffRtiePdgTGirUDd2+HZ7OzvhxxQqYVN1MyUoiQURMDOYEBsKtY8dG2Drd4WdJ3YABA4TPQ121hH4hIiJqCnpdOvXrr78iNjYWGzZsqHOb2NhY9O3bV+UopZubG4YMGYK9e/cKZcePH0d2djZCahyJnjlzJoqKivDdd989dvxNQSQSwcnJCXl5ebXWi42NRfv27TFmzBihzM7ODuPHj8fBgwdRWloKAEhMTERiYiKCg4NV7jIZEhKCiooKYYfJ0IlEIjjZ2iKvqKjWerGnT6O9tTXGPP+8UGbXpg3Gv/ACDsbFobSsDACQePcuEu/dQ/DQoUKSAQAhw4ZV9svp042zITrCz5J2hYWF9VoW2FL6haiuZDIZPvvsMy4LNBA///knVnz9NbIKCvQdikEpLS3F/Pnzcfz4cVRUVOg7HKqit0RDLpdj1qxZmDZtGnr16lWnNgqFApcuXUKfGke3AaBfv364ceMGCgsLAQAXLlwAALW63t7eMDY2Fp43REVFRcjKysKNGzcQGRmJo0ePYsiQIbW2uXDhAry8vGBsrPpf2q9fPxQXFyM5OVmoB6j3i6OjIzp27GjY/VJSgqyCAtxIT0fkwYM4Gh+PIc88U2ubCzdvwqtbN/V+cXFBcWkpklNThXoA0KfGshrHdu3Q0dZWeN4Q8bOk3ZQpU2BlZQULCwsMGjQI586dq7V+S+kXorpQJhjdunXD9OnTsWzZMn2HRAB2HjuGsD174PTWW1i8cycTjipJSUmIjIzE4MGDMWDAACYcBkJvicann36KO3fuYOXKlXVuk5OTg9LSUjg4OKg9pyxLS0sDAKSnp0MkEsHe3l6lnpmZGdq1ayfUM0QLFiyAnZ0dunfvjtDQUIwePRqbNm2qtU16enqd+6V6ec26Bt0v27fDbuJEdJ8xA6Gff47RPj7YNGNGrW3Sc3PhoGH5ioONDQAgLSdHqAdAc922bYV6hoifJXVmZmYYO3YsPv74Yxw8eBDh4eG4fPkyBgwYUGsC8KT3C1Fd1EwwUqsOyHCnzXAYGxmhRCbDhwcOMOHQ4Pfff2fCYSD0co5GdnY2li5dig8++AB2dnZ1bieVSgEA5ubmas9ZWFio1JFKpTAzM9P4OhYWFkI9QzR37lwEBQUhLS0Ne/fuhVwuh0wmq7WNVCqtc78A2vuwwIC/qOYGBiLI1xdpOTnY+9tvkCsUkFUtfdJGKpPB3NRUrdyiqkxa1a/SqqVl2uoWGOj7hZ8lzXx9feHr6ys8DgwMRFBQEDw9PbF48WJ8//33Gts96f1CjUMulyMhIQEikQg9e/aEkZGRvkNqELlcjs8//xzLly9HSkqKsB3KnbS8vDy0b9++3q9pbW2NX3/9FY7VLr5BD435z39gZlL33bH84mLAyAioqIBCoRASjg2HD2PuiBFYOGYMbFq3bsSIm8Yvv/xSr/dbWbX9AXnV1SKVCUf//v2xatUqvPjiizqPk2qnl0Tj/fffh42NDWbNmlWvdmKxGACE8w2qKykpUakjFou17pyXlJQI9QyRm5ubcEL2pEmT4O/vjxEjRiAuLk7rACYWi+vcL4D2PjTofunYUTghe9LgwfBfuhQjwsMR99FH2vvFzEw4D6O6kqoycdWOorhqx1JbXbGWHUp942ep7rp3746RI0di//79kMvlEFU7F0epJfYLPb6dO3di6tSpAIBvv/0WAQEBeo6oYY4dO4bp06cLj2seBa6oqEBmZma9Xzc7OxujR49GXFzcY8f4pGhd7fvhUeca1oUy4Vi9bx/KFQqsnTLlsV9TH8zNzWFsbAyFQgGZTNag91t1yoTj9OnTePnll5GdnQ2JRKKLUJtc62rJY6tWrfQYSf00eaJx7do1REdHY8OGDSpLC0pKSlBWVobbt2/DysoKNlVLW6qzsbGBubm5sPynOmWZ8oiJg4MD5HI5MjMzVZY2yGQyZGdnN6sjK0FBQZgxYwaSk5PRo0cPjXUcHBzq3C/KcicnJ7W6/fr102XojSrI1xczNm9Gcmoqemi5IpRD27bCsqjq0quWQjlWvc+US6bSc3PhVGNmID03F/2qXR7YUPCzVH9OTk6QyWQoKiqClZWV2vMttV/o8SjP26n5e3MzePBgbN26VWVGo3qyYWRkVK+ZUwDCjmJxcbFOY23uQkePRlxyMk4lJsLEuH6r2POLi1Eml0OhUAhlxsbGMDMxEWY0mitLS0vs27cPM2bMQEVFRb1mB8vKypBbY7wXiUSQy+XCjEZzTTIAYPXq1Zg8eTKsrKzU9t8MWZMnGqmpqVAoFJg9ezZmz56t9nyXLl0wZ84cjVfPMTY2Rq9evTSe0BkXF4euXbsKGV/v3r0BAOfOnVO5ZOy5c+egUCiE55sD5RKM/Px8rXV69+6NU6dOQaFQqJz4HBcXB4lEAldXV6EeUNkP1ZOKtLQ0pKSk1HoJXUOjXPaUX8sA1rtLF5xKTFTvl+RkSMzN4dqhQ2W9rl0BAOeuX0e/qr4CgLTsbKRkZSHY378xNuGx8LNUfzdv3oSFhYXWo0EttV+IgMqdsmnTpmHSpEnYtWuXWsJhbW2Nv//+u16vaWJiArlcrnZBjpbO1MQE37z3XoPavrF+PfacPAlANcFYUHWvqOZu1KhRGDVqVL3bXbx4UfjeVSYYPj4+WLlyJfz8/JrtkkYlsVgMLy8vfYdRb03+yffw8MCBAwfU/vXs2ROdOnXCgQMHhCnou3fvIikpSaV9UFAQ/vjjD5Udgb/++gvHjh3DuHHjhLLBgwfDxsZG7QZ0UVFRkEgkBjm1rWmKsKysDLt27YJYLMbTTz8NoPLIalJSksp6xKCgIPz999/Yv3+/UJaVlYWYmBiMGDFCWHPes2dPuLm5ITo6WphSBCr7xcjIqN73HGgKmRou7VtWXo5dx49DbGaGp6sy+/ScHCSlpKCs2qVMg/r3x995edj/v/8JZVkFBYg5fRoj+vUTzsno2akT3Dp2RPQPP6j2y9Gjlf1SdQ8OQ8LPknb3799XK7t48SIOHToEf39/YaenpfULUV2YmZlh2rRpuHHjBrZu3YoOVQdkmvuO2pNEUVEBCzMz/Hv0aNzbvh3/mTz5iUgydMXHxwfHjh3DqVOnMGjQIL539ajJZzRsbW01ZqrKo67Vn5s0aRJOnjypMnUbEhKCrVu3IiAgAKGhoTA1NcX69evRvn17LFiwQKgnFouxcuVKzJw5E+PGjcPQoUNx6tQpfPnll1i1apXG5ST6NmPGDBQUFGDgwIHo0KEDMjIysHv3biQlJWHdunXCUdjFixdj586duHXrFpydnQFU7hz5+PhgypQpSExMFO4MLpfLsXz5cpW/s3btWgQGBsLf3x+vvvoqEhISsGnTJkybNg3u7u5NvdmPNGPzZhQUF2Ngz57o0K4dMnJzsfvkSSSlpGDdW2+hVdVa18W7dmHnsWO4tXUrnKtOIAvy9YVPjx6Y8sknSLx3T7gzuFyhwPLXXlP5O2unTEFgeDj8w8Lw6oABSLh7F5u++w7T/vEPuBvgNCU/S9pNmDABYrEYvr6+sLe3R2JiIqKjoyGRSLB69WqhXkvrF6L6UCYcyhkOXuDAMEwePBgujo4IGT6cyUU1bm5umDdvHkaMGPFEzGA8KfR+Z/D6at26NU6cOIF58+YhPDwcCoUCfn5+iIyMVFs7GhISAlNTU6xbtw6HDh2Ck5MTIiMjMWfOHD1FX7sJEyZg27ZtiIqKQnZ2Nlq3bg1vb2+sWbMGgYGBtbYViUQ4cuQIFi5ciE8++QRSqRR9+/bFjh071M7reOWVV7B//34sX74cs2bNgp2dHZYsWYKlS5c25uY12IQXXsC2n35C1NGjyC4sRGuxGN7dumHN5MkIfO65WtuKRCIcCQvDws8/xyeHD0Mqk6Gviwt2zJmjdl7HK337Yv/ixVj+1VeYFR0NuzZtsCQoCEtffbUxN09vnuTP0qhRo7B7926sX78eBQUFsLOzw5gxYxAWFobuNe6VUtOT3C9EDaFMOMgwvNS7N17i0kw15ubmWL9+vb7DoBqMKup4cWFlZpiZmVnvk8GeVOfPn4e3tzfi4+Ob5bq5xnT+5El4+/khPiICXh4e+g7HYJxPSID3kiWIP3ECXrzMngp+nlTdv39fOMmc14DXzhDGpo8//hhz584FAHz11Vd4Vc8HJwzps6Q8R8PT0xMXL17UaywAxyZtODZpZ0ifJ0NQ37GJZ2cREREREZHOMdEgIiIiIiKdY6JBREREREQ6x0SDiIiIiIh0jokGERERERHpHBMNIiIiIiLSOSYaRERERESkc0w0iIiIiIhI55rdncGJiIgMgUwmQ3l5OWQymVBWWlqK4uJimJqawtTUVI/R6U9FRQWkUqnwOwDI5XIUFxcDACQSid5iI6KmxRkNIiKiesrIyIC1tTUsLS3x73//W7hD+ZtvvglLS0vY2dmpJCAtyb/+9S9YWlrC0tISCoUCAHDlyhWhLCoqSs8RElFTYaJBRERUT7a2trC3txceK4/cA4CRkRFcXFxa7IxGr169an3ew8OjiSIhIn1jokFERFRPJiYmWLZsmcbnKioqEB4eLsxytDRTp05F+/bt1cpFIhH8/PwwYMAAPURFRPrARIOIiKgBJk6ciM6dO6skFCKRCH369IG/v78eI9MvCwsLfPDBB2qJllwux4oVK/QUFRHpAxMNIiKiBlDOalRfNiWXy1v0bIbS1KlTVZaWcTaDqGViokFERNRA1Wc1RCIRvL29W/RshlLNWQ3OZhC1TEw0iIiIGqj6rIZcLseqVata/GyGUvVZDc5mELVMTDSIiIgew8SJE+Hs7Iy+fftyNqMaCwsLLF26FAA4m0HUQvGGfURERI/BxMQEcXFxMDc352xGDf/617/g7++P7t276zsUItIDJhpERESPqfqJz/SQkZERkwyiFoxLp4iIiIiISOeYaBARERERkc4x0SAiIiIiIp1jokFERERERDrHRIOIiIiIiHSOiQYREREREekcEw0iIiIiItI5JhpERERERKRzTDSIiIiIiEjnmGgQEREREZHOMdEgIiIiIiKdY6JBREREREQ6Z1LfBpcuXULbtm0bI5Zm5+rVqyo/6aGrycmVP2/f1m8gBkbZH1f//FOvcRgi4T3DzxMAIDc3V98hNCscmx7i2KQdxybNODZpx7FJVX3HJqOKioqKOlU0MmpQQERE9Hjq+DXdInFsIiLSj7qMTfWe0fj555951KjK1atXMXHiRHz55Zdwd3fXdzgGReib6Gi4u7rqOxyDcfXPPzFx7lx8GRwMd2dnfYdjUK7evo2J0dGV7xlvb32Ho3e5ubl46aWX9B1Gs8Gx6SGOTdpxbNKMY5N2HJtU1Xdsqnei4enpCTs7u/o2e6K5u7vDy8tL32EYJHdvb/aNBu7OzvDy8NB3GAbJ3dWV7xkA9+/f13cIzQrHJnUcm7Tj2KQZxybtODZVqu/YxJPBiYiIiIhI55hoEBERERGRzjHRICIiIiIinWOiQUREREREOsdEg4iIiIiIdI6JBhERERER6RwTDSIiIiIi0jkmGkREREREpHNMNIiIiIiISOeYaBARERERkc4x0SAiIiIiIp1jokFEREREBkEul+NmRoa+wzBId+7cQVlZmb7DqBcmGkRERERkEJZ//TVc334bSSkp+g7FoGRkZMDNzQ0zZ87Udyj1wkSDiIiIiPQup7AQ6775BnKFAiu+/lrf4RiUNWvWoKSkBNu3b8edO3f0HU6dMdEgIiIiIr2LPHgQJTIZAODrU6c4q1ElIyMDmzdvFh6vWrVKj9HUDxMNIiIiItKrnMJCrD94EIqKCgCAyMgIyzmrAaByNkMulwOoPIelOc1qMNEgIiIiIr3acOiQMJsBAOUKBf7LWQ1hNkOZaCg1l1kNJhpEREREpDc5hYVY/803wmyGksjIqMWfq1F9NkOpOc1qMNEgIiIiIr3ZcOgQikpL1crLFYoWfa6GttkMoDLZaA6zGkw0iIiIiEhvLt66pfW5iooKXLl7twmjMRzJyckoLy/X+vzFixebMJqGMdF3AERERETUch1YsgQlZWXIKSyE01tvAQD8n30WB5YsgREAsbm5fgPUk4EDB6KoqAgKhQJvv/02vvjiCwBAfHw83NzcYN4M+oWJBhERERHpjbGxMSTm5iong4uqylo6CwsLAICJycNddrFYDIlEoq+Q6oVLp4iIiIiISOeYaBARERERkc4x0SAiIiIiIp1jokFERERERDrHRIOIiIiIiHSuyRONEydOwMjISOO/33///ZHtU1NTMX78eFhbW8PKygojR47EzZs3Ndbdtm0b3N3dYWFhARcXF2zcuFHXm9OoVq1aBSMjI3h4eNSp/pPaN1euXMG4cePQtWtXSCQS2NraYuDAgTh8+HCd2ufl5SE4OBh2dnawtLTEoEGDcP78eY11Dx06BC8vL1hYWKBTp04ICwur9RrW+nTl7l2MW70aXadPhyQoCLavv46B776Lw2fP1ql93oMHCN60CXYTJ8Jy3DgMeu89nL9xQ2PdQ3Fx8Jo7FxZjx6LTW28hbM8elGu4gZCh4meJiIio6ent8razZ89G3759Vcq6d+9ea5sHDx5g0KBByM/Px5IlS2BqaorIyEi8+OKL+PPPP9GuXTuh7pYtW/D2229j7NixmD9/Pk6dOoXZs2ejuLgYixYtapRt0qWUlBRERETA0tKyTvWf5L65c+cOCgsLMXnyZDg6OqK4uBj79u1DYGAgtmzZguDgYK1tFQoFAgICcPHiRSxcuBC2trbYvHkz/Pz8EB8fDxcXF6Hu0aNHMWrUKPj5+WHjxo24fPkywsPDkZmZiaioqKbY1Hq5k5mJQqkUkwcPhqONDYpLS7HvzBkEhodjS0gIgl9+WWtbhUKBgBUrcPH2bSwcPRq2VlbYfOQI/JYsQXxkJFwcHYW6R+PjMSoiAn4eHtgYHIzLd+4gfO9eZOblISokpCk29bHws0RERKQfeks0BgwYgKCgoHq12bx5M65du4azZ88KScqwYcPg4eGBdevWISIiAgAglUrx3nvvISAgALGxsQCA6dOnQ6FQYOXKlQgODkbbtm11u0E6FhoaCh8fH8jlcmRlZT2y/pPcN8OHD8fw4cNVyt555x14e3tj/fr1tSYasbGxOHPmDGJiYoT32/jx4+Hq6oqwsDDs2bNHqBsaGgpPT0/8+OOPwvWqraysEBERgTlz5sDNza0Rtq7hhvfpg+F9+qiUvRMQAO/587H+4MFaE43YM2dwJikJMYsWIah/fwDA+BdegOvbbyNszx7sCQ0V6oZu3w5PZ2f8uGIFTEQiAICVRIKImBjMCQyEW8eOjbB1usPPEhERkX7o9RyNwsLCei1LiY2NRd++fVVmQtzc3DBkyBDs3btXKDt+/Diys7MRUuNo68yZM1FUVITvvvvu8YNvRL/++itiY2OxYcOGOrdpKX2jJBKJ4OTkhLy8vFrrxcbGon379hgzZoxQZmdnh/Hjx+PgwYMoLS0FACQmJiIxMRHBwcEqN8UJCQlBRUWFsCNp6EQiEZxsbZFXVFRrvdjTp9He2hpjnn9eKLNr0wbjX3gBB+PiUFpWBgBIvHsXiffuIXjoUCHJAICQYcMq++X06cbZEB3hZ4mIiEh/9JZoTJkyBVZWVrCwsMCgQYNw7ty5WusrFApcunQJfWocwQWAfv364caNGygsLAQAXLhwAQDU6np7e8PY2Fh43hDJ5XLMmjUL06ZNQ69everUpqX0TVFREbKysnDjxg1ERkbi6NGjGDJkSK1tLly4AC8vLxgbq77V+/Xrh+LiYiQnJwv1APV+cXR0RMeOHQ27X0pKkFVQgBvp6Yg8eBBH4+Mx5Jlnam1z4eZNeHXrpt4vLi4oLi1FcmqqUA8A+tRY1ujYrh062toKzxsifpaIdEMmk+Gzzz7j+UcaJCQkYP78+UhJSdF3KC1eaVkZ5m/bhuOXLqGiokLf4VCVJl86ZWZmhrFjx2L48OGwtbVFYmIiPvroIwwYMABnzpzBs88+q7FdTk4OSktL4eDgoPacsiwtLQ09evRAeno6RCIR7O3t1f52u3btkJaWpvsN05FPP/0Ud+7cwc8//1znNi2lbxYsWIAtW7YAAIyNjTFmzBhs2rSp1jbp6ekYOHCgWnn1funVqxfS09NVymvWNeh+2b4dW77/HkBVv/j4YNOMGbW2Sc/NxcCePdXKHWxsAABpOTno5eyM9NzcynINS4Ac2rZFWk7O44bfaPhZIno8MpkMu3btwvLly5GSkgIbGxvMmjVL32EZhISEBCxbtgz79u0DADz77LN444039BxVy5aUkoLIgwcRefAg+ru7Y+Xrr8OvVy8YGRnpO7QWrckTDV9fX/j6+gqPAwMDERQUBE9PTyxevBjfV+0w1SSVSgEA5ubmas9ZWFio1JFKpTAzM9P4OhYWFkI9Q5OdnY2lS5figw8+gJ2dXZ3btYS+AYC5c+ciKCgIaWlp2Lt3L+RyOWQyWa1tpFJpnfsF0N6HBQUFjxt+o5kbGIggX1+k5eRg72+/Qa5QQFa19EkbqUwGc1NTtXKLqjJpVb9Kq5aWaatbYKDvF36WiBquZoKh3FHjUWLVBKP6MlsyLL//9RcGv/8+Ew4DYBCfku7du2PkyJHYv38/5HI5RNXWgiuJxWIAENbUV1dSUqJSRywWa90BLSkpEeoZmvfff79BR4xaQt8AlevklSdkT5o0Cf7+/hgxYgTi4uK0foGIxeI69wugvQ8Nul86dhROyJ40eDD8ly7FiPBwxH30kfZ+MTMTzsOorqSqTFy1Ay2u2uHWVlesZUdb3/hZoqYkl8uRkJAAkUiEnj17NtsdGrlcjs8//1xrgpGXl4f27dvX+zWtra3x66+/wrHa1eyam8TERCxdulQlwah+junMmTMRWu0iGo9SUVEBuVyORYsWYeHChc32PdOYfrl4Ee3rMUtUVu2S63KFAoBqwrHqjTfwYh0vcU66YxCJBgA4OTlBJpOhqKgIVlZWas/b2NjA3NxcWOJSnbJM+SXm4OAAuVyOzMxMlWUNMpkM2dnZBvlld+3aNURHR2PDhg0qSy5KSkpQVlaG27dvw8rKCjZVS1uqe9L7RpugoCDMmDEDycnJ6NGjh8Y6Dg4Ode4XZbmTk5Na3X79+uky9EYV5OuLGZs3Izk1FT20XBHKoW1bYVlUdelVS6Ecq95nyiVT6bm5cKoxM5Cem4t+1S4PbCj4WaKmtnPnTkydOhUA8O233yIgIEDPETXMsWPHMH36dOFxzRmMiooKZGZm1vt1s7OzMXr0aMTFxT12jPoyYcIEJCQkAIDGi9gUFhYK52/Vx6JFizB48GCN54W1ROampjA2MoKiogKy8nJk5uc/1uspE47TV6/i5WXLkL17NyQaZqybg9atWwu/t2rVSo+R1I/B3Bn85s2bsLCw0Np5xsbG6NWrl8aTxuPi4tC1a1fhP6F3794AoFb33LlzUCgUwvOGJDU1FQqFArNnz0aXLl2Ef3FxcUhOTkaXLl2wYsUKjW2f9L7RRrk0Jb+WL6LevXvj/PnzUFR92SjFxcVBIpHA1dVVqAeo90taWhpSUlKaV79UHWXPLy7WWqd3ly44f+OGer8kJ0Nibg7XDh0q63XtCgA4d/26Sr207GykZGWhd5cuugxdJ/hZoqZWfQezITubhmLw4MHYunUrOlYdoKh5lN3IyAj29vb1+qdUXMv3UXOwd+9eBAUFwcjISOOSqdatW9erXyQSidC2Ob9ndM3SwgL7Fi+GfZs2sLOygn2bNnX+11bD/qOo6oIn/d3d8f2yZc02yQCA1atXIz4+HteuXVM7IGrImnxG4/79+2prpi9evIhDhw5h2LBhwlVw7t69i+LiYpV7FwQFBeHdd9/FuXPnhOz/r7/+wrFjx1SmLAcPHgwbGxtERUWp3H8hKioKEonEII82eXh44MCBA2rl77//PgoLC/Hxxx+jW7duAFpe39Q8YgwAZWVl2LVrF8RiMZ5++mkAlUec8/Pz0a1bN5hWnVMQFBSE2NhY7N+/X7iPRlZWFmJiYjBixAhhLX7Pnj3h5uaG6OhozJgxQ1i+FxUVBSMjo3rf86UpZOblwd7aWqWsrLwcu44fh9jMDE9XfRGl5+Qgv7gY3Z56CqZVA2RQ//6IPXMG+//3P+E+GlkFBYg5fRoj+vUTzsno2akT3Dp2RPQPP2DG0KEP++Xo0cp+qWprSPhZImoYkUiEadOmYfLkydi1axeWLVsmLKGqqKiAtbU1/v7773q9pomJCeRyudoV7pobd3d3xMTEqJ2joZzd+L//+796nQy+ZMkS/Oc//2mscJu1UT4+GOXjU+92F2/dQu85cwBUJhhyhQI+PXogfOJE+NXxyoOGTCwWw8vLS99h1FuTJxoTJkyAWCyGr68v7O3tkZiYiOjoaEgkEqxevVqoN2nSJJw8eVJl6jYkJARbt25FQEAAQkNDYWpqivXr16N9+/ZYsGCBUE8sFmPlypWYOXMmxo0bh6FDh+LUqVP48ssvsWrVKo1LJvTN1tYWo0aNUitXXv+/+nMtrW9mzJiBgoICDBw4EB06dEBGRgZ2796NpKQkrFu3TpgFW7x4MXbu3Ilbt27B2dkZQOVOo4+PD6ZMmYLExEThzuByuRzLly9X+Ttr165FYGAg/P398eqrryIhIQGbNm3CtGnT4O7u3tSb/UgzNm9GQXExBvbsiQ7t2iEjNxe7T55EUkoK1r31FlpVnSOweNcu7Dx2DLe2boVz1frqIF9f+PTogSmffILEe/eEO4PLFQosf+01lb+zdsoUBIaHwz8sDK8OGICEu3ex6bvvMO0f/4C7AR5V4WeJ6PGYmppi6tSpeOONNzSeFN6SeXh4IDY2Vu2qU2RYnqQEo7lr8kRj1KhR2L17N9avX4+CggLY2dlhzJgxCAsLQ/ca1+qvqXXr1jhx4gTmzZuH8PBwKBQK+Pn5ITIyUm2WJCQkBKampli3bh0OHToEJycnREZGYk5VtvukeZL7ZsKECdi2bRuioqKQnZ2N1q1bw9vbG2vWrEFgYGCtbUUiEY4cOYKFCxfik08+gVQqRd++fbFjxw618zpeeeUV7N+/H8uXL8esWbNgZ2eHJUuWYOnSpY25eQ024YUXsO2nnxB19CiyCwvRWiyGd7duWDN5MgKfe67WtiKRCEfCwrDw88/xyeHDkMpk6Ovigh1z5qid1/FK377Yv3gxln/1FWZFR8OuTRssCQrC0ldfbczN05sn+bNEVB9mZmaYNm0aJk2ahF27dvFKatVUTzi2b9+OQYMG6TukFs+tY0fMGzkSgf36McEwIEYVdbxenfJIRmZmZr0uF/kkO3/+PLy9vREfH98sp7MaE/tGs/MnT8Lbzw/xERHw4tUvVJxPSID3kiWIP3ECXi++qO9w9O7+/fvCkkFeVlQ7QxibPv74Y8ydOxcA8NVXX+FVPSfhhvT9q1w65enpiYsXL+o1FsBw+qb60qljx47pPVHh2KQdxyZV9R2bmveiSSIiIiIiMkhMNIiIiIiISOeYaBARERERkc4x0SAiIiIiIp1jokFERERERDrHRIOIiIiIiHSOiQYREREREekcEw0iIiIiItI5JhpERERERKRzJvoOgIiIqDmSyWQoLy+HTCYTykpLS1FcXAxTU1OYmprqMTr9qaiogFQqFX4HALlcjuLiYgCARCLRW2z6VlpaCrlcjrKyMqGspKQExcXFMDMzg4kJd8voycIZDSIionrKyMiAtbU1LC0t8e9//xtGRkYAgDfffBOWlpaws7NTSUBakn/961+wtLSEpaUlFAoFAODKlStCWVRUlJ4j1I8rV66gdevWsLS0xEcffSS8Z4YPHw5LS0t0795dzxES6R4TDSIionqytbWFvb298Fh55B4AjIyM4OLi0mJnNHr16lXr8x4eHk0UiWHp3LkzLC0thcc13zOenp76CIuoUTHRICIiqicTExMsW7ZM43MVFRUIDw8Xjli3NFOnTkX79u3VykUiEfz8/DBgwAA9RKV/rVq1wrvvvqvxfVFRUYHly5frISqixsVEg4iIqAEmTpyIzp07q+w4ikQi9OnTB/7+/nqMTL8sLCzwwQcfqO1Qy+VyrFixQk9RGYaZM2eiTZs2KmUikQgjRozAs88+q6eoiBoPEw0iIqIGUM5qVF8CI5fLsXLlyhY7m6E0depUlaVlLX02Q0nTrIZcLtc6O0bU3DHRICIiaqDqsxrK2YyhQ4fqOyy9qzmrwdmMh6rPaihnM7y8vPQcFVHjYKJBRETUQNVnNeRyeYs+N6Om6rManM14SDmrAVQmYDw3g55kTDSIiIgew8SJE+Hs7Iy+ffu26HMzarKwsMDSpUsBgLMZNcycORM2NjY8N4OeeLwzDBER0WMwMTFBXFwczM3NOZtRw7/+9S/4+/vzHhE1tGrVCpcuXYKNjY2+QyFqVEw0iIiIHlP1E5/pISMjIyYZWnTo0EHfIRA1Oi6dIiIiIiIinWOiQUREREREOsdEg4iIiIiIdI6JBhERERER6RwTDSIiIiIi0jkmGkREREREpHNMNIiIiIiISOeYaBARERERkc4x0SAiIiIiIp1jokFERERERDrHRIOIiIiIiHSOiQYREREREemcSX0bnPnjDKzbWjdCKM1P8l/JAICrV6/qORLDo+wT9o2qq8lV75nbt/UbiAFS9snVP//UaxyGIjcvT98hNCscmx7i2KQdxybNODZpx7FJVX3HJqOKioqKOlU0MmpIPERE9Jjq+DXdInFsIiLSj7qMTfWe0VgSvQStrFo1KKAnze3rtxH9fjSiP4+Gt6e3vsMxKFevXsXEiRPx5Zdfwt3dXd/hGAyhX6Kj4e7qqu9wDMrVP//ExLlz8WVwMNydnfUdjt7lPniAlyIi9B1Gs8Gx6SGOTdpxbNKMY5N2HJtU1Xdsqnei4e7hjjY2berb7Inm2sMVXl5e+g7DILm7u7NvNHD39ma/aOHu7AwvDw99h6F39/Pz9R1Cs8KxSR3HJu04NmnGsUk7jk2V6js28WRwIiIiIiLSOSYaRERERESkc0w0iIiIiIhI55hoEBERERGRzjHRICIiIiIinWOiQUREREREOsdEg4iIiIiIdI6JBhERERER6RwTDSIiIiIi0jkmGkREREREpHNMNIiIiIiISOeYaBARERERGbg7mZkoKy/Xdxj1wkSDiIiIiMiAZeTmwi0kBDM//VTfodQLEw0iIiIiIgO2Zt8+lMhk2P7zz7iTmanvcOqMiQYRERERkYHKyM3F5iNHhMer9u7VYzT1w0SDiIiIiMhArdm3D3KFAgAgVyia1awGEw0iIiIiIgOknM1QJhpKzWVWg4kGEREREZEBqj6bodScZjWYaBARERERGRhtsxlAZbLRHGY1mGgQERERERmY5NRUlMvlWp+/eOtWE0bTMCb6DoCIiKg5ykwVI+6XpxB/yh4ZdyXIyzZHeZkxrNuVwqZ9CZ55Pgv9BmXApVe+vkMlomZooIcHimJioKiowNubN+OL48cBAPGRkXDr2BHmJoa/G2/4ERIRERmQglwz7Nnoih/2doa8XH1hQGaaBJlpEiRdsMF/N7vi2Rcy8da/E9HZtVAP0RJRc2ZhZgYAMDF++F0jNjODxNxcXyHVCxMNIiKiOrp7vRWWTfdBVrpYKBOZKNDjmVzYOUphaqZATqYFki60RfEDUwDAhd/sMf9sOyxYewG+Q9P1FToRUZNjokFERFQHd6+3wruv98eD/KojjKYKjJpyA6Om3IBV2zKVumUyY5w64ojPP3wa+TnmKJOJ8OE8b8xdcwF+I1L1ET4RUZPjyeBERESPICs1xtp53kKSYS4ux4ptv2PS/CS1JAMATM0UGDwqBR9/cxKOnR8AABQKI2wO80TqLcsmjZ2ISF+YaBARET1CzBYX3LlmJTyet+YCPPplP7KdjX0pVmz/HWLLymSkpNgEmz54ptHiJCIyJEw0iIiIalEqFeHIbmfhsc9L6fD1z6hze/sOUvxzVrLw+Mq5dki+ZK3DCImIDBMTDSIiolr89r0DCquWTAHAyDdv1vs1/MfdgVhSLjw++lVnncRGRGTI9JJolMnKsOOjHXhzwJsIeiYIoeNDceH0hTq1zf47G2vmrsFrfV/DBO8JCA8JR8Y9zUeWfoz9ESHDQzDWcyxmDJ2Bb7/4VpeboXMnTpyAkZGRxn+///77I9unpqZi/PjxsLa2hpWVFUaOHImbNzUPiNu2bYO7uzssLCzg4uKCjRs36npzGtWqVatgZGQEDw+POtV/EvvmypUrGDduHLp27QqJRAJbW1sMHDgQhw8frlP7vLw8BAcHw87ODpaWlhg0aBDOnz+vse6hQ4fg5eUFCwsLdOrUCWFhYSgvL9dY1xBcuXsX41avRtfp0yEJCoLt669j4Lvv4vDZs3Vqn/fgAYI3bYLdxImwHDcOg957D+dv3NBY91BcHLzmzoXF2LHo9NZbCNuzp9YbLFHzcznOVvjd1kGKnn1y6v0aYks5+g15OFZdPmtbS20ioieDXq46teHdDTjz4xkETgqEQ2cHHDtwDCtmrMCqnavwtPfTWttJi6R4b9J7KHpQhKAZQTAxMcHBnQex5I0l2HBgA6zaPlw/+/3X32Pzss3w9ffFyDdHIjE+EdGrolFaUoqx08c2xWY22OzZs9G3b1+Vsu7du9fa5sGDBxg0aBDy8/OxZMkSmJqaIjIyEi+++CL+/PNPtGvXTqi7ZcsWvP322xg7dizmz5+PU6dOYfbs2SguLsaiRYsaZZt0KSUlBREREbC0rNsJlU9q39y5cweFhYWYPHkyHB0dUVxcjH379iEwMBBbtmxBcHCw1rYKhQIBAQG4ePEiFi5cCFtbW2zevBl+fn6Ij4+Hi4uLUPfo0aMYNWoU/Pz8sHHjRly+fBnh4eHIzMxEVFRUU2xqvd3JzEShVIrJgwfD0cYGxaWl2HfmDALDw7ElJATBL7+sta1CoUDAihW4ePs2Fo4eDVsrK2w+cgR+S5YgPjISLo6OQt2j8fEYFREBPw8PbAwOxuU7dxC+dy8y8/IQFRLSFJtKTSDxvI3wu6tnboNfp4dnLk4e7ggAyEyVIPtvC7RrX/LY8RERGaomTzSSLyXj1JFTmLJwCkZPHQ0AGDxqMN4Z8Q52rN2BD7/+UGvbI18dQdqdNKyLWQeXXpU7Qt4DvfHOiHfwzeffYNL8SQCA0pJSfLHhC/R5sQ/e/eRdAMDQ8UOhUCjw36j/Yuj4oWjVplUjb2nDDRgwAEFBQfVqs3nzZly7dg1nz54VkpRhw4bBw8MD69atQ0REBABAKpXivffeQ0BAAGJjYwEA06dPh0KhwMqVKxEcHIy2bdvqdoN0LDQ0FD4+PpDL5cjKynpk/Se1b4YPH47hw4erlL3zzjvw9vbG+vXra000YmNjcebMGcTExAjvtfHjx8PV1RVhYWHYs2ePUDc0NBSenp748ccfYVJ1F1IrKytERERgzpw5cHNza4StezzD+/TB8D59VMreCQiA9/z5WH/wYK2JRuyZMziTlISYRYsQ1L8/AGD8Cy/A9e23EbZnD/aEhgp1Q7dvh6ezM35csQImIhEAwEoiQURMDOYEBsKtY8dG2DpqavfTHt4zo7NLw2+616lG279TxUw0iOiJ1uRLp07/cBrGImMMnTBUKDMzN8M/xv4DSX8m4X76fa1tz/xwBi69XIQkAwA6du2IZ3yewW/f/yaUXY67jMK8Qgz/p+pOWMA/A1BSXII/Tv6hwy1qHIWFhfVamhIbG4u+ffuqzIS4ublhyJAh2Lt3r1B2/PhxZGdnI6TG0daZM2eiqKgI33333eMH34h+/fVXxMbGYsOGDXVu01L6BgBEIhGcnJyQl5dXa73Y2Fi0b98eY8aMEcrs7Owwfvx4HDx4EKWlpQCAxMREJCYmIjg4WEgyACAkJAQVFRVCQtYciEQiONnaIq+oqNZ6sadPo721NcY8/7xQZtemDca/8AIOxsWhtKzy6kGJd+8i8d49BA8dKiQZABAybFhl35w+3TgbQk2q+IGJyt2/W1mpX8q2rlq1UW37IM9MS03DIpPJ8NlnnxnsMlJ9SkhIwPz585GSkqLvUAyKVCrFJ598gp07d+o7FNKzJk80bl69iQ7OHSBpJVEpd/V0BQDcunpLYzuFQoHbf91Gdw/1JUQuni7IuJuB4gfFwt8AoFa3W89uMDY2xs3E+p/I15SmTJkCKysrWFhYYNCgQTh37lyt9RUKBS5duoQ+NY7gAkC/fv1w48YNFBZWHkm7cKHyXJiadb29vWFsbCw8b4jkcjlmzZqFadOmoVevXnVq0xL6pqioCFlZWbhx4wYiIyNx9OhRDBkypNY2Fy5cgJeXF4yNVb8C+vXrh+LiYiQnJwv1APU+cXR0RMeOHQ22T5SKSkqQVVCAG+npiDx4EEfj4zHkmdovLXrh5k14deum3jcuLiguLUVyaqpQDwD61FjW6NiuHTra2grPU/MmLVKd+DeXNPzcJIsabYuLDPueucoEo1u3bpg+fTqWLVum75AMRkJCAoKCgtCrVy9ERkbi+PHj+g7JIEilUmzYsAGdOnXCnDlzsHr1an2HRHrW5N9yufdz0dZOffmJsiwnU/NJdg/yH6BMVqaxrY2djdBW0kqCnMwcGIuMYd3OWqWeqZkpWlu31vo39M3MzAxjx47F8OHDYWtri8TERHz00UcYMGAAzpw5g2effVZju5ycHJSWlsLBwUHtOWVZWloaevTogfT0dIhEItjb26v97Xbt2iEtLU33G6Yjn376Ke7cuYOff/65zm1aQt8sWLAAW7ZsAQAYGxtjzJgx2LRpU61t0tPTMXDgQLXy6n3Sq1cvpKenq5TXrGuofaK0YPt2bPn+ewBVfePjg00zZtTaJj03FwN79lQrd7Cp/J5Jy8lBL2dnpOdWrtV30LCczqFtW6TlGOb3DNWP2FI1OSgtbviwWVKjrcTSMC+oIJPJsGvXLixfvhwpKSkwMjICAFRUVOg5Mv1LSEjAsmXLsG/fPpVZ3pZOKpViy5YtWLVqFbKzs4X3Ct8z1OSfElmJDKZmpmrlZuaVU8jKJRs1lZZUlmtqa2peWSYrlQk/TUw1b5qpualQz9D4+vrC19dXeBwYGIigoCB4enpi8eLF+L5qh6kmqVQKADA3N1d7zsLCQqWOVCqFmZnm6XoLCwuhnqHJzs7G0qVL8cEHH8DOzq7O7VpC38ydOxdBQUFIS0vD3r17IZfLIZPV/h6XSqV17hNAe/8VFBQ8bviNam5gIIJ8fZGWk4O9v/0GuUIBWVntS1+kMhnMTdW/ZyyqyqRVfSut+q7SVrfAQN8vVD+SVuUQmSiE5VOF+er/33X1oEbbVtaGNRbJ5XJ8/vnnWhOMvLw8tG/fvt6va29vj7Nnz0IsFj+6soFKTEzE0qVLVRKM6subZ86cidBq52/V1bp16zBx4kSdxdnUSktLERUVpZZgKF2/fr1B75lu3brh9OnTwnuQmq8mTzTMLMxQJlMf6JU7/5p2aADA3KKyXFPbstLKMmWyYmZuhvIyzUeKykrLhHrNQffu3TFy5Ejs378fcrkcomprwZWUX96akrSSkhKVOmKxWOtOaElJicEOBO+//z5sbGwwa9aserVrCX3j5uYmnJA9adIk+Pv7Y8SIEYiLi9P6JS0Wi+vcJ4D2/jPUPlFy69hROCF70uDB8F+6FCPCwxH30Ufa+8bMTDgPo7qSqjJxVTIqrvqu0lZXrCVppebHzlGKjLuVV7m7e611g1+nZtv2HQwrGT127BimT58uPK6501hRUYHMzMx6v25mZiZycnLQoUOHx45RXyZMmICEhAQA0Hj+ZGFhobAMtz7i4+ObdaIRExODefPmaX1eLpc3+D2jUCg07vO0VK0lD085aFV1ULA5aPJEo61dW+T8rb6kIPd+5TIEG3sbtecAoFWbVjA1MxXqVZdzP0elrY29DRRyBfKy81SWT5XJylCYV6j1bxgqJycnyGQyFBUVwcrKSu15GxsbmJubC8tcqlOWOVZdktPBwUH44FdfIiSTyZCdnS3UMyTXrl1DdHQ0NmzYoLJUp6SkBGVlZbh9+zasrKxgY6P+//qk940mQUFBmDFjBpKTk9GjRw+NdRwcHOrcJ8pyJycntbr9+vXTZeiNLsjXFzM2b0Zyaip6aLkilEPbtsKyqOrSq5ZCOVa9z5RLptJzc+FUY5YtPTcX/apdIpiaN/dnc4REI/lSw68891e1tvYdig3uilODBw/G1q1bVWY0qicbRkZG9ZpRVrK3t9f4/dyc7N27V5jREIlEaslG69at633gxcjICN7e3roMs8mNGzcO2dnZCA8P1zijIRKJVC4hX1fdNJwn19KtnjQJkwcPhpVYrDbmGLIm/1/s6tYVqbdThRO3lf66+BcAoIt7F43tjI2N0dm1M64nXFd7LvliMp5yeko4wbyLW+Vr1Kx7PeE6FAoFurp3feztaEo3b96EhYUFWrXSfEleY2Nj9OrVS+NJ43FxcejatStat648kta7d28AUKt77tw5KBQK4XlDkpqaCoVCgdmzZ6NLly7Cv7i4OCQnJ6NLly5YsWKFxrZPet9oolzulJ+fr7VO7969cf78eSgUCpXyuLg4SCQSuLq6CvUA9T5JS0tDSkpKs+kTJeWyp/ziYq11enfpgvM3bqj3TXIyJObmcK06Ktu7a+X3yLnrqt8zadnZSMnKQu8umr/LqPnxfO7hZbSzMsRI+KP+O83SIhHO/vKU8LjXc4++NHdTE4lEmDZtGm7evInPPvtMmIFQzv5ZW1vj77//rve/y5cvG/zs56O4u7sjJiYGly5dwsiRIwFA5RyN//u//6t3v2RkZDTr2QygchXKnDlzcPfuXURGRsLW1lZltrh79+4Nes+cOXOGy6ZqEJubw6tbN3RvJgc9lZo80fAd6guFXIEf/vuDUFYmK8MvB36B6zOusHOozNLup91Hyk3Vy8X1H9of1y5fw7XL14SylJspuBR3Cf1f7i+Uefp4onWb1jj61VGV9ke/PgpzsTn6vKh+BSJDcP+++qV9L168iEOHDsHf31/I7u/evYukpCSVekFBQfjjjz9Udgj/+usvHDt2DOPGjRPKBg8eDBsbG7UbrUVFRUEikSAgIECXm6QTHh4eOHDggNq/nj17olOnTjhw4ACmTp0KoGX1jabp6LKyMuzatQtisRhPP11588v09HQkJSWhrNoSn6CgIPz999/Yv3+/UJaVlYWYmBiMGDFCWMLYs2dPuLm5ITo6GvJqd7uOioqCkZFRve/30lQyNVzet6y8HLuOH4fYzAxPV83OpOfkICklBWXVjk4G9e+Pv/PysP9//xPKsgoKEHP6NEb06yeck9GzUye4deyI6B9+UO2bo0cr+6b/w+8kat76v5yOVm0eLqs8uKNbvV/jx5jOkFY7GXzYq3d0EltjMDU1xdSpU3Hjxg1s3bpVLeFoyTw8PBAbG4vLly8LCQdVLrOtmXAAfM+QHpZO9XimB/q/3B+7InchLycPDp0ccOybY8hMzcSs8Ifr7yMXRSLhjwQcSjoklA17bRh+iPkBK95egdFvjYbIRISDOw7Cup01Rk0ZJdQztzDH63Nex6crPsXqOavh9YIXrpy7ghOHTmDi3Ilobd3wNbaNacKECRCLxfD19YW9vT0SExMRHR0NiUSicom4SZMm4eTJkypTlCEhIdi6dSsCAgIQGhoKU1NTrF+/Hu3bt8eCBQuEemKxGCtXrsTMmTMxbtw4DB06FKdOncKXX36JVatWGeT0tq2tLUaNGqVWrryXRvXnWlLfzJgxAwUFBRg4cCA6dOiAjIwM7N69G0lJSVi3bp0wA7Z48WLs3LkTt27dgrOzM4DKRMPHxwdTpkxBYmKicGdwuVyO5cuXq/ydtWvXIjAwEP7+/nj11VeRkJCATZs2Ydq0aXB3d2/qza6TGZs3o6C4GAN79kSHdu2QkZuL3SdPIiklBeveegutqo6uLt61CzuPHcOtrVvhXHXCYpCvL3x69MCUTz5B4r17wp3B5QoFlr/2msrfWTtlCgLDw+EfFoZXBwxAwt272PTdd5j2j3/AvcZSM2q+LCRyDH/tNvZ+WjnTF/fLU/jfT0/h+X9k1Kl9ZqoYez55uIyxZ59suHrmNUaoOmVmZoZp06Zh0qRJ2LVrl8FeEEMflAlHQkICtm/fjkGDBuk7JIOgTDiCg4OxdetWtGnTRt8hkZ7p5dps89bMw+6Pd+PEoRN4kP8Azj2c8UHUB/Do61FrO0krCSJ2ReCz/3yGvVF7UaGogEc/D0xdPBVtbFTfzMP/ORwiExG++fwbnD12FrYOtpi6eCoCJwU25qY9llGjRmH37t1Yv349CgoKYGdnhzFjxiAsLAzdu6vfP6S61q1b48SJE5g3bx7Cw8OhUCjg5+eHyMhItTW1ISEhMDU1xbp163Do0CE4OTkhMjISc+bMaczN05sntW8mTJiAbdu2ISoqCtnZ2WjdujW8vb2xZs0aBAbW/j4XiUQ4cuQIFi5ciE8++QRSqRR9+/bFjh071M7reOWVV7B//34sX74cs2bNgp2dHZYsWYKlS5c25uY9lgkvvIBtP/2EqKNHkV1YiNZiMby7dcOayZMR+NxztbYViUQ4EhaGhZ9/jk8OH4ZUJkNfFxfsmDNH7byOV/r2xf7Fi7H8q68wKzoadm3aYElQEJa++mpjbh7pwbgZ1/G/nx1w73rlgar1/34WYdFx8Ohb+2WMc++bI2yqjzCbYSEpxzsrLzZ6vLqkTDhInYeHB9avX6/vMAyOWCzG7Nmz9R0GGQCjijpe5Fg5/fXFmS/UdupbqoQ/E7Dk1SU4ceYEXnz+RX2HY1DOnz8Pb29vxMfHw8vLS9/hGAz2i3bnT56Et58f4iMi4OVR+0GHluB+fj7s33gDAK9FX5umHJvuJLfGu6/3R1Fh5fI5E1MFRr91AyPfvAGrtqpXHyuTGeHUkQ74fO3TyM+uXIpobFyBuWsuwG9EaqPGybFJO34Ha8Z+0Y5jk6r6jk282wwREVEddHYtxOrdp7E8+DlkZYhRXmaMmC0u2L+tG3r0zoW9oxQmpgrk3jdH0gUbISEBAFMzOeZ9eAEvvKx+tTcioicVEw0iIqI66uxaiPX7fsWejT3wU2wnyMuNIS83RuK5dkjU0ubZ/pmY8u9EOPeo/30WiIiaMyYaRERE9WDdToaQZZcxdtp1/P7LUzh/yh7pdy2Rn20GebkxrGxksG0vhadPFp4bkgGXXtovNU1E9CRjokFERNQA7TtKMXLyLYycfEvfoRARGSTedpGIiIiIiHSOiQYREREREekcEw0iIiIiItI5JhpERERERKRzTDSIiIiIiEjnmGgQEREREZHOMdEgIiIiIiKdY6JBREREREQ6x0SDiIiIiIh0jokGERERERHpHBMNIiIiIiLSOSYaRERERESkc0w0iIiIiIhI55hoEBERERGRzjHRICIiIiIinWOiQUREREREOsdEg4iIiIiIdI6JBhERERER6RwTDSIiIiIi0jkmGkREREREpHNMNIiIiIiISOeYaBARERERkc4x0SAiIiIiIp1jokFERERERDrHRIOIiIiIiHTOpL4NCnILGiOOZulBwQMAQF5uHu7fv6/naAxLbm6u8JN98xD7RbvcvLzKnw8e4H5+vn6DMQBZBfyurQ+OTQ9xbNKO38GasV+049ikqr5jk1FFRUVFnSoaGTUoICIiejx1/JpukTg2ERHpR13GJi6dIiIiIiIinavzjAYREREREVFdcUaDiIiIiIh0jokGERERERHpHBMNIiIiIiLSOSYaRERERESkc0w0iIiIiIhI55hoEBERERGRzjHRICIiIiIinWOiQUREREREOsdEg4iIiIiIdO7/AdXCUNxhk38vAAAAAElFTkSuQmCC\n"
          },
          "metadata": {}
        }
      ],
      "source": [
        "size = 5\n",
        "goal_states = [(0, size - 1), (1, size - 1), (size - 1, 0)]\n",
        "error_states = [(o, o) for o in range(size)]\n",
        "grid = gridworld((size, size), goal_states, error_states, finite_horizon=True)\n",
        "\n",
        "solution = Solution(grid, gamma=1.0, epsilon=0.2)\n",
        "q_table = solution.q_learning()\n",
        "\n",
        "Q_star_dict = {}\n",
        "pi_star_dict = {}\n",
        "for i in range(grid.rows):\n",
        "  for j in range(grid.columns):\n",
        "    Q_star_dict[(i, j)] = np.max(q_table[i, j])\n",
        "    # Find the indices where the values are approximately equal to the maximum value\n",
        "    indices = np.argwhere(np.isclose(q_table[i, j], Q_star_dict[(i, j)], atol=1e-4))\n",
        "    indices = [arr.item() for arr in indices]\n",
        "    pi_star_dict[(i, j)] = [grid.actions[index] for index in indices]\n",
        "\n",
        "for (i, j) in goal_states:\n",
        "    pi_star_dict.pop((i, j))\n",
        "\n",
        "print(\"Optimal Q function and optimal policy in the finite horizon case: \")\n",
        "create_grid(rows=5, columns=5, cell_size=20, line_width=1, border_width=3,\n",
        "            error_states=error_states, goal_states=goal_states, arrows=pi_star_dict,\n",
        "            Q_values=Q_star_dict, save_path='grid_with_arrows_colored.png')\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [],
      "metadata": {
        "id": "KDZ-0jOGZmQN"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1Cab4jvYKIiu"
      },
      "source": [
        "# 2. Infinite Horizon Problem  \n",
        "In an **infinite horizon** problem, the agent interacts with the environment over an **unbounded number of time steps**.  \n",
        "\n",
        "### 🎯 Mathematical Objective:  \n",
        "$$\n",
        "J = \\mathbb{E} \\left[ \\sum_{t=0}^{\\infty} \\gamma^t r_t \\right]\n",
        "$$\n",
        "where:  \n",
        "- $\\gamma \\in (0, 1)$ → Discount factor to reduce the value of future rewards.  \n",
        "<!-- - The agent never \"knows\" when the episode will end.   -->\n",
        "\n",
        "### Key Characteristics:  \n",
        "- The agent focuses on **long-term returns** instead of immediate gains.  \n",
        "- Terminal states may not exist — the task could continue forever.  \n",
        "<!-- - The policy is **stationary** (same strategy regardless of time).   -->\n",
        "- The agent balances **exploitation** (short-term rewards) and **exploration** (future benefits).  \n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 439
        },
        "id": "cuGrAPVsGOmz",
        "outputId": "7de9072b-c4c2-4d02-c13d-af61b83116df"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Optimal Q function and optimal policy in the infinite horizon case: \n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 1000x500 with 2 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxoAAAGVCAYAAAB5OYd2AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAk1tJREFUeJzs3XdYFNf7NvCbXkRQFCyIEQWFiCWiCWIB0dgIqAiaYiQ2sGIjKqCYCBI0UUgsKN8Yu29sv8Ro1MSaoCQYa9RAsAQVFFGkLn2X9w/YkZVFd3FhQe/PdXmpZ87MnDk7uzPPnDIaZWVlZSAiIiIiIlIhTXUXgIiIiIiIXj0MNIiIiIiISOUYaBARERERkcox0CAiIiIiIpVjoEFERERERCrHQIOIiIiIiFSOgQYREREREakcAw0iIiIiIlI5BhpERERERKRy2opm1NDQqM1yEBFRNcrKytRdhHqL1yYiIvVQ5NrEFg0iIiIiIlI5hVs0pP5Ztw7NjY1roywNzt8JCRgUHo7jP/6Irk5O6i5OvfL3339j0KBBOH78OLp27aru4tQbrJfqxf0Vh5FuIxEUEwQ7ezt1F0ftcjJzMMNthrqL0WDw2vQUr03V42+wfKyX6vHaJEvZa5PSgUZzY2OYmZgou9orqamRUfnfTZrAzMxMzaWpX5o2bSr8zbp5ivVSvSZNmwAAjIyNYGLK3xhSDq9NT/HaVD3+BsvHeqker00vh12niIiIiIhI5RhoEBERERGRyjHQICIiIiIilWOgQUREREREKsdAg4iIiIiIVI6BBhERERERqRwDDSIiIiIiUjkGGkREREREpHIMNIiIiIiISOUYaBARERERkcox0CAiIiIiIpVjoEFERERERCrHQIOIiIiIiFSOgQYREREREakcAw0iIiIiIlI5BhpERERERKRyDDSIiIiIiEjlGGgQEREREZHKMdAgIiIiIiKVY6BBREREREQqx0CDiIiIiIhUjoEGERERERGpHAMNIiIiIiJSOQYaRERERESkcgw0iIiIiIhI5RhoEBERERGRyjHQICIiIiIilavVQOP63bvwjohA+ylTYOjlheYffYT+ixbh4LlzMvn+98svcA4MRIuPP4aepyesJk/GhK+/RvLDhy/cR/LDh9Dw8Kj2z5S1a6tdd/mePdDw8ID9zJkvfazKuH79Ory9vdG+fXsYGhqiefPm6N+/Pw4ePFgl7549e+Do6IgmTZqgWbNmcHZ2xs8//6zwvnJzc7FgwQJYWVlBT08PFhYW8PLyQn5+vky+rKws+Pr6wszMDI0aNcKAAQNw8eLFlz7Wl7V8+XJoaGjA3t6+2jxZWVkwNzeHhoYG9u3bp/C2N23aBDs7O+jr68PGxgZr1qyRmy81NRVjxoxBkyZNYGxsjBEjRuD27dtKH8vLOH36NDQ0NOT++fPPP2XyxsXFoW/fvjA0NETLli3h7++PvLw8hffVkOoFAK7GX4WHrYfcP4mXE2XyJlxMwMIPF8KruxfG9x2PmLAYFIgKFN7Xr/t+xfTh0zG662j4DfHDoe2H5ObLeJiBFXNW4INeH2Csw1iETQ9D2r20lzpOIiKihka7Njd+Jz0duQUF8HF1RWtTU+QXFWF/XBw8wsKwcfp0+A4dCgC4dPs2rFq0gMfbb6OpkRH+e/gQ//v1Vxz66y9c+fprtG7WrNp9mJmYYPvcuVXSj168iJ2//YbB3bvLXS/l8WOE792LRvr6KjlWZdy5cwe5ubnw8fFB69atkZ+fj/3798PDwwMbN26Er68vAGDNmjXw9/eHm5sbIiIiUFhYiC1btuC9997D/v374enp+dz9ZGdnw9nZGSkpKfD19YW1tTUePXqE2NhYFBUVwdDQEAAgkUjg5uaGK1eu4NNPP0Xz5s2xfv16uLi44MKFC7Cxsan1OpEnJSUF4eHhaNSo0XPzhYSEVAmcXmTjxo2YOnUqRo8ejXnz5iE2Nhb+/v7Iz8/HwoULhXx5eXkYMGAAsrOzERQUBB0dHURGRsLZ2RmXL19Gs+ecm7XB398fvXr1kkmztrYW/n358mUMHDgQdnZ2WL16NVJSUvDVV1/hxo0bOHLkyAu331DrBQDcP3aHdRdrmbTWb7QW/n074TaWTFiCNh3aYNKiSchIy8AP3/2A+3fu47P/ffbC7R/9/ijWf7YeToOdMOKTEfjnwj+IWR6DosIijJ4yWshXICpA8PhgiPJE8PLzgra2Ng5sPYCgj4MQ9UMUjJsaq+yYiYiI6rNaDTSG9+yJ4T17yqTNdHODw7x5WH3ggBBorJ82rcq6Ix0d0XPePGw7dQqLvLyq3UcjfX2MGzCgSvqWkydhbGgI97fflrtewObNcOzUCWKJBI9zcpQ5rJc2fPhwDB8+XCZt5syZcHBwwOrVq2UCjV69euHgwYPQ0NAAAEycOBEWFhbYunXrCwONwMBA3LlzBxcvXoSVlZWQXvmGEQD27duHuLg47N27F14VdT1mzBh07NgRS5cuxa5du176mGsiICAAjo6OEIvFePz4sdw8165dQ3R0NEJCQhASEqLQdgsKChAcHAw3NzehBWTKlCmQSCQIDQ2Fr68vmjZtCgBYv349bty4gXPnzgk3+MOGDYO9vT1WrVqF8PBwFRyp4vr16yd8RvIEBQWhadOmOH36NIyNy29o27VrhylTpuDXX3/F4MGDq123IdcLALzp8Cb6DO1T7fLtkdthZGyE8G3hMDQqD7LNLcyxdslaXDpzCW/1favadYsKi7A9ajt6OvfEom8WAQCGjBkCiUSC3dG7MWTMEBiZGAEADv+/w7h/5z5W7V0Fmy7lQbpDfwfMdJ+JHzf/iPHzxqvqkImIiOq1Oh+joaWlBcvmzZElEj03XztzcwB4YT55Hjx5glNXr8Kzd2/o6+pWWf77tWvYd/YsoiZPVnrbtUVLSwuWlpbIysoS0nJycoQuQVLGxsYwMjKCgYHBc7eXlZWFzZs3w9fXF1ZWViguLkZRUZHcvPv27UOLFi1kAhczMzOMGTMGBw4cqHa92vT7779j3759iIqKem6+2bNnY9SoUejXr5/C2z516hQyMjIwffp0mfQZM2ZAJBLJdE3bt28fevXqJdOKYGtri4EDB2LPnj0K71OVcnNzUVpaWiU9JycHx44dw7hx44QgAwDGjx8PIyOjF5a3odcLAOTn5UNcKpabfjnuMpw9nIUgAwAGjBgAA0MDnDl65rnbvRp/FblZuRj+oewDArcP3VCYX4i/fvtLSIv7JQ42XWyEIAMA2rRvg26O3V64H3r9xP/7L4K3b0dqRoa6i1KvSCQSBAYG4uDBgygrK1N3ceqV48ePY9myZdU+gHtdFRUVYd68eTh16hTPmXqkTgINUWEhHufk4NaDB4g8cABHLlzAwG7dquTLyMlBelYWzt+4gQnffAMAGNi1q9L7+z42FhKJBB85O1dZJhaLMSsmBpMHD0aXdu2U3rYqiUQiPH78GLdu3UJkZCSOHDmCgQMHCstdXFxw9OhRrFmzBsnJyUhMTMSMGTOQnZ2N2bNnP3fbZ86cQWFhIaytreHl5QVDQ0MYGBigT58+uHz5skzeS5cuoUePHtDUlD0d3n77beTn5yMpKUllx6wIsViMWbNmYfLkyejSpUu1+fbu3Yu4uDisXLlSqe1funQJANDzmdY2BwcHaGpqCsslEgn+/vvvKvmA8rq5desWcnNzldr3y5owYQKMjY2hr6+PAQMG4Pz588Kyq1evorS0tEp5dXV10b17d+G4qtOQ6wUAvgn6Bu/3fB+ju41G8Phg3Lh6Q1iWnJQMcakYNvay3QB1dHVgZWeF2/88f2zJ7YTy5db2sl2zOnTuAE1NTWF9iUSC5H+Tq+QDAJuuNki7m4b8POW6+dGrbe/ZswjfuxdWkydj1saNDDgqpKenIyIiAh4eHujevTsDjkq2bt2KpUuXwtLSEoGBgQw4KiQmJiIyMhKurq7o168fA456ok4CjfnffQezceNg7eeHgM2bMcrREWv9/Krks5gwAS3Gj0ev+fMRl5CAb3x98e5b1XdnqM7O335DK1NTuMoJUjYcPYo7jx4h9KOPanQsqjR//nyYmZnB2toaAQEBGDVqFNZWGrz+zTffwMXFBf7+/rCysoKdnR327NmDEydOoHfv3s/d9o0b5TdZgYGBuHfvHrZt24Z169bh1q1bcHV1xYMHD4S8Dx48QKtWrapsQ5p2//59VRyuwjZs2IA7d+4gNDS02jwFBQUICAjA3Llz0U7JgPHBgwfQ0tKCeUWrmZSuri6aNWsmHO+TJ09QVFRUL+pGV1cXo0ePxtdff40DBw4gLCwMV69eRb9+/YQAQPqZVlfeF5W1IdYLAGjraMNpsBMmB09G8PpgjJs9DslJyQgcF4hb/9wCAGSmZwIAmpo1rbJ+U7OmePLoyXP38ST9CTS1NNGkWROZdB1dHTRu0hhP0svXz8vOQ0lxidz9mJqZCtsiqkxbSwslYjGijxxhwCHHtWvX4OHhgbfeeguHDh3izSMATU1NFBYWYuXKlQw45Pjzzz8ZcNQTtTpGQ2qOhwe8nJxw/8kT7DlzBmKJBMUlJVXyHVm6FIUlJUi4dw87Tp+GqLBQ6X0lpabiws2bmDtiRJUn9Bk5OQjZtQtLxoyBmYlJjY9HVebMmQMvLy/cv38fe/bsgVgsRnFxsbDc0NAQnTp1Qps2bfDee+8hNzcXkZGR8PT0RGxsrMwg4GdJZxnS0NDAiRMnYGRU3n/8rbfeQu/evbFu3TqEhYUBKL9p19PTq7IN/YqB8gUFis/K87IyMjIQEhKCJUuWwMzMrNp8ERERKCkpQVBQkNL7KCgogK6cLnVA+TFLj1f6d32oGycnJzg5OQn/9/DwgJeXF7p27YrAwEAcPXr0heV9UVkbYr0AgF0PO9j1sBP+/47rO3Aa4gT/Ef7YtnobPv/2cxQXlX+vdHR1qqyvq6eL4sLiKumVFRcVQ1tH/s+ljp6OsP2iwqJq96OjpyNsi149Qdu2YdOxY0qvl1vpuyKWSCAGEH3kCDYePQrfoUMR7O2NVqamKixp3UpKSsKQIUOUnrBDLH7aBVIikQAob7V1d3dHt27dsHz5cri5uam0rHVt0qRJOHRI/sx1z5OdnS38WyKRCAFHVFQU5syZg08//RSmDficiY+Ph5eXl8z9kCJKKt1XSs8facDRp08fLF++HM5yero0FEWFRUi5lQIDIwOZiU7quzoJNGzbtIFtmzYAgPGurhgcEgL3sDDEf/WVzPiDARUtEMMcHDDinXdgP2sWjPT1MfO99xTe187ffgMAud2mFu/YAVMjI8xSYnu1ydbWFra2tgDK+9EPHjwY7u7uiI+Ph4aGBry9vaGtrS0z7e2IESNgY2OD4OBg7N69u9ptS8dwuLu7C0EGADg6OsLKygpxcXEyeeWNwyisCPReNB5ElRYvXgxTU1PMmjWr2jzJycn48ssvsW7dOpljU5SBgUG1P2CFhYXC8Ur/ri918yxra2uMGDEC//d//wexWPzC8r6orK9KvQDls0294/oO/jj2B8RiMXT1ygOokuKqDziKi4qhqy8/wJLS1dNFaUnVcTEAUFJUImxfT1+v2v2UFJUI26JXT1xiItIr3QC+DGnAse7nn5EtEmH7vHkq2a463Lp1C8nJySrZljTguHLlCtzd3XH//n20bNlSJdtWh99//x3p6ekq2ZY04IiIiEBpaSm+/PJLlWxXHRISEpCSkqKSbUkDjrNnz2Lo0KHIyMgQZtxsaLat2oaD28vvBzed2gSzVtU/jK1P6iTQeJaXkxP81q9HUmoqOlUEIM/q0KoV3mrfHjt/+02pQGPXb7+hk4UFHJ552n/j/n3E/PoroiZPxv0nT7suFBYXo0QsRvLDhzA2NIRp48Y1OygV8PLygp+fH5KSkqCjo4OjR48iJiZGJo+pqSn69u2Ls2fPPndbrVuXR7stWrSosszc3ByZmZnC/1u1aiXTlUpKmibdVm27ceMGYmJiEBUVJdP1prCwECUlJUhOToaxsTFCQkJgYWEBFxcX4QKWllb+joJHjx4hOTkZbdu2rdKiJdWqVSuIxWKkp6fLdBMqLi5GRkaGcLympqbQ09OrF3VTHUtLSxQXF0MkEgndlqor74vK+irVCwA0b9UcpSWlKCooQlPz8q5MmY8yq+TLfJQpdGuqjqm5KSRiCbIysmS6T5UUlyA3Kxem5uXrG5kYQUdXR+5+pN2zpHnp1eJka4uEe/eUXi+3oAAlYjFKKz3B19LUhKaGBvyGDkXwmDGqLGad69ChA9q1a1ejFo2MZ7qPaWpqQiKRCC0aDTnIAID+/fsjpwazXmZnZ6OkpEQIvIDyutHT0xNaNBoyOzs7tGnTpkYtGpXvbYDyiXbEYrHQotFQgwwAyBc9/Q4V5ivf40dd1BJoFFScPNkv+OEpKCpCkZzZdaoT/++/uPngAZZ9+GGVZakZGZBIJPCPiYH/MzfvAGA1ZQpmu7sjasoUhfenatLuJtnZ2UIUXrn5WKqkpETurEOVOTg4ACh/qdqz7t+/L7SkAED37t0RWzGAvvLNeXx8PAwNDdGxY0flD6YGUlNTyz8jf3/4+/tXWW5lZYXZs2fj7t27uHnzJtq3b18lj3TGpMzMTDRp0kTufrpXvFvl/PnzMtMMnz9/HhKJRFiuqamJLl26yAy4loqPj0f79u3RWI2BKQDcvn0b+vr6MDIygr29PbS1tXH+/HmMqXRzUlxcjMuXL8ukyfMq1QsAPLz3ELp6utA31McbNm9AS1sLN67dQN9hfYU8JcUl+C/hP/QZVv20uABgZVs+PfTNazfR0/npIPib125CIpGgvV35uaipqYk3Or6Bm9duVtlG0pUktLRsKTPrFb06wsePR/h45acuDvjuO3xd0WpdOcBY5OUFCzW8j0bVOnbsiP/++0/p9dLS0oSHJ9IAo0uXLggLC4Obm5tMb4iGatOmTTVa7+OPPxamndfU1ISuri7mzp2LefPmoXnz5qosolq88847uFeDoP3KlSvCdUoaYDg6OiIsLAwuLi6qLSQprFYHg6dXmqpVqqS0FNtOnYKBri7etLREqViMTDlvLT6XlISrd+6g5zMtE4kpKbj76JHc/e36/XcAwIdyuk3Zv/EGfggKqvKnc9u2aGtmhh+CgjDp3XdrcJTKk9dUWlJSgm3btsHAwABvvvkmrK2toampid27d8sMYkpJSUFsbCzeqjRIvqSkBImJiTJPmDt16oRu3brhwIEDMgPEfv31V9y7dw/vVjpWLy8vPHz4EP/3f/8npD1+/Bh79+6Fu7u73L74tcHe3h4//PBDlT+dO3dG27Zt8cMPP2DSpEkICwurkkc6cHzBggX44YcfhJf85efnIzExUaYOXF1dYWpqiujoaJn9R0dHw9DQUKbfr5eXF/766y+Zm+p///0XJ0+ehLe3d21Wh4xHcs75K1eu4KeffsLgwYOhqakJExMTDBo0CDt27JCZ9Wn79u3Iy8uTKe+rUi8AkP2kaneV/xL/w7lT59C9T3doamqiUeNG6Na7G3776TeZWZ9O/XQKBfkFMu/fKCooQsrtFORkPn3S2NWxKxqbNMaR/yf70sMj3x+BnoGeTPDRZ0gf3Lh6Q2bWq5TbKfg7/u/nvueDXl+lYjF0tLQwbdgw/Pftt1jj5/dKBBmq0qVLFxw8eBCXLl3Ce++990oEGS9LIpHAwMAACxcuxL179xAeHv5KBBmq4ujoiFOnTuHMmTMMMtSsVls0/NavR05+Pvp37gyLZs2QlpmJnb/9hsSUFKyaOBFGBgbIysuD5cSJGNu3Lzq3bYtG+vq4mpyMzSdOwKRRIywZO1Zmm3bTp8PZ3h6nn3khmFgsxu7YWDh26oQOcmbDaW5sjJGOjlXSo376CQDkLqstfn5+yMnJQf/+/WFhYYG0tDTs3LkTiYmJWLVqFYyMjGBkZISJEyfi22+/xcCBA+Hp6Ync3FysX78eBQUFCAwMFLaXmpoKOzs7+Pj4YMuWLUJ6ZGQk3n33XfTt2xd+fn7Izs7G6tWr0bFjR0yr9JJELy8vODo6YsKECfjnn3+EN4OLxWJ8/vnndVYvzZs3x8iRI6ukS9+lIW+ZlLT1olevXjL5zp07hwEDBmDp0qX47LPPAJSPHwgNDcWMGTPg7e2NIUOGIDY2Fjt27MDy5ctlBtFNnz4d//vf/+Dm5oaAgADo6Ohg9erVaNGiBebPn/+SR6y4sWPHwsDAAE5OTjA3N8c///yDmJgYGBoaIiIiQsi3fPlyODk5wdnZGb6+vkhJScGqVaswePBgDK14QearVC8AsHLuSujp68H2LVuYmJrg3q17+GXPL9DT14PPfB8h38dzPsaCDxYgaHwQhowZgoy0DPy4+Ue81ectOPRzEPIl/Z2EYJ9gvD/jfXw4q7x1VE9fDx/N/ggblm1AxOwI9OjbA9fPX8fpn05j3JxxaNzkaQvOsA+G4Ze9v2DZ1GUYNXEUtLS1cGDLATRp1gQjJ4yss3qhhsG7Tx/o6+pi2rBhDC4qMTc3x6JFi9CnT59XpgVDVXx8fGBjY4Pp06czuKjE1tYWc+fOhYeHB4OLeqRWA42xffti07FjiD5yBBm5uWhsYACHDh2wwscHHu+8AwAw1NPD5HffxamrV7EvLg4FxcVobWqKD/r3x+IxY9BOzhgDeY5fuYKHWVkNoj/r2LFjsWnTJkRHRyMjIwONGzeGg4MDVqxYAQ8PDyFfdHQ0unXrhk2bNgmBRa9evbBt2zb079//hfsZMGAAjh49iiVLliAoKAiGhoYYOXIkVq5cKTOIWktLC4cPH8ann36Kb775BgUFBejVqxe2bNmCTp06qb4C6oHp06dDR0cHq1atwk8//QRLS0tERkZWeT9J48aNcfr0acydOxdhYWGQSCRwcXFBZGTkc2fFUrWRI0di586dWL16NXJycmBmZgZPT08sXbpUZvaxHj164Pjx41i4cCHmzp2Lxo0bY9KkSfjiiy8U2k9DqxcAcBzoiN8O/YYDmw8gX5QPk6Ym6P1ub7w/432ZmTk6dO6A0M2h2PLVFmz6YhMMGhlg0OhBCr+pe/iHw6GlrYUfN/+IcyfPoXmr5pgUOAke4z1k8hkaGSJ8Wzi+/eJb7InegzJJGezftsekwEkwMVX/bHdUv7zTqRPeeUV/Z1+Gpqamwr9br5tBgwZh0KBB6i5GvaOnp4fVq1eruxj0DI0yBScXlj5NSN++vV5MDVsfXLx2DQ5BQbhw+jR6NOAp02rDxYsX4eDggAsXLqBHjx7qLk69wXqp3m9//AYXJxeEfx8O++726i6O2mU/ycbHTh8DAOeAfw5em6rital6/A2Wj/VSvfpybfo66Guc+L8TAIB1P6+DZQdLtZRD2WtTnbywj4iIiIiIXi8MNIiIiIiISOUYaBARERERkcox0CAiIiIiIpVjoEFERERERCrHQIOIiIiIiFSOgQYREREREakcAw0iIiIiIlK5Wn0zOBERERER1UxxUTHKJGWQiCVP0wqLUVRQBG1dbWhpaamxdC/GQIOIiIiIqJ659tc1BI8PrvIG7rmj5wIAbLrYYNXeVeoomsLYdYqIiIiIqJ6xaGcBLe3qWyysbK3qsDQ1w0CDiIiIiKieaWrWFMM/HA5Nzaq365qamhgzdYwaSqUcBhpERERERPXQ6Mmjoakle7uuqaWJQaMHwdzCXE2lUhwDDSIiIiKiekhuq0YZGkRrBsBAg4iIiIio3qrcqtGQWjMABhpERERERPWWtFUDQINqzQAYaBARERER1WujJ4+Grp5ug2rNAPgeDSIiIiKieq2pWVOsP7wepuam6i6KUhhoEBERERHVcw2pJUOKXaeIiIiIiEjlGGgQEREREZHKMdAgIiIiIiKVY6BBREREREQqx0CDiIiIiIhUjoEGERERERGpHAMNIiIiIiJSOQYaRERERESkcgw0iIiIiIhI5RhoEBERERGRyjHQICIiIiIilWOgQUREREREKqet7Ap/JySgqZFRbZSlwUlITi7/+/JltZajPkpISir/OyFBzSWpX6T1wXqpKunf8nPm33//RWFpoZpLo36iHJG6i9Cg8Nr0FK9N1eO1ST5em6rHa5MsZa9NGmVlZWUKZdTQqFGBiIjo5Sj4M/1a4rWJiEg9FLk2Kd2icfzHH9G0SZOalOeVk3D5MsbNmYMdvr6wa9dO3cWpVxKSkzEuJgY7YmJg5+Cg7uLUGwkJCRg3bhx27NgBOzs7dRenXrnw9wX4TvCFzzIfvNHhDXUXR+1EOSKsmrZK3cVoMHhteorXpurx2iQfr03V47VJlrLXJqUDja5OTjAzM1N2tVeaXbt26GFvr+5i1Et2HTuiR48e6i5GvWNnZ8d6eUZuUS4AoFPHTrDvzu9T9pNsdRehQeG1qSpem6rHa5N8vDZVxWuTLGWvTRwMTkREREREKsdAg4iIiIiIVI6BBhERERERqRwDDSIiIiIiUjkGGkREREREpHIMNIiIiIiISOUYaBARERERkcox0CAiIiIiIpVjoEFERERERCrHQIOIiIiIiFSOgQYREREREakcAw0iIiIionouPTUdpSWl6i6GUhhoEBERERHVY5mPMjF9+HRsWLZB3UVRCgMNIiIiIqJ6bP+3+1FcVIzj+48jPTVd3cVRGAMNIiIiIqJ6KvNRJg7vOlz+Hw1gz4Y96i2QEhhoEBERERHVU/u/3Q+JWAIAkIglDapVg4EGEREREVE9JG3NkEgkTxMbUKsGAw0iIiIionqocmuGVENq1WCgQURERERUz8htzaggkUgaRKsGAw0iIiIionomNTkV4lJxtcv/S/yvDktTM9rqLgAREREREcmy72WPvZf3okxShvWfrcepA6cAAJH7I9GmfRto69b/2/j6X0IiIiIioteQrp4uAEBT62knJF19XegZ6KmrSEph1ykiIiIiIlI5BhpERERERKRyDDSIiIiIiEjlGGgQEREREZHKMdAgIiIiIiKVq9VA4/r16/D29kb79u1haGiI5s2bo3///jh48KCQRyKRYMuWLfDw8IClpSUaNWoEe3t7hIWFobCw8IX7SE5OhoaGRrV/pkyZUu26y5cvh4aGBuzt7VVyvIq6fvcuvCMi0H7KFBh6eaH5Rx+h/6JFOHjuXJW8e86cgWNAAJp88AGaffQRnAMD8fNffym8r9z8fCzYvBlWkydDz9MTFp98Aq+ICOQXFcnky8rLg+/atTAbNw6NvL0xIDgYF2/deuljfVmKfEZZWVkwNzeHhoYG9u3bp/C2N23aBDs7O+jr68PGxgZr1qyRmy81NRVjxoxBkyZNYGxsjBEjRuD27dtKH8vLOH36dLXn+J9//imTNy4uDn379oWhoSFatmwJf39/5OXlKbyvhlQvAHA1/io8bD3k/km8nCiTN+FiAhZ+uBBe3b0wvu94xITFoEBUoPC+ft33K6YPn47RXUfDb4gfDm0/JDdfxsMMrJizAh/0+gBjHcYibHoY0u6lvdRxEhERNTS1Or3tnTt3kJubCx8fH7Ru3Rr5+fnYv38/PDw8sHHjRvj6+iI/Px8TJkyAo6Mjpk6dCnNzc/zxxx9YunQpTpw4gZMnT0JDQ6PafZiZmWH79u1V0o8ePYqdO3di8ODBctdLSUlBeHg4GjVqpLLjVdSd9HTkFhTAx9UVrU1NkV9UhP1xcfAIC8PG6dPhO3QoAGDNoUPwj4mBW8+eiBg4EIXFxdhy8iTeCw3F/kWL4Onk9Nz9ZItEcA4KQsrjx/AdMgTWrVrhUU4OYq9fR1FJCQz1yqdGk0gkcFu2DFeSk/HpqFFobmyM9YcPwyUoCBciI2HTunWt14k8in5GISEhyM/PV2rbGzduxNSpUzF69GjMmzcPsbGx8Pf3R35+PhYuXCjky8vLw4ABA5CdnY2goCDo6OggMjISzs7OuHz5Mpo1a1ajY6spf39/9OrVSybN2tpa+Pfly5cxcOBA2NnZYfXq1UhJScFXX32FGzdu4MiRIy/cfkOtFwBw/9gd1l2sZdJav/H03L2dcBtLJixBmw5tMGnRJGSkZeCH737A/Tv38dn/Pnvh9o9+fxTrP1sPp8FOGPHJCPxz4R/ELI9BUWERRk8ZLeQrEBUgeHwwRHkiePl5QVtbGwe2HkDQx0GI+iEKxk2NVXbMRERE9VmtBhrDhw/H8OHDZdJmzpwJBwcHrF69Gr6+vtDV1cXZs2fhVOmmecqUKWjXrp0QbAwaNKjafTRq1Ajjxo2rkr5lyxYYGxvD3d1d7noBAQFwdHSEWCzG48ePa3iENTO8Z08M79lTJm2mmxsc5s3D6gMHZAKNXjY2OLhkiRBsTXz3XVhMmICtJ0++MNAI3LYNd9LTcTEyElYtWwrpC0ePlsm3Ly4OcYmJ2LtwIbz69AEAjOnbFx2nTsXSXbuwKyDgpY+5JhT5jK5du4bo6GiEhIQgJCREoe0WFBQgODgYbm5uQgvIlClTIJFIEBoaCl9fXzRt2hQAsH79ety4cQPnzp0TbvCHDRsGe3t7rFq1CuHh4So4UsX169cPXl5e1S4PCgpC06ZNcfr0aRgbl9/QtmvXDlOmTMGvv/5abeANNOx6AYA3Hd5En6F9ql2+PXI7jIyNEL4tHIZGhgAAcwtzrF2yFpfOXMJbfd+qdt2iwiJsj9qOns49seibRQCAIWOGQCKRYHf0bgwZMwRGJkYAgMP/7zDu37mPVXtXwaaLDQDAob8DZrrPxI+bf8T4eeNVdchERET1Wp2P0dDS0oKlpSWysrIAALq6ujJBhtSoUaMAAAkJCUrv48GDBzh16hQ8PT2hr69fZfnvv/+Offv2ISoqSult1xYtLS1YNm+OLJFISMvJz4e5iYlMi46xoSGM9PVhoPf8F7Vk5eVh84kT8B0yBFYtW6K4pARFJSVy8+47exYtmjSBZ+/eQpqZiQnG9O2LA/Hx1a5XmxT9jGbPno1Ro0ahX79+Cm/71KlTyMjIwPTp02XSZ8yYAZFIhJ9//llI27dvH3r16iXTimBra4uBAwdiz549Cu9TlXJzc1FaWlolPScnB8eOHcO4ceOEIAMAxo8fDyMjoxeWt6HXCwDk5+VDXCqWm3457jKcPZyFIAMABowYAANDA5w5eua5270afxW5WbkY/qHsgxO3D91QmF+Iv3572p0x7pc42HSxEYIMAGjTvg26OXZ74X6IiIheJXUSaIhEIjx+/Bi3bt1CZGQkjhw5goEDBz53nbS08v7MzZs3V3p/33//PSQSCT766KMqy8RiMWbNmoXJkyejS5cuSm9blUSFhXick4NbDx4g8sABHLlwAQO7dROWu3TpgqMXL2LNoUNIfvgQiSkpmLFhA7JFIsyupqVG6kxCAgqLi2HdqhW8IiJg6O0NAy8v9FmwAJef6Ud/6fZt9OjQAZqasqfD2zY2yC8qQlJqquoOWgGKfkZ79+5FXFwcVq5cqdT2L126BADo+UyrkoODAzQ1NYXlEokEf//9d5V8APD222/j1q1byM3NVWrfL2vChAkwNjaGvr4+BgwYgPPnzwvLrl69itLS0irl1dXVRffu3YXjqk5DrhcA+CboG7zf832M7jYaweODcePqDWFZclIyxKVi2NjbyKyjo6sDKzsr3P7n+WNLbieUL7e2l+2a1aFz+fdGur5EIkHyv8lV8gGATVcbpN1NQ36ect386NUWHx+P4OBgpNbx72x9J5FIELh1Kw6eO4eysjJ1F6deOX78OJYtW1bnvTHqu6KiIsybNw+nTp3iOVOP1EmgMX/+fJiZmcHa2hoBAQEYNWoU1q5d+9x1Vq5cCWNjYwwbNkzp/e3cuROtWrWCq6trlWUbNmzAnTt3EBoaqvR2VW3+d9/BbNw4WPv5IWDzZoxydMRaPz9h+TdTpsClSxf4x8TAasoU2E2fjj1nzuBEWBh629o+d9s37t8HUN596t7jx9g2dy7W+fnhVloaXBcvxoMnT4S8DzIz0aqiS0xlrUxNAQD3K+WtC4p8RgUFBQgICMDcuXPRrl07pbb/4MEDaGlpwdzcXCZdV1cXzZo1w/2Kunvy5AmKiorQqlWrKtuQpknz1jZdXV2MHj0aX3/9NQ4cOICwsDBcvXoV/fr1EwKABw8eyJTt2fK+qKwNsV4AQFtHG06DnTA5eDKC1wdj3OxxSE5KRuC4QNz6p3xCg8z0TABAU7Oq53lTs6Z48uj55/iT9CfQ1NJEk2ZNZNJ1dHXQuEljPEkvXz8vOw8lxSVy92NqZipsi0hq7969CA8Ph5WVFWbNmsWAo0J6djYi9u+HR1gYus+ezYCjkq1bt2Lp0qWwtLREYGAgA44KiYmJiIyMhKurK/r168eAo56ok0Bjzpw5OHbsGLZu3Yphw4ZBLBajuLi42vzh4eE4fvw4IiIi0KRJE6X2lZSUhAsXLuD999+v8oQ+IyMDISEhWLJkCczMzGpyKCo1x8MDx5Ytw9Y5czCsRw+IJRIUV+qmZKinh04WFvBxdcXehQvxnb8/WjVtCs8vvsDNF9zI5RWUz6SjoaGBE6Gh+NDZGdOGD8ePQUHIzMvDusOHhbwFxcXQ09Gpsg39irSC53xWqqboZxQREYGSkhIEBQUpvY+CggLo6urKXaavr4+CirqT/q0np5uatEueNE9tc3Jywr59+zBx4kR4eHhg0aJF+PPPP6GhoYHAwECFyvuisjbEegEAux52WPTNIrw7+l284/oOvHy98OXuLwENYNvqbQCA4qLyc1hHt+p5rquni+LC55/jxUXF0NaRP6RNR09H2H5RYVG1+9HR05EpC5GUtrY2SkpKEB0dzYBDjmt378IjLAxvzZmDQ3/9xZtHAJqamigsLMTKlSsZcMjx559/MuCoJ2p1MLiUra0tbCuewI8fPx6DBw+Gu7s74uPjq8wotXv3bixevBiTJk3CtGnTlN7Xzp07AUBut6nFixfD1NQUs2bNqsFRqJ5tmzawbdMGADDe1RWDQ0LgHhaG+K++goaGBrxXrIC2lhYOLlkirDPinXdgM3UqgnfswO4FC6rdtnQMh3uvXjAyMBDSHW1tYdWiBeIqjX0x0NWVOw6jsCLNoJqbz9qgyGeUnJyML7/8EuvWrYORkZHS+zAwMKg20C0sLIRBRX1J/y56Zipgab7KedTB2toaI0aMwP/93/9BLBa/sLwvKuurUi9A+WxT77i+gz+O/QGxWAxdvfJzuKS46nleXFQMXf3nn+O6erooLak6LgYASopKhO3r6etVu5+SohJhW/TqCQoKwqZNm5Rer3I3Q7FYDLFYjOjoaGFmxuDgYLmthw1FUmoqhixdWmVK9RcRSyTCvyUV/7565w7cQ0PRzcoKy8eNg9szM/A1NJMmTcKhQ/KnyH6e7Oxs4d8SiUQIOKKiojBnzhx8+umnMK3okdAQxcfHw8vL67kPpOUpqXQfIxaXj9WTBhx9+vTB8uXL4ezsrNKy0ovVSaDxLC8vL/j5+SEpKQmdOnUS0o8dO4bx48fDzc0NGzZsqNG2d+3ahU6dOsHBwUEm/caNG4iJiUFUVJRMt47CwkKUlJQgOTkZxsbGav1yejk5wW/9eiSlpkJHWxtHL15EzIwZMnlMGzdGXzs7nH3BIPnWFcfRQk6LkLmJCTIrvVehVdOmeJCZWSWftHtV6zqqE0U/o5CQEFhYWMDFxQXJyckAno7pefToEZKTk9G2bdsqLVpSrVq1glgsRnp6ukw3oeLiYmRkZKB1xXS+pqam0NPTE7okVSZNa62mqX+lLC0tUVxcDJFIJNyMVFfeF5X1VaoXAGjeqjlKS0pRVFCEpublXZkyH1U9zzMfZQrdmqpjam4KiViCrIwsme5TJcUlyM3Khal5+fpGJkbQ0dWRux9p9yxpXnq1xMXFIT09XSXbkgYc69atQ3Z2ttwp3BuKW2lpSFZRvUgDjiv//Qf3sDDc37IFLeV0+20ofv/9d5WdM9KAIyIiAqWlpfjyyy9Vsl11SEhIQEpKikq2JQ04zp49i6FDhyIjIwOGhoYvWKt+Mmz0tNz6hlUnOqqv1BJoSLtVVI7K4+PjMWrUKPTs2RN79uyBtrbyRYuPj8fNmzexbNmyKstSU1MhkUjg7+8Pf3//KsutrKwwe/Zstc5EJe2ilJ2fLzzNqfxUR6pELEapuOrMOpU5dOgAAEiVM77i/pMnQksKAHS3skLsP/9AIpHI3JzHJyXBUE8PHS0slD+YGlD0M7p79y5u3ryJ9u3bV8kjnTEpMzOz2m533bt3BwCcP39eZvrl8+fPQyKRCMs1NTXRpUsXmQHXUvHx8Wjfvj0aN26s5FGq1u3bt6Gvrw8jIyPY29tDW1sb58+fx5gxY4Q8xcXFuHz5skyaPK9SvQDAw3sPoaunC31Dfbxh8wa0tLVw49oN9B3WV8hTUlyC/xL+Q59h1U+LCwBWtlYAgJvXbqKn89NB8Dev3YREIkF7u/JzUVNTE290fAM3r92sso2kK0loadlSZtYrenU4OTnVaJbE3NxclJSUyMwkp6WlBU1NTfj5+SE4OFiVxaxzHVq2RDtz8xq1aGQ8M6mEpqYmJBKJ0KLRkIMMAOjfvz9ycnKUXi87OxslJSVC4AWU142enp7QotGQ2dnZoU2bNjVq0ch85qGplpYWxGKx0KLRUIMMABg/fzxcR7rCwMgAZq3U3/1fUbUaaDz7ZBQoPxG2bdsGAwMDvPnmmwDKo1c3Nze0a9cOhw4dem63i8TERBgaGqJt27ZVlu3atQsA8OGHH1ZZZm9vjx9++KFK+uLFi5Gbm4uvv/4aHSpuzmtbelYWzJ+5CS4pLcW2U6dgoKuLNy0tUVBcDE1NTew+cwZ+Q4cKXcxSHj9G7D//oK+dncy6t9LSYGJoKAzg7tSmDbpZWeFAfDwe5+SgecV0p79euoR7jx9j1nvvCet79emDfXFx+L8//hDeo/E4Jwd7z56F+9tvyx2/URsU/Yyys7Or9EW9du0alixZggULFqB3797CS/7y8/Nx9+5dNG/eXJjBzNXVFaampoiOjpa5oY6OjoahoSHc3NyENC8vLyxatAjnz58XZln6999/cfLkSQTU4ftFHj16VGXMypUrV/DTTz9h2LBh0NTUhImJCQYNGoQdO3ZgyZIlws3+9u3bkZeXB29vb2HdV6VeACD7STZMTE1k0v5L/A/nTp1Dj349oKmpiUaNG6Fb72747affMHbaWOFm/9RPp1CQXyDz/o2igiI8evAIxk2NhZfrdXXsisYmjXHk/x2RCTSOfH8EegZ6Mml9hvTB1lVbcePqDWGK25TbKfg7/m+Mmjiq1uqB1Cs8PLxG748JCAjA119/DUA2wFi0aBEs6ughT23qaGGB/779Vun10jIz0crHB8DTAKPLG28gbNw4uPXs+dwX+TYUNelqBwAff/yxcL+jqakJXV1dzJ07F/PmzavRTJ31zTvvvIN79+4pvd6VK1eEB2LSAMPR0RFhYWFwcXFRbSHVQE9fDx061819qirVaqDh5+eHnJwc9O/fHxYWFkhLS8POnTuRmJiIVatWwcjICLm5uRgyZAgyMzPx6aefyszVDwAdOnRA70rvd7Czs4OzszNOnz4tk08sFmP37t1wdHSUGzA0b94cI0eOrJIubcGQt6y2+K1fj5z8fPTv3BkWzZohLTMTO3/7DYkpKVg1cSKMDAxgZGCAiYMG4dtff8XAxYvh2bs3cgsKsP7IERQUFSGw0kvbUjMyYDd9OnxcXbFlzhwhPXLSJLwbEoK+CxfCb+hQZItEWH3gADpaWGBapdm8vJyc4NipEyZ88w3+uXdPeDO4WCLB5x98UGf18jKfkbT1olevXjL5zp07hwEDBmDp0qX47LPPAJSPHwgNDcWMGTPg7e2NIUOGIDY2Fjt27MDy5ctlus9Nnz4d//vf/+Dm5oaAgADo6Ohg9erVaNGiBebPn/+SR6y4sWPHwsDAAE5OTjA3N8c///yDmJgYGBoaIiIiQsi3fPlyODk5wdnZGb6+vkhJScGqVaswePBgDK14EeSrVC8AsHLuSujp68H2LVuYmJrg3q17+GXPL9DT14PPfB8h38dzPsaCDxYgaHwQhowZgoy0DPy4+Ue81ectOPR72tUy6e8kBPsE4/0Z7+PDWeUPLfT09fDR7I+wYdkGRMyOQI++PXD9/HWc/uk0xs0Zh8ZNnrbgDPtgGH7Z+wuWTV2GURNHQUtbCwe2HECTZk0wcsLIOqsXajhKS0uho6PzSgUYqvSqBRiqIJFIYGBggDlz5rwyAYYqvUoBRkNXq4HG2LFjsWnTJkRHRyMjIwONGzeGg4MDVqxYAQ8PDwDlswxJI9dFixZV2YaPj49MoFGd48eP4+HDhw2imXls377YdOwYoo8cQUZuLhobGMChQwes8PGBxzvvCPmip01Dt3btsOnYMQRW9NHtZW2NbXPmoL+9/Qv3M6BrVxz97DMs2bkTQdu3w1BPDyMdHbHyk09kBohraWnh8NKl+HTzZnxz8CAKiovRy8YGW2bPRqdKXaxeJdOnT4eOjg5WrVqFn376CZaWloiMjMTs2bNl8jVu3BinT5/G3LlzERYWBolEAhcXF0RGRtbpzGUjR47Ezp07sXr1auTk5MDMzAyenp5YunQprK2fvrOhR48eOH78OBYuXIi5c+eicePGmDRpEr744guF9tPQ6gUAHAc64rdDv+HA5gPIF+XDpKkJer/bG+/PeB+t33g6VqRD5w4I3RyKLV9twaYvNsGgkQEGjR6k8Ju6h384HFraWvhx8484d/IcmrdqjkmBk+Ax3kMmn6GRIcK3hePbL77Fnug9KJOUwf5te0wKnFSl5YXI29sb+vr6mDZtGgOMSsxNTLBo9Gj0efNNBhjP8PHxgY2NDaZPn84AoxJbW1vMnTsXHh4eDDDqEY0yBef8kn7J09PT68XUsPXBxd9+g4OLCy6Eh6OHAjf+r5OL167BISgIF06fRg/O8iC4ePEiHBwccOHCBfTo0UPdxalXfvvjN7g4uSD8+3DYd+f3KftJNj52+hgAODXjc/DaVBWvTdXjtUk+Xpuqx2uTLGWvTXXyHg0iIiIiInq9MNAgIiIiIiKVY6BBREREREQqx0CDiIiIiIhUjoEGERERERGpHAMNIiIiIiJSOQYaRERERESkcgw0iIiIiIhI5RhoEBERERGRyjHQICIiIiIilWOgQUREREREKsdAg4iIiIiIVI6BBhERERERqRwDDSIiIiIiUjkGGkREREREpHIMNIiIiIiISOUYaBARERERkcox0CAiIiIiIpVjoEFERERERCrHQIOIiIiIiFSOgQYREREREakcAw0iIiIiIlI5BhpERERERKRyDDSIiIiIiEjlGGgQEREREZHKMdAgIiIiIiKVY6BBREREREQqx0CDiIiIiIhUjoEGERERERGpHAMNIiIiIiJSOQYaRERERESkcgw0iIiIiIhI5bSVXeHvv/9G06ZNa6MsDU5CUlL538nJ6i1IPSStk4TLl9VajvpGOGcSEtRckvon6d/yukm+mazegtQTeTl56i5Cg8Jr01O8NlWP1yb5eG2qHq9NspS9NmmUlZWVKZRRQ6NGBSIiopej4M/0a4nXJiIi9VDk2qR0i8bx48f51KhCQkICxo0bhx0xMbDr2FHdxalXEi5fxrg5c7DD1xd27dqpuzj1RkJyMsbFxJSfMw4O6i5OvSL9PsVsjkHHTvw+ZWVmYaTbSHUXo8HgtekpXpuqx2uTfLw2VY/XJlnKXpuUDjS6du0KMzMzZVd7pdk5OKBHjx7qLka9ZNeuHXrY26u7GPWOXceOPGeq4dCV3ycAePTokbqL0KDw2lQVr03V47VJPl6bqsdrUzllr00cDE5ERERERCrHQIOIiIiIiFSOgQYREREREakcAw0iIiIiIlI5BhpERERERKRyDDSIiIiIiEjlGGgQEREREZHKMdAgIiIiIiKVY6BBREREREQqx0CDiIiIiIhUjoEGERERERGpHAMNIiKil5Seno7s7Gx1F6PeKSsrw82bN9VdjHopNSMD+UVF6i5GvSMWi3H79m11F6NeunPnDkpKStRdDKUw0CAiInoJpaWleOeddzBo0CCUlZWpuzj1SnR0NGxsbBAbG6vuotQreQUF6DprFt5fuVLdRal3Pv/8c3Ts2BGJiYnqLkq9kpaWBltbW8yYMUPdRVEKAw0iIqKXsGPHDiQnJ+P8+fP49ddf1V2ceqOwsBDLli0DAISEhKi5NPXLusOH8SQvDwf/+guXbt1Sd3HqjSdPnmDVqlUQi8XCuUPlVqxYgcLCQnz33Xe4c+eOuoujMAYaRERENVRaWorPPvsMGhoa0NLSQnBwMFs1KmzatAnp6ekAgNOnT7NVo0JeQQEi9u0DAGhpamLprl1qLlH9ERkZicLCQgDA999/z1aNCmlpaVi/fr3w/+XLl6uxNMphoEFERFRDO3bswJ07d1BWVgaxWIwLFy6wVQPlrRmhoaFC0KWlpcVWjQrrDh9Gdn4+AEAskeDgX3/hIls18OTJE6xevRoSiQRA+Tnz+eefq7lU9cOKFSsgFosBlI9haUitGgw0iIiIaqBya4aUlpYWFi9e/Nq3alRuzQDKb47YqvG0NaPy+aGlqYnP2KqBqKgooTUDKP9+7d69+7Vv1ZC2ZkgDDamG0qrBQIOIiKgGKrdmSInF4td+rMazrRlSbNWQbc2QkrZqvM5jNZ5tzZDS0tJ67cdqVG7NkGpIrRoMNIiIiJQkbc2QR0ND47Vu1di0aRMePnxYJf11b9WQ15ohpQG81mM1oqKiIBKJqqSXlpa+1mM1qmvNAMq/Tw2hVYOBBhERkZIeP36MR48eCf+v3H2qrKwMN27caHDz3avK1atXn7v82rVrdVSS+uXuo0cQVeoapFFpWRmAvxvA0+nacuXKlWqXlZWV4fr163VYmvojKSkJpaWl1S5/Xr3VFww0iIiIlNSyZUtkZmZCJBJh5cqVwlPqLVu2QCQS4dGjR9DV1VVzKdUjOjoaIpEIIpEImprltxmdO3cW0qZNm6bmEqrHm23bInfPHoj27kXAqFGQtmscXroUor17cXPjRrWWT51++OEHiEQi3Lt3T0gbPHgwRCIR8vPzMXr0aDWWTn369+8vfG8+/vhjIf3ChQsQiUSIi4tTY+kUo63uAhARETVEurq6wh8pPT09GBoaqrFU6qehoSHUgbSlR0tL67WvFwDQ09EBAOhoaQlp+jo6MNTTU1eR6gVNTU0YGhrKDAbnOVNOX18fAKCt/fSW3cDAoMHUDVs0iIiIiIhI5RhoEBERERGRyjHQICIiIiIilWOgQUREREREKsdAg4iIiIiIVI6BBhERERERqVytBhp//fUXZs6cic6dO6NRo0Zo27YtxowZg6SkpCp59+zZA0dHRzRp0gTNmjWDs7Mzfv75Z6X3eevWLejr60NDQwPnz5+XWebi4gINDQ25f3QqppyrC9evX4e3tzfat28PQ0NDNG/eHP3798fBgweFPBKJBFu2bIGHhwcsLS3RqFEj2NvbIywsTGb6t+okJydXe6waGhqYMmVKtesuX74cGhoasLe3V8nxKuP63bvwjohA+ylTYOjlheYffYT+ixbh4LlzVfLuOXMGjgEBaPLBB2j20UdwDgzEz3/9pfC+cvPzsWDzZlhNngw9T09YfPIJvCIikF9UJJMvKy8PvmvXwmzcODTy9saA4GBcvHXrpY/1ZSnyOWVlZcHc3BwaGhrYt2+fwtvetGkT7OzsoK+vDxsbG6xZs0ZuvtTUVIwZMwZNmjSBsbExRowYgdu3byt9LC/j9OnT1Z7nf/75p0zeuLg49O3bF4aGhmjZsiX8/f2Rl5en8L4aUr0QERGpW62+R2PFihU4e/YsvL290bVrV6SlpWHt2rXo0aMH/vzzT+EGac2aNfD394ebmxsiIiJQWFiILVu24L333sP+/fvh6emp8D7nzp0LbW1tFD1zswgAwcHBmDx5skyaSCTC1KlTMXjw4Jc7WCXcuXMHubm58PHxQevWrZGfn4/9+/fDw8MDGzduhK+vL/Lz8zFhwgQ4Ojpi6tSpMDc3xx9//IGlS5fixIkTOHnypMybaJ9lZmaG7du3V0k/evQodu7cWe3xpqSkIDw8HI0aNVLZ8SrjTno6cgsK4OPqitampsgvKsL+uDh4hIVh4/Tp8B06FACw5tAh+MfEwK1nT0QMHIjC4mJsOXkS74WGYv+iRfB0cnrufrJFIjgHBSHl8WP4DhkC61at8CgnB7HXr6OopESY01wikcBt2TJcSU7Gp6NGobmxMdYfPgyXoCBciIyETevWtV4n8ij6OYWEhCA/P1+pbW/cuBFTp07F6NGjMW/ePMTGxsLf3x/5+flYuHChkC8vLw8DBgxAdnY2goKCoKOjg8jISDg7O+Py5cto1qxZjY6tpvz9/dGrVy+ZNGtra+Hfly9fxsCBA2FnZ4fVq1cjJSUFX331FW7cuIEjR468cPsNtV6IiIjUpVYDjXnz5mHXrl0yLzMaO3YsunTpgoiICOzYsQNAeaDRq1cvHDx4ULh5njhxIiwsLLB161aFA41ffvkFv/zyCxYsWICwsLAqy999990qadIyfPTRR0ofX00NHz4cw4cPl0mbOXMmHBwcsHr1avj6+kJXVxdnz56FU6Ub5ilTpqBdu3ZCsDFo0KBq99GoUSOMGzeuSvqWLVtgbGwMd3d3uesFBATA0dERYrEYjx8/ruER1tzwnj0xvGdPmbSZbm5wmDcPqw8ckAk0etnY4OCSJU/PmXffhcWECdh68uQLA43AbdtwJz0dFyMjYdWypZC+8Jm3j+6Li0NcYiL2LlwIrz59AABj+vZFx6lTsXTXLuwKCHjpY64JRT6na9euITo6GiEhIQgJCVFouwUFBQgODoabm5vQAjJlyhRIJBKEhobC19cXTZs2BQCsX78eN27cwLlz54Qb/GHDhsHe3h6rVq1CeHi4Co5Ucf369YOXl1e1y4OCgtC0aVOcPn0axsbGAIB27dphypQp+PXXX5/7sKEh1wsREZG61GrXKScnJ5kgAwBsbGzQuXNnJCQkCGk5OTlC9w4pY2NjGBkZwcDAQKF9lZSUYPbs2Zg9ezY6dOigcBl37dqFRo0aYcSIEQqvUxu0tLRgaWmJrKwsAOVvnHWSc7M8atQoAJCpP0U9ePAAp06dgqenp/Cmycp+//137Nu3D1FRUUpvuzZpaWnBsnlzZIlEQlpOfj7MTUxkzxlDQxjp68PgBW9YzcrLw+YTJ+A7ZAisWrZEcUkJikpK5Obdd/YsWjRpAs/evYU0MxMTjOnbFwfi46tdrzYp+jnNnj0bo0aNQr9+/RTe9qlTp5CRkYHp06fLpM+YMQMikUimO+O+ffvQq1cvmVYEW1tbDBw4EHv27FF4n6qUm5uL0tLSKuk5OTk4duwYxo0bJwQZADB+/HgYGRm9sLwNvV6o/oqPj0dwcDBSU1PVXZR6RSKRIDAwEAcPHkRZWZm6i1OvHL98Gcu+/x6Pc3LUXZR6paioCPPmzcOpU6d4ztQjdT4YvKysDA8fPkTz5s2FNBcXFxw9ehRr1qxBcnIyEhMTMWPGDGRnZ2P27NkKbTcqKgqZmZlYvHixwmV59OgRjh07hpEjR6qlq5BIJMLjx49x69YtREZG4siRIxg4cOBz10lLSwMAmfpT1Pfffw+JRCK39UYsFmPWrFmYPHkyunTpovS2VU1UWIjHOTm49eABIg8cwJELFzCwWzdhuUuXLjh68SLWHDqE5IcPkZiSghkbNiBbJMLsalprpM4kJKCwuBjWrVrBKyICht7eMPDyQp8FC3D5mX70l27fRo8OHaCpKftVedvGBvlFRUiq45sDRT+nvXv3Ii4uDitXrlRq+5cuXQIA9HymVcnBwQGamprCcolEgr///rtKPgB4++23cevWLeTm5iq175c1YcIEGBsbQ19fHwMGDJAZo3X16lWUlpZWKa+uri66d+8uHFd1GnK9UP22d+9ehIeHw8rKCrNmzWLAUSE9PR0RERHw8PBA9+7dGXBUsvXkSSzdtQuWEycicOtWBhwVEhMTERkZCVdXV/Tr148BRz1R54HGzp07kZqairFjxwpp33zzDVxcXODv7w8rKyvY2dlhz549OHHiBHpXepJcnbS0NISGhiI0NFTmaeWL7N69G6WlpXXabaqy+fPnw8zMDNbW1ggICMCoUaOwdu3a566zcuVKGBsbY9iwYUrvb+fOnWjVqhVcXV2rLNuwYQPu3LmD0NBQpbdbG+Z/9x3Mxo2DtZ8fAjZvxihHR6z18xOWfzNlCly6dIF/TAyspkyB3fTp2HPmDE6EhaG3re1zt33j/n0A5d2n7j1+jG1z52Kdnx9upaXBdfFiPHjyRMj7IDMTrSq6xFTWytQUAHC/Ut66oMjnVFBQgICAAMydOxft2rVTavsPHjyAlpYWzM3NZdJ1dXXRrFkz3K+ouydPnqCoqAitWrWqsg1pmjRvbdPV1cXo0aPx9ddf48CBAwgLC8PVq1fRr18/IQB48OCBTNmeLe+LytoQ64UaDm1tbZSUlCA6OpoBhxzXrl1jwPEMTQ0NFBYXY+UPPzDgkOPPP/9kwFFP1OoYjWdJWyp69+4NHx8fId3Q0BCdOnVCmzZt8N577yE3NxeRkZHw9PREbGyszIBOeRYuXIj27dtXGej9Irt27YKZmZncsRt1Yc6cOfDy8sL9+/exZ88eiMViFBcXV5s/PDwcx48fx/r169GkSROl9pWUlIQLFy5g7ty5VZ7OZ2RkICQkBEuWLIGZmVlNDkXl5nh4wMvJCfefPMGeM2cglkhQXKmbkqGeHjpZWKBNs2Z4r1cv5BYUIPLAAXh+8QViv/gC1s8ZpJ1XUAAA0NDQwInQUBhVdM97q3179F6wAOsOH0ZYxfiWguJi6MmZkUy/Iq3gOZ+Xqin6OUVERKCkpARBQUFK76OgoKBKd0cpfX19FFTUnfRvPTnd1KTd8qR5apuTk5NMN0MPDw94eXmha9euCAwMxNGjR19Y3heVtSHWC6mHn5+fwi3xAGRauMRiMcRiMaKjo4WJQYKDg+UGrg2F9Abv+vXraNGihcLricVi4d8SiQTA04Cja9euCA8Ph5ubm2oLqyaeX3wBXW3Fb8ey8/MBDQ2grAwSiUQIOKIOHsQcd3d86ukJ08aNa7HEdePEiRNKnTMlle4RpOePNODo06cPli9fDmdnZ5WXk56vzgKNtLQ0uLm5wcTEBPv27YOWlpawzNvbG9ra2jLTu44YMQI2NjYIDg7G7t27q93un3/+ie3bt+PEiRNVbqCf5/bt2/jjjz8wc+ZMaCvxBVclW1tb2FY8fR8/fjwGDx4Md3d3xMfHV5lRavfu3Vi8eDEmTZqEadOmKb2vnTt3ApA/6H3x4sUwNTXFrFmzanAUtcO2TRvYtmkDABjv6orBISFwDwtD/FdfQUNDA94rVkBbSwsHlywR1hnxzjuwmToVwTt2YPeCBdVuWzqGw71XLyHIAABHW1tYtWiBuErjXwx0deWOwyisSDOo5uazNijyOSUnJ+PLL7/EunXrYGRkpPQ+DAwMqg12CwsLhTFT0r/lze4mnX5Z0fFVtcHa2hojRozA//3f/0EsFr+wvC8q66tSL1Q7Gle6qcvJyUHOSz5ZlgYc69atQ3Z2ttwZBBsKfX195OfnQywWIz09/aW2JQ04/v77b7i7u+P+/ftoWWkyj4akcaXfgcrjD2tKGnBE7N+PUokEX06Y8NLbVAc9PT1oampCIpGguLj4pc8ZacBx9uxZDB06FBkZGTA0NFRFUetc5d+Zmlzf1aVO7rCzs7MxbNgwZGVlITY2Fq0rPW2+ffs2jh49ipiYGJl1TE1N0bdvX5w9e/a5216wYAH69esHKysrJCcnA4AwC8+DBw9w9+5dtG3btsp6u3btAlC3s029iJeXF/z8/JCUlIROnToJ6ceOHcP48ePh5uaGDRs21Gjbu3btQqdOneDg4CCTfuPGDcTExCAqKkqmS0dhYSFKSkqQnJwMY2NjmFZ0FVIXLycn+K1fj6TUVOhoa+PoxYuImTFDJo9p48boa2eHsy8YKN+64lhayGkVMjcxQWal9yq0atoUDzIzq+STdq9qXUf1oujnFBISAgsLC7i4uAjfB+m4nkePHiE5ORlt27atNihv1aqVcENQuZtQcXExMjIyhO+uqakp9PT0hC5JlUnTWqtp6l8pS0tLFBcXQyQSCU+Eqyvvi8r6KtULqZ6Pjw/i4uLwww8/KP3gKjc3FyUlJTKTGGhpaUFTUxN+fn4IDg5WdXHr1NGjRzF27FiUlJQo9TBQLBYjIyNDJk16A9qtWzcsX768wQYZABAwahTik5IQ+88/0FaiXoDyFo0SsVgIvIDyutHV1hZaNBqqRo0aYf/+/fDz80NZWdlzp/F/VklJCTKfuV5raWlBLBYLLRoNNcgAynsr+Pj4wNjYGJaWluoujsJqPdAoLCyEu7s7kpKScPz4cbz55psyyx8+fAhAtplU6tkfX3nu3r2LO3fuwMrKqsoyDw8PmJiYCDM5VbZr1y506NABjo6OShxN7ZJ2qcjOzhbS4uPjMWrUKPTs2RN79uypUetLfHw8bt68iWXLllVZlpqaColEAn9/f/j7+1dZbmVlhdmzZ6t9JippF6Xs/HyIK35cxZV+ZKVKxGKUyjmXKnOomJUsVc74ivtPnggtKQDQ3coKsf/8A4lEInORjE9KgqGeHjpaWCh/MDWg6Od09+5d3Lx5E+3bt6+SRzpjUmZmZrVd77p37w4AOH/+vMwUzOfPn4dEIhGWa2pqokuXLlVeigmUn2/t27eXefqiDrdv34a+vj6MjIxgb28PbW1tnD9/HmPGjBHyFBcX4/LlyzJp8rxK9UKqp6WlhW+//Rbffvut0usGBATg66+/FrYjDTAWLVoEizr6falN/fr1q9G4pLS0NOEBgTTA6NKlC8LCwuDm5qbUDWh9pKOtjR9rGER+vHo1dv32GwDZAGN+xbueGrqRI0di5MiRSq935coV4bdYGmA4OjoiNDRUeGFzQ2ZgYIAePXqouxhKq9VAQywWY+zYsfjjjz9w4MABuQO7ra2toampid27d8PPz084EVJSUhAbG4u+ffsKeUtKSnDr1i2YmJgIP0AxMTFVXkh28uRJrFmzBl999ZXQNamyS5cuISEhAUsqdbupS88+FQXKj23btm0wMDAQgrGEhAS4ubmhXbt2OHTo0HO7XCQmJsLQ0PC5rTcffvhhlWX29vb44YcfqqQvXrwYubm5+Prrr5WaLvhlpWdlwfyZm+CS0lJsO3UKBrq6eNPSEgXFxeXnzJkz8Bs69Ok58/gxYv/5B33t7GTWvZWWBhNDQ2EAd6c2bdDNygoH4uPxOCdH+GH+9dIl3Hv8GLPee09Y36tPH+yLi8P//fGH8B6Nxzk52Hv2LNzfflvu+I3aoOjnlJ2dXeW9GteuXcOSJUuwYMEC9O7dW5hhLT8/H3fv3kXz5s2FWcxcXV1hamqK6OhomRvq6OhoGBoayvSJ9vLywqJFi3D+/HlhlqV///0XJ0+eREAdvl/k0aNHVcasXLlyBT/99BOGDRsGTU1NmJiYYNCgQdixYweWLFki3Oxv374deXl58Pb2FtZ9VeqFGo7S0lLo6Oi8UgGGKr1KAYaqSMrKoK+r+0oFGKr0KgUYDV2tBhrz58/HTz/9BHd3dzx58kR4OZ7UuHHjYGZmhokTJ+Lbb7/FwIED4enpidzcXKxfvx4FBQUIDAwU8qempsLOzg4+Pj7YsmULAMh9yZa0BcPZ2VnuNJPPG69QF/z8/JCTk4P+/fvDwsICaWlp2LlzJxITE7Fq1SoYGRkhNzcXQ4YMQWZmJj799FOZefoBoEOHDjKBm52dHZydnXH69GmZfGKxGLt374ajo6PcgKF58+ZynxxIWzBq8lThZfitX4+c/Hz079wZFs2aIS0zEzt/+w2JKSlYNXEijAwMYGRggImDBuHbX3/FwMWL4dm7N3ILCrD+yBEUFBUhsNJL21IzMmA3fTp8XF2xZc4cIT1y0iS8GxKCvgsXwm/oUGSLRFh94AA6WlhgWqUZvbycnODYqRMmfPMN/rl3T3gzuFgiwecffFBn9fIyn5O09aJXr14y+c6dO4cBAwZg6dKl+OyzzwCUPzEJDQ3FjBkz4O3tjSFDhiA2NhY7duzA8uXLZbrQTZ8+Hf/73//g5uaGgIAA6OjoYPXq1WjRogXmz5//kkesuLFjx8LAwABOTk4wNzfHP//8g5iYGBgaGiIiIkLIt3z5cjg5OcHZ2Rm+vr5ISUnBqlWrMHjwYAyteBHkq1Qv1DB4e3tDX18f06ZNY4BRibm5ORYtWoQ+ffowwHiGj6srbFq3xvThwxlgVGJra4u5c+fC3d2dAUY9UquBxuXLlwEABw8elBnoLSV9c3V0dDS6deuGTZs2CYFFr169sG3bNvTv31+lZZJIJPj+++/Ro0cPmXEQdWns2LHYtGkToqOjkZGRgcaNG8PBwQErVqyAh4cHgPIZhu7duwcAWLRoUZVt+Pj4KDT17/Hjx/Hw4cMG0893bN++2HTsGKKPHEFGbi4aGxjAoUMHrPDxgcc77wj5oqdNQ7d27bDp2DEEVgyU7GVtjW1z5qC/vf0L9zOga1cc/ewzLNm5E0Hbt8NQTw8jHR2x8pNPZAaIa2lp4fDSpfh082Z8c/AgCoqL0cvGBltmz0anSl2sXiXTp0+Hjo4OVq1ahZ9++gmWlpaIjIysMpNO48aNcfr0acydOxdhYWGQSCRwcXFBZGRknc5eNnLkSOzcuROrV69GTk4OzMzM4OnpiaVLl8rMWNejRw8cP34cCxcuxNy5c9G4cWNMmjQJX3zxhUL7aWj1Qg3DO++8g3cq/bZROU1NTYW/m6+bQd27Y1BFFyF6Sk9PD6tXr1Z3MegZGmUKTi4sjQzT09N5saxw8eJFODg44MKFCw2y31xtuvjbb3BwccGF8HD0UODG/3Vx8do1OAQF4cLp0+jBafZk8Psk69GjR0IXS84BXz1em6rid6l6vDbJx2tT9fh9kqXstanOX9hHRERERESvPgYaRERERESkcgw0iIiIiIhI5RhoEBERERGRyjHQICIiIiIilWOgQUREREREKsdAg4iIiIiIVI6BBhERERERqRwDDSIiIiIiUjkGGkREREREpHIMNIiIiIiISOUYaBARERERkcox0CAiIiIiIpVjoEFERERERCrHQIOIiIiIiFSOgQYREREREakcAw0iIiIiIlI5BhpERERERKRyDDSIiIiIiEjlGGgQEREREZHKMdAgIiIiIiKVY6BBREREREQqx0CDiIiIiIhUjoEGERERERGpHAMNIiIiIiJSOQYaRERERESkcgw0iIiIiIhI5RhoEBERERGRyjHQICIiIiIilWOgQUREREREKsdAg4iIiIiIVE5b2RX+/vtvNG3atDbK0uAkJCTI/E1PJSQllf+dnKzegtQz0vpIuHxZreWoj4Rzht8nAEBmZqa6i9Cg8Nr0FK9N1eO1ST5em6rHa5MsZa9NGmVlZWUKZdTQqFGBiIjo5Sj4M/1a4rWJiEg9FLk2Kd2icfz4cT41qpCQkIBx48Zhx44dsLOzU3dx6hWhbmJiYNexo7qLU28kXL6McXPmYIevL+zatVN3ceqVhORkjIuJKT9nHBzUXRy1y8zMxKBBg9RdjAaD16aneG2qHq9N8vHaVD1em2Qpe21SOtDo2rUrzMzMlF3tlWZnZ4cePXqouxj1kp2DA+tGDrt27dDD3l7dxaiX7Dp25DkD4NGjR+ouQoPCa1NVvDZVj9cm+Xhtqh6vTeWUvTZxMDgREREREakcAw0iIiIiIlI5BhpERERERKRyDDSIiIiIiEjlGGgQEREREZHKMdAgIiIiIiKVY6BBREREREQqx0CDiIiIiIhUjoEGERERERGpHAMNIiIiIiJSOQYaRERERESkcgw0iIiIXlJ6ejqys7PVXYx6p6ysDDdv3lR3Meql1NRU5Ofnq7sY9Y5YLMbttDR1F6NeunPnDkpKStRdDKUw0CAiInoJpaWleOeddzBo0CCUlZWpuzj1SnR0NGxsbBAbG6vuotQreXl56Nq1K95//311F6Xe+fz779Fx6lQkpqSouyj1SlpaGmxtbTFjxgx1F0UpDDSIiIhewo4dO5CcnIzz58/j119/VXdx6o3CwkIsW7YMABASEqLm0tQv69atw5MnT3Dw4EFcunRJ3cWpN57k5mLVjz9CLJFg2fffq7s49cqKFStQWFiI7777Dnfu3FF3cRTGQIOIiKiGSktL8dlnn0FDQwNaWlpYvHgxWzUqbNq0Cenp6QCA06dPs1WjQl5eHiIiIgAAWlpaWLp0qZpLVH9EHjiAwuJiAMD3sbFs1aiQlpaG9evXC/9fvny5GkujHAYaRERENbRjxw7cuXMHZWVlEIvFOH/+PH755Rd1F0vtCgsLERoaKgRdWlpabNWosG7dOmE8j1gsxsGDB3Hx4kU1l0r9nuTmYvWBA5BIzxkNDXzOVg0A5a0ZYrEYQPk505BaNRhoEBER1UDl1gwpLS0tLFmy5LVv1ajcmgGU3xyxVeNpa0bl80NLSwufffaZ+gpVT0T99JPQmgEApRIJdrNVQ2jNkAYaUg2lVYOBBhERUQ1Ubs2QkrZqvM5jNZ5tzZBiq4Zsa4aUtFXjdR6r8SQ3F6t//FFozZDS0tB47cdqVG7NkGpIrRoMNIiIiJQkbc2QR0ND47Ueq7Fp0yY8fPiwSvrr3qohrzVDSkND47UeqxH1008QFRVVSS+VSF7rsRrVtWYA5d+nhtCqwUCDiIhISY8fP8ajR4+E/1fuPlVWVoYbN240uPnuVeXq1avPXX7t2rU6Kkn9cvfuXYhEIuH/z54zf//9tzqKVS9c+e+/apeVlZXh+t27dVia+iMpKQmlpaXVLr9y5UodlqZmGGgQEREpqWXLlsjMzIRIJMLKlSuFp9RbtmyBSCTCo0ePoKurq+ZSqkd0dDREIhFEIhE0NctvMzp37iykTZs2Tc0lVI8333wTubm5EIlECAgIEM6Zw4cPQyQSvdYvNvwhKAiivXtx77vvhLTBb70F0d69yN+7F6OdnNRYOvXp37+/8L35+OOPhfQLFy5AJBIhLi5OjaVTjLa6C0BERNQQ6erqCn+k9PT0YGhoqMZSqZ+GhoZQB9Kn9lpaWq99vQDl5wcA6OjoCGn6+vqvfd1oamrCUE9PZjC4VkXa605fXx8AoK399JbdwMCgwZwzbNEgIiIiIiKVY6BBREREREQqx0CDiIiIiIhUjoEGERERERGpHAMNIiIiIiJSuVoNNP766y/MnDkTnTt3RqNGjdC2bVuMGTMGSUlJ1a5TUlKCN998ExoaGvjqq68U3ldubi4WLFgAKysr6OnpwcLCAl5eXsjPzxfynDhxAhMnTkTHjh1haGiI9u3bY/LkyXjw4MFLHaeylKmXPXv2wNHREU2aNEGzZs3g7OyMn3/+Wel93rp1C/r6+tDQ0MD58+dllrm4uEBDQ0Pun8ozY9SF69evw9vbG+3bt4ehoSGaN2+O/v374+DBg0IeiUSCLVu2wMPDA5aWlmjUqBHs7e0RFhaGwsLCF+4jOTm52uPV0NDAlClTql13+fLl0NDQgL29vUqOV1HX796Fd0QE2k+ZAkMvLzT/6CP0X7QIB8+dq5J3z5kzcAwIQJMPPkCzjz6Cc2Agfv7rL4X3lZufjwWbN8Nq8mToeXrC4pNP4BURgfxnXqaUlZcH37VrYTZuHBp5e2NAcDAu3rr10sf6shT5jLKysmBubg4NDQ3s27dP4W1v2rQJdnZ20NfXh42NDdasWSM3X2pqKsaMGYMmTZrA2NgYI0aMwO3bt5U+FiIiooasVqe3XbFiBc6ePQtvb2907doVaWlpWLt2LXr06IE///xT7o3AmjVrcFfJF7NkZ2fD2dkZKSkp8PX1hbW1NR49eoTY2FgUFRUJU4AtXLgQT548gbe3N2xsbHD79m2sXbsWhw4dwuXLl9GyZUuVHPeLKFova9asgb+/P9zc3BAREYHCwkJs2bIF7733Hvbv3w9PT0+F9zl37lxoa2ujSM6bN4ODgzF58mSZNJFIhKlTp2Lw4MEvd7BKunPnDnJzc+Hj44PWrVsjPz8f+/fvh4eHBzZu3AhfX1/k5+djwoQJcHR0xNSpU2Fubo4//vgDS5cuxYkTJ3Dy5EmZFyE9y8zMDNu3b6+SfvToUezcubPaY05JSUF4eDgaNWqksuNV1J30dOQWFMDH1RWtTU2RX1SE/XFx8AgLw8bp0+E7dCgAYM2hQ/CPiYFbz56IGDgQhcXF2HLyJN4LDcX+RYvg+YK5yLNFIjgHBSHl8WP4DhkC61at8CgnB7HXr6OopESYalAikcBt2TJcSU7Gp6NGobmxMdYfPgyXoCBciIyETevWtV4n8ij6GYWEhMg8hFDExo0bMXXqVIwePRrz5s1DbGws/P39kZ+fj4ULFwr58vLyMGDAAGRnZyMoKAg6OjqIjIyEs7MzLl++jGbNmtXo2IiIiBqaWg005s2bh127dsnMMT527Fh06dIFERER2LFjh0z+9PR0LFu2DAsXLkRISIjC+wkMDMSdO3dw8eJFWFlZCemVL/4AsHr1avTt21d4gRAADB06FM7Ozli7di3CwsKUPcQaUbRe1qxZg169euHgwYPCjfPEiRNhYWGBrVu3Khxo/PLLL/jll1+wYMECucf47rvvVkmTluGjjz5S+vhexvDhwzF8+HCZtJkzZ8LBwQGrV6+Gr68vdHV1cfbsWThVummeMmUK2rVrJwQbgwYNqnYfjRo1wrhx46qkb9myBcbGxnB3d5e7XkBAABwdHSEWi/H48eMaHmHNDO/ZE8N79pRJm+nmBod587D6wAGZQKOXjQ0OLlny9Jx5911YTJiArSdPvjDQCNy2DXfS03ExMhJWlQLvhaNHy+TbFxeHuMRE7F24EF59+gAAxvTti45Tp2Lprl3YFRDw0sdcE4p8RteuXUN0dDRCQkIU/p0pKChAcHAw3NzchBaQKVOmQCKRIDQ0FL6+vmjatCkAYP369bhx4wbOnTuHXr16AQCGDRsGe3t7rFq1CuHh4So4UiIiovqvVrtOOTk5VXkzqo2NDTp37oyEhIQq+RctWoROnTrJvQmsTlZWFjZv3gxfX19YWVmhuLhY7lN7oPwNi5WDDGmaqamp3PLUFkXrJScnR+jeIWVsbAwjIyMYGBgotK+SkhLMnj0bs2fPRocOHRQu465du9CoUSOMGDFC4XVqi5aWFiwtLZGVlQWg/CVZTnJumEeNGgUANfosHzx4gFOnTsHT01N4OU5lv//+O/bt24eoqCilt11btLS0YNm8ObJEIiEtJz8f5iYmsueMoSGM9PVh8IIXH2Xl5WHziRPwHTIEVi1borikBEUlJXLz7jt7Fi2aNIFn795CmpmJCcb07YsD8fHVrlebFP2MZs+ejVGjRqFfv34Kb/vUqVPIyMjA9OnTZdJnzJgBkUgk051x37596NWrlxBkAICtrS0GDhyIPXv2KLxPIiKihq7OB4OXlZXh4cOHaN68uUz6uXPnsHXrVkRFRT2328uzzpw5g8LCQlhbW8PLywuGhoYwMDBAnz59cPny5Reun5eXh7y8vCrlqWvy6sXFxQVHjx7FmjVrkJycjMTERMyYMQPZ2dmYPXu2QtuNiopCZmYmFi9erHBZHj16hGPHjmHkyJFq6SYElHfdevz4MW7duoXIyEgcOXIEAwcOfO46aWlpAFCjz/L777+HRCKR24IjFosxa9YsTJ48GV26dFF626okKizE45wc3HrwAJEHDuDIhQsY2K2bsNylSxccvXgRaw4dQvLDh0hMScGMDRuQLRJhdjUtNVJnEhJQWFwM61at4BURAUNvbxh4eaHPggW4/Mz4gku3b6NHhw5VAve3bWyQX1SEpNRU1R20AhT9jPbu3Yu4uDisXLlSqe1funQJANDzmVYlBwcHaGpqCsslEgn+/vvvKvkA4O2338atW7eQm5ur1L7p1RYfH4/g4GCk1vF3pr6TSCQIDAzEwYMHUVZWpu7i1CvHjx/HsmXL6rxlvb4rKinBvE2bcOrvv3nO1CN1Hmjs3LkTqampGDt2rJBWVlaGWbNmYezYsehd6QmpIm7cuAGgvPvUvXv3sG3bNqxbtw63bt2Cq6vrCwd6R0VFobi4WKY86iCvXr755hu4uLjA398fVlZWsLOzw549e3DixAmF6iktLQ2hoaEIDQ2FsbGxwmXZvXs3SktL67zbVGXz58+HmZkZrK2tERAQgFGjRmHt2rXPXWflypUwNjbGsGHDlN7fzp070apVK7i6ulZZtmHDBty5cwehoaFKb1fV5n/3HczGjYO1nx8CNm/GKEdHrPXzE5Z/M2UKXLp0gX9MDKymTIHd9OnYc+YMToSFobet7XO3feP+fQDl3afuPX6MbXPnYp2fH26lpcF18WI8ePJEyPsgMxOtKroKVdbK1BQAcL9S3rqgyGdUUFCAgIAAzJ07F+3atVNq+w8ePICWlhbMzc1l0nV1ddGsWTPcr6i7J0+eoKioCK1ataqyDWmaNC8RUB78hoeHo127dpg5cyYDjgrp6emIiIiAh4cHunfvzoCjkq1bt2Lp0qWwtLREYGAgA44KiSkpiDxwAK6LF6PfokUMOOqJOg00pE/ke/fuDR8fHyF9y5YtuHr1KlasWKH0NvPy8gAAGhoaOHHiBD788ENMmzYNP/74IzIzM7Fu3bpq1/3999/x+eefY8yYMXJvMOtKdfViaGiITp06wcfHB3v37sV3332HVq1awdPTEzdv3nzhdhcuXCjMrKWMXbt2wczMTO7YjboyZ84cHDt2DFu3bsWwYcMgFotRXFxcbf7w8HAcP34cERERaNKkiVL7SkpKwoULF/D+++9XeUKfkZGBkJAQLFmyBGZmZjU5FJWa4+GBY8uWYeucORjWowfEEgmKK3VTMtTTQycLC/i4umLvwoX4zt8frZo2hecXX+DmC25w8woKAFR8l0JD8aGzM6YNH44fg4KQmZeHdYcPC3kLiouhJ2dGMv2KtILnfFaqpuhnFBERgZKSEgQFBSm9j4KCgirdHaX09fVRUFF30r/15HRTk3bJk+YhktLW1kZpaSk2bNjAgEOOa9euwcPDA2+99RYOHTrEm0cAmpqaKCwsxMqVKxlwyPHnv/8y4KgnanUweGVpaWlwc3ODiYkJ9u3bBy0tLQDl4xACAwPx6aefwtLSUuntSscquLu7w8jISEh3dHSElZUV4uLi5K6XmJiIUaNGwd7eHt9++20Njkg1qqsXAPD29oa2trbM1K4jRoyAjY0NgoODsXv37mq3++eff2L79u04ceJElZvn57l9+zb++OMPzJw5E9radXZ6VGFrawvbiifw48ePx+DBg+Hu7o74+PgqXet2796NxYsXY9KkSZg2bZrS+9q5cycA+QPfFy9eDFNTU8yaNasGR6F6tm3awLZNGwDAeFdXDA4JgXtYGOK/+goaGhrwXrEC2lpaOLhkibDOiHfegc3UqQjesQO7FyyodtvSMRzuvXrBqNIYIEdbW1i1aIG4SmNfDHR15Y7DKKxIM6jmprw2KPIZJScn48svv8S6detkficUZWBgUG2gW1hYKPwOSf+WN05MOvWyouOrqGHy8/NTuGsrAJmudGKxGEB5C93GjRvh5+eH4OBguS1kDYX0Bu/69eto0aKFwutJ6wIo70YFAFevXoW7uzu6deuG5cuXw83NTbWFVRNPT89qH2TIk52dLfxbIpEIAUdUVBTmzJmDTz/9FKYVrcsN2YkrV9Di448Vzl9S6ZwRV5wz0oCjj50dln/8MZzreGp6qqNAIzs7G8OGDUNWVhZiY2PRutLUl1999ZXQdSk5ORlA+RSVAJCZmYnk5GS0bt262i+hdFvyfsDMzc2RmZlZJf3evXsYPHgwTExMcPjwYTRu3PhlD7FGnlcvt2/fxtGjRxETEyOzjqmpKfr27YuzZ88+d9sLFixAv379YGVlJdSr9GnHgwcPcPfuXbRt27bKert27QJQ97NNvYiXlxf8/PyQlJSETp06CenHjh3D+PHj4ebmhg0bNtRo27t27UKnTp3g4OAgk37jxg3ExMQgKipKprtLYWEhSkpKkJycDGNjY7X+oHs5OcFv/XokpaZCR1sbRy9eRMyMGTJ5TBs3Rl87O5x9wSD51hXH0UJOi5C5iQkyK1oPAaBV06Z4IOe7Je1e1bqO6kTRzygkJAQWFhZwcXERvg/SMT2PHj1CcnIy2rZtW21Q3qpVK4jFYqSnp8t0nyouLkZGRobw3TU1NYWenp7cLpvStNZqmvqXak/la0hOTg5ycnJeanvSm+x169YhOztb7nTcDYW+vj7y8/OF78/LkAYcV65cgbu7O+7fv19n09KrWuVzRjrRycuQBhwREREoLS3Fl19++dLbVAc9HR1oamhAUlaG4tJSpFcKqmpCGnCcTUjA0M8+Q8bOncI07Q1N5XOmJg/M1KXWA43CwkK4u7sjKSkJx48fx5tvvimz/O7du8jMzETnzp2rrBseHo7w8HBcunQJ3bt3l7t96c2hvGbm+/fvC0/FpTIyMjB48GAUFRXhxIkTantS9KJ6efjwIQDZpzpSJSUlKC0tfe727969izt37shM9yvl4eEBExMTuT9uu3btQocOHeDo6KjE0dQ+aXeTyk9y4uPjMWrUKPTs2RN79uypUQtMfHw8bt68iWXLllVZlpqaColEAn9/f/j7+1dZbmVlhdmzZ6t1JippF6Xs/HzhB1X6d2UlYjFK5ZxLlTlUzEqWKmd8xf0nT4SWFADobmWF2H/+gUQikbk5j09KgqGeHjpaWCh/MDWg6Gd09+5d3Lx5E+3bt6+SRzqTVGZmZrXd7qS/P+fPn5eZfvn8+fOQSCTCck1NTXTp0qXKSzGB8nOtffv2anuwQbXHx8cHcXFx+OGHH5T+HcrNza3ym66lpQVNTU2hRaMhO3r0KMaOHYuSkhKlWtfFYjEyMjJk0jQ1NSGRSIQWjYYaZADlU3HHx8cjNjZW6XMmOzsbJSUlQuAFlNeNnp6e0KLRUDXS18f+wED4rVuHsrIypSYHKhGLZR6IAYCWpibEEonQotFQgwygvPuvj48PjI2Na9QDSF1qNdAQi8UYO3Ys/vjjDxw4cEDuAGZ/f3+MHDlSJi09PR1+fn745JNPMGLECOFmuaSkBLdu3YKJiYkQIHTq1AndunXDgQMH8PjxY2HGoV9//RX37t2T6U4hEokwfPhwpKam4tSpU7CxsamlI38+RerF2toampqa2L17N/z8/IQvW0pKCmJjY9G3b18hr7x6iYmJqfJCspMnT2LNmjX46quvqgRgQPnMOgkJCVhSqdtNXXv2iTFQfnzbtm2DgYGBEJAlJCTAzc0N7dq1w6FDh57bHSUxMRGGhobPbcH58MMPqyyzt7fHDz/8UCV98eLFyM3Nxddff63UlMEvIz0rC+bP3ASXlJZi26lTMNDVxZuWligoLi4/Z86cgd/QoU/PmcePEfvPP+hrZyez7q20NJgYGgoDuDu1aYNuVlY4EB+Pxzk5aF4xgcCvly7h3uPHmPXee8L6Xn36YF9cHP7vjz+E92g8zsnB3rNn4f7223LHb9QGRT+j7OzsKv2Xr127hiVLlmDBggXo3bu3MMNafn4+7t69i+bNmwu/J66urjA1NUV0dLRMoBEdHQ1DQ0OZLhxeXl5YtGgRzp8/L8w+9e+//+LkyZMIUNP7Rah2aWlp4dtvv61RN9yAgAB8/fXXwnakAcaiRYtgUUcBe23q169fjSZASEtLE65n0gCjS5cuCAsLg5ubm1I3oPWRjo4Ofvzxxxqt+/HHHwvXLk1NTejq6mLu3LmYN2+e2mfQVIWRjo4YWYOHnVf++w/dK7otSgMMx06dEDZuHFzUPGOkKhgYGKBHjx7qLobSajXQmD9/Pn766Se4u7vjyZMnVV7QN27cOPTo0aNKxUm7NnTu3FkmCElNTYWdnR18fHywZcsWIT0yMhLvvvsu+vbtCz8/P2RnZ2P16tXo2LGjTJ/9jz76COfOncPEiRORkJAg874FIyOjKgFPbVGkXszMzDBx4kR8++23GDhwIDw9PZGbm4v169ejoKAAgYGBQn559SLv7dbSFgxnZ2e5028+b6xCXfHz80NOTg769+8PCwsLpKWlYefOnUhMTMSqVatgZGSE3NxcDBkyBJmZmfj0009l3mEAAB06dJAJ3uzs7ODs7IzTp0/L5BOLxdi9ezccHR3lBgzNmzeXe05IWzDq6nwBAL/165GTn4/+nTvDolkzpGVmYudvvyExJQWrJk6EkYEBjAwMMHHQIHz7668YuHgxPHv3Rm5BAdYfOYKCoiIEenkJ20vNyIDd9OnwcXXFljlzhPTISZPwbkgI+i5cCL+hQ5EtEmH1gQPoaGGBaZVm8/JycoJjp06Y8M03+OfePeHN4GKJBJ9/8EGd1cvLfEbS1otevXrJ5Dt37hwGDBiApUuX4rPPPgNQ/gMfGhqKGTNmwNvbG0OGDEFsbCx27NiB5cuXy3Sfmz59Ov73v//Bzc0NAQEB0NHRwerVq9GiRQvMnz//JY+YXkWlpaXQ0dF5pQIMVXqVAgxVkUgkMDAwwJw5c16ZAEOVXqUAo6Gr1UBD+h6LgwcPygxollLmxXzPM2DAABw9ehRLlixBUFAQDA0NMXLkSKxcuVKmH5u0PN999x2+++47mW288cYbdXbjqGi9REdHo1u3bti0aZMQWPTq1Qvbtm1D//79VVomiUSC77//Hj169JAZA1HXxo4di02bNiE6OhoZGRlo3LgxHBwcsGLFCnh4eAAo7/527949AOUveXyWj4+PQtP/Hj9+HA8fPmwQXRPG9u2LTceOIfrIEWTk5qKxgQEcOnTACh8feLzzjpAveto0dGvXDpuOHUNgRb/uXtbW2DZnDvorMAhuQNeuOPrZZ1iycyeCtm+HoZ4eRjo6YuUnn8gMENfS0sLhpUvx6ebN+ObgQRQUF6OXjQ22zJ6NTpW6WL1Kpk+fDh0dHaxatQo//fQTLC0tERkZWWXgb+PGjXH69GnMnTsXYWFhkEgkcHFxQWRkZL2YuYzqF29vb+jr62PatGkMMCoxNzfHokWL0KdPHwYYz/Dx8YGNjQ2mT5/OAKMS2zZtMHfECHi8/TYDjHpEo0zBOb+kX/L09HReLCtcvHgRDg4OuHDhQoNszqpNrBv5Lv72GxxcXHAhPBw9OPuFjIvXrsEhKAgXTp9GD2dndRdH7R49eiR0I+TUjNXjtakq/v5Wj3UjH69N1eO1SZay16Y6f2EfERERERG9+hhoEBERERGRyjHQICIiIiIilWOgQUREREREKsdAg4iIiIiIVI6BBhERERERqRwDDSIiIiIiUjkGGkREREREpHIMNIiIiIiISOUYaBARERERkcox0CAiIiIiIpVjoEFERERERCrHQIOIiIiIiFSOgQYREREREakcAw0iIiIiIlI5BhpERERERKRyDDSIiIiIiEjlGGgQEREREZHKMdAgIiIiIiKVY6BBREREREQqx0CDiIiIiIhUjoEGERERERGpHAMNIiIiIiJSOQYaRERERESkcgw0iIiIiIhI5RhoEBERERGRyjHQICIiIiIilWOgQUREREREKsdAg4iIiIiIVI6BBhERERERqRwDDSIiIiIiUjkGGkREREREpHLayq4Q91ccmjRtUgtFaXiS/k0CACQkJKi5JPWPtE5YN7ISkirOmeRk9RakHpLWScLly2otR32RmZWl7iI0KLw2PcVrU/V4bZKP16bq8dokS9lrk0ZZWVmZQhk1NGpSHiIiekkK/ky/lnhtIiJSD0WuTUq3aATFBMHI2KhGBXrVJN9MRsziGMRsjoFDVwd1F6deSUhIwLhx47Bjxw7Y2dmpuzj1hlAvMTGw69hR3cWpVxIuX8a4OXOww9cXdu3aqbs4apeZl4dB4eHqLkaDwWvTU7w2VY/XJvl4baoer02ylL02KR1o2NnbwcTURNnVXmkdO3VEjx491F2MesnOzo51I4edgwPrpRp27dqhh729uouhdo+ys9VdhAaF16aqeG2qHq9N8vHaVD1em8ope23iYHAiIiIiIlI5BhpERERERKRyDDSIiIiIiEjlGGgQEREREZHKMdAgIiIiIiKVY6BBREREREQqx0CDiIiIiIhUjoEGERERERGpHAMNIiIiIiJSOQYaRERERESkcgw0iIiIiIhI5RhoEBERvaSsjCyIckXqLka9U1ZWhps3b6q7GPVSamoq8vPz1V2MekcsFuP27dvqLka9dCc9HSWlpeouhlIYaBAREb0EcakYAWMCsGTCEpSVlam7OPVKdHQ0bGxsEBsbq+6i1Ct5eXno2rUr3n//fXUXpd75/PPP0bFjRyQmJqq7KPVKWmYmbKdPx4wNG9RdFKUw0CAiInoJp386jfTUdNy8dhOXzlxSd3HqjcLCQixbtgwAEBISoubS1C/r1q3DkydPcPDgQVy6xHNG6smTJ1i1ahXEYrFw7lC5Ffv3o7C4GN8dP4476enqLo7CGGgQERHVkLhUjF1rdwEagKamJnZ8vYOtGhU2bdqE9IobotOnT7NVo0JeXh4iIiIAAFpaWli6dKmaS1R/REZGorCwEADw/fffs1WjQlpmJtYfPiz8f/mePWosjXIYaBAREdXQ6Z9O49H9R0AZIJFIcPPaTVw8c1HdxVK7wsJChIaGCkGXlpYWWzUqrFu3DtnZ2QDKxyMcPHgQFy/ynHny5AlWr14NiUQCoPyc+fzzz9Vcqvphxf79EFfUi1giaVCtGgw0iIiIaqBya4aUpqYmdn6987Vv1ajcmgGU31CzVeNpa0bl80NLSwufffaZ+gpVT0RFRQmtGQBQWlqK3bt3v/atGtLWDGmgIdVQWjUYaBAREdVA5dYMKWmrxus8VuPZ1gwptmrItmZISVs1XuexGs+2ZkhpaWm99mM1KrdmSDWkVg0GGkREREoSWjPk0cBrPVZj06ZNePjwYZX0171VQ15rhpSGhsZrPVYjKioKIlHV6aFLS0tf67Ea1bVmAOXBRkNo1WCgQUREpKSczBzkPMkR/q+hUan/VBlw/859lJY0rPnuVeXq1avPXX7t2rU6Kkn9cvfuXZmb6crnTFlZGf7++291FKteuHLlSrXLysrKcP369TosTf2RlJqKUrG42uVX/vuvDktTMww0iIiIlNTUrCl2nduFvZf24pNPPxGeUs/+Yjb2XtqLHXE7oKOro+ZSqkd0dDREIhFEIhE0NctvMzp37iykTZs2Tc0lVI8333wTubm5EIlECAgIEM6Zw4cPQyQSvdYvNvzhhx8gEolw7949IW3w4MEQiUTIz8/H6NGj1Vg69elvbw/R3r0Q7d2LjwcMENIvREZCtHcv4lauVGPpFKOt7gIQERE1RNJAQltHWyZNz0BPXUWqFzQ0NGBoaCj8Gyjvay9Ne53p6ZWfGzo6T4NQfX39175uNDU1YWhoKDMYnOdMOX1dXQCAtubTtgEDXV0Y6jWM3xm2aBARERERkcox0CAiIiIiIpVjoEFERERERCrHQIOIiIiIiFSOgQYREREREakcAw0iIiIiIlK5Wg00bly9gQ3LNmDGezPg/ZY3Jg6YiBVzViD1v1SZfEl/JyH682jM9ZyLUfaj4GHrodR+Lp25hG+Cv8FM95kY+eZITHad/Nz8D+4+wFfzv8LHTh/Dq5sX/Ib4YXvkdqWPr6b++usvzJw5E507d0ajRo3Qtm1bjBkzBklJSdWuU1JSgjfffBMaGhr46quvFN5Xbm4uFixYACsrK+jp6cHCwgJeXl7Iz88X8pw4cQITJ05Ex44dYWhoiPbt22Py5Ml48ODBSx1nTShTN3v27IGjoyOaNGmCZs2awdnZGT///LPS+7x16xb09fWhoaGB8+fPyyxzcXGBhoaG3D+VpyesbdevX4e3tzfat28PQ0NDNG/eHP3798fBgweFPBKJBFu2bIGHhwcsLS3RqFEj2NvbIywsTGbKwOokJydXe6waGhqYMmVKtesuX74cGhoasLe3V8nxKuP63bvwjohA+ylTYOjlheYffYT+ixbh4LlzVfLuOXMGjgEBaPLBB2j20UdwDgzEz3/9pfC+cvPzsWDzZlhNngw9T09YfPIJvCIikF9UJJMvKy8PvmvXwmzcODTy9saA4GBcvHXrpY+ViIioIanV92js/99+JFxKQJ8hfdCuUztkPs7Ezzt/xtzRc/Hl91/ijY5vAADO/3Yex/YdQ7uO7dCyTUukJqe+YMuyfjv0G84cOYP2b7aHqbnpc/PeTriN4PHBMG1hipETRqJxk8Z49OARHj94XOPjVNaKFStw9uxZeHt7o2vXrkhLS8PatWvRo0cP/Pnnn3Jv1tasWYO7d+8qtZ/s7Gw4OzsjJSUFvr6+sLa2xqNHjxAbG4uioiJhfuqFCxfiyZMn8Pb2ho2NDW7fvo21a9fi0KFDuHz5Mlq2bKmS41aEonWzZs0a+Pv7w83NDRERESgsLMSWLVvw3nvvYf/+/fD09FR4n3PnzoW2tjaKnrlZBIDg4GBMniwbuIpEIkydOhWDBw9+uYNVwp07d5CbmwsfHx+0bt0a+fn52L9/Pzw8PLBx40b4+voiPz8fEyZMgKOjI6ZOnQpzc3P88ccfWLp0KU6cOIGTJ0/Kvr34GWZmZti+vWrAffToUezcubPa401JSUF4eDgaNWqksuNVxp30dOQWFMDH1RWtTU2RX1SE/XFx8AgLw8bp0+E7dCgAYM2hQ/CPiYFbz56IGDgQhcXF2HLyJN4LDcX+RYvg6eT03P1ki0RwDgpCyuPH8B0yBNatWuFRTg5ir19HUUmJMKe5RCKB27JluJKcjE9HjUJzY2OsP3wYLkFBuBAZCZvWrWu9ToiIiOqDWg00RnwyAvO/mi/zdtR+w/phlscs7PvfPsz/cj4AYNgHwzB6ymjo6ethw7INSgcaH8/9GDNDZ0JbRxvL/Jbh7g35N+QSiQSRCyJh0d4Cy7cuh56+el52Mm/ePOzatQu6FS9hAYCxY8eiS5cuiIiIwI4dO2Typ6enY9myZVi4cCFCQkIU3k9gYCDu3LmDixcvwsrKSkhfuHChTL7Vq1ejb9++whtcAWDo0KFwdnbG2rVrERYWpuwh1piidbNmzRr06tULBw8eFG6eJ06cCAsLC2zdulXhQOOXX37BL7/8ggULFsg9znfffbdKmrQMH330kdLHV1PDhw/H8OHDZdJmzpwJBwcHrF69Gr6+vtDV1cXZs2fhVOmGecqUKWjXrp0QbAwaNKjafTRq1Ajjxo2rkr5lyxYYGxvD3d1d7noBAQFwdHSEWCzG48d1F7BLDe/ZE8N79pRJm+nmBod587D6wAGZQKOXjQ0OLlny9Jx5911YTJiArSdPvjDQCNy2DXfS03ExMhJWlYLvhc+8sXZfXBziEhOxd+FCePXpAwAY07cvOk6diqW7dmFXQMBLHzMREVFDUKtdp+x62MkEGQDQul1rtLVui5RbKUJa0+ZNX+qmv1mLZjJvZq3OpbOXcOfGHXww4wPo6euhqKAIYrG4xvutKScnJ5kbaQCwsbFB586dkZCQUCX/okWL0KlTJ7k3gdXJysrC5s2b4evrCysrKxQXF8t9Yg8A/fv3lwkypGmmpqZyy1ObFK2bnJwcmJubyzyhNzY2hpGREQwMDBTaV0lJCWbPno3Zs2ejQ4cOCpdx165daNSoEUaMGKHwOrVBS0sLlpaWyMrKAgDo6urKBBlSo0aNAoAafZYPHjzAqVOn4OnpCX19/SrLf//9d+zbtw9RUVFKb7s2aWlpwbJ5c2SJREJaTn4+zE1MZM8ZQ0MY6evD4AVvWM3Ky8PmEyfgO2QIrFq2RHFJCYpKSuTm3Xf2LFo0aQLP3r2FNDMTE4zp2xcH4uOrXY9eT/9e+RfbI7cj42GGuotSr0gkEgQGBuLgwYMoKytTd3HqlePHj2PZsmVqebBTnxUVFWHevHk4deoUz5l6pM4Hg5eVlSErIwvGTY3rete4EncFAKCtq415o+fB+y1veHf3xpfzvkRuVm6dl6eysrIyPHz4EM2bN5dJP3fuHLZu3YqoqKjndnt51pkzZ1BYWAhra2t4eXnB0NAQBgYG6NOnDy5fvvzC9fPy8pCXl1elPOogr25cXFxw9OhRrFmzBsnJyUhMTMSMGTOQnZ2N2bNnK7TdqKgoZGZmYvHixQqX5dGjRzh27BhGjhyplq5CIpEIjx8/xq1btxAZGYkjR45g4MCBz10nLS0NAGr0WX7//feQSCRyW2/EYjFmzZqFyZMno0uXLkpvW9VEhYV4nJODWw8eIPLAARy5cAEDu3UTlrt06YKjFy9izaFDSH74EIkpKZixYQOyRSLMrqa1RupMQgIKi4th3aoVvCIiYOjtDQMvL/RZsACXb9+WyXvp9m306NChSvD+to0N8ouKkJSqXIstvdrOHj2LvRv3YvLAydgYupEBR4X09HRERETAw8MD3bt3Z8BRydatW7F06VJYWloiMDCQAUeFxMREREZGwtXVFf369WPAUU/UeaBx+uBpZDzMQN/hfet617h/5z4AYOWclbBob4FFXy/C6CmjEfdrHEKnhar1hNy5cydSU1MxduxYIa2srAyzZs3C2LFj0bvS01FF3LhxA0B596l79+5h27ZtWLduHW7dugVXV9cXDvSOiopCcXGxTHnURV7dfPPNN3BxcYG/vz+srKxgZ2eHPXv24MSJEwrVVVpaGkJDQxEaGgpjY8WD3t27d6O0tLROu01VNn/+fJiZmcHa2hoBAQEYNWoU1q5d+9x1Vq5cCWNjYwwbNkzp/e3cuROtWrWCq6trlWUbNmzAnTt3EBoaqvR2a8P8776D2bhxsPbzQ8DmzRjl6Ii1fn7C8m+mTIFLly7wj4mB1ZQpsJs+HXvOnMGJsDD0trV97rZv3C//7Qjctg33Hj/Gtrlzsc7PD7fS0uC6eDEePHki5H2QmYlWTZtW2UYr0/LxY/cr5SUCylvgxKViHPn+CAMOOa5duwYPDw+89dZbOHToEG8eAWhqaqKwsBArV65kwCHHn3/+yYCjnqjVMRrPSrmdgo3LNsK2uy1cR1a9calthfnlM+/YdLERxoc4DXGCnr4etq3ehit/XEF3p+51Xi7p0/jevXvDx8dHSN+yZQuuXr2Kffv2Kb3NvLw8AICGhgZOnDgBIyMjAMBbb72F3r17Y926ddWOvfj999/x+eefY8yYMXJvMOtSdXVjaGiITp06oU2bNnjvvfeQm5uLyMhIeHp6IjY2FtbW1s/d7sKFC4XZtZSxa9cumJmZyR27URfmzJkDLy8v3L9/H3v27IFYLEZxcXG1+cPDw3H8+HGsX78eTZo0UWpfSUlJuHDhAubOnVvl6XxGRgZCQkKwZMkSmJmZ1eRQVG6Ohwe8nJxw/8kT7DlzBmKJBMWVuikZ6umhk4UF2jRrhvd69UJuQQEiDxyA5xdfIPaLL2D9nEHaeQUFACq+T6GhMKronvdW+/bovWAB1h0+jLCKro0FxcXQkzMjmX5FWsFzPi9q+NYvXY//Lf+fwvkL8guAisZqiVgCADjy/REc3X0UQ8cOhbef9wsnOanPpDd4169fR4sWLRRer3K3ZomkvF6uXr0Kd3d3dOvWDcuXL4ebm5tqC6smnp6eVboMP092drbwb4lEIgQcUVFRmDNnDj799FOYmjbcc0bqxIkTSp0zJZV+76XnjzTg6NOnD5YvXw5nZ2eVl5Oer84CjcxHmVjmtwyGjQ2x8OuF0NLSqqtdC3T1yr/I/d36y6Q7v+eMbau3IfFSYp0HGmlpaXBzc4OJiQn27dsn1EtOTg4CAwPx6aefwtLSUuntSscpuLu7C0EGADg6OsLKygpxcXFy10tMTMSoUaNgb2+Pb7/9tgZHpDrV1Q0AeHt7Q1tbW2Z61xEjRsDGxgbBwcHYvXt3tdv9888/sX37dpw4caLKDfTz3L59G3/88QdmzpwJbe06jdEFtra2sK14+j5+/HgMHjwY7u7uiI+Pr9K1bvfu3Vi8eDEmTZqEadOmKb2vnTt3ApA/6H3x4sUwNTXFrFmzanAUtcO2TRvYtmkDABjv6orBISFwDwtD/FdfQUNDA94rVkBbSwsHlywR1hnxzjuwmToVwTt2YPeCBdVuWzqGw71XLyHIAABHW1tYtWiBuErjXwx0deWOwyisSDNQ4oaCGgaDRk/Pify8/OfkVIw04Ph5588Q5Yowb+W8l96muujr6yM/Px9isRjp6ekvtS1pwHHlyhW4u7vj/v37dToroio1btxY+Ld0nN3LkAYcERERKC0txZdffvnS21QHPT09aGpqQiKRoLi4+KXPGWnAcfbsWQwdOhQZGRnCjJsNTeNK5TaSM2ayvqqTuyVRrgif+X4GUY4IX+z8As1aNKuL3VYhfSrUpFkTmXSTZiYAgLycvDotT3Z2NoYNG4asrCzExsaidaUnql999ZXQdSk5ORlA+TSiAJCZmYnk5GS0bt262qcg0m3Jexpgbm6OzMzMKun37t3D4MGDYWJigsOHD8v8ENa159XN7du3cfToUcTExMisY2pqir59++Ls2bPP3faCBQvQr18/WFlZCXUrbXJ+8OAB7t69i7Zt21ZZb9euXQDqdrapF/Hy8oKfnx+SkpLQqVMnIf3YsWMYP3483NzcsGHDhhpte9euXejUqRMcHBxk0m/cuIGYmBhERUXhfkWXIgAoLCxESUkJkpOTYWxsrPYnal5O/7+9uw+qqkzgOP4TVPYSRpayo9Ymo4gIhoEvlISCWKaJ6GjYi+HS7ty0EbXGKd/KTcPVSR1tFmZMyMHFydxe0FwZfEGH0oEZtWZKyNHcBAwRREAlvVzcP/BeuXJVrh5etO/nH4Z77rmc88zlPM/vPC/naZmTk3WspESdOnZU1uHDWv/mmw7vebhLF4UHBOi720yU73ntXP7spFfIx9tblReuXzt6dO2q35z8f9mGV/W8D+40wlFUbJQKDxfq4O6DLt9Eq71UK2udVda663fw3dzd1KFDB42ZOkYvml80+nBbVVZWluLi4mSxWFy6sWO1WlVR4Th8zNYAtfVo3KshQ2pYrS8vL0+5ubku37iqqqqSxWKxBy+poWw8PDzsPRr3qgceeEBffPGFzGazrl696tLcVIvF0qRt4+7uLqvVau/RuFdDhiT987XXFB8VpQdNJj3WTkYSNEeLB40rl69o6YylOv2/01qatlR/6du0Adda+gb2VfbWbFWUOV68zpU1NABac4L677//rvHjx+vYsWPavXu3BgwY4LD91KlTqqysVGBgYJN9k5KSlJSUpCNHjmjQoEFOP9/WOCxxMvH09OnT9rviNhUVFXr22Wd1+fJl7dmzRz169LjDM7t7tyubM2fOSJLTFcMsFovq6upu+fmnTp3Sr7/+6rDkr01MTIy8vb2d3mHavHmz+vTpo7CwMBfOpmXVXhvS07grPS8vTxMnTtTgwYP1+eef31HvS15eno4fP64PPvigybaSkhLV19crMTFRiYmJTbb7+vpq9uzZbb4SlW2IUtWlS7Jeq5CtjSpmG4vVqrrbrD4Xem1VshIn8ytOnztn70mRpEG+vso9elT19fUODau8Y8fk6eGhfr16uX4yaNfc3d0168NZmvWh6z18aSvStD29oWe2ccCY/PfJbXZTzkjPPPOMww2J5iotLbXXQ7aAMXDgQC1btkzjxo1zqQHaHnXq1Elff/31He07bdo0+40vNzc3de7cWXPnztVbb73VLhZwuVuxsbGKjY11eb8ffvjB3iayBYywsDAtW7ZMI0eONPQY24LJw0MhLqyQ2V60aNCwWq1aOXelfv7+Zy3810L1f/LWEy6bo/iXYnn8yUPde7qe5oaNGqZPkj7Rni/3aNTEUfZGQPbWbEnSk8OfvOvjaw6r1aq4uDgdPHhQmZmZTicvJyYmNvlHKysrk9ls1vTp0zVhwgR7Q9lisejEiRPy9va2X5j9/f0VHByszMxMlZeX2y8+2dnZKioqchjycvHiRY0dO1YlJSXKycmRn59fC5357TWnbPr27Ss3Nzdt2bJFZrPZXuEUFxcrNzdX4eHXFxpwVjbr1693eDK6JO3du1cff/yxPvrooyYhTJKOHDmigoICLW407KY1lZWVycfHx+E1i8Wi9PR0mUwmexgrKCjQuHHj1Lt3b33zzTe3XOq3sLBQnp6et+y9efnll5tsCwoK0ldffdXk9UWLFqmmpkZr1651abngu1V2/rx8buhpsNTVKT0nR6bOnTXgscdUe+VKw3fm229lHjPm+nemvFy5R48qPCDAYd8TpaXy9vS0T+D2f/RRBfv6KjMvT+XV1ep2bQGB7CNHVFRerlkvvGDff/Lw4frPgQP68uBB+3M0yqurtfW77zR+6FCn8zfwx2a1WuXe0f2+ChhGup8ChlHq6+tlMpk0Z86c+yZgGOl+Chj3uhYNGmkr0pS/N19DI4eqpqpGOdtyHLZHxkRKkspKyuzbjv90XJK0JaVhjL1PTx9FToi07zNz7EwFDQlS0qYk+2snfz6p/L35kqTfTv2mixcu2vf39ffV0KihkqSu3btqyhtTtHndZi352xKFRYfpZOFJZW/NVsS4CPkNbJ0G9ttvv61t27Zp/PjxOnfuXJMH9L366qsKCQlRSEiIw+u2YT6BgYEOIaSkpEQBAQGKj4/Xxo0b7a+vWbNGo0ePVnh4uMxms6qqqrR69Wr169fPYcz+K6+8ovz8fCUkJKigoMDheQteXl53dGfhTjWnbLp3766EhARt2LBBo0aN0qRJk1RTU6Pk5GTV1tZq/vz59vc7KxtnT7i29WCMGDFCg294+Jt06/kKrcFsNqu6uloRERHq1auXSktLlZGRocLCQq1atUpeXl6qqanRc889p8rKSs2bN087duxw+Iw+ffo4BLeAgACNGDFC+/btc3if1WrVli1bFBYW5jQwdOvWzel3wtaD0ZrfF0kyJyer+tIlRQQGqtcjj6i0slIZ+/ersLhYqxIS5GUyyctkUkJ0tDZkZ2vUokWa9NRTqqmtVfLOnaq9fFnzJ0+2f15JRYUCZs5UfFSUNs6ZY399zeuva/R77yn8nXdkHjNGVRcvanVmpvr16qUZjVb0mvz00wrz99df163T0aIi+5PBrfX1+sdLL7Vm0eAeMHzMcHX26KznX3qegNGIj4+P3n33XQ0fPpyAcYP4+Hj5+flp5syZBIxG+vfvr7lz5yomJoaA0Y60aNA4WXBSkpSfk6/8nPwm221B40zxGWWszXDYZvs9aEiQQ9Bw5peffrnp/lGxUfagIUlxM+Lk9aCXdvx7hzYs36CHuj2kKW9M0dSZU108uztne47F9u3bHSYz27jyYL5biYyMVFZWlhYvXqwFCxbI09NTsbGxWrlypcMEcdvxpKWlKS0tzeEzHn/88VZtODa3bFJSUhQcHKzU1FR7sBgyZIjS09MVERHRZL+7UV9fr88++0whISEO8yBaU1xcnFJTU5WSkqKKigp16dJFoaGhWrFihWJiYiQ1DH8rKiqS1PCQxxvFx8c3a+nf3bt368yZM1q4cKGxJ9FC4sLDlbprl1J27lRFTY26mEwK7dNHK+LjFTNsmP19KTNmKLh3b6Xu2qX5mzZJkob07av0OXMUERR0278T+cQTylqyRIszMrRg0yZ5engoNixMK6dPd5gg7u7urv++/77mffqp1m3frtorVzTEz08bZ8+Wf6MhVoAk+Qf7yz+4ba4r7Zmbm5uWL1/e1ofRLkVHRys6OrqtD6Pd8fDw0OrVq9v6MHCDDlebubiw7W7CpgOb5P2wd4se1L3ix+9/1IKpC7TvwD6NeIol0xo7fPiwQkNDdejQoSY9M39klMvNHd6/X6EjR+pQUpJCmtHwv9+draqSz7RpksQa8LdA3dQUddPNcQ12jnK5OeomR67WTa3+wD4AAAAA9z+CBgAAAADDETQAAAAAGI6gAQAAAMBwBA0AAAAAhiNoAAAAADAcQQMAAACA4QgaAAAAAAxH0AAAAABgOIIGAAAAAMMRNAAAAAAYjqABAAAAwHAEDQAAAACGI2gAAAAAMBxBAwAAAIDhCBoAAAAADEfQAAAAAGA4ggYAAAAAwxE0AAAAABiOoAEAAADAcAQNAAAAAIYjaAAAAAAwHEEDAAAAgOEIGgAAAAAMR9AAAAAAYDiCBgAAAADDETQAAAAAGI6gAQAAAMBwBA0AAAAAhuvo6g7VldUtcRz3pAvVFyRJ5yvP6+zZs218NO1LZWWl/Sdlcx3lcnOV5883/LxwQWerqtr2YNqB8mquta6gbrqOuunmuAY7R7ncHHWTI1frpg5Xr1692qw3duhwRwcEALg7zbxM/yFRNwFA22hO3cTQKQAAAACGa3aPBgAAAAA0Fz0aAAAAAAxH0AAAAABgOIIGAAAAAMMRNAAAAAAYjqABAAAAwHAEDQAAAACGI2gAAAAAMBxBAwAAAIDhCBoAAAAADPd/XMRTUEf5D+0AAAAASUVORK5CYII=\n"
          },
          "metadata": {}
        }
      ],
      "source": [
        "grid = gridworld((size, size), goal_states, error_states, finite_horizon=False)\n",
        "solution = Solution(grid, gamma=0.9, epsilon=0.2)\n",
        "q_table = solution.q_learning()\n",
        "\n",
        "Q_star_dict = {}\n",
        "pi_star_dict = {}\n",
        "for i in range(grid.rows):\n",
        "  for j in range(grid.columns):\n",
        "    Q_star_dict[(i, j)] = np.max(q_table[i, j])\n",
        "    # Find the indices where the values are approximately equal to the maximum value\n",
        "    indices = np.argwhere(np.isclose(q_table[i, j], Q_star_dict[(i, j)], atol=1e-3))\n",
        "    indices = [arr.item() for arr in indices]\n",
        "    pi_star_dict[(i, j)] = [grid.actions[index] for index in indices]\n",
        "\n",
        "print(\"Optimal Q function and optimal policy in the infinite horizon case: \")\n",
        "create_grid(rows=5, columns=5, cell_size=20, line_width=1, border_width=3,\n",
        "            error_states=error_states, goal_states=goal_states, arrows=pi_star_dict,\n",
        "            Q_values=Q_star_dict, save_path='grid_with_arrows_colored.png')"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}