Upload 45 files

.gitignore

@@ -0,0 +1,158 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+pip-wheel-metadata/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pycharm
+.idea/
+
+# Eye-Tracker
+gui/
+media/
+subjects/
+build/
+dist/
+codes/p1.py
+codes/temp1.py
+codes/.ipynb_checkpoints/
+codes/__pycache__/
+codes/backup_codes/
+models/io/raw/
+models/io/trained/*
+!models/io/trained/mdl1.h5
+!models/io/trained/mdl1.pickle
+models/et/raw/
+models/et/trained/*
+!models/et/trained/mdl1-hrz.h5
+!models/et/trained/mdl1-vrt.h5
+!models/et/trained/mdl1.pickle
+!crt_exe_gui.spec
+dataset/
+codes/jupyter_notebook/FinalPlots.ipynb
+codes/jupyter_notebook/temp1.ipynb
+temp1.py

README.md

@@ -1,5 +1,143 @@
----
-license: mit
-language:
-- en
----
+# Owleye
+## Intro
+Owleye gives you the posibiliy to transform your webcam to an eye tracker. At first, you should calibrate owleye to know you, then it detects which point you are looking on your monitor.
+
+**Demo: Click on the image below and see the video explanation.**
+
+[![IMAGE ALT TEXT HERE](https://github.com/owleye/Owleye/assets/53625380/bcddb7a9-fe92-48a5-8b21-de40e7f99efa)](https://www.youtube.com/watch?v=E6OmSAgI)
+___
+## Installation
+
+### 1.Use source code:
+
+Open terminal, download the repo:
+`git clone <repo address>`
+
+(or just download the zip file)
+
+Go to the project directory:
+`cd Owleye`
+
+make a virtual environment:
+`python -m venv env` or `virtualenv env`
+
+activate the virtual environment:
+
+Windows:
+`.\env\Scripts\activate`
+
+Linux:
+`source env/bin/activate`
+
+Install required libraries:
+`pip install -r requirements.txt`
+
+### 2. Use .exe file
+
+Download the release file. It is tested on Windows 10.
+
+## Usage
+
+If you are using the source code, after activating the virtual environment, run main.py or main_gui.py: `python main.py` or `python main_gui.py`
+
+main_gui.py is a simplified version of main.py. There are some methods in main.py that are not in main_gui.py. Also, main_gui.py has GUI. Usage of main_gui.py is much easier.
+
+If you have downloaded the Owleye.exe, run it. This file is exactly main_gui.py with the needed libraries.
+
+This is the opened window:
+
+![Screenshot 2024-03-25 141144](https://github.com/owleye/Owleye/assets/53625380/70e6ce5f-00af-4db8-8fd2-60207f7274a8)
+
+You can learn about the program's usage in [this tutorial](https://github.com/owleye/Owleye/blob/main/docs/USE_APP.md).
+
+## Method
+
+**Owleye's structure:**
+
+![Owleye structure](https://github.com/owleye/Owleye/blob/main/docs/images/Owleye%20structure.png)
+
+
+While the camera is streaming, Owleye gets the images and extracts head and eyes features (blocks 1-5 in above image). Then it feeds this data to the neural networks (NN) models to calculate the user's eye viewpoint (block 6 in above image).
+
+### Calculating the sixth block's inputs
+
+As in the first block of the [Owleye's structure](https://github.com/owleye/Owleye/blob/main/README.md#:~:text=Method-,Owleye%27s%20structure%3A,-While%20the%20camera) is visible, it receives the user's images during time and after detecting thier face, in the second block it extracts their [468 landmarks/keypoints](https://github.com/owleye/Owleye/blob/main/docs/images/468_landmarks.jpg). It's done by canonical face model which is in the world coordinates. Owleye uses Mediapipe package to implement these steps. Then in the third block, Owleye computes the face rotation and position vectors by extracted landmarks. In the fourth block, Owleye extracts the eyes' images using landmarks and gives them to the fifth block to calculate iris positions. The image below shows the landmarks and the rotaion vector on face:
+
+![Screenshot 2024-03-27 035946](https://github.com/owleye/Owleye/assets/53625380/cc31d4c2-b359-41aa-824f-03f4ec075714)
+
+
+Finlaly, three type of inputs are ready to be fed to sixth block which is eye viewpoint predictive model:
+- **Head rotation and position vectors:** (r1, r2, r3), (x, y, z). Rotation and position, world coordinates.
+- **Left and right eyes iris:** (xl, yl), (xr, yr). These are calculated respect to the eyes (image coordinates).
+- **Eyes images:** Two images are concatenated together in rows.
+
+We will consider the first and the second inputs as the **face vector** which has a length of 10.
+
+![Screenshot 2024-03-14 034920](https://github.com/owleye/Owleye/assets/53625380/b1f44929-a867-45eb-b5be-211c5f41f08c)
+
+### Sixth block's output
+
+The output of Owleye is a vector of user's eye view points on screen (xp, yp) per sample (an image and a vector). During time, this output will be a matrix. The matrix's shape is n by 2. The values are normalized between 0 and 1. For example, the program tracks the user for 10 seconds, with an FPS of 15, we have a matirx with a shape of 150 by 2. The first column is for the horizontal direction and the second is for the vertical direction.
+
+### Calibration
+The calibration process consists of looking at a white point in a black screen for a certain time. Then, the point's position changes and the user must look at it again. This process is repeated until the calibration ends. During this procedure, Owleye collects data (input and output). It means each sample data entails one image, one face vector and one location point. This is because we already have the first five blocks, and the models and calculations have been prepared. Just the sixth block should be made.
+
+### Dataset
+
+We implemented the calibration on 20 male subjects and collected 221000 samples (eyes images and face vectors as inputs and the appeared point's locations as outputs). The dataset is collected in an environment like the image below. The subjects were seated in a driving simulator and they werer instructed for Owleye's calibration. The camera in front of them is Microsoft Lifecam VX-6000 (in the red area). The distance between the camera and the participants was nearly 80 cm. There were three monitors (1280 x 720) that were located 170 cm further the user.
+
+![20220628_144415crop](https://github.com/owleye/Owleye/assets/53625380/65ccd86f-18d4-43d7-b833-d498bccb60da)
+
+
+### Modeling
+
+For the sixth block in [Owleye's structure](https://github.com/owleye/Owleye/blob/main/README.md#:~:text=Method-,Owleye%27s%20structure%3A,-While%20the%20camera), Two Convolutional Neural Network (CNN) models are used to predict the user's eye view point in the [horizonal](https://github.com/owleye/Owleye/blob/main/models/et/trained/mdl1-hrz.h5) and [vertical](https://github.com/owleye/Owleye/blob/main/models/et/trained/mdl1-vrt.h5) directions on the monitor. These models are trained using the aforementioned [dataset](https://github.com/owleye/Owleye/tree/main#:~:text=should%20be%20made.-,Dataset,-We%20implemented%20calibration). We called them "base models" or "et models" (Abbreviation of eye tracking). They are located in the models folder.
+
+**Network architecture:**
+![Screenshot 2024-03-16 163427](https://github.com/owleye/Owleye/assets/53625380/02d196c2-c9c2-497d-b1e5-d3d7b2a29160)
+
+Right side of the above picture illustrates the CNN model's structure. In this model there is two branches. The left branch is for the eyes image, and the right branch is for a vector with a length of 10. six value for head's rotation and position and 4 value for iris position.
+
+The horizontal and vertical base models' accuracy on the test data were 95.9% and 85.4%, respectively.
+
+### Fine-tuning
+
+Owleye and its base models are built using the dataset that aleardy we explained. The features like light condition, camera quality, camera position, and monitors position were stationary in the dataset. They are specific to the environment. A rich dataset is the one that is robust to all of the mentioned situations and features. It should be collected in various light conditions, with different cameras and different positions for camera, user and monitors. But, due to the possible costs, it was not an appropriate way for us. So we decided to provide a calibration step. In thes way, we can collect a little amount of data from the person that we want detect their eye movement. Then we can retrain the [base models in the sixth block](https://github.com/owleye/Owleye/blob/main/README.md#:~:text=Method-,Owleye%27s%20structure%3A,-While%20the%20camera) using the new collected data. Actually we are customizing Owleye for each person. So, the last layer's weights of the base models change based on the new collected data. In this way, the network maintains its original shape and just is calibrated a little for each person. Finally, Owleye gets familiar with the new light condition and the devices positions. So, despite Owleye is built using a specific dataset with specific features, you can use it in different conditions (e.g. your own computer with different features).
+
+### Fixations
+
+The [IV-T method](https://tobii.23video.com/the-tobii-pro-fixation-filters-eye-movement) is used to extract user's fixations. A [fixation](https://en.wikipedia.org/wiki/Fixation_(visual)) is a series of eye view points that are close together. So, first of all we removed the outliers using median filter. Then we merged close fixations and removed short ones. below image shows the fixations in a monitor during a certain time. In [this file](https://github.com/owleye/Owleye/blob/main/codes/jupyter_notebook/fixations_in_AOIs.ipynb) you can see the way of calculating the fixations.
+
+![Screenshot 2024-03-16 195233](https://github.com/owleye/Owleye/assets/53625380/57b9e984-5f54-48a0-984e-110c65b2cc20)
+
+
+### Blinking
+
+Indeed, while the user is blinking, they aren't seeing anywhere. So, the data in that short time should be removed. We've calculated the blinking using Eye Aspect Ratio (EAR) method. In this way, when the user's EAR goes lower than a certain threshold, it is considered as a blink. So, the output (x, y) will be deleted in the next computations. Also in this periods, we can interpolate the outputs during time.
+
+![Screenshot 2024-03-16 201348](https://github.com/owleye/Owleye/assets/53625380/9ba0751f-ac96-46cd-a878-053a7e55158c)
+
+### In-out model
+
+A model called io is trained to see whether the user is looking into the screen or not. This is because the et model always predict a point, no matter the user is looking inside of the screen or outside of it. So, because the et (base models) are trained using the data of inside of the screen, the et models can't extrapolate when the user is looking outside of the screen. This model is in the "models/io" folder.
+
+## Limitations and future works
+**1) Recunstructing whole code:** The structure of the code is terrible:)) Owleye is made in 2021 and I have not dedicated so much of time to improve it since then. Therefore, a lot of things have changed. The structure of the code totally can be redesigned to reach a better performance. The code can be more object oriented. the libraries (mediapipe and tensorflow) have changed a lot. So, the algorithm can be rewritten considering the changes.
+
+**2) Changing the calibration algorithm:** The calibration duration time is really long. Using methods like image morphing makes it unnecessary to collect images in all positions and angles of the head and eyes.
+
+**3) Changing the fine-tuning method:** In the current method, to retrain the algorithm, we considered to just change the weights in the last layer of the network. This fine-tuning process can be improved by implementing better solutions.
+
+**4) Adding camera calibration:** The computed head angles and positions are meaningless when the camera is not calibrated. By calibrating and having angles and positions of the head, we can calculate the real eyes' angles and positions. So, using these parameters, we can implement better methods to reach to the eyes view point. Maybe just with a simple linear regression model and real parameters of the head and eyes we could get to the target.
+
+**5) Creating a python library:** It can be desired to create a package from the code. So, everybody could just install and import the library and use it as they want.
+
+**6) Providing real-time usage:** For now, it isn't possible to use the program in real-time. Because the FPS goes down in this way. the program's FPS for a camera that is 30 FPS reaches to 15. So, by optimizing some packages, we can get to a better result.
+
+## Contributing
+
+Feel free to improve the project. I'll appreciate your pull requests.
+
+## About project
+
+Owleye is a subsection of my master thesis: "Driver's Hazard Perception Assessment in a Driving Simulator".

codes/base/core.py

@@ -0,0 +1,168 @@
+import numpy as np
+import tensorflow as tf
+
+
+def tflite_inference(inputs, model_path, dtype=np.float32):
+
+    if not isinstance(inputs, (list, tuple)):
+        inputs = (inputs,)
+
+    # Load the TFLite model and allocate tensors.
+    interpreter = tf.lite.Interpreter(model_path=model_path)
+    interpreter.allocate_tensors()
+
+    # Get input and output tensors.
+    input_details = interpreter.get_input_details()
+    output_details = interpreter.get_output_details()
+
+    # Test the model on random input data.
+    for inp, inp_det in zip(inputs, input_details):
+        interpreter.set_tensor(inp_det["index"], np.array(inp[None, ...], dtype=dtype))
+
+    interpreter.invoke()
+
+    # The function `get_tensor()` returns a copy of the tensor data.
+    # Use `tensor()` in order to get a pointer to the tensor.
+    outputs = [interpreter.get_tensor(out["index"]) for out in output_details]
+
+    return outputs
+
+
+def landmarks_to_detections(landmarks):
+    """
+    landmarks: (3, N) landmarks
+    """
+    x_min = np.amin(landmarks[0, :])
+    x_max = np.amax(landmarks[0, :])
+    y_min = np.amin(landmarks[1, :])
+    y_max = np.amax(landmarks[1, :])
+
+    bbox = dict()
+    bbox["x_min"] = x_min
+    bbox["y_min"] = y_min
+    bbox["width"] = x_max - x_min
+    bbox["height"] = y_max - y_min
+
+    detections = dict()
+    detections["bboxs"] = bbox
+    detections["keypoints"] = landmarks[:2, :]
+
+    return detections
+
+
+def detections_to_rect(
+    detections,
+    image_size,
+    rotation_vector_start_end=None,
+    rotation_vector_target_angle=0,
+):
+
+    keypoints = detections["keypoints"]
+    x_min = np.amin(keypoints[0, :])
+    x_max = np.amax(keypoints[0, :])
+    y_min = np.amin(keypoints[1, :])
+    y_max = np.amax(keypoints[1, :])
+
+    rect = dict()
+    rect["x_center"] = (x_min + x_max) / 2
+    rect["y_center"] = (y_min + y_max) / 2
+    rect["width"] = x_max - x_min
+    rect["height"] = y_max - y_min
+
+    if rotation_vector_start_end is not None:
+        rect["rotation"] = compute_rotation(
+            detections,
+            image_size,
+            rotation_vector_start_end,
+            rotation_vector_target_angle,
+        )
+    else:
+        rect["rotation"] = None
+
+    return rect
+
+
+def compute_rotation(detections, image_size, rotation_vector_start_end, target_angle):
+
+    keypoints = detections["keypoints"]
+
+    x0 = keypoints[0, rotation_vector_start_end[0]] * image_size[0]
+    y0 = keypoints[1, rotation_vector_start_end[0]] * image_size[1]
+    x1 = keypoints[0, rotation_vector_start_end[1]] * image_size[0]
+    y1 = keypoints[1, rotation_vector_start_end[1]] * image_size[1]
+
+    rotation = normalize_radians(target_angle - np.arctan2(-(y1 - y0), x1 - x0))
+
+    return rotation
+
+
+def normalize_radians(angle):
+    return angle - 2 * np.pi * np.floor((angle - (-np.pi)) / (2 * np.pi))
+
+
+def transform_rect(
+    rect,
+    image_size,
+    scale_x=1,
+    scale_y=1,
+    shift_x=0,
+    shift_y=0,
+    square_long=True,
+    square_short=False,
+    opt_rotation=None,
+):
+    width = rect["width"]
+    height = rect["height"]
+    rotation = rect["rotation"]
+    image_width = image_size[0]
+    image_height = image_size[1]
+
+    if rotation is not None and opt_rotation is not None:
+        rotation += opt_rotation
+        rotation = normalize_radians(rotation)
+
+    if rotation is None:
+        rect["x_center"] = rect["x_center"] + width * shift_x
+        rect["y_center"] = rect["y_center"] + height * shift_y
+    else:
+        x_shift = (
+            image_width * width * shift_x * np.cos(rotation)
+            - image_height * height * shift_y * np.sin(rotation)
+        ) / image_width
+        y_shift = (
+            image_width * width * shift_x * np.sin(rotation)
+            + image_height * height * shift_y * np.cos(rotation)
+        ) / image_height
+
+        rect["x_center"] = rect["x_center"] + x_shift
+        rect["y_center"] = rect["y_center"] + y_shift
+
+    if square_long:
+        long_side = np.max((width * image_width, height * image_height))
+        width = long_side / image_width
+        height = long_side / image_height
+    elif square_short:
+        short_side = np.min((width * image_width, height * image_height))
+        width = short_side / image_width
+        height = short_side / image_height
+
+    rect["width"] = width * scale_x
+    rect["height"] = height * scale_y
+
+    return rect
+
+
+def slice_from_roi(roi, image_size, horizontal_side=True):
+    if horizontal_side:
+        center = roi["x_center"]
+        norm_side = roi["width"]
+        image_side = image_size[0]
+    else:
+        center = roi["y_center"]
+        norm_side = roi["height"]
+        image_side = image_size[1]
+
+    first_id = int((center - norm_side / 2) * image_side)
+    second_id = int((center + norm_side / 2) * image_side)
+
+    return (first_id, second_id)

codes/base/eyeing.py

@@ -0,0 +1,821 @@
+"""This module is like a tool for just eyes and also the alll of the program. There are a lot of functions here"""
+
+import os
+import shutil
+import numpy as np
+import mediapipe as mp
+import cv2
+import pickle
+from sklearn.utils import shuffle
+from screeninfo import get_monitors
+from codes.base.face_geometry import PCF, procrustes_landmark_basis, get_metric_landmarks
+from codes.base.iris_lm_depth import from_landmarks_to_depth as fl2d
+import time
+import math
+from screeninfo import get_monitors
+
+
+STATIC_IMAGE_MODE = False
+MIN_TRACKING_CONFIDENCE = 0.5
+MIN_DETECTION_CONFIDENCE = 0.5
+EYE_SIZE = (100, 50)
+Y_SCALER = 1000.0
+X1_SCALER = 255.0
+WHITE = (220, 220, 220)
+BLACK = (0, 0, 0)
+GRAY = (70, 70, 70)
+RED = (0, 0, 220)
+BLUE = (220, 0, 0)
+GREEN = (0, 220, 0)
+PATH2ROOT = ""
+PATH2ROOT_ABS = os.path.dirname(__file__) + "/../../"
+CLB = "clb"
+IO = "io"
+LTN = "ltn"
+ACC = "acc"
+SMP = "smp"
+MDL = "mdl"
+RAW = "raw"
+TRAINED = "trained"
+T = "t"
+X1 = "x1"
+X2 = "x2"
+Y = "y"
+ER = "er"
+FV = "fv"
+DEFAULT_BLINKING_THRESHOLD = 4.5
+LATENCY_WAITING_TIME = 50
+
+
+def get_mesh():
+    """
+    Creating face mesh model
+
+    Parameters:
+        None
+    
+    Returns:
+        face_mesh: The face mesh model
+    """
+    print("Configuring face detection model...")
+    face_mesh = mp.solutions.face_mesh.FaceMesh(
+        static_image_mode=STATIC_IMAGE_MODE,
+        min_tracking_confidence=MIN_TRACKING_CONFIDENCE,
+        min_detection_confidence=MIN_DETECTION_CONFIDENCE)
+
+    return face_mesh
+
+
+def get_clb_win_prp(clb_win_align=(0, 0)):
+    """
+    Creating calibration window
+
+    Parameters:
+        clb_win_align: The window's top-left location
+    
+    Returns:
+        clb_win_size: The window's size
+    """
+    clb_win_w_align, clb_win_h_align = clb_win_align
+    screen_w = None
+    screen_h = None
+    for m in get_monitors():
+        screen_w = m.width
+        screen_h = m.height
+
+    clb_win_w = screen_w - clb_win_w_align
+    clb_win_h = screen_h - clb_win_h_align
+    clb_win_size = (clb_win_w, clb_win_h)
+
+    return clb_win_size
+
+
+def get_some_landmarks_ids():
+    """
+    Getting some landmarks that are needed for calculation of the face rotation and position vectors
+
+    Parameters:
+        None
+    
+    Returns:
+        some_landmarks_ids: The landmarks numbers
+    """
+    jaw_landmarks_ids = [61, 291, 199]
+    some_landmarks_ids = jaw_landmarks_ids + [
+        key for key, _ in procrustes_landmark_basis
+    ]
+    some_landmarks_ids.sort()
+    return some_landmarks_ids
+
+
+def get_camera_properties(camera_id):
+    """
+    Getting the camera properties.
+
+    Parameters:
+        camera_id: camera ID
+    
+    Returns:
+        fr_size: The frame size
+        camera_matrix: The intrinsic matrix of the camera
+        dst_cof: distortion coefficients of the camera
+        pcf: An object that is needed for later calculations
+    """
+    print("Getting camera properties...")
+    fr_w, fr_h = 1280, 720
+    cap = cv2.VideoCapture(camera_id)  # (tp.CAMERA_ID, cv2.CAP_DSHOW)
+    cap.set(cv2.CAP_PROP_FRAME_WIDTH, fr_w)
+    cap.set(cv2.CAP_PROP_FRAME_HEIGHT, fr_h)
+
+    new_fr_w = cap.get(cv2.CAP_PROP_FRAME_WIDTH)
+    new_fr_h = cap.get(cv2.CAP_PROP_FRAME_HEIGHT)
+    cap.release()
+    fr_size = new_fr_w, new_fr_h
+
+    fr_center = (new_fr_w // 2, new_fr_h // 2)
+    focal_length = new_fr_w
+    camera_matrix = np.array([
+        [focal_length, 0, fr_center[0]],
+        [0, focal_length, fr_center[1]],
+        [0, 0, 1]], dtype="double")
+    dst_cof = np.zeros((4, 1))
+
+    pcf = PCF(
+        frame_height=fr_h,
+        frame_width=fr_w,
+        fy=fr_w)
+    return fr_size, camera_matrix, dst_cof, pcf
+
+
+def get_camera(camera_id, frame_size):
+    """
+    Setting the camera
+    
+    Parameters:
+        camera_id: Camera ID
+        frame_size: The frame size
+    
+    Returns:
+        cap: The capture object"""
+    frame_w, frame_h = frame_size
+    cap = cv2.VideoCapture(camera_id)  # (camera_id, cv2.CAP_DSHOW)
+    cap.set(cv2.CAP_PROP_FRAME_WIDTH, frame_w)
+    cap.set(cv2.CAP_PROP_FRAME_HEIGHT, frame_h)
+    return cap
+
+
+def get_frame(cap):
+    """
+    Getting the frame
+
+    Parameters:
+        cap: The capture object
+    
+    Returns:
+        success: whether or not the frame is received
+        img: the frame (BGR)
+        img_rgb: the frame (RGB). It is needed for face mesh model
+    """
+    success, img = cap.read()
+    if success:
+        img = cv2.flip(img, 1)
+        img_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
+
+    else:
+        img = None
+        img_rgb = None
+
+    return success, img, img_rgb
+
+
+def get_eyes_pixels(eye_pixels):
+    """
+    Get eyes locations
+
+    Parameters:
+        eyes_pixels: eyes pixels
+    
+    Returns:
+        eye_top_left: eye top left
+        eye_bottom_right: eyes bottom right
+    """
+    pxl = np.min(eye_pixels[:, 0])
+    pxr = np.max(eye_pixels[:, 0])
+    pyt = np.min(eye_pixels[:, 1])
+    pyb = np.max(eye_pixels[:, 1])
+    ew = max(pxr - pxl, 25)
+    ht = int(0.35 * ew)
+    hb = int(0.25 * ew)
+    wl = int(0.2 * ew)
+    wr = int(0.1 * ew)
+    eye_top_left = pxl - wl, pyt - ht
+    eye_bottom_right = pxr + wr, pyb + hb
+    return eye_top_left, eye_bottom_right
+
+
+def get_face(all_landmarks_pixels):
+    """
+    Getting the face
+
+    Parameters:
+        all_landmarks_pixels: the landmarks
+    
+    Retruns:
+        face_left: face left
+        face_right: face_right
+        face_top: face top
+        face_bottom: face bottom
+    """
+    face_left = all_landmarks_pixels[:, 0].min()
+    face_right = all_landmarks_pixels[:, 0].max()
+    face_top = all_landmarks_pixels[:, 1].min()
+    face_bottom = all_landmarks_pixels[:, 1].max()
+
+    return face_left, face_right, face_top, face_bottom
+
+
+def get_eyes_ratio(all_landmarks):
+    """
+    Getting the eyes ratio
+
+    Parameters:
+        all_landmarks: all of the landmarks
+    
+    Returns:
+        ear: The eyes aspect ratio
+    """
+    wl = np.sqrt(((all_landmarks[33,:2]-all_landmarks[133,:2])**2).sum())
+    hl1 = np.sqrt(((all_landmarks[159,:2]-all_landmarks[145,:2])**2).sum())
+    hl2 = np.sqrt(((all_landmarks[158,:2]-all_landmarks[153,:2])**2).sum())
+    hl = (hl1 + hl2) / 2
+    
+    wr = np.sqrt(((all_landmarks[362,:2]-all_landmarks[263,:2])**2).sum())
+    hr1 = np.sqrt(((all_landmarks[385,:2]-all_landmarks[380,:2])**2).sum())
+    hr2 = np.sqrt(((all_landmarks[386,:2]-all_landmarks[374,:2])**2).sum())
+    hr = (hr1 + hr2) / 2
+
+    ear = ((wl / hl + wr / hr) / 2)
+    
+    return ear
+
+
+def get_model_inputs(
+        image,
+        image_rgb,
+        face_mesh,
+        camera_matrix,
+        pcf,
+        image_size,
+        dst_cof,
+        some_landmarks_ids,
+        show_features=False,
+        return_face=False
+):
+    """
+    Preparing the models inputs. Eyes images, face rotation, face position, iris locations in image
+
+    Parameters:
+        image: the frame (BGR)
+        image_rgb: the frame (RGB) for face mesh model
+        face_mesh: The face mesh model
+        camera_matrix: The intrinsic matrix
+        pcf: pcf object for calculating the face vectors
+        image_size: frame size
+        dst_cof: distortion coefficients of the camera
+        some_landmarks_ids: the landmarks needed for calculation of face vectors
+        show_features: whether or not show the inputs
+        return_face: whether or not return the face image
+    
+    Returns:
+        success: whether or not the eyes extraction was successful
+        image: the frame
+        eyes_gray: the eyes image which is gray scale
+        features_vector: 10 values for face rotation, face position and iris locations
+        eye_ratio: The eyes aspect ratio
+        face_img: face image
+    """
+    left_eye_landmarks_ids = (33, 133)
+    right_eye_landmarks_ids = (362, 263)
+    jaw_landmarks_ids = (61, 291, 199)
+    focal_length = camera_matrix[0, 0]
+    success = False
+    eyes_gray = None
+    features = []
+    features_vector = []
+    eye_ratio = 0.0
+    face_img = []
+    face_size = (300, 350)
+
+    mfl = face_mesh.multi_face_landmarks
+    # If there is any landmark
+    if mfl:
+        all_landmarks = np.array([(lm.x, lm.y, lm.z) for lm in mfl[0].landmark])
+        all_landmarks_pixels = np.array(all_landmarks[:,:2] * image_size, np.uint32)
+
+        eye_ratio = get_eyes_ratio(all_landmarks)
+
+        head_pose_landmarks = all_landmarks.T
+        metric_landmarks, _ = get_metric_landmarks(head_pose_landmarks.copy(), pcf)
+
+        some_landmarks_model = metric_landmarks[:, some_landmarks_ids].T
+        some_landmarks_image = (all_landmarks[some_landmarks_ids, :2] * image_size)
+
+        # Caluculating the face vector and face position
+        (
+            _,
+            rotation_vector,
+            translation_vector
+        ) = cv2.solvePnP(
+            some_landmarks_model,
+            some_landmarks_image,
+            camera_matrix,
+            dst_cof,
+            flags=cv2.SOLVEPNP_ITERATIVE
+        )
+
+        features.append(rotation_vector.reshape((3,)))
+        features.append(translation_vector.reshape((3,)))
+
+        # calculating iris location
+        (
+            success_left,
+            _,
+            _,
+            left_iris_landmarks,
+            _,
+            left_iris_landmarks_respect_face
+        ) = fl2d(
+            image_rgb,
+            all_landmarks[left_eye_landmarks_ids, :].T,
+            image_size,
+            is_right_eye=False,
+            focal_length=focal_length
+        )
+
+        (
+            success_right,
+            _,
+            _,
+            right_iris_landmarks,
+            _,
+            right_iris_landmarks_respect_face
+        ) = fl2d(
+            image_rgb,
+            all_landmarks[right_eye_landmarks_ids, :].T,
+            image_size,
+            is_right_eye=True,
+            focal_length=focal_length
+        )
+
+        if success_left and success_right:
+            left_eye_pixels = all_landmarks_pixels[left_eye_landmarks_ids, :2]
+            left_eye_tl, left_eye_br = get_eyes_pixels(left_eye_pixels)
+            eye_left = image_rgb[left_eye_tl[1]:left_eye_br[1], left_eye_tl[0]:left_eye_br[0]]
+
+            right_eye_pixels = all_landmarks_pixels[right_eye_landmarks_ids, :2]
+            right_eye_tl, right_eye_br = get_eyes_pixels(right_eye_pixels)
+            eye_right = image_rgb[right_eye_tl[1]:right_eye_br[1], right_eye_tl[0]:right_eye_br[0]]
+
+            if eye_left.any() and eye_right.any():
+                success = True
+                
+                features.append(left_iris_landmarks_respect_face[0, :2])
+                features.append(right_iris_landmarks_respect_face[0, :2])
+
+                for feats in features:
+                    for feat in feats:
+                        features_vector.append(feat)
+                features_vector = np.array(features_vector)
+
+                eye_left_resize = cv2.resize(eye_left, EYE_SIZE, interpolation=cv2.INTER_AREA)
+                eye_right_resize = cv2.resize(eye_right, EYE_SIZE, interpolation=cv2.INTER_AREA)
+
+                eyes = np.concatenate([eye_left_resize, eye_right_resize])
+                eyes_gray = np.expand_dims(cv2.cvtColor(eyes, cv2.COLOR_RGB2GRAY), 2)
+
+                if return_face:
+                    fp = get_face(all_landmarks_pixels)
+                    face_img = image[fp[2]:fp[3], fp[0]:fp[1]]
+                    face_img = cv2.resize(face_img, face_size, interpolation=cv2.INTER_AREA)
+
+                if show_features:
+                    cv2.rectangle(image, left_eye_tl, left_eye_br, (190, 100, 40), 2)
+                    cv2.rectangle(image, right_eye_tl, right_eye_br, (190, 100, 40), 2)
+
+                    jaw_landmarks_pixels = all_landmarks_pixels[jaw_landmarks_ids, :2]
+                    for pix in jaw_landmarks_pixels:
+                        cv2.circle(image, pix, 2, (0, 255, 255), cv2.FILLED)
+
+                    left_eye_landmarks_pixels = all_landmarks_pixels[left_eye_landmarks_ids, :2]
+                    for pix in left_eye_landmarks_pixels:
+                        cv2.circle(image, pix, 2, (255, 0, 255), cv2.FILLED)
+
+                    right_eye_landmarks_pixels = all_landmarks_pixels[right_eye_landmarks_ids, :2]
+                    for pix in right_eye_landmarks_pixels:
+                        cv2.circle(image, pix, 2, (255, 0, 255), cv2.FILLED)
+
+                    left_iris_pixel = np.array(
+                        left_iris_landmarks[0, :2] * image_size).astype(np.uint32)
+                    cv2.circle(image, left_iris_pixel, 4, (255, 255, 0), cv2.FILLED)
+
+                    right_iris_pixel = np.array(
+                        right_iris_landmarks[0, :2] * image_size).astype(np.uint32)
+                    cv2.circle(image, right_iris_pixel, 4, (255, 255, 0), cv2.FILLED)
+
+                    (nose_end_point2D, _) = cv2.projectPoints(
+                        np.array([(0.0, 0.0, 25.0)]),
+                        rotation_vector,
+                        translation_vector,
+                        camera_matrix,
+                        dst_cof,
+                    )
+
+                    p1 = (int(some_landmarks_image[0][0]), int(some_landmarks_image[0][1]))
+                    p2 = (int(nose_end_point2D[0][0][0]), int(nose_end_point2D[0][0][1]))
+
+                    cv2.line(image, p1, p2, (127, 64, 255), 2)
+
+    return success, image, eyes_gray, features_vector, eye_ratio, face_img
+
+
+def get_time(i, t, print_time=False):
+    """
+    getting time
+
+    Parameters:
+        i: the iterator
+        t: the time
+        print_time: whether or not print the time
+
+    Returns:
+        fps: Frame per second
+    """
+    el_t = time.perf_counter() - t
+    fps = round(i / el_t, 2)
+    if print_time:
+        print(f"Elapsed time: {int(el_t / 60)}:{int(el_t % 60)}")
+    return fps
+
+
+def create_dir(folders_list):
+    """
+    creating direcotry
+
+    Parameters:
+        folders_list: folders' list
+
+    Returns:
+        fol_dir: folder directory
+    """
+    fol_dir = ""
+    for fol in folders_list:
+        if fol[-1] != "/":
+            fol += "/"
+        fol_dir += fol
+        if not os.path.exists(fol_dir):
+            os.mkdir(fol_dir)
+
+    return fol_dir
+
+
+def load(fol_dir, data_name):
+    """
+    Loading the the data
+
+    Parameters:
+        fol_dir: folder directory
+        data_name: the data name
+    
+    Returns:
+        data: the loaded data
+    """
+    print("Loading data from " + fol_dir)
+    data = []
+    for dn in data_name:
+        with open(fol_dir + dn + ".pickle", 'rb') as f:
+            data.append(pickle.load(f))
+    return data
+
+
+def save(data, fol_dir, data_name):
+    """
+    Saving the data
+
+    Parameters:
+        data: the data that we want to save
+        fol_dir: the folder directory
+        data_nmae: name of the file that we want put
+    
+    Returns:
+        None
+    """
+    print("Saving data in " + fol_dir)
+    for (d, dn) in zip(data, data_name):
+        with open(fol_dir + dn + ".pickle", 'wb') as f:
+            pickle.dump(d, f)
+
+
+def remove(fol_dir, files=None):
+    """
+    Removing the files
+
+    Parameters:
+        fol_dir: folder directory
+        files: the file that we want to remove
+
+    Returns:
+        None
+    """
+    if files:
+        for fn in files:
+            file_dir = fol_dir + fn + ".pickle"
+            print("Removing " + file_dir)
+            os.remove(file_dir)
+    else:
+        print("Removing " + fol_dir)
+        shutil.rmtree(fol_dir)
+
+
+def file_existing(fol_dir, file_name):
+    """
+    Checking the existance of the file
+
+    Parameters:
+        fol_dir: folder directory
+        file_name: file name
+    
+    Returns:
+        file_exist: whether or not the file exists
+    """
+    files = os.listdir(fol_dir)
+    file_exist = False
+    if files:
+        for f in files:
+            if f == file_name:
+                file_exist = True
+
+    return file_exist
+
+
+def pass_frames(cap, n_frames=5):
+    """
+    Skipping the some frames
+
+    Parameters:
+        cap: camera objec
+        n_frames: number of frames that we want to pass
+    
+    Returns:
+        None
+    """
+    for _ in range(n_frames):
+        get_frame(cap)
+
+
+def show_clb_win(
+    win_name,
+    pnt=None,
+    pnt_prd=None,
+    texts=None,
+    win_color=BLACK,
+    win_size=(640, 480),
+    pnt_color=WHITE,
+    pnt_prd_color=BLUE
+    ):
+    """
+    Showing the calibration window
+
+    Parameters:
+        win_name: the windows name
+        pnt: the point for calibration
+        pnt_prd: the predicted point
+        texts: the texts that we want to put in the window
+        win_color: the window color
+        win_size: window size
+        pnt_color: the calibration point color
+        pnt_prd_color: the predicted point color
+    
+    Returns:
+        None
+    """
+    pnt_d = int(win_size[0] / 80.0)
+    clb_img = np.ones((win_size[1], win_size[0], 3))
+    clb_img[:, :, 0] = clb_img[:, :, 0] * win_color[0]
+    clb_img[:, :, 1] = clb_img[:, :, 1] * win_color[1]
+    clb_img[:, :, 2] = clb_img[:, :, 2] * win_color[2]
+    clb_img = clb_img.astype(np.uint8)
+    if np.array(pnt).any():
+        pxl = (np.array(pnt) * np.array(win_size)).astype(np.uint32)
+        cv2.circle(clb_img, pxl, pnt_d, pnt_color, cv2.FILLED)
+    if np.array(pnt_prd).any():
+        pxl_prd = (np.array(pnt_prd) * np.array(win_size)).astype(np.uint32)
+        cv2.circle(clb_img, pxl_prd, int(pnt_d / 2), pnt_prd_color, cv2.FILLED)
+    if texts:
+        for tx in texts:
+            cv2.putText(clb_img,
+                tx[0],
+                (int(win_size[0]*tx[1][0]), int(win_size[1]*tx[1][1])),
+                cv2.FONT_HERSHEY_SIMPLEX,
+                tx[2],
+                tx[3],
+                tx[4])
+    cv2.imshow(win_name, clb_img)
+
+
+def big_win(win_name="", x_disp=0, y_disp=0):
+    """
+    Make the calibration window full size
+
+    Paramters:
+        win_name: window name
+        x_disp: x coordinate
+        y_disp: y coordinate
+    
+    Returns:
+        None
+    """
+    cv2.namedWindow(win_name, cv2.WND_PROP_FULLSCREEN)
+    cv2.moveWindow(win_name, x_disp, y_disp)
+    cv2.setWindowProperty(win_name, cv2.WND_PROP_FULLSCREEN, cv2.WINDOW_FULLSCREEN)
+
+
+def get_blink_v(t_vec, eyes_ratio):
+    """
+    Calculating the blinking vector's velocity
+
+    Parameters:
+        t_vec: time vector
+        eyes_ratio: eyes aspect ratio vector
+    
+    Returns:
+        blink_v: velocity of eyes aspect ratio vector
+    """
+    blink_v = eyes_ratio.copy()
+
+    blink_v[1:] = (eyes_ratio[1:] - eyes_ratio[:-1]) / (t_vec[1:] - t_vec[:-1])
+    blink_v[0] = blink_v[1]
+
+    return blink_v
+
+
+def get_blink_duration(t_vec, blinking_period):
+    """
+    Calculating the blink duration
+
+    Parameters:
+        t_vec: time vector
+        blinking_period: blinking period
+    
+    Returns:
+        before_closing: before closing sample
+        after_closing: after closing sample
+    """
+    dt = 1 / (t_vec[1:] - t_vec[:-1])
+    fps = dt.mean()
+    sampling_period = 1/fps
+    n_smp_blink = round(blinking_period/sampling_period)
+    before_closing = math.floor(n_smp_blink / 3)
+    after_closing = math.floor(2 * n_smp_blink / 3) - 1
+
+    return before_closing, after_closing
+
+
+def get_blinking_vec(eyes_ratio_v, bc, ac, threshold):
+    """
+    getting blinking vector
+
+    Parameters:
+        eyes_ratio_v: the vector of eye aspect ratio velocity
+        bc: before closing
+        ac: after closing
+        threshold: blinking threshold
+    
+    Returns:
+        blinking: whether or not the user is blinking
+        eys_ratio_v_blink: the vector of blinking
+    """
+    closed_eyes = (eyes_ratio_v > threshold)
+    blinking = closed_eyes.copy()
+    n_smp = blinking.shape[0]
+    eyes_ratio_v_blink = np.zeros((n_smp,))
+
+    for (i, ce) in enumerate(closed_eyes):
+        if ce and (i > bc) and (i < n_smp-ac):
+            for j in range(1, bc+1):
+                blinking[i-j] = True
+            for j in range(1, ac+1):
+                blinking[i+j] = True
+
+    eyes_ratio_v_blink[blinking] = threshold
+
+    return blinking, eyes_ratio_v_blink
+
+
+def get_blinking(t_mat, eyes_ratio_mat, threshold=DEFAULT_BLINKING_THRESHOLD, normal_blinking_period=0.4):
+    """
+    Getting blinking
+
+    Parameters:
+        t_mat: a list of time vectors
+        eyes_ratio_mat: a list of eyes aspect ratio vectors
+        threshold: blinking threshold
+        normal_blinking_period: normal blinking threshold
+    
+    Returns:
+        eyes_ratio_v_mat: a list of eyes aspect ratio velocity vector
+        blinking_mat: a list of blinking vectors
+        eyes_ratio_v_blink_mat: a list of eyes aspect ratio boolians vectors
+    """
+    eyes_ratio_v_mat = []
+    blinking_mat = []
+    eyes_ratio_v_blink_mat = []
+
+    for (k, eyes_ratio) in enumerate(eyes_ratio_mat):
+        t_vec = t_mat[k]
+
+        eyes_ratio_v_mat.append(get_blink_v(t_vec, eyes_ratio).copy())
+
+        bc, ac = get_blink_duration(t_vec, normal_blinking_period)
+
+        blinking, eyes_ratio_v_blink = get_blinking_vec(eyes_ratio_v_mat[-1], bc, ac, threshold)
+
+        blinking_mat.append(blinking)
+        eyes_ratio_v_blink_mat.append(eyes_ratio_v_blink)
+    
+    return eyes_ratio_v_mat, blinking_mat, eyes_ratio_v_blink_mat
+
+
+def find_max_mdl(fol_dir, a=3, b=-3):
+    """
+    finding the maximum model number
+
+    Parameters:
+        fol_dir: folder directory
+        a: the first index of the model number
+        b: the last index of the model number
+    
+    Returns:
+        max_num: maximum model number
+    """
+    mdl_numbers = []
+    mdl_name = os.listdir(fol_dir)
+    if mdl_name:
+        for mn in mdl_name:
+            if mn[-3:] == ".h5":
+                mdl_num = int(mn[a:b])
+                mdl_numbers.append(mdl_num)
+        max_num = max(mdl_numbers)
+    else:
+        max_num = 0
+
+    return max_num
+
+
+def get_threshold(er_dir, threshold):
+    """
+    Getting the threshold. default, user offered or application offered
+
+    Parameters:
+        er_dir: directory of er file
+        thereshold: the threshold, 'd', 'ao', or 'uo'
+
+    Returns:
+        threshold: the threshold value
+    """
+    if threshold == "d":
+        threshold = DEFAULT_BLINKING_THRESHOLD
+    elif threshold == "ao":
+        oth = "oth_app"
+        fe = file_existing(er_dir, oth + ".pickle")
+        if fe:
+            threshold = load(er_dir , [oth])[0]
+        else:
+            print("App offered threshold does not exist!! We use default threshold.")
+            threshold = DEFAULT_BLINKING_THRESHOLD
+    elif threshold == "uo":
+        oth = "oth_usr"
+        fe = file_existing(er_dir, oth + ".pickle")
+        if fe:
+            threshold = load(er_dir , [oth])[0]
+        else:
+            print("User offered threshold does not exist!! We use default threshold.")
+            threshold = DEFAULT_BLINKING_THRESHOLD
+    else:
+        threshold = float(threshold)
+
+    print(f"blinking threshold: {threshold}")
+    return threshold
+
+
+# Getting some directories
+models_dir = create_dir([PATH2ROOT_ABS+"models"])
+io_dir = create_dir([models_dir+"io"])
+et_dir = create_dir([models_dir+"et"])
+io_raw_dir = create_dir([io_dir, RAW])
+io_trained_dir = create_dir([io_dir, TRAINED])
+et_raw_dir = create_dir([et_dir, RAW])
+et_trained_dir = create_dir([et_dir, TRAINED])
+files_dir = create_dir([PATH2ROOT_ABS+"other_files"])
+scalers_dir = create_dir([files_dir, "scalers"])
+subjects_dir = create_dir([PATH2ROOT+"subjects"])
+monitors = get_monitors()

codes/base/face_geometry.py

@@ -0,0 +1,2678 @@
+# Many parts taken from the cpp implementation from github.com/google/mediapipe
+#
+# Copyright 2020 The MediaPipe Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import numpy as np
+
+
+class Singleton(type):
+    _instances = {}
+
+    def __call__(cls, *args, **kwargs):
+        if cls not in cls._instances:
+            cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)
+        return cls._instances[cls]
+
+
+class Debugger(metaclass=Singleton):
+    def set_debug(self, debug):
+        self.debug = debug
+
+    def toggle(self):
+        self.debug = not self.debug
+
+    def get_debug(self):
+        return self.debug
+
+
+DEBUG = Debugger()
+DEBUG.set_debug(False)
+
+
+class PCF:
+    def __init__(
+        self,
+        near=1,
+        far=10000,
+        frame_height=1920,
+        frame_width=1080,
+        fy=1074.520446598223,
+    ):
+
+        self.near = near
+        self.far = far
+        self.frame_height = frame_height
+        self.frame_width = frame_width
+        self.fy = fy
+
+        fov_y = 2 * np.arctan(frame_height / (2 * fy))
+        
+        # kDegreesToRadians = np.pi / 180.0 # never used
+        height_at_near = 2 * near * np.tan(0.5 * fov_y)
+        width_at_near = frame_width * height_at_near / frame_height
+        # print(height_at_near)
+        
+        self.fov_y = fov_y
+        self.left = -0.5 * width_at_near
+        self.right = 0.5 * width_at_near
+        self.bottom = -0.5 * height_at_near
+        self.top = 0.5 * height_at_near
+
+
+canonical_metric_landmarks = np.array(
+    [
+        0.000000,
+        -3.406404,
+        5.979507,
+        0.499977,
+        0.652534,
+        0.000000,
+        -1.126865,
+        7.475604,
+        0.500026,
+        0.547487,
+        0.000000,
+        -2.089024,
+        6.058267,
+        0.499974,
+        0.602372,
+        -0.463928,
+        0.955357,
+        6.633583,
+        0.482113,
+        0.471979,
+        0.000000,
+        -0.463170,
+        7.586580,
+        0.500151,
+        0.527156,
+        0.000000,
+        0.365669,
+        7.242870,
+        0.499910,
+        0.498253,
+        0.000000,
+        2.473255,
+        5.788627,
+        0.499523,
+        0.401062,
+        -4.253081,
+        2.577646,
+        3.279702,
+        0.289712,
+        0.380764,
+        0.000000,
+        4.019042,
+        5.284764,
+        0.499955,
+        0.312398,
+        0.000000,
+        4.885979,
+        5.385258,
+        0.499987,
+        0.269919,
+        0.000000,
+        8.261778,
+        4.481535,
+        0.500023,
+        0.107050,
+        0.000000,
+        -3.706811,
+        5.864924,
+        0.500023,
+        0.666234,
+        0.000000,
+        -3.918301,
+        5.569430,
+        0.500016,
+        0.679224,
+        0.000000,
+        -3.994436,
+        5.219482,
+        0.500023,
+        0.692348,
+        0.000000,
+        -4.542400,
+        5.404754,
+        0.499977,
+        0.695278,
+        0.000000,
+        -4.745577,
+        5.529457,
+        0.499977,
+        0.705934,
+        0.000000,
+        -5.019567,
+        5.601448,
+        0.499977,
+        0.719385,
+        0.000000,
+        -5.365123,
+        5.535441,
+        0.499977,
+        0.737019,
+        0.000000,
+        -6.149624,
+        5.071372,
+        0.499968,
+        0.781371,
+        0.000000,
+        -1.501095,
+        7.112196,
+        0.499816,
+        0.562981,
+        -0.416106,
+        -1.466449,
+        6.447657,
+        0.473773,
+        0.573910,
+        -7.087960,
+        5.434801,
+        0.099620,
+        0.104907,
+        0.254141,
+        -2.628639,
+        2.035898,
+        3.848121,
+        0.365930,
+        0.409576,
+        -3.198363,
+        1.985815,
+        3.796952,
+        0.338758,
+        0.413025,
+        -3.775151,
+        2.039402,
+        3.646194,
+        0.311120,
+        0.409460,
+        -4.465819,
+        2.422950,
+        3.155168,
+        0.274658,
+        0.389131,
+        -2.164289,
+        2.189867,
+        3.851822,
+        0.393362,
+        0.403706,
+        -3.208229,
+        3.223926,
+        4.115822,
+        0.345234,
+        0.344011,
+        -2.673803,
+        3.205337,
+        4.092203,
+        0.370094,
+        0.346076,
+        -3.745193,
+        3.165286,
+        3.972409,
+        0.319322,
+        0.347265,
+        -4.161018,
+        3.059069,
+        3.719554,
+        0.297903,
+        0.353591,
+        -5.062006,
+        1.934418,
+        2.776093,
+        0.247792,
+        0.410810,
+        -2.266659,
+        -7.425768,
+        4.389812,
+        0.396889,
+        0.842755,
+        -4.445859,
+        2.663991,
+        3.173422,
+        0.280098,
+        0.375600,
+        -7.214530,
+        2.263009,
+        0.073150,
+        0.106310,
+        0.399956,
+        -5.799793,
+        2.349546,
+        2.204059,
+        0.209925,
+        0.391353,
+        -2.844939,
+        -0.720868,
+        4.433130,
+        0.355808,
+        0.534406,
+        -0.711452,
+        -3.329355,
+        5.877044,
+        0.471751,
+        0.650404,
+        -0.606033,
+        -3.924562,
+        5.444923,
+        0.474155,
+        0.680192,
+        -1.431615,
+        -3.500953,
+        5.496189,
+        0.439785,
+        0.657229,
+        -1.914910,
+        -3.803146,
+        5.028930,
+        0.414617,
+        0.666541,
+        -1.131043,
+        -3.973937,
+        5.189648,
+        0.450374,
+        0.680861,
+        -1.563548,
+        -4.082763,
+        4.842263,
+        0.428771,
+        0.682691,
+        -2.650112,
+        -5.003649,
+        4.188483,
+        0.374971,
+        0.727805,
+        -0.427049,
+        -1.094134,
+        7.360529,
+        0.486717,
+        0.547629,
+        -0.496396,
+        -0.475659,
+        7.440358,
+        0.485301,
+        0.527395,
+        -5.253307,
+        3.881582,
+        3.363159,
+        0.257765,
+        0.314490,
+        -1.718698,
+        0.974609,
+        4.558359,
+        0.401223,
+        0.455172,
+        -1.608635,
+        -0.942516,
+        5.814193,
+        0.429819,
+        0.548615,
+        -1.651267,
+        -0.610868,
+        5.581319,
+        0.421352,
+        0.533741,
+        -4.765501,
+        -0.701554,
+        3.534632,
+        0.276896,
+        0.532057,
+        -0.478306,
+        0.295766,
+        7.101013,
+        0.483370,
+        0.499587,
+        -3.734964,
+        4.508230,
+        4.550454,
+        0.337212,
+        0.282883,
+        -4.588603,
+        4.302037,
+        4.048484,
+        0.296392,
+        0.293243,
+        -6.279331,
+        6.615427,
+        1.425850,
+        0.169295,
+        0.193814,
+        -1.220941,
+        4.142165,
+        5.106035,
+        0.447580,
+        0.302610,
+        -2.193489,
+        3.100317,
+        4.000575,
+        0.392390,
+        0.353888,
+        -3.102642,
+        -4.352984,
+        4.095905,
+        0.354490,
+        0.696784,
+        -6.719682,
+        -4.788645,
+        -1.745401,
+        0.067305,
+        0.730105,
+        -1.193824,
+        -1.306795,
+        5.737747,
+        0.442739,
+        0.572826,
+        -0.729766,
+        -1.593712,
+        5.833208,
+        0.457098,
+        0.584792,
+        -2.456206,
+        -4.342621,
+        4.283884,
+        0.381974,
+        0.694711,
+        -2.204823,
+        -4.304508,
+        4.162499,
+        0.392389,
+        0.694203,
+        -4.985894,
+        4.802461,
+        3.751977,
+        0.277076,
+        0.271932,
+        -1.592294,
+        -1.257709,
+        5.456949,
+        0.422552,
+        0.563233,
+        -2.644548,
+        4.524654,
+        4.921559,
+        0.385919,
+        0.281364,
+        -2.760292,
+        5.100971,
+        5.015990,
+        0.383103,
+        0.255840,
+        -3.523964,
+        8.005976,
+        3.729163,
+        0.331431,
+        0.119714,
+        -5.599763,
+        5.715470,
+        2.724259,
+        0.229924,
+        0.232003,
+        -3.063932,
+        6.566144,
+        4.529981,
+        0.364501,
+        0.189114,
+        -5.720968,
+        4.254584,
+        2.830852,
+        0.229622,
+        0.299541,
+        -6.374393,
+        4.785590,
+        1.591691,
+        0.173287,
+        0.278748,
+        -0.672728,
+        -3.688016,
+        5.737804,
+        0.472879,
+        0.666198,
+        -1.262560,
+        -3.787691,
+        5.417779,
+        0.446828,
+        0.668527,
+        -1.732553,
+        -3.952767,
+        5.000579,
+        0.422762,
+        0.673890,
+        -1.043625,
+        -1.464973,
+        5.662455,
+        0.445308,
+        0.580066,
+        -2.321234,
+        -4.329069,
+        4.258156,
+        0.388103,
+        0.693961,
+        -2.056846,
+        -4.477671,
+        4.520883,
+        0.403039,
+        0.706540,
+        -2.153084,
+        -4.276322,
+        4.038093,
+        0.403629,
+        0.693953,
+        -0.946874,
+        -1.035249,
+        6.512274,
+        0.460042,
+        0.557139,
+        -1.469132,
+        -4.036351,
+        4.604908,
+        0.431158,
+        0.692366,
+        -1.024340,
+        -3.989851,
+        4.926693,
+        0.452182,
+        0.692366,
+        -0.533422,
+        -3.993222,
+        5.138202,
+        0.475387,
+        0.692366,
+        -0.769720,
+        -6.095394,
+        4.985883,
+        0.465828,
+        0.779190,
+        -0.699606,
+        -5.291850,
+        5.448304,
+        0.472329,
+        0.736226,
+        -0.669687,
+        -4.949770,
+        5.509612,
+        0.473087,
+        0.717857,
+        -0.630947,
+        -4.695101,
+        5.449371,
+        0.473122,
+        0.704626,
+        -0.583218,
+        -4.517982,
+        5.339869,
+        0.473033,
+        0.695278,
+        -1.537170,
+        -4.423206,
+        4.745470,
+        0.427942,
+        0.695278,
+        -1.615600,
+        -4.475942,
+        4.813632,
+        0.426479,
+        0.703540,
+        -1.729053,
+        -4.618680,
+        4.854463,
+        0.423162,
+        0.711846,
+        -1.838624,
+        -4.828746,
+        4.823737,
+        0.418309,
+        0.720063,
+        -2.368250,
+        -3.106237,
+        4.868096,
+        0.390095,
+        0.639573,
+        -7.542244,
+        -1.049282,
+        -2.431321,
+        0.013954,
+        0.560034,
+        0.000000,
+        -1.724003,
+        6.601390,
+        0.499914,
+        0.580147,
+        -1.826614,
+        -4.399531,
+        4.399021,
+        0.413200,
+        0.695400,
+        -1.929558,
+        -4.411831,
+        4.497052,
+        0.409626,
+        0.701823,
+        -0.597442,
+        -2.013686,
+        5.866456,
+        0.468080,
+        0.601535,
+        -1.405627,
+        -1.714196,
+        5.241087,
+        0.422729,
+        0.585985,
+        -0.662449,
+        -1.819321,
+        5.863759,
+        0.463080,
+        0.593784,
+        -2.342340,
+        0.572222,
+        4.294303,
+        0.372120,
+        0.473414,
+        -3.327324,
+        0.104863,
+        4.113860,
+        0.334562,
+        0.496073,
+        -1.726175,
+        -0.919165,
+        5.273355,
+        0.411671,
+        0.546965,
+        -5.133204,
+        7.485602,
+        2.660442,
+        0.242176,
+        0.147676,
+        -4.538641,
+        6.319907,
+        3.683424,
+        0.290777,
+        0.201446,
+        -3.986562,
+        5.109487,
+        4.466315,
+        0.327338,
+        0.256527,
+        -2.169681,
+        -5.440433,
+        4.455874,
+        0.399510,
+        0.748921,
+        -1.395634,
+        5.011963,
+        5.316032,
+        0.441728,
+        0.261676,
+        -1.619500,
+        6.599217,
+        4.921106,
+        0.429765,
+        0.187834,
+        -1.891399,
+        8.236377,
+        4.274997,
+        0.412198,
+        0.108901,
+        -4.195832,
+        2.235205,
+        3.375099,
+        0.288955,
+        0.398952,
+        -5.733342,
+        1.411738,
+        2.431726,
+        0.218937,
+        0.435411,
+        -1.859887,
+        2.355757,
+        3.843181,
+        0.412782,
+        0.398970,
+        -4.988612,
+        3.074654,
+        3.083858,
+        0.257135,
+        0.355440,
+        -1.303263,
+        1.416453,
+        4.831091,
+        0.427685,
+        0.437961,
+        -1.305757,
+        -0.672779,
+        6.415959,
+        0.448340,
+        0.536936,
+        -6.465170,
+        0.937119,
+        1.689873,
+        0.178560,
+        0.457554,
+        -5.258659,
+        0.945811,
+        2.974312,
+        0.247308,
+        0.457194,
+        -4.432338,
+        0.722096,
+        3.522615,
+        0.286267,
+        0.467675,
+        -3.300681,
+        0.861641,
+        3.872784,
+        0.332828,
+        0.460712,
+        -2.430178,
+        1.131492,
+        4.039035,
+        0.368756,
+        0.447207,
+        -1.820731,
+        1.467954,
+        4.224124,
+        0.398964,
+        0.432655,
+        -0.563221,
+        2.307693,
+        5.566789,
+        0.476410,
+        0.405806,
+        -6.338145,
+        -0.529279,
+        1.881175,
+        0.189241,
+        0.523924,
+        -5.587698,
+        3.208071,
+        2.687839,
+        0.228962,
+        0.348951,
+        -0.242624,
+        -1.462857,
+        7.071491,
+        0.490726,
+        0.562401,
+        -1.611251,
+        0.339326,
+        4.895421,
+        0.404670,
+        0.485133,
+        -7.743095,
+        2.364999,
+        -2.005167,
+        0.019469,
+        0.401564,
+        -1.391142,
+        1.851048,
+        4.448999,
+        0.426243,
+        0.420431,
+        -1.785794,
+        -0.978284,
+        4.850470,
+        0.396993,
+        0.548797,
+        -4.670959,
+        2.664461,
+        3.084075,
+        0.266470,
+        0.376977,
+        -1.333970,
+        -0.283761,
+        6.097047,
+        0.439121,
+        0.518958,
+        -7.270895,
+        -2.890917,
+        -2.252455,
+        0.032314,
+        0.644357,
+        -1.856432,
+        2.585245,
+        3.757904,
+        0.419054,
+        0.387155,
+        -0.923388,
+        0.073076,
+        6.671944,
+        0.462783,
+        0.505747,
+        -5.000589,
+        -6.135128,
+        1.892523,
+        0.238979,
+        0.779745,
+        -5.085276,
+        -7.178590,
+        0.714711,
+        0.198221,
+        0.831938,
+        -7.159291,
+        -0.811820,
+        -0.072044,
+        0.107550,
+        0.540755,
+        -5.843051,
+        -5.248023,
+        0.924091,
+        0.183610,
+        0.740257,
+        -6.847258,
+        3.662916,
+        0.724695,
+        0.134410,
+        0.333683,
+        -2.412942,
+        -8.258853,
+        4.119213,
+        0.385764,
+        0.883154,
+        -0.179909,
+        -1.689864,
+        6.573301,
+        0.490967,
+        0.579378,
+        -2.103655,
+        -0.163946,
+        4.566119,
+        0.382385,
+        0.508573,
+        -6.407571,
+        2.236021,
+        1.560843,
+        0.174399,
+        0.397671,
+        -3.670075,
+        2.360153,
+        3.635230,
+        0.318785,
+        0.396235,
+        -3.177186,
+        2.294265,
+        3.775704,
+        0.343364,
+        0.400597,
+        -2.196121,
+        -4.598322,
+        4.479786,
+        0.396100,
+        0.710217,
+        -6.234883,
+        -1.944430,
+        1.663542,
+        0.187885,
+        0.588538,
+        -1.292924,
+        -9.295920,
+        4.094063,
+        0.430987,
+        0.944065,
+        -3.210651,
+        -8.533278,
+        2.802001,
+        0.318993,
+        0.898285,
+        -4.068926,
+        -7.993109,
+        1.925119,
+        0.266248,
+        0.869701,
+        0.000000,
+        6.545390,
+        5.027311,
+        0.500023,
+        0.190576,
+        0.000000,
+        -9.403378,
+        4.264492,
+        0.499977,
+        0.954453,
+        -2.724032,
+        2.315802,
+        3.777151,
+        0.366170,
+        0.398822,
+        -2.288460,
+        2.398891,
+        3.697603,
+        0.393207,
+        0.395537,
+        -1.998311,
+        2.496547,
+        3.689148,
+        0.410373,
+        0.391080,
+        -6.130040,
+        3.399261,
+        2.038516,
+        0.194993,
+        0.342102,
+        -2.288460,
+        2.886504,
+        3.775031,
+        0.388665,
+        0.362284,
+        -2.724032,
+        2.961810,
+        3.871767,
+        0.365962,
+        0.355971,
+        -3.177186,
+        2.964136,
+        3.876973,
+        0.343364,
+        0.355357,
+        -3.670075,
+        2.927714,
+        3.724325,
+        0.318785,
+        0.358340,
+        -4.018389,
+        2.857357,
+        3.482983,
+        0.301415,
+        0.363156,
+        -7.555811,
+        4.106811,
+        -0.991917,
+        0.058133,
+        0.319076,
+        -4.018389,
+        2.483695,
+        3.440898,
+        0.301415,
+        0.387449,
+        0.000000,
+        -2.521945,
+        5.932265,
+        0.499988,
+        0.618434,
+        -1.776217,
+        -2.683946,
+        5.213116,
+        0.415838,
+        0.624196,
+        -1.222237,
+        -1.182444,
+        5.952465,
+        0.445682,
+        0.566077,
+        -0.731493,
+        -2.536683,
+        5.815343,
+        0.465844,
+        0.620641,
+        0.000000,
+        3.271027,
+        5.236015,
+        0.499923,
+        0.351524,
+        -4.135272,
+        -6.996638,
+        2.671970,
+        0.288719,
+        0.819946,
+        -3.311811,
+        -7.660815,
+        3.382963,
+        0.335279,
+        0.852820,
+        -1.313701,
+        -8.639995,
+        4.702456,
+        0.440512,
+        0.902419,
+        -5.940524,
+        -6.223629,
+        -0.631468,
+        0.128294,
+        0.791941,
+        -1.998311,
+        2.743838,
+        3.744030,
+        0.408772,
+        0.373894,
+        -0.901447,
+        1.236992,
+        5.754256,
+        0.455607,
+        0.451801,
+        0.000000,
+        -8.765243,
+        4.891441,
+        0.499877,
+        0.908990,
+        -2.308977,
+        -8.974196,
+        3.609070,
+        0.375437,
+        0.924192,
+        -6.954154,
+        -2.439843,
+        -0.131163,
+        0.114210,
+        0.615022,
+        -1.098819,
+        -4.458788,
+        5.120727,
+        0.448662,
+        0.695278,
+        -1.181124,
+        -4.579996,
+        5.189564,
+        0.448020,
+        0.704632,
+        -1.255818,
+        -4.787901,
+        5.237051,
+        0.447112,
+        0.715808,
+        -1.325085,
+        -5.106507,
+        5.205010,
+        0.444832,
+        0.730794,
+        -1.546388,
+        -5.819392,
+        4.757893,
+        0.430012,
+        0.766809,
+        -1.953754,
+        -4.183892,
+        4.431713,
+        0.406787,
+        0.685673,
+        -2.117802,
+        -4.137093,
+        4.555096,
+        0.400738,
+        0.681069,
+        -2.285339,
+        -4.051196,
+        4.582438,
+        0.392400,
+        0.677703,
+        -2.850160,
+        -3.665720,
+        4.484994,
+        0.367856,
+        0.663919,
+        -5.278538,
+        -2.238942,
+        2.861224,
+        0.247923,
+        0.601333,
+        -0.946709,
+        1.907628,
+        5.196779,
+        0.452770,
+        0.420850,
+        -1.314173,
+        3.104912,
+        4.231404,
+        0.436392,
+        0.359887,
+        -1.780000,
+        2.860000,
+        3.881555,
+        0.416164,
+        0.368714,
+        -1.845110,
+        -4.098880,
+        4.247264,
+        0.413386,
+        0.692366,
+        -5.436187,
+        -4.030482,
+        2.109852,
+        0.228018,
+        0.683572,
+        -0.766444,
+        3.182131,
+        4.861453,
+        0.468268,
+        0.352671,
+        -1.938616,
+        -6.614410,
+        4.521085,
+        0.411362,
+        0.804327,
+        0.000000,
+        1.059413,
+        6.774605,
+        0.499989,
+        0.469825,
+        -0.516573,
+        1.583572,
+        6.148363,
+        0.479154,
+        0.442654,
+        0.000000,
+        1.728369,
+        6.316750,
+        0.499974,
+        0.439637,
+        -1.246815,
+        0.230297,
+        5.681036,
+        0.432112,
+        0.493589,
+        0.000000,
+        -7.942194,
+        5.181173,
+        0.499886,
+        0.866917,
+        0.000000,
+        -6.991499,
+        5.153478,
+        0.499913,
+        0.821729,
+        -0.997827,
+        -6.930921,
+        4.979576,
+        0.456549,
+        0.819201,
+        -3.288807,
+        -5.382514,
+        3.795752,
+        0.344549,
+        0.745439,
+        -2.311631,
+        -1.566237,
+        4.590085,
+        0.378909,
+        0.574010,
+        -2.680250,
+        -6.111567,
+        4.096152,
+        0.374293,
+        0.780185,
+        -3.832928,
+        -1.537326,
+        4.137731,
+        0.319688,
+        0.570738,
+        -2.961860,
+        -2.274215,
+        4.440943,
+        0.357155,
+        0.604270,
+        -4.386901,
+        -2.683286,
+        3.643886,
+        0.295284,
+        0.621581,
+        -1.217295,
+        -7.834465,
+        4.969286,
+        0.447750,
+        0.862477,
+        -1.542374,
+        -0.136843,
+        5.201008,
+        0.410986,
+        0.508723,
+        -3.878377,
+        -6.041764,
+        3.311079,
+        0.313951,
+        0.775308,
+        -3.084037,
+        -6.809842,
+        3.814195,
+        0.354128,
+        0.812553,
+        -3.747321,
+        -4.503545,
+        3.726453,
+        0.324548,
+        0.703993,
+        -6.094129,
+        -3.205991,
+        1.473482,
+        0.189096,
+        0.646300,
+        -4.588995,
+        -4.728726,
+        2.983221,
+        0.279777,
+        0.714658,
+        -6.583231,
+        -3.941269,
+        0.070268,
+        0.133823,
+        0.682701,
+        -3.492580,
+        -3.195820,
+        4.130198,
+        0.336768,
+        0.644733,
+        -1.255543,
+        0.802341,
+        5.307551,
+        0.429884,
+        0.466522,
+        -1.126122,
+        -0.933602,
+        6.538785,
+        0.455528,
+        0.548623,
+        -1.443109,
+        -1.142774,
+        5.905127,
+        0.437114,
+        0.558896,
+        -0.923043,
+        -0.529042,
+        7.003423,
+        0.467288,
+        0.529925,
+        -1.755386,
+        3.529117,
+        4.327696,
+        0.414712,
+        0.335220,
+        -2.632589,
+        3.713828,
+        4.364629,
+        0.377046,
+        0.322778,
+        -3.388062,
+        3.721976,
+        4.309028,
+        0.344108,
+        0.320151,
+        -4.075766,
+        3.675413,
+        4.076063,
+        0.312876,
+        0.322332,
+        -4.622910,
+        3.474691,
+        3.646321,
+        0.283526,
+        0.333190,
+        -5.171755,
+        2.535753,
+        2.670867,
+        0.241246,
+        0.382786,
+        -7.297331,
+        0.763172,
+        -0.048769,
+        0.102986,
+        0.468763,
+        -4.706828,
+        1.651000,
+        3.109532,
+        0.267612,
+        0.424560,
+        -4.071712,
+        1.476821,
+        3.476944,
+        0.297879,
+        0.433176,
+        -3.269817,
+        1.470659,
+        3.731945,
+        0.333434,
+        0.433878,
+        -2.527572,
+        1.617311,
+        3.865444,
+        0.366427,
+        0.426116,
+        -1.970894,
+        1.858505,
+        3.961782,
+        0.396012,
+        0.416696,
+        -1.579543,
+        2.097941,
+        4.084996,
+        0.420121,
+        0.410228,
+        -7.664182,
+        0.673132,
+        -2.435867,
+        0.007561,
+        0.480777,
+        -1.397041,
+        -1.340139,
+        5.630378,
+        0.432949,
+        0.569518,
+        -0.884838,
+        0.658740,
+        6.233232,
+        0.458639,
+        0.479089,
+        -0.767097,
+        -0.968035,
+        7.077932,
+        0.473466,
+        0.545744,
+        -0.460213,
+        -1.334106,
+        6.787447,
+        0.476088,
+        0.563830,
+        -0.748618,
+        -1.067994,
+        6.798303,
+        0.468472,
+        0.555057,
+        -1.236408,
+        -1.585568,
+        5.480490,
+        0.433991,
+        0.582362,
+        -0.387306,
+        -1.409990,
+        6.957705,
+        0.483518,
+        0.562984,
+        -0.319925,
+        -1.607931,
+        6.508676,
+        0.482483,
+        0.577849,
+        -1.639633,
+        2.556298,
+        3.863736,
+        0.426450,
+        0.389799,
+        -1.255645,
+        2.467144,
+        4.203800,
+        0.438999,
+        0.396495,
+        -1.031362,
+        2.382663,
+        4.615849,
+        0.450067,
+        0.400434,
+        -4.253081,
+        2.772296,
+        3.315305,
+        0.289712,
+        0.368253,
+        -4.530000,
+        2.910000,
+        3.339685,
+        0.276670,
+        0.363373,
+        0.463928,
+        0.955357,
+        6.633583,
+        0.517862,
+        0.471948,
+        4.253081,
+        2.577646,
+        3.279702,
+        0.710288,
+        0.380764,
+        0.416106,
+        -1.466449,
+        6.447657,
+        0.526227,
+        0.573910,
+        7.087960,
+        5.434801,
+        0.099620,
+        0.895093,
+        0.254141,
+        2.628639,
+        2.035898,
+        3.848121,
+        0.634070,
+        0.409576,
+        3.198363,
+        1.985815,
+        3.796952,
+        0.661242,
+        0.413025,
+        3.775151,
+        2.039402,
+        3.646194,
+        0.688880,
+        0.409460,
+        4.465819,
+        2.422950,
+        3.155168,
+        0.725342,
+        0.389131,
+        2.164289,
+        2.189867,
+        3.851822,
+        0.606630,
+        0.403705,
+        3.208229,
+        3.223926,
+        4.115822,
+        0.654766,
+        0.344011,
+        2.673803,
+        3.205337,
+        4.092203,
+        0.629906,
+        0.346076,
+        3.745193,
+        3.165286,
+        3.972409,
+        0.680678,
+        0.347265,
+        4.161018,
+        3.059069,
+        3.719554,
+        0.702097,
+        0.353591,
+        5.062006,
+        1.934418,
+        2.776093,
+        0.752212,
+        0.410805,
+        2.266659,
+        -7.425768,
+        4.389812,
+        0.602918,
+        0.842863,
+        4.445859,
+        2.663991,
+        3.173422,
+        0.719902,
+        0.375600,
+        7.214530,
+        2.263009,
+        0.073150,
+        0.893693,
+        0.399960,
+        5.799793,
+        2.349546,
+        2.204059,
+        0.790082,
+        0.391354,
+        2.844939,
+        -0.720868,
+        4.433130,
+        0.643998,
+        0.534488,
+        0.711452,
+        -3.329355,
+        5.877044,
+        0.528249,
+        0.650404,
+        0.606033,
+        -3.924562,
+        5.444923,
+        0.525850,
+        0.680191,
+        1.431615,
+        -3.500953,
+        5.496189,
+        0.560215,
+        0.657229,
+        1.914910,
+        -3.803146,
+        5.028930,
+        0.585384,
+        0.666541,
+        1.131043,
+        -3.973937,
+        5.189648,
+        0.549626,
+        0.680861,
+        1.563548,
+        -4.082763,
+        4.842263,
+        0.571228,
+        0.682692,
+        2.650112,
+        -5.003649,
+        4.188483,
+        0.624852,
+        0.728099,
+        0.427049,
+        -1.094134,
+        7.360529,
+        0.513050,
+        0.547282,
+        0.496396,
+        -0.475659,
+        7.440358,
+        0.515097,
+        0.527252,
+        5.253307,
+        3.881582,
+        3.363159,
+        0.742247,
+        0.314507,
+        1.718698,
+        0.974609,
+        4.558359,
+        0.598631,
+        0.454979,
+        1.608635,
+        -0.942516,
+        5.814193,
+        0.570338,
+        0.548575,
+        1.651267,
+        -0.610868,
+        5.581319,
+        0.578632,
+        0.533623,
+        4.765501,
+        -0.701554,
+        3.534632,
+        0.723087,
+        0.532054,
+        0.478306,
+        0.295766,
+        7.101013,
+        0.516446,
+        0.499639,
+        3.734964,
+        4.508230,
+        4.550454,
+        0.662801,
+        0.282918,
+        4.588603,
+        4.302037,
+        4.048484,
+        0.703624,
+        0.293271,
+        6.279331,
+        6.615427,
+        1.425850,
+        0.830705,
+        0.193814,
+        1.220941,
+        4.142165,
+        5.106035,
+        0.552386,
+        0.302568,
+        2.193489,
+        3.100317,
+        4.000575,
+        0.607610,
+        0.353888,
+        3.102642,
+        -4.352984,
+        4.095905,
+        0.645429,
+        0.696707,
+        6.719682,
+        -4.788645,
+        -1.745401,
+        0.932695,
+        0.730105,
+        1.193824,
+        -1.306795,
+        5.737747,
+        0.557261,
+        0.572826,
+        0.729766,
+        -1.593712,
+        5.833208,
+        0.542902,
+        0.584792,
+        2.456206,
+        -4.342621,
+        4.283884,
+        0.618026,
+        0.694711,
+        2.204823,
+        -4.304508,
+        4.162499,
+        0.607591,
+        0.694203,
+        4.985894,
+        4.802461,
+        3.751977,
+        0.722943,
+        0.271963,
+        1.592294,
+        -1.257709,
+        5.456949,
+        0.577414,
+        0.563167,
+        2.644548,
+        4.524654,
+        4.921559,
+        0.614083,
+        0.281387,
+        2.760292,
+        5.100971,
+        5.015990,
+        0.616907,
+        0.255886,
+        3.523964,
+        8.005976,
+        3.729163,
+        0.668509,
+        0.119914,
+        5.599763,
+        5.715470,
+        2.724259,
+        0.770092,
+        0.232021,
+        3.063932,
+        6.566144,
+        4.529981,
+        0.635536,
+        0.189249,
+        5.720968,
+        4.254584,
+        2.830852,
+        0.770391,
+        0.299556,
+        6.374393,
+        4.785590,
+        1.591691,
+        0.826722,
+        0.278755,
+        0.672728,
+        -3.688016,
+        5.737804,
+        0.527121,
+        0.666198,
+        1.262560,
+        -3.787691,
+        5.417779,
+        0.553172,
+        0.668527,
+        1.732553,
+        -3.952767,
+        5.000579,
+        0.577238,
+        0.673890,
+        1.043625,
+        -1.464973,
+        5.662455,
+        0.554692,
+        0.580066,
+        2.321234,
+        -4.329069,
+        4.258156,
+        0.611897,
+        0.693961,
+        2.056846,
+        -4.477671,
+        4.520883,
+        0.596961,
+        0.706540,
+        2.153084,
+        -4.276322,
+        4.038093,
+        0.596371,
+        0.693953,
+        0.946874,
+        -1.035249,
+        6.512274,
+        0.539958,
+        0.557139,
+        1.469132,
+        -4.036351,
+        4.604908,
+        0.568842,
+        0.692366,
+        1.024340,
+        -3.989851,
+        4.926693,
+        0.547818,
+        0.692366,
+        0.533422,
+        -3.993222,
+        5.138202,
+        0.524613,
+        0.692366,
+        0.769720,
+        -6.095394,
+        4.985883,
+        0.534090,
+        0.779141,
+        0.699606,
+        -5.291850,
+        5.448304,
+        0.527671,
+        0.736226,
+        0.669687,
+        -4.949770,
+        5.509612,
+        0.526913,
+        0.717857,
+        0.630947,
+        -4.695101,
+        5.449371,
+        0.526878,
+        0.704626,
+        0.583218,
+        -4.517982,
+        5.339869,
+        0.526967,
+        0.695278,
+        1.537170,
+        -4.423206,
+        4.745470,
+        0.572058,
+        0.695278,
+        1.615600,
+        -4.475942,
+        4.813632,
+        0.573521,
+        0.703540,
+        1.729053,
+        -4.618680,
+        4.854463,
+        0.576838,
+        0.711846,
+        1.838624,
+        -4.828746,
+        4.823737,
+        0.581691,
+        0.720063,
+        2.368250,
+        -3.106237,
+        4.868096,
+        0.609945,
+        0.639910,
+        7.542244,
+        -1.049282,
+        -2.431321,
+        0.986046,
+        0.560034,
+        1.826614,
+        -4.399531,
+        4.399021,
+        0.586800,
+        0.695400,
+        1.929558,
+        -4.411831,
+        4.497052,
+        0.590372,
+        0.701823,
+        0.597442,
+        -2.013686,
+        5.866456,
+        0.531915,
+        0.601537,
+        1.405627,
+        -1.714196,
+        5.241087,
+        0.577268,
+        0.585935,
+        0.662449,
+        -1.819321,
+        5.863759,
+        0.536915,
+        0.593786,
+        2.342340,
+        0.572222,
+        4.294303,
+        0.627543,
+        0.473352,
+        3.327324,
+        0.104863,
+        4.113860,
+        0.665586,
+        0.495951,
+        1.726175,
+        -0.919165,
+        5.273355,
+        0.588354,
+        0.546862,
+        5.133204,
+        7.485602,
+        2.660442,
+        0.757824,
+        0.147676,
+        4.538641,
+        6.319907,
+        3.683424,
+        0.709250,
+        0.201508,
+        3.986562,
+        5.109487,
+        4.466315,
+        0.672684,
+        0.256581,
+        2.169681,
+        -5.440433,
+        4.455874,
+        0.600409,
+        0.749005,
+        1.395634,
+        5.011963,
+        5.316032,
+        0.558266,
+        0.261672,
+        1.619500,
+        6.599217,
+        4.921106,
+        0.570304,
+        0.187871,
+        1.891399,
+        8.236377,
+        4.274997,
+        0.588166,
+        0.109044,
+        4.195832,
+        2.235205,
+        3.375099,
+        0.711045,
+        0.398952,
+        5.733342,
+        1.411738,
+        2.431726,
+        0.781070,
+        0.435405,
+        1.859887,
+        2.355757,
+        3.843181,
+        0.587247,
+        0.398932,
+        4.988612,
+        3.074654,
+        3.083858,
+        0.742870,
+        0.355446,
+        1.303263,
+        1.416453,
+        4.831091,
+        0.572156,
+        0.437652,
+        1.305757,
+        -0.672779,
+        6.415959,
+        0.551868,
+        0.536570,
+        6.465170,
+        0.937119,
+        1.689873,
+        0.821442,
+        0.457556,
+        5.258659,
+        0.945811,
+        2.974312,
+        0.752702,
+        0.457182,
+        4.432338,
+        0.722096,
+        3.522615,
+        0.713757,
+        0.467627,
+        3.300681,
+        0.861641,
+        3.872784,
+        0.667113,
+        0.460673,
+        2.430178,
+        1.131492,
+        4.039035,
+        0.631101,
+        0.447154,
+        1.820731,
+        1.467954,
+        4.224124,
+        0.600862,
+        0.432473,
+        0.563221,
+        2.307693,
+        5.566789,
+        0.523481,
+        0.405627,
+        6.338145,
+        -0.529279,
+        1.881175,
+        0.810748,
+        0.523926,
+        5.587698,
+        3.208071,
+        2.687839,
+        0.771046,
+        0.348959,
+        0.242624,
+        -1.462857,
+        7.071491,
+        0.509127,
+        0.562718,
+        1.611251,
+        0.339326,
+        4.895421,
+        0.595293,
+        0.485024,
+        7.743095,
+        2.364999,
+        -2.005167,
+        0.980531,
+        0.401564,
+        1.391142,
+        1.851048,
+        4.448999,
+        0.573500,
+        0.420000,
+        1.785794,
+        -0.978284,
+        4.850470,
+        0.602995,
+        0.548688,
+        4.670959,
+        2.664461,
+        3.084075,
+        0.733530,
+        0.376977,
+        1.333970,
+        -0.283761,
+        6.097047,
+        0.560611,
+        0.519017,
+        7.270895,
+        -2.890917,
+        -2.252455,
+        0.967686,
+        0.644357,
+        1.856432,
+        2.585245,
+        3.757904,
+        0.580985,
+        0.387160,
+        0.923388,
+        0.073076,
+        6.671944,
+        0.537728,
+        0.505385,
+        5.000589,
+        -6.135128,
+        1.892523,
+        0.760966,
+        0.779753,
+        5.085276,
+        -7.178590,
+        0.714711,
+        0.801779,
+        0.831938,
+        7.159291,
+        -0.811820,
+        -0.072044,
+        0.892441,
+        0.540761,
+        5.843051,
+        -5.248023,
+        0.924091,
+        0.816351,
+        0.740260,
+        6.847258,
+        3.662916,
+        0.724695,
+        0.865595,
+        0.333687,
+        2.412942,
+        -8.258853,
+        4.119213,
+        0.614074,
+        0.883246,
+        0.179909,
+        -1.689864,
+        6.573301,
+        0.508953,
+        0.579438,
+        2.103655,
+        -0.163946,
+        4.566119,
+        0.617942,
+        0.508316,
+        6.407571,
+        2.236021,
+        1.560843,
+        0.825608,
+        0.397675,
+        3.670075,
+        2.360153,
+        3.635230,
+        0.681215,
+        0.396235,
+        3.177186,
+        2.294265,
+        3.775704,
+        0.656636,
+        0.400597,
+        2.196121,
+        -4.598322,
+        4.479786,
+        0.603900,
+        0.710217,
+        6.234883,
+        -1.944430,
+        1.663542,
+        0.812086,
+        0.588539,
+        1.292924,
+        -9.295920,
+        4.094063,
+        0.568013,
+        0.944565,
+        3.210651,
+        -8.533278,
+        2.802001,
+        0.681008,
+        0.898285,
+        4.068926,
+        -7.993109,
+        1.925119,
+        0.733752,
+        0.869701,
+        2.724032,
+        2.315802,
+        3.777151,
+        0.633830,
+        0.398822,
+        2.288460,
+        2.398891,
+        3.697603,
+        0.606793,
+        0.395537,
+        1.998311,
+        2.496547,
+        3.689148,
+        0.589660,
+        0.391062,
+        6.130040,
+        3.399261,
+        2.038516,
+        0.805016,
+        0.342108,
+        2.288460,
+        2.886504,
+        3.775031,
+        0.611335,
+        0.362284,
+        2.724032,
+        2.961810,
+        3.871767,
+        0.634038,
+        0.355971,
+        3.177186,
+        2.964136,
+        3.876973,
+        0.656636,
+        0.355357,
+        3.670075,
+        2.927714,
+        3.724325,
+        0.681215,
+        0.358340,
+        4.018389,
+        2.857357,
+        3.482983,
+        0.698585,
+        0.363156,
+        7.555811,
+        4.106811,
+        -0.991917,
+        0.941867,
+        0.319076,
+        4.018389,
+        2.483695,
+        3.440898,
+        0.698585,
+        0.387449,
+        1.776217,
+        -2.683946,
+        5.213116,
+        0.584177,
+        0.624107,
+        1.222237,
+        -1.182444,
+        5.952465,
+        0.554318,
+        0.566077,
+        0.731493,
+        -2.536683,
+        5.815343,
+        0.534154,
+        0.620640,
+        4.135272,
+        -6.996638,
+        2.671970,
+        0.711218,
+        0.819975,
+        3.311811,
+        -7.660815,
+        3.382963,
+        0.664630,
+        0.852871,
+        1.313701,
+        -8.639995,
+        4.702456,
+        0.559100,
+        0.902632,
+        5.940524,
+        -6.223629,
+        -0.631468,
+        0.871706,
+        0.791941,
+        1.998311,
+        2.743838,
+        3.744030,
+        0.591234,
+        0.373894,
+        0.901447,
+        1.236992,
+        5.754256,
+        0.544341,
+        0.451584,
+        2.308977,
+        -8.974196,
+        3.609070,
+        0.624563,
+        0.924192,
+        6.954154,
+        -2.439843,
+        -0.131163,
+        0.885770,
+        0.615029,
+        1.098819,
+        -4.458788,
+        5.120727,
+        0.551338,
+        0.695278,
+        1.181124,
+        -4.579996,
+        5.189564,
+        0.551980,
+        0.704632,
+        1.255818,
+        -4.787901,
+        5.237051,
+        0.552888,
+        0.715808,
+        1.325085,
+        -5.106507,
+        5.205010,
+        0.555168,
+        0.730794,
+        1.546388,
+        -5.819392,
+        4.757893,
+        0.569944,
+        0.767035,
+        1.953754,
+        -4.183892,
+        4.431713,
+        0.593203,
+        0.685676,
+        2.117802,
+        -4.137093,
+        4.555096,
+        0.599262,
+        0.681069,
+        2.285339,
+        -4.051196,
+        4.582438,
+        0.607600,
+        0.677703,
+        2.850160,
+        -3.665720,
+        4.484994,
+        0.631938,
+        0.663500,
+        5.278538,
+        -2.238942,
+        2.861224,
+        0.752033,
+        0.601315,
+        0.946709,
+        1.907628,
+        5.196779,
+        0.547226,
+        0.420395,
+        1.314173,
+        3.104912,
+        4.231404,
+        0.563544,
+        0.359828,
+        1.780000,
+        2.860000,
+        3.881555,
+        0.583841,
+        0.368714,
+        1.845110,
+        -4.098880,
+        4.247264,
+        0.586614,
+        0.692366,
+        5.436187,
+        -4.030482,
+        2.109852,
+        0.771915,
+        0.683578,
+        0.766444,
+        3.182131,
+        4.861453,
+        0.531597,
+        0.352483,
+        1.938616,
+        -6.614410,
+        4.521085,
+        0.588371,
+        0.804441,
+        0.516573,
+        1.583572,
+        6.148363,
+        0.520797,
+        0.442565,
+        1.246815,
+        0.230297,
+        5.681036,
+        0.567985,
+        0.493479,
+        0.997827,
+        -6.930921,
+        4.979576,
+        0.543283,
+        0.819255,
+        3.288807,
+        -5.382514,
+        3.795752,
+        0.655317,
+        0.745515,
+        2.311631,
+        -1.566237,
+        4.590085,
+        0.621009,
+        0.574018,
+        2.680250,
+        -6.111567,
+        4.096152,
+        0.625560,
+        0.780312,
+        3.832928,
+        -1.537326,
+        4.137731,
+        0.680198,
+        0.570719,
+        2.961860,
+        -2.274215,
+        4.440943,
+        0.642764,
+        0.604338,
+        4.386901,
+        -2.683286,
+        3.643886,
+        0.704663,
+        0.621530,
+        1.217295,
+        -7.834465,
+        4.969286,
+        0.552012,
+        0.862592,
+        1.542374,
+        -0.136843,
+        5.201008,
+        0.589072,
+        0.508637,
+        3.878377,
+        -6.041764,
+        3.311079,
+        0.685945,
+        0.775357,
+        3.084037,
+        -6.809842,
+        3.814195,
+        0.645735,
+        0.812640,
+        3.747321,
+        -4.503545,
+        3.726453,
+        0.675343,
+        0.703978,
+        6.094129,
+        -3.205991,
+        1.473482,
+        0.810858,
+        0.646305,
+        4.588995,
+        -4.728726,
+        2.983221,
+        0.720122,
+        0.714667,
+        6.583231,
+        -3.941269,
+        0.070268,
+        0.866152,
+        0.682705,
+        3.492580,
+        -3.195820,
+        4.130198,
+        0.663187,
+        0.644597,
+        1.255543,
+        0.802341,
+        5.307551,
+        0.570082,
+        0.466326,
+        1.126122,
+        -0.933602,
+        6.538785,
+        0.544562,
+        0.548376,
+        1.443109,
+        -1.142774,
+        5.905127,
+        0.562759,
+        0.558785,
+        0.923043,
+        -0.529042,
+        7.003423,
+        0.531987,
+        0.530140,
+        1.755386,
+        3.529117,
+        4.327696,
+        0.585271,
+        0.335177,
+        2.632589,
+        3.713828,
+        4.364629,
+        0.622953,
+        0.322779,
+        3.388062,
+        3.721976,
+        4.309028,
+        0.655896,
+        0.320163,
+        4.075766,
+        3.675413,
+        4.076063,
+        0.687132,
+        0.322346,
+        4.622910,
+        3.474691,
+        3.646321,
+        0.716482,
+        0.333201,
+        5.171755,
+        2.535753,
+        2.670867,
+        0.758757,
+        0.382787,
+        7.297331,
+        0.763172,
+        -0.048769,
+        0.897013,
+        0.468769,
+        4.706828,
+        1.651000,
+        3.109532,
+        0.732392,
+        0.424547,
+        4.071712,
+        1.476821,
+        3.476944,
+        0.702114,
+        0.433163,
+        3.269817,
+        1.470659,
+        3.731945,
+        0.666525,
+        0.433866,
+        2.527572,
+        1.617311,
+        3.865444,
+        0.633505,
+        0.426088,
+        1.970894,
+        1.858505,
+        3.961782,
+        0.603876,
+        0.416587,
+        1.579543,
+        2.097941,
+        4.084996,
+        0.579658,
+        0.409945,
+        7.664182,
+        0.673132,
+        -2.435867,
+        0.992440,
+        0.480777,
+        1.397041,
+        -1.340139,
+        5.630378,
+        0.567192,
+        0.569420,
+        0.884838,
+        0.658740,
+        6.233232,
+        0.541366,
+        0.478899,
+        0.767097,
+        -0.968035,
+        7.077932,
+        0.526564,
+        0.546118,
+        0.460213,
+        -1.334106,
+        6.787447,
+        0.523913,
+        0.563830,
+        0.748618,
+        -1.067994,
+        6.798303,
+        0.531529,
+        0.555057,
+        1.236408,
+        -1.585568,
+        5.480490,
+        0.566036,
+        0.582329,
+        0.387306,
+        -1.409990,
+        6.957705,
+        0.516311,
+        0.563054,
+        0.319925,
+        -1.607931,
+        6.508676,
+        0.517472,
+        0.577877,
+        1.639633,
+        2.556298,
+        3.863736,
+        0.573595,
+        0.389807,
+        1.255645,
+        2.467144,
+        4.203800,
+        0.560698,
+        0.395332,
+        1.031362,
+        2.382663,
+        4.615849,
+        0.549756,
+        0.399751,
+        4.253081,
+        2.772296,
+        3.315305,
+        0.710288,
+        0.368253,
+        4.530000,
+        2.910000,
+        3.339685,
+        0.723330,
+        0.363373,
+    ]
+)
+canonical_metric_landmarks = np.reshape(
+    canonical_metric_landmarks, (canonical_metric_landmarks.shape[0] // 5, 5)
+).T
+canonical_metric_landmarks = canonical_metric_landmarks[:3, :]
+
+procrustes_landmark_basis = [
+    (4, 0.070909939706326),
+    (6, 0.032100144773722),
+    (10, 0.008446550928056),
+    (33, 0.058724168688059),
+    (54, 0.007667080033571),
+    (67, 0.009078059345484),
+    (117, 0.009791937656701),
+    (119, 0.014565368182957),
+    (121, 0.018591361120343),
+    (127, 0.005197994410992),
+    (129, 0.120625205338001),
+    (132, 0.005560018587857),
+    (133, 0.05328618362546),
+    (136, 0.066890455782413),
+    (143, 0.014816547743976),
+    (147, 0.014262833632529),
+    (198, 0.025462191551924),
+    (205, 0.047252278774977),
+    (263, 0.058724168688059),
+    (284, 0.007667080033571),
+    (297, 0.009078059345484),
+    (346, 0.009791937656701),
+    (348, 0.014565368182957),
+    (350, 0.018591361120343),
+    (356, 0.005197994410992),
+    (358, 0.120625205338001),
+    (361, 0.005560018587857),
+    (362, 0.05328618362546),
+    (365, 0.066890455782413),
+    (372, 0.014816547743976),
+    (376, 0.014262833632529),
+    (420, 0.025462191551924),
+    (425, 0.047252278774977),
+]
+landmark_weights = np.zeros((canonical_metric_landmarks.shape[1],))
+for idx, weight in procrustes_landmark_basis:
+    landmark_weights[idx] = weight
+
+
+def log(name, f):
+    if DEBUG.get_debug():
+        print(f"{name} logged:", f)
+        print()
+
+
+def cpp_compare(name, np_matrix):
+    if DEBUG.get_debug():
+        # reorder cpp matrix as memory alignment is not correct
+        cpp_matrix = np.load(f"{name}_cpp.npy")
+        rows, cols = cpp_matrix.shape
+        cpp_matrix = np.split(np.reshape(cpp_matrix, -1), cols)
+        cpp_matrix = np.stack(cpp_matrix, 1)
+
+        print(f"{name}:", np.sum(np.abs(cpp_matrix - np_matrix[:rows, :cols]) ** 2))
+        print()
+
+
+def get_metric_landmarks(screen_landmarks, pcf):
+    screen_landmarks = project_xy(screen_landmarks, pcf)
+    depth_offset = np.mean(screen_landmarks[2, :])
+    
+    intermediate_landmarks = screen_landmarks.copy()
+    intermediate_landmarks = change_handedness(intermediate_landmarks)
+    first_iteration_scale = estimate_scale(intermediate_landmarks)
+
+    intermediate_landmarks = screen_landmarks.copy()
+    intermediate_landmarks = move_and_rescale_z(
+        pcf, depth_offset, first_iteration_scale, intermediate_landmarks
+    )
+    intermediate_landmarks = unproject_xy(pcf, intermediate_landmarks)
+    intermediate_landmarks = change_handedness(intermediate_landmarks)
+    second_iteration_scale = estimate_scale(intermediate_landmarks)
+
+    metric_landmarks = screen_landmarks.copy()
+    total_scale = first_iteration_scale * second_iteration_scale
+    metric_landmarks = move_and_rescale_z(
+        pcf, depth_offset, total_scale, metric_landmarks
+    )
+    metric_landmarks = unproject_xy(pcf, metric_landmarks)
+    metric_landmarks = change_handedness(metric_landmarks)
+
+    pose_transform_mat = solve_weighted_orthogonal_problem(
+        canonical_metric_landmarks, metric_landmarks, landmark_weights
+    )
+    cpp_compare("pose_transform_mat", pose_transform_mat)
+
+    inv_pose_transform_mat = np.linalg.inv(pose_transform_mat)
+    inv_pose_rotation = inv_pose_transform_mat[:3, :3]
+    inv_pose_translation = inv_pose_transform_mat[:3, 3]
+
+    metric_landmarks = (
+        inv_pose_rotation @ metric_landmarks + inv_pose_translation[:, None]
+    )
+
+    return metric_landmarks, pose_transform_mat
+
+
+def project_xy(landmarks, pcf):
+    x_scale = pcf.right - pcf.left
+    y_scale = pcf.top - pcf.bottom
+    x_translation = pcf.left
+    y_translation = pcf.bottom
+
+    landmarks[1, :] = 1.0 - landmarks[1, :]
+
+    landmarks = landmarks * np.array([[x_scale, y_scale, x_scale]]).T
+    landmarks = landmarks + np.array([[x_translation, y_translation, 0]]).T
+
+    return landmarks
+
+
+def change_handedness(landmarks):
+    landmarks[2, :] *= -1.0
+
+    return landmarks
+
+
+def move_and_rescale_z(pcf, depth_offset, scale, landmarks):
+    landmarks[2, :] = (landmarks[2, :] - depth_offset + pcf.near) / scale
+
+    return landmarks
+
+
+def unproject_xy(pcf, landmarks):
+    landmarks[0, :] = landmarks[0, :] * landmarks[2, :] / pcf.near
+    landmarks[1, :] = landmarks[1, :] * landmarks[2, :] / pcf.near
+
+    return landmarks
+
+
+def estimate_scale(landmarks):
+    transform_mat = solve_weighted_orthogonal_problem(
+        canonical_metric_landmarks, landmarks, landmark_weights
+    )
+
+    return np.linalg.norm(transform_mat[:, 0])
+
+
+def extract_square_root(point_weights):
+    return np.sqrt(point_weights)
+
+
+def solve_weighted_orthogonal_problem(source_points, target_points, point_weights):
+    sqrt_weights = extract_square_root(point_weights)
+    transform_mat = internal_solve_weighted_orthogonal_problem(
+        source_points, target_points, sqrt_weights
+    )
+    return transform_mat
+
+
+def internal_solve_weighted_orthogonal_problem(sources, targets, sqrt_weights):
+    cpp_compare("sources", sources)
+    cpp_compare("targets", targets)
+
+    # tranposed(A_w).
+    weighted_sources = sources * sqrt_weights[None, :]
+    # tranposed(B_w).
+    weighted_targets = targets * sqrt_weights[None, :]
+
+    cpp_compare("weighted_sources", weighted_sources)
+    cpp_compare("weighted_targets", weighted_targets)
+
+    # w = tranposed(j_w) j_w.
+    total_weight = np.sum(sqrt_weights * sqrt_weights)
+    log("total_weight", total_weight)
+
+    # Let C = (j_w tranposed(j_w)) / (tranposed(j_w) j_w).
+    # Note that C = tranposed(C), hence (I - C) = tranposed(I - C).
+    #
+    # tranposed(A_w) C = tranposed(A_w) j_w tranposed(j_w) / w =
+    # (tranposed(A_w) j_w) tranposed(j_w) / w = c_w tranposed(j_w),
+    #
+    # where c_w = tranposed(A_w) j_w / w is a k x 1 vector calculated here:
+    twice_weighted_sources = weighted_sources * sqrt_weights[None, :]
+    source_center_of_mass = np.sum(twice_weighted_sources, axis=1) / total_weight
+    log("source_center_of_mass", source_center_of_mass)
+
+    # tranposed((I - C) A_w) = tranposed(A_w) (I - C) =
+    # tranposed(A_w) - tranposed(A_w) C = tranposed(A_w) - c_w tranposed(j_w).
+    centered_weighted_sources = weighted_sources - np.matmul(
+        source_center_of_mass[:, None], sqrt_weights[None, :]
+    )
+    cpp_compare("centered_weighted_sources", centered_weighted_sources)
+
+    design_matrix = np.matmul(weighted_targets, centered_weighted_sources.T)
+    cpp_compare("design_matrix", design_matrix)
+    log("design_matrix_norm", np.linalg.norm(design_matrix))
+
+    rotation = compute_optimal_rotation(design_matrix)
+
+    scale = compute_optimal_scale(
+        centered_weighted_sources, weighted_sources, weighted_targets, rotation
+    )
+    log("scale", scale)
+
+    rotation_and_scale = scale * rotation
+
+    pointwise_diffs = weighted_targets - np.matmul(rotation_and_scale, weighted_sources)
+    cpp_compare("pointwise_diffs", pointwise_diffs)
+
+    weighted_pointwise_diffs = pointwise_diffs * sqrt_weights[None, :]
+    cpp_compare("weighted_pointwise_diffs", weighted_pointwise_diffs)
+
+    translation = np.sum(weighted_pointwise_diffs, axis=1) / total_weight
+    log("translation", translation)
+
+    transform_mat = combine_transform_matrix(rotation_and_scale, translation)
+    cpp_compare("transform_mat", transform_mat)
+
+    return transform_mat
+
+
+def compute_optimal_rotation(design_matrix):
+    if np.linalg.norm(design_matrix) < 1e-9:
+        print("Design matrix norm is too small!")
+
+    u, _, vh = np.linalg.svd(design_matrix, full_matrices=True)
+
+    postrotation = u
+    prerotation = vh
+
+    if np.linalg.det(postrotation) * np.linalg.det(prerotation) < 0:
+        postrotation[:, 2] = -1 * postrotation[:, 2]
+
+    cpp_compare("postrotation", postrotation)
+    cpp_compare("prerotation", prerotation)
+
+    rotation = np.matmul(postrotation, prerotation)
+
+    cpp_compare("rotation", rotation)
+
+    return rotation
+
+
+def compute_optimal_scale(
+    centered_weighted_sources, weighted_sources, weighted_targets, rotation
+):
+    rotated_centered_weighted_sources = np.matmul(rotation, centered_weighted_sources)
+
+    numerator = np.sum(rotated_centered_weighted_sources * weighted_targets)
+    denominator = np.sum(centered_weighted_sources * weighted_sources)
+
+    if denominator < 1e-9:
+        print("Scale expression denominator is too small!")
+    if numerator / denominator < 1e-9:
+        print("Scale is too small!")
+
+    return numerator / denominator
+
+
+def combine_transform_matrix(r_and_s, t):
+    result = np.eye(4)
+    result[:3, :3] = r_and_s
+    result[:3, 3] = t
+    return result

codes/base/iris_lm_depth.py

@@ -0,0 +1,157 @@
+import os
+import cv2
+import numpy as np
+from codes.base.core import (
+    detections_to_rect,
+    landmarks_to_detections,
+    slice_from_roi,
+    tflite_inference,
+    transform_rect,
+)
+
+
+PATH2ROOT_ABS = os.path.dirname(__file__) + "/../../"
+
+
+def from_landmarks_to_depth(
+    frame_rgb, eye_landmarks, image_size, is_right_eye=False, focal_length=None
+):
+    """
+    Predicting the iris position using eyes landmarks
+
+    Parameters:
+        frame_rgb: the frame (RGB)
+        eye_landmarks: the landmarks of the eyes
+        image_size: the image size
+        is_right_eye: whether or not it's the right eye
+        focal_length: the focal length of the camera
+
+    Returns:
+        success: whether or not the iris is detected
+        depth: distance to the camera
+        iris_size: the size of the iris
+        iris_landmarks: iris landmarks
+        eye_contours: eye contours
+        iris_landmarks_respect: iris landmarks respect to the eyes
+    """
+    if focal_length is None:
+        focal_length = frame_rgb.shape[1]
+
+    detections = landmarks_to_detections(eye_landmarks)
+    rect = detections_to_rect(detections, image_size, rotation_vector_start_end=(0, 1))
+    roi = transform_rect(rect, image_size, scale_x=2.3, scale_y=2.3)
+
+    slice_y = slice_from_roi(roi, image_size, False)
+    slice_x = slice_from_roi(roi, image_size, True)
+    eye_image = frame_rgb[slice(*slice_y), slice(*slice_x), :]
+    position_in_frame = np.array((slice_x[0], slice_y[0], 0))
+
+    if eye_image.any():
+        success = True
+        eye_contours, iris_landmarks, eye_frame_low = detect_iris(
+            eye_image.copy(), is_right_eye=is_right_eye
+        )
+
+        iris_landmarks_respect = iris_landmarks.copy()
+
+        eye_contours[:, 0] = eye_contours[:, 0] * eye_image.shape[0]
+        eye_contours[:, 1] = eye_contours[:, 1] * eye_image.shape[1]
+        eye_contours = eye_contours + position_in_frame
+
+        eye_contours[:, 0] = eye_contours[:, 0] / frame_rgb.shape[1]
+        eye_contours[:, 1] = eye_contours[:, 1] / frame_rgb.shape[0]
+
+        iris_landmarks[:, 0] = iris_landmarks[:, 0] * eye_image.shape[0]
+        iris_landmarks[:, 1] = iris_landmarks[:, 1] * eye_image.shape[1]
+        iris_landmarks = iris_landmarks + position_in_frame
+
+        iris_landmarks[:, 0] = iris_landmarks[:, 0] / frame_rgb.shape[1]
+        iris_landmarks[:, 1] = iris_landmarks[:, 1] / frame_rgb.shape[0]
+
+        depth, iris_size = calculate_iris_depth(iris_landmarks, image_size, focal_length)
+        
+    else:
+        success = False
+        depth = None
+        iris_size = None
+        iris_landmarks = None
+        eye_contours = None
+        iris_landmarks_respect = None
+
+    return success, depth, iris_size, iris_landmarks, eye_contours, iris_landmarks_respect
+
+
+def detect_iris(eye_frame, is_right_eye=False):
+    side_low = 64
+    eye_frame_low = cv2.resize(
+        eye_frame, (side_low, side_low), interpolation=cv2.INTER_AREA
+    )
+
+    model_path = PATH2ROOT_ABS + "models/iris/iris_landmark.tflite"
+
+    if is_right_eye:
+        eye_frame_low = np.fliplr(eye_frame_low)
+
+    outputs = tflite_inference(eye_frame_low / 127.5 - 1.0, model_path)
+    eye_contours_low = np.reshape(outputs[0], (71, 3))
+    iris_landmarks_low = np.reshape(outputs[1], (5, 3))
+
+    eye_contours = eye_contours_low / side_low
+    iris_landmarks = iris_landmarks_low / side_low
+
+    if is_right_eye:
+        eye_contours[:, 0] = 1 - eye_contours[:, 0]
+        iris_landmarks[:, 0] = 1 - iris_landmarks[:, 0]
+
+    return eye_contours, iris_landmarks, eye_frame_low
+
+
+def calculate_iris_depth(iris_landmarks, image_size, focal_length_pixel):
+    """
+    iris_landmarks should be normalized to the complete image frame
+    depth in mm
+    """
+    iris_size = calculate_iris_diameter(iris_landmarks, image_size)
+    depth = calculate_depth(
+        iris_landmarks[0, :], focal_length_pixel, iris_size, image_size
+    )
+
+    return depth, iris_size
+
+
+def get_depth(x0, y0, x1, y1):
+    return np.sqrt((x0 - x1) ** 2 + (y0 - y1) ** 2)
+
+
+def get_landmark_depth(ld0, ld1, image_size):
+    return get_depth(
+        ld0[0] * image_size[0],
+        ld0[1] * image_size[1],
+        ld1[0] * image_size[0],
+        ld1[1] * image_size[1],
+    )
+
+
+def calculate_iris_diameter(iris_landmarks, image_size):
+    dist_vert = get_landmark_depth(
+        iris_landmarks[1, :], iris_landmarks[3, :], image_size
+    )
+    dist_hori = get_landmark_depth(
+        iris_landmarks[2, :], iris_landmarks[4, :], image_size
+    )
+
+    return (dist_hori + dist_vert) / 2.0
+
+
+def calculate_depth(center_landmark, focal_length_pixel, iris_size, image_size):
+    # Average fixed iris size across human beings.
+    human_iris_size_in_mm = 11.8
+    origin = np.array(image_size) / 2.0
+    center_landmark_pixel = center_landmark[:2] * np.array(image_size)
+    y = get_depth(
+        origin[0], origin[1], center_landmark_pixel[0], center_landmark_pixel[1]
+    )
+    x = np.sqrt(focal_length_pixel ** 2 + y ** 2)
+    depth = human_iris_size_in_mm * x / iris_size
+
+    return depth

codes/calibration.py

@@ -0,0 +1,582 @@
+"""This module is for calibration of the Owleye. The module includes the code to collect data from the user,
+while they are looking at the white point. The molude contains one class called Clb. To understand this module, you should know
+about Mediapipe landmark detection."""
+
+
+import numpy as np
+import cv2
+import time
+from codes.base import eyeing as ey
+import os
+from datetime import datetime
+if os.name == "nt":
+    import winsound
+elif os.name == "posix":
+    pass
+from sklearn.utils import shuffle
+import math
+
+
+INFO = ("", "M", 25, "Email: ")      # The information that goes to information.txt
+CALIBRATION_GRID = (4, 200, 6, 100)         # Calibration grid
+
+# Class for calibration
+class Clb(object):
+    running = True
+
+    @staticmethod
+    def create_grid(clb_grid):
+        """
+        This method creates the desired grid points.
+
+        Parameters:
+            clb_grid: A list
+            
+        Returns:
+            points: A list that contains n lists
+        """
+        point_ratio = 0.012
+        if len(clb_grid) == 2:
+            # For going through just rows
+            rows = clb_grid[0]
+            points_in_row = clb_grid[1]
+            points = []
+            dy_rows = (1 - rows * point_ratio) / (rows - 1)
+            dx = (1 - points_in_row * point_ratio) / (points_in_row - 1)
+
+            for j in range(rows):
+                if j == 0:
+                    p_y = j * (point_ratio + dy_rows) + 4.0 * point_ratio / 3.0
+                elif j == rows-1:
+                    p_y = j * (point_ratio + dy_rows) - point_ratio / 3.0
+                else:
+                    p_y = j * (point_ratio + dy_rows) + point_ratio / 2
+                smp_in_p = []
+                for i in range(points_in_row):
+                    if i == 0:
+                        p_x = i * (point_ratio + dx) + point_ratio
+                    elif i == points_in_row - 1:
+                        p_x = i * (point_ratio + dx)
+                    else:
+                        p_x = i * (point_ratio + dx) + point_ratio / 2
+                    smp_in_p.append([p_x, p_y])
+                if j % 2 == 0:
+                    points.append(smp_in_p)
+                else:
+                    smp_in_p.reverse()
+                    points.append(smp_in_p)
+
+        elif len(clb_grid) == 3:
+            # For appearing stationary (not moving)
+            rows = clb_grid[0]
+            cols = clb_grid[1]
+            smp_in_pnt = clb_grid[2]
+            points = []
+            dy = (1 - rows * point_ratio) / (rows - 1)
+            dx = (1 - cols * point_ratio) / (cols - 1)
+
+            for j in range(rows):
+                if j == 0:
+                    p_y = j * (point_ratio + dy) + 4.0 * point_ratio / 3.0
+                elif j == rows - 1:
+                    p_y = j * (point_ratio + dy) - point_ratio / 3.0
+                else:
+                    p_y = j * (point_ratio + dy) + point_ratio / 2
+                for i in range(cols):
+                    if i == 0:
+                        p_x = i * (point_ratio + dx) + point_ratio
+                    elif i == cols - 1:
+                        p_x = i * (point_ratio + dx)
+                    else:
+                        p_x = i * (point_ratio + dx) + point_ratio / 2
+                    smp_in_p = []
+                    for k in range(smp_in_pnt):
+                        smp_in_p.append([p_x, p_y])
+                    points.append(smp_in_p)
+
+        elif len(clb_grid) == 4:
+            # For going through rows and columns. It is suggested
+            rows = clb_grid[0]
+            points_in_row = clb_grid[1]
+            cols = clb_grid[2]
+            points_in_col = clb_grid[3]
+            points = []
+
+            d_rows = (1 - rows * point_ratio) / (rows - 1)
+            dx = (1 - points_in_row * point_ratio) / (points_in_row - 1)
+            d_cols = (1 - cols * point_ratio) / (cols - 1)
+            dy = (1 - points_in_col * point_ratio) / (points_in_col - 1)
+
+            for j in range(rows):
+                if j == 0:
+                    p_y = j * (point_ratio + d_rows) + 4.0 * point_ratio / 3.0
+                elif j == rows - 1:
+                    p_y = j * (point_ratio + d_rows) - point_ratio / 3.0
+                else:
+                    p_y = j * (point_ratio + d_rows) + point_ratio / 2
+                smp_in_p = []
+                for i in range(points_in_row):
+                    if i == 0:
+                        p_x = i * (point_ratio + dx) + point_ratio
+                    elif i == points_in_row - 1:
+                        p_x = i * (point_ratio + dx)
+                    else:
+                        p_x = i * (point_ratio + dx) + point_ratio / 2
+                    smp_in_p.append([p_x, p_y])
+                if j % 2 == 0:
+                    points.append(smp_in_p)
+                else:
+                    smp_in_p.reverse()
+                    points.append(smp_in_p)
+            for i in range(cols):
+                if i == 0:
+                    p_x = i * (point_ratio + d_cols) + point_ratio
+                elif i == cols - 1:
+                    p_x = i * (point_ratio + d_cols)
+                else:
+                    p_x = i * (point_ratio + d_cols) + point_ratio / 2
+                smp_in_p = []
+                for j in range(points_in_col):
+                    if j == 0:
+                        p_y = j * (point_ratio + dy) + 4.0 * point_ratio / 3.0
+                    elif j == points_in_col - 1:
+                        p_y = j * (point_ratio + dy) - point_ratio / 3.0
+                    else:
+                        p_y = j * (point_ratio + dy) + point_ratio / 2
+                    smp_in_p.append([p_x, p_y])
+                if i % 2 == 0:
+                    points.append(smp_in_p)
+                else:
+                    smp_in_p.reverse()
+                    points.append(smp_in_p)
+
+        else:
+            print("\nPlease Enter a vector with length of 2-4!!")
+            points = None
+            quit()
+
+        return points
+
+
+    def et(self, num, camera_id=0, info=INFO, clb_grid=CALIBRATION_GRID):
+        """
+        Collecting the data (inputs and outputs of the models)
+        
+        Parameters:
+            num: Subject's number
+            camera_id: Camera ID
+            info: Subject's information
+            clb_grid: Calibration grid
+        
+        Retruns:
+            None
+        
+        """
+        
+        print("\nCalibration started!")
+        
+        # Some interactions with user
+        name, descriptions = info
+        tx0 = [["Follow WHITE point", (0.05, 0.25), 1.5, ey.RED, 3],
+        ["SPACE --> start", (0.05, 0.5), 1.5, ey.RED, 3],
+        ["ESC --> Stop", (0.05, 0.75), 1.5, ey.RED, 3]]
+        run_app = True
+
+        sbj_dir = ey.subjects_dir + f"{num}/"
+        if os.path.exists(sbj_dir):
+            tx1 = [["There is a subject in", (0.05, 0.2), 1.3, ey.RED, 2],
+            [f"{sbj_dir}.", (0.05, 0.4), 1.3, ey.RED, 2],
+            ["Do you want to", (0.05, 0.6), 1.3, ey.RED, 2],
+            ["remove it (y/n)?", (0.05, 0.8), 1.3, ey.RED, 2]]
+
+            win_name = "Subject exists"
+            ey.big_win(win_name, 0)
+            ey.show_clb_win(win_name, texts=tx1, win_color=ey.WHITE)
+            button = cv2.waitKey(0)
+            if button == 27 or (button == ord("q")) or (button == ord("Q")) or (button == ord("n")) or (button == ord("N")):
+                run_app = False
+            cv2.destroyWindow(win_name)
+
+        if run_app:
+            sbj_dir = ey.create_dir([sbj_dir])
+            clb_points = self.create_grid(clb_grid)
+
+            # Some landmarks needed for calculation of face vectors
+            some_landmarks_ids = ey.get_some_landmarks_ids()
+
+            (
+                frame_size,
+                camera_matrix,
+                dst_cof,
+                pcf
+            ) = ey.get_camera_properties(camera_id)
+
+            face_mesh = ey.get_mesh()
+
+            fps_vec = []
+            t_mat = []
+            eyes_mat = []
+            inp_scalars_mat = []
+            points_loc_mat = []
+            eyes_ratio_mat = []
+
+            t0 = time.perf_counter()
+            cap = ey.get_camera(camera_id, frame_size)
+            ey.pass_frames(cap, 100)
+
+            win_name = "Information"
+            ey.big_win(win_name, math.floor(len(ey.monitors) / 2)*ey.monitors[0].width)
+            ey.show_clb_win(win_name, texts=tx0, win_color=ey.WHITE)
+            cv2.waitKey(10000)
+            cv2.destroyWindow(win_name)
+
+            # Going through monitors
+            for (i_m, m) in enumerate(ey.monitors):
+                win_name = f"Calibration-{i_m}"
+                ey.big_win(win_name, i_m * m.width)
+                # Going to each series of points (for example, one row of points)
+                for item in clb_points:
+                    if not self.running and (i_m != 0):
+                        break
+                    t_vec = []
+                    eyes_vec = []
+                    inp_scalars_vec = []
+                    points_loc_vec = []
+                    eyes_ratio_vec = []
+                    
+                    pnt = item[0]
+                    ey.show_clb_win(win_name, pnt, win_color=ey.GRAY)
+                    button = cv2.waitKey(0)
+                    if button == 27 or (button == ord("q")) or (button == ord("Q")):
+                        break
+                    elif button == ord(' '):
+                        ey.pass_frames(cap)
+                        t1 = time.perf_counter()
+                        s = len(item)
+
+                        # Going through each point in each series
+                        for pnt in item:
+                            ey.show_clb_win(win_name, pnt)
+                            button = cv2.waitKey(1)
+                            if button == 27:
+                                break
+                            while True:
+                                frame_success, frame, frame_rgb = ey.get_frame(cap)     # Get image
+                                if frame_success:
+                                    results = face_mesh.process(frame_rgb) # Predicting the landmarks using image
+                                    
+                                    # Getting the inputs of the models
+                                    (
+                                        features_success,
+                                        _,
+                                        eyes_frame_gray,
+                                        features_vector,
+                                        eyes_ratio,
+                                        _
+                                    ) = ey.get_model_inputs(
+                                        frame,
+                                        frame_rgb,
+                                        results,
+                                        camera_matrix,
+                                        pcf,
+                                        frame_size,
+                                        dst_cof,
+                                        some_landmarks_ids
+                                    )
+                                    if features_success:
+                                        # Putting the inputs of the models into lists
+                                        t_vec.append(round(time.perf_counter() - t1, 3))
+                                        eyes_vec.append(eyes_frame_gray)
+                                        inp_scalars_vec.append(features_vector)
+                                        points_loc_vec.append([(pnt[0] + i_m)/len(ey.monitors), pnt[1]])
+                                        eyes_ratio_vec.append(eyes_ratio)
+                                        break
+                            if not self.running:
+                                break
+                        fps_vec.append(ey.get_time(s, t1))
+                        t_mat.append(np.array(t_vec))
+                        eyes_mat.append(np.array(eyes_vec))
+                        inp_scalars_mat.append(np.array(inp_scalars_vec))
+                        points_loc_mat.append(np.array(points_loc_vec))
+                        eyes_ratio_mat.append(np.array(eyes_ratio_vec))
+                        
+                    if not self.running:
+                        break
+                    if button == 27 or (button == ord("q")) or (button == ord("Q")):
+                        break
+                if button == 27 or (button == ord("q")) or (button == ord("Q")):
+                    break
+                cv2.destroyWindow(win_name)
+            cap.release()
+            cv2.destroyAllWindows()
+
+            if button != 27 and (button != ord("q")) and (button != ord("Q")):
+                ey.get_time(0, t0, True)
+                print(f"Mean FPS : {np.array(fps_vec).mean()}")
+
+                f = open(sbj_dir + "Information.txt", "w+")
+                f.write(name + "\n" + descriptions + "\n" + str(datetime.now())[:16])
+                f.close()
+
+                et_dir = ey.create_dir([sbj_dir, ey.CLB])
+                ey.save([t_mat, eyes_mat, inp_scalars_mat, points_loc_mat, eyes_ratio_mat], et_dir, [ey.T, ey.X1, ey.X2, ey.Y, ey.ER])
+
+        else:
+            self.running = False
+
+
+    @staticmethod
+    def make_io(num, data_out):
+        """
+        Mixing the data of calibration and out looking, to create a dataset of in-out
+        
+        Parameters:
+            data_out: data of user's looking at outside of the screen
+        
+        Returns:
+            None
+        """
+        sbj_dir = ey.create_dir([ey.subjects_dir, f"{num}"])
+        et_dir = ey.create_dir([sbj_dir, ey.CLB])
+
+        x1_et0, x2_et0 = ey.load(et_dir, [ey.X1, ey.X2])
+        x1_et = []
+        x2_et = []
+        for (x1_vec, x2_vec) in zip(x1_et0, x2_et0):
+            for (x10, x20) in zip(x1_vec, x2_vec):
+                x1_et.append(x10)
+                x2_et.append(x20)
+        x1_et = np.array(x1_et)
+        x2_et = np.array(x2_et)
+
+        x1_o, x2_o, y_o = data_out
+        smp_in_cls = int(x1_o.shape[0])
+
+        x1_et_shf, x2_et_shf = shuffle(x1_et, x2_et)
+
+        x1_i, x2_i = x1_et_shf[:smp_in_cls], x2_et_shf[:smp_in_cls]
+        y_i = np.zeros((smp_in_cls,))
+
+        x1_io = [np.concatenate((x1_i, x1_o))]
+        x2_io = [np.concatenate((x2_i, x2_o))]
+        y_io = [np.concatenate((y_i, y_o))]
+
+        io_dir = ey.create_dir([sbj_dir, ey.IO])
+        ey.save([x1_io, x2_io, y_io], io_dir, [ey.X1, ey.X2, ey.Y])
+
+
+    def out(self, num, camera_id=0, n_smp_in_cls=300):
+        """
+        Collecting data while the user is looking out of the screen
+        
+        Parameters:
+            num: Subject number
+            camera_id: Camera ID
+            n_smp_in_cls: The number of samples for each class
+        
+        Returns:
+            None
+        """
+        print("Getting out data...")
+        out_class_num = 1
+
+        some_landmarks_ids = ey.get_some_landmarks_ids()
+
+        (
+            frame_size,
+            camera_matrix,
+            dst_cof,
+            pcf
+        ) = ey.get_camera_properties(camera_id)
+
+        face_mesh = ey.get_mesh()
+
+        t0 = time.perf_counter()
+        eyes_data_gray = []
+        vector_inputs = []
+        output_class = []
+        fps_vec = []
+        cap = ey.get_camera(camera_id, frame_size)
+        ey.pass_frames(cap, 100)
+        tx0 = [["Look everywhere ", (0.05, 0.25), 1.3, ey.RED, 3],
+        ["'out' of screen", (0.05, 0.5), 1.3, ey.RED, 3],
+        ["SPACE --> start sampling", (0.05, 0.75), 1.3, ey.RED, 3]]
+
+        win_name = "out of screen"
+        ey.big_win(win_name, 0)
+        ey.show_clb_win(win_name, texts=tx0, win_color=ey.WHITE)
+        button = cv2.waitKey(0)
+        if button == 27 or (button == ord("q")) or (button == ord("Q")):
+            quit()
+        cv2.destroyWindow(win_name)
+        i = 0
+        ey.pass_frames(cap)
+        t1 = time.perf_counter()
+
+        # Going through frames
+        while True:
+            frame_success, frame, frame_rgb = ey.get_frame(cap)
+            if frame_success:
+                # Predicting the face landmarks
+                results = face_mesh.process(frame_rgb)
+
+                # Calculating the face features
+                (
+                    features_success,
+                    _,
+                    eyes_frame_gray,
+                    features_vector,
+                    _,
+                    _
+                ) = ey.get_model_inputs(
+                    frame,
+                    frame_rgb,
+                    results,
+                    camera_matrix,
+                    pcf,
+                    frame_size,
+                    dst_cof,
+                    some_landmarks_ids
+                )
+                if features_success:
+                    eyes_data_gray.append(eyes_frame_gray)
+                    vector_inputs.append(features_vector)
+                    output_class.append(out_class_num)
+
+                    i += 1
+                    if i == n_smp_in_cls:
+                        break
+        fps_vec.append(ey.get_time(i, t1))
+        print("Data collected")
+        if os.name == "nt":
+            winsound.PlaySound("SystemExit", winsound.SND_ALIAS)
+
+        cap.release()
+        cv2.destroyAllWindows()
+        ey.get_time(0, t0, True)
+        print(f"Mean FPS : {np.array(fps_vec).mean()}")
+
+        x1 = np.array(eyes_data_gray)
+        x2 = np.array(vector_inputs)
+        y = np.array(output_class)
+
+        print("Data collection finished!")
+
+        self.make_io(num, [x1, x2, y])
+
+    def calculate_threshold(self, num, camera_id=0):
+        """
+        Calculating the blinking threshold automatically. Here we collect data and tell the user to blink during a certain time.
+        Then we gain the maximum value for thier eye movement velocity and it's considered as a blink. So, we tune the threshold
+        base don that.
+
+        Parameters:
+            num: subject number
+            camera_id: Camera ID
+        
+        Returns:
+            None
+        """
+
+        print("\nGetting eyes ratio...")
+        tx0 = [["Look somewhere", (0.02, 0.3), 1.1, ey.RED, 2],
+        ["SPACE --> start/pause", (0.02, 0.6), 1.1, ey.RED, 2]]
+        tx1 = [["Blink", (0.39, 0.5), 1.6, ey.RED, 3]]
+        some_landmarks_ids = ey.get_some_landmarks_ids()
+
+        (
+            frame_size,
+            camera_matrix,
+            dst_cof,
+            pcf
+        ) = ey.get_camera_properties(camera_id)
+
+        face_mesh = ey.get_mesh()
+
+        eyes_ratio_mat = []
+        t_mat = []
+        t0 = time.perf_counter()
+        cap = ey.get_camera(camera_id, frame_size)
+        ey.pass_frames(cap, 100)
+
+        # Going through frames, if the user pressed 'SPACE', the program will be paused, if they press 'q', the program will be stopped.
+        i = 0
+        while self.running:
+            win_name = f"Calibration-{i}"
+            ey.big_win(win_name, math.floor(len(ey.monitors) / 2)*ey.monitors[0].width)
+
+            eyes_ratio_vec = []
+            t_vec = []
+            ey.show_clb_win(win_name, win_color=ey.WHITE, texts=tx0)
+
+            button = cv2.waitKey(0)
+            if (button == ord('q')) or (button == ord('Q')) or (button == 27):
+                break
+            elif button == ord(' '):
+                ey.pass_frames(cap)
+                t1 = time.perf_counter()
+                while self.running:
+                    ey.show_clb_win(win_name, texts=tx1, win_color=ey.GRAY)
+                    button = cv2.waitKey(1)
+                    if (button == ord('q')) or (button == ord('Q')) or (button == 27) or (button == ord(' ')):
+                        break
+                    frame_success, frame, frame_rgb = ey.get_frame(cap)
+                    if frame_success:
+                        # Predicting the face landmarks
+                        results = face_mesh.process(frame_rgb)
+
+                        # Calculating the face features
+                        (
+                            features_success,
+                            _,
+                            _,
+                            _,
+                            eyes_ratio,
+                            _
+                        ) = ey.get_model_inputs(
+                            frame,
+                            frame_rgb,
+                            results,
+                            camera_matrix,
+                            pcf,
+                            frame_size,
+                            dst_cof,
+                            some_landmarks_ids,
+                            False,
+
+                        )
+                        if features_success:
+                            t_vec.append(round(time.perf_counter() - t1, 3))
+                            eyes_ratio_vec.append(eyes_ratio)
+                            
+                    if not self.running:
+                        break
+            t_mat.append(np.array(t_vec))
+            eyes_ratio_mat.append(np.array(eyes_ratio_vec))
+            if (button == ord('q')) or (button == ord('Q')) or (button == 27):
+                break
+            if not self.running:
+                break
+            cv2.destroyWindow(win_name)
+        cap.release()
+        cv2.destroyAllWindows()
+        ey.get_time(0, t0, True)
+
+        eyes_ratio_v_mat = ey.get_blinking(t_mat, eyes_ratio_mat)[0]
+
+        offered_threshold = ey.DEFAULT_BLINKING_THRESHOLD
+        if len(eyes_ratio_v_mat) > 1:
+            max_values = []
+            for eyes_ratio_v_vec in eyes_ratio_v_mat:
+                max_values.append(eyes_ratio_v_vec.max())
+            offered_threshold = min(max_values) * 0.99
+        else:
+            if eyes_ratio_v_mat:
+                offered_threshold = eyes_ratio_v_mat[0].max() * 0.6
+        print(f"Offered Threshold: {offered_threshold}")
+            
+        er_dir = ey.create_dir([ey.subjects_dir, f"{num}", ey.ER])
+
+        ey.save([t_mat, eyes_ratio_mat, offered_threshold], er_dir, [ey.T, ey.ER, "oth_app"])
+
+

codes/crt_train_models.py

@@ -0,0 +1,561 @@
+"""This file is for creating and training the neural network models for eye movement prediction. Also, this is for creating and training
+The in-out model which is for predicting whether the subject is looking inside of the screen or outside of the screen. To understand this
+module, you should know about how to build neural network models with keras and tensorflow"""
+
+from tensorflow.keras.layers import (Input, Conv2D, Flatten, MaxPooling2D,
+                                     Dense, Concatenate)
+from tensorflow.keras.models import Model
+import numpy as np
+import os
+from tensorflow.keras.callbacks import EarlyStopping
+from tensorflow.keras.models import load_model
+from sklearn.preprocessing import StandardScaler
+from sklearn.utils import shuffle
+from joblib import dump as j_dump
+from joblib import load as j_load
+import random
+from codes.base import eyeing as ey
+from openpyxl import Workbook
+
+
+class Modeling():
+    @staticmethod
+    def create_io():
+        """
+        creating in-out model (a CNN model) using tensorflow and keras
+        
+        Parameters:
+            None
+        
+        Returns:
+            None
+        """
+        
+        print("Starting to create an empty in_out model...")
+        inp1_shape = (ey.EYE_SIZE[0], ey.EYE_SIZE[1]*2, 1)
+        x2_chosen_features = (0, 1, 2, 3, 4, 5, 6, 7, 8, 9)
+        inp2_shape = (len(x2_chosen_features),)
+
+        inp1 = Input(inp1_shape)
+        layer = Conv2D(16, (11, 11), (1, 1), 'same', activation='relu')(inp1)
+        layer = MaxPooling2D((2, 2), (2, 2))(layer)
+        layer = Conv2D(32, (7, 7), (1, 1), 'same', activation='relu')(layer)
+        layer = MaxPooling2D((2, 2), (2, 2))(layer)
+        layer = Conv2D(64, (5, 5), (1, 1), 'same', activation='relu')(layer)
+        layer = MaxPooling2D((2, 2), (2, 2))(layer)
+        layer = Conv2D(128, (3, 3), (1, 1), activation='relu')(layer)
+        layer = MaxPooling2D((2, 2), (2, 2))(layer)
+        layer = Flatten()(layer)
+        inp2 = Input(inp2_shape)
+        layer = Concatenate()([layer, inp2])
+        layer = Dense(256, 'relu')(layer)
+        layer = Dense(128, 'relu')(layer)
+        layer = Dense(32, 'relu')(layer)
+        layer = Dense(8, 'relu')(layer)
+        output_layer = Dense(1, "sigmoid")(layer)
+        input_layers = [inp1, inp2]
+        model = Model(inputs=input_layers, outputs=output_layer)
+        model.compile(optimizer="adam", loss="binary_crossentropy", metrics="acc")
+        print(model.summary())
+        n_weights = np.sum([np.prod(v.get_shape()) for v in model.trainable_weights])
+
+        mdl_num = ey.find_max_mdl(ey.io_raw_dir) + 1
+        info = {"n_weights": n_weights,
+                "input1_shape": inp1_shape,
+                "input2_shape": inp2_shape,
+                "x2_chosen_features": x2_chosen_features}
+        mdl_name = ey.MDL + f"{mdl_num}"
+        mdl_dir = ey.io_raw_dir + mdl_name + ".h5"
+        model.save(mdl_dir)
+        ey.save([info], ey.io_raw_dir, [mdl_name])
+        print("\nEmpty in_out model created and saved to " + mdl_dir)
+
+
+    @staticmethod
+    def create_et():
+        """
+        Creating eye tracking model (CNN model) using tensorflow and keras. You can change the structure in following, as you want.
+        
+        Parameters:
+            None
+        
+        Returns:
+            None
+        """
+        
+
+        print("Starting to create empty eye_tracking models...")
+        inp1_shape = (ey.EYE_SIZE[0], ey.EYE_SIZE[1]*2, 1)
+        x2_chosen_features = (0, 1, 2, 3, 4, 5, 6, 7, 8, 9)
+        inp2_shape = (len(x2_chosen_features),)
+
+        inp1 = Input(inp1_shape)
+        layer = Conv2D(16, (11, 11), (1, 1), 'same', activation='relu')(inp1)
+        layer = MaxPooling2D((2, 2), (2, 2))(layer)
+        layer = Conv2D(32, (7, 7), (1, 1), 'same', activation='relu')(layer)
+        layer = MaxPooling2D((2, 2), (2, 2))(layer)
+        layer = Conv2D(64, (5, 5), (1, 1), 'same', activation='relu')(layer)
+        layer = MaxPooling2D((2, 2), (2, 2))(layer)
+        layer = Conv2D(128, (3, 3), (1, 1), activation='relu')(layer)
+        layer = MaxPooling2D((2, 2), (2, 2))(layer)
+        layer = Flatten()(layer)
+        inp2 = Input(inp2_shape)
+        layer = Concatenate()([layer, inp2])
+        layer = Dense(256, 'relu')(layer)
+        layer = Dense(128, 'relu')(layer)
+        layer = Dense(32, 'relu')(layer)
+        layer = Dense(8, 'relu')(layer)
+        out = Dense(1, 'linear')(layer)
+        input_layers = [inp1, inp2]
+        model = Model(inputs=input_layers, outputs=out)
+        model.compile(optimizer='adam', loss='mse')
+        print(model.summary())
+        n_weights = np.sum([np.prod(v.get_shape()) for v in model.trainable_weights])
+
+        mdl_num = ey.find_max_mdl(ey.et_raw_dir) + 1
+        info = {"n_weights": n_weights,
+                "input1_shape": inp1_shape,
+                "input2_shape": inp2_shape,
+                "x2_chosen_features": x2_chosen_features}
+
+        mdl_name = ey.MDL + f"{mdl_num}"
+        mdl_dir = ey.et_raw_dir + mdl_name + ".h5"
+        model.save(mdl_dir)
+        ey.save([info], ey.et_raw_dir, [mdl_name])
+        print("\nEmpty eye_tracking model created and saved to " + mdl_dir)
+
+    @staticmethod
+    def train_io(
+        subjects,
+        models_list,
+        min_max_brightness_ratio=[[0.65, 1.45]],
+        r_train_list=[0.85],
+        n_epochs_patience=[[160, 10]],
+        save_scaler=False,
+        show_model=False
+        ):
+        """
+        Training the io models. This method uses the dataset in the io folder of subject's number folder. The parameters should be lists.
+        So, you can train each model with several parameters and hyper parameters to see which one works better.
+
+        Parameters:
+            subjects: a list of subject numbers that you want to train the model with them.
+            models_list: You can train several models at a same time. So, you can enter a list of model numbers
+            min_max_brightness_ratio: To make the models robust to the brightness, the eyes images are multiplies into a number between two considered numbers
+            r_train_list: The ratio for train dataset
+            n_epochs_patience: The number of epochs and patience to intrupt training
+            save_scaler: To save the scaler
+            show_model: To show the model
+        
+        Returns:
+            None
+        """
+        print("Starting to train in_out model...")
+        # Loading all subjects
+        x1_load = []
+        x2_load = []
+        y_load = []
+        for sbj in subjects:
+            data_io_dir = ey.create_dir([ey.subjects_dir, f"{sbj}", ey.IO])
+            x1_load0, x2_load0, y_load0 = ey.load(data_io_dir, [ey.X1, ey.X2, ey.Y])
+            for (x10, x20, y10) in zip(x1_load0[0], x2_load0[0], y_load0[0]):
+                x1_load.append(x10)
+                x2_load.append(x20)
+                y_load.append(y10)
+
+        x1_load = np.array(x1_load)
+        x2_load = np.array(x2_load)
+        y_load = np.array(y_load)
+
+        n_smp = x1_load.shape[0]
+        print(f"\nNumber of samples : {n_smp}")
+
+        # Going through each brightness in min_max_brightness_ratio list
+        j = 1
+        for mbr in min_max_brightness_ratio:
+            x1_new = x1_load.copy()
+            for (i, _) in enumerate(x1_load):
+                r = random.uniform(mbr[0], mbr[1])
+                x1_new[i] = (x1_new[i] * r).astype(np.uint8)
+
+            # Going through each model
+            for raw_mdl_num in models_list:
+                info = ey.load(ey.io_raw_dir, [ey.MDL + f"{raw_mdl_num}"])[0]
+                x2_chosen_features = info["x2_chosen_features"]
+                x2_new = x2_load[:, x2_chosen_features]
+
+                x1_shf, x2_shf, y_shf = shuffle(x1_new, x2_new, y_load)
+
+                x1_scaler = ey.X1_SCALER
+                x1 = x1_shf / x1_scaler
+
+                x2_scaler = StandardScaler()
+                x2 = x2_scaler.fit_transform(x2_shf)
+
+                scalers = [x1_scaler, x2_scaler]
+                if save_scaler:
+                    j_dump(scalers, ey.scalers_dir + f"scl_io_{len(x2_chosen_features)}.bin")
+
+                # Going through each training ratio in r_train_list
+                for rt in r_train_list:
+                    n_train = int(rt * n_smp)
+                    x1_train, x2_train = x1[:n_train], x2[:n_train]
+                    x1_val, x2_val = x1[n_train:], x2[n_train:]
+                    
+                    y_train = y_shf[:n_train]
+                    y_val = y_shf[n_train:]
+                    print("\nTrain and val data shape:")
+                    print(x1_train.shape, x1_val.shape, x2_train.shape, x2_val.shape,
+                          y_train.shape, y_val.shape)
+
+                    x_train = [x1_train, x2_train]
+                    x_val = [x1_val, x2_val]
+
+                    # Going throught each epoch and patience in n_epochs_patience
+                    for nep in n_epochs_patience:
+                        # Training the models
+                        info["min_max_brightness_ratio"] = mbr
+                        info["r_train"] = rt
+                        info["n_epochs_patience"] = nep
+                        cb = EarlyStopping(patience=nep[1], verbose=1, restore_best_weights=True)
+
+                        raw_model_dir = ey.io_raw_dir + ey.MDL + f"{raw_mdl_num}.h5"
+                        print("\nLoading blink_in_out model from " + raw_model_dir)
+                        model = load_model(raw_model_dir)
+                        if show_model:
+                            print(model.summary())
+
+                        print(f"\n<<<<<<< {j}-model:{raw_mdl_num}-min_max_ratio:{mbr}-r_train:{rt}-epoch_patience:{nep} >>>>>>>>")
+                        model.fit(x_train,
+                                  y_train,
+                                  validation_data=(x_val, y_val),
+                                  epochs=nep[0],
+                                  callbacks=cb)
+                        train_loss = model.evaluate(x_train, y_train)
+                        val_loss = model.evaluate(x_val, y_val)
+
+                        info["train_loss"] = train_loss
+                        info["val_loss"] = val_loss
+
+                        trained_mdl_num = ey.find_max_mdl(ey.io_trained_dir) + 1
+                        mdl_name = ey.MDL + f'{trained_mdl_num}'
+                        ey.save([info], ey.io_trained_dir, [mdl_name])
+                        mdl_tr_dir = ey.io_trained_dir + mdl_name + ".h5"
+                        model.save(mdl_tr_dir)
+                        print("\nSaving in_out model in " + mdl_tr_dir)
+                        j += 1
+        
+
+    @staticmethod
+    def train_et(
+        subjects,
+        models_list,
+        min_max_brightness_ratio=[[0.65, 1.45]],
+        r_train_list=[0.8],
+        n_epochs_patience=[[100, 15]],
+        shift_samples=None,
+        blinking_threshold="d",
+        save_scaler=False,
+        show_model=False
+        ):
+        """
+        Training the et (base) models. This method uses the dataset in the et folder of subject's number folder. The parameters should be lists.
+        So, you can train each model with several parameters and hyper parameters to see which one works better.
+
+        Parameters:
+            subjects: a list of subject numbers that you want to train the model with them.
+            models_list: You can train several models at a same time. So, you can enter a list of model numbers
+            min_max_brightness_ratio: To make the models robust to the brightness, the eyes images are multiplies into a number between two considered numbers
+            r_train_list: The ratio for train dataset
+            n_epochs_patience: The number of epochs and patience to intrupt training
+            shift_samples: To shift sample if there is a high delay
+            blinking_threshold: It can have three types --> d: default, ao: app offered, uo: user offered
+            save_scaler: To save the scaler
+            show_model: To show the model
+        
+        Returns:
+            None
+        """
+        print("Starting to train eye_tracking models...")
+        
+        # Loading all subjects
+        x1_load = []
+        x2_load = []
+        y_load = []
+        kk = 0
+        for sbj in subjects:
+            sbj_dir = ey.create_dir([ey.subjects_dir, f"{sbj}"])
+            sbj_clb_dir = ey.create_dir([sbj_dir, ey.CLB])
+
+            (
+                sbj_x1_load,
+                sbj_x2_load,
+                sbj_y_load,
+                sbj_t_mat,
+                sbj_eyes_ratio
+            ) = ey.load(sbj_clb_dir, [ey.X1, ey.X2, ey.Y, ey.T, ey.ER])
+
+            # If there is any shifting samples, doing that
+            if shift_samples:
+                if shift_samples[kk]:
+                    ii = 0
+                    for (x11, x21, y1, t1, eyr1) in zip(sbj_x1_load, sbj_x2_load, sbj_y_load, sbj_t_mat, sbj_eyes_ratio):
+                        sbj_t_mat[ii] = t1[:-shift_samples[kk]]
+                        sbj_x1_load[ii] = x11[shift_samples[kk]:]
+                        sbj_x2_load[ii] = x21[shift_samples[kk]:]
+                        sbj_y_load[ii] = y1[:-shift_samples[kk]]
+                        sbj_eyes_ratio[ii] = eyr1[shift_samples[kk]:]
+                        ii += 1
+
+            kk += 1
+            sbj_er_dir = ey.create_dir([sbj_dir, ey.ER])
+
+            # Removing the samples that are during blinking
+            sbj_blinking_threshold = ey.get_threshold(sbj_er_dir, blinking_threshold)
+
+            sbj_blinking = ey.get_blinking(sbj_t_mat, sbj_eyes_ratio, sbj_blinking_threshold)[1]
+
+            for (x11, x21, y1, b1) in zip(sbj_x1_load, sbj_x2_load, sbj_y_load, sbj_blinking):
+                for (x10, x20, y0, b0) in zip(x11, x21, y1, b1):
+                    if not b0:
+                        x1_load.append(x10)
+                        x2_load.append(x20)
+                        y_load.append(y0)
+        x1_load = np.array(x1_load)
+        x2_load = np.array(x2_load)
+        y_load = np.array(y_load)
+        n_smp = x1_load.shape[0]
+        print(f"\nNumber of samples : {n_smp}")
+
+        # Going through each brightness in min_max_brightness_ratio list
+        j = 1
+        for mbr in min_max_brightness_ratio:
+            x1_new = x1_load.copy()
+            for (i, _) in enumerate(x1_load):
+                r = random.uniform(mbr[0], mbr[1])
+                x1_new[i] = (x1_new[i] * r).astype(np.uint8)
+
+            # Going through each model
+            for raw_mdl_num in models_list:
+                info = ey.load(ey.et_raw_dir, [ey.MDL + f"{raw_mdl_num}"])[0]
+                x2_chosen_features = info["x2_chosen_features"]
+                x2_new = x2_load[:, x2_chosen_features]
+
+                x1_shf, x2_shf, y_hrz_shf, y_vrt_shf = shuffle(x1_new, x2_new, y_load[:, 0], y_load[:, 1])
+
+                x1_scaler = ey.X1_SCALER
+                x1 = x1_shf / x1_scaler
+
+                x2_scaler = StandardScaler()
+                x2 = x2_scaler.fit_transform(x2_shf)
+                y_scaler = ey.Y_SCALER
+
+                scalers = [x1_scaler, x2_scaler, y_scaler]
+
+                if save_scaler:
+                    j_dump(scalers, ey.scalers_dir + f"scl_et_{len(x2_chosen_features)}.bin")
+
+                # Going through each training ratio in r_train_list
+                for rt in r_train_list:
+                    n_train = int(rt * n_smp)
+                    x1_train, x2_train = x1[:n_train], x2[:n_train]
+                    x1_val, x2_val = x1[n_train:], x2[n_train:]
+                    
+                    y_hrz_train, y_vrt_train = y_hrz_shf[:n_train], y_vrt_shf[:n_train]
+                    y_hrz_val, y_vrt_val = y_hrz_shf[n_train:], y_vrt_shf[n_train:]
+                    print("\nTrain and val data shape:")
+                    print(x1_train.shape, x1_val.shape, x2_train.shape, x2_val.shape,
+                          y_hrz_train.shape, y_hrz_val.shape, y_vrt_train.shape, y_vrt_val.shape)
+
+                    x_train = [x1_train, x2_train]
+                    x_val = [x1_val, x2_val]
+
+                    # Going throught each epoch and patience in n_epochs_patience
+                    for nep in n_epochs_patience:
+                        # Training the models
+                        info["min_max_brightness_ratio"] = mbr
+                        info["r_train"] = rt
+                        info["n_epochs_patience"] = nep
+                        cb = EarlyStopping(patience=nep[1], verbose=1, restore_best_weights=True)
+
+                        raw_model_dir = ey.et_raw_dir + ey.MDL + f"{raw_mdl_num}.h5"
+                        print("\nLoading eye_tracking model from " + raw_model_dir)
+                        model_hrz = load_model(raw_model_dir)
+                        model_vrt = load_model(raw_model_dir)
+                        if show_model:  
+                            print(model_hrz.summary())
+
+                        trained_mdl_num = ey.find_max_mdl(ey.et_trained_dir, b=-7) + 1
+
+                        print(f"\n<<<<<<< {j}-model-hrz:{raw_mdl_num}-min_max_ratio:{mbr}-r_train:{rt}-epoch_patience:{nep} >>>>>>>>")
+                        model_hrz.fit(x_train,
+                                      y_hrz_train * y_scaler,
+                                      validation_data=(x_val, y_hrz_val * y_scaler),
+                                      epochs=nep[0],
+                                      callbacks=cb)
+                        mdl_name = ey.MDL + f"{trained_mdl_num}"
+                        mdl_hrz_tr_dir = ey.et_trained_dir + mdl_name + "-hrz.h5"
+                        print("\nSaving horizontally eye_tracking model in " + mdl_hrz_tr_dir)
+                        model_hrz.save(mdl_hrz_tr_dir)
+                        hrz_train_loss = model_hrz.evaluate(x_train, y_hrz_train * y_scaler)
+                        hrz_val_loss = model_hrz.evaluate(x_val, y_hrz_val * y_scaler)
+                        info["hrz_train_loss"] = hrz_train_loss
+                        info["hrz_val_loss"] = hrz_val_loss
+
+                        print(f"\n<<<<<<< {j}-model-vrt:{raw_mdl_num}-min_max_ratio:{mbr}-r_train:{rt}-epoch_patience:{nep} >>>>>>>>")
+                        model_vrt.fit(x_train,
+                                      y_vrt_train * y_scaler,
+                                      validation_data=(x_val, y_vrt_val * y_scaler),
+                                      epochs=nep[0],
+                                      callbacks=cb)
+                        tr_model_vrt_dir = ey.et_trained_dir + mdl_name + f"-vrt.h5"
+                        print("Saving vertically eye_tracking model in " + tr_model_vrt_dir)
+                        model_vrt.save(tr_model_vrt_dir)
+                        vrt_train_loss = model_vrt.evaluate(x_train, y_vrt_train * y_scaler)
+                        vrt_val_loss = model_vrt.evaluate(x_val, y_vrt_val * y_scaler)
+                        info["vrt_train_loss"] = vrt_train_loss
+                        info["vrt_val_loss"] = vrt_val_loss
+
+                        ey.save([info], ey.et_trained_dir, [mdl_name])
+
+                        j += 1
+
+
+    @staticmethod
+    def get_models_information(io=True, raw=True, show_model=False):
+        """
+        To write the models information in an excel file. It gets the information from attached pickle file for each model.
+        There are raw models and trained models in the io and the et.
+
+        Parameters:
+            io: If it's io or et
+            raw: If the model is trained or not
+            show_model: If you want to show the model
+
+        Returns:
+            None
+        """
+        wb = Workbook()
+        ws = wb.active
+        ws['A1'] = "# of model"
+        ws['B1'] = "# of weights"
+        ws['C1'] = "input 1 shape"
+        ws['D1'] = "input 2 shape"
+        ws['E1'] = "x2 chosen features"
+        if io:
+            if raw:
+                files_name = os.listdir(ey.io_raw_dir)
+                if files_name:
+                    for fn in files_name:
+                        if fn[-7:] == ".pickle":
+                            mdl_num = int(fn[3:-7])
+                            mdl_name = ey.MDL + f"{mdl_num}"
+                            if show_model:
+                                mdl_dir = ey.io_raw_dir + mdl_name + ".h5"
+                                mdl = load_model(mdl_dir)
+                                print(f"<<<<<<<<<<<<<< {mdl_dir} >>>>>>>>>>>>>>")
+                                print(mdl.summary())
+                            info = ey.load(ey.io_raw_dir, [mdl_name])[0]
+
+                            ws[f'A{mdl_num+1}'] = str(mdl_num)
+                            ws[f'B{mdl_num+1}'] = str(info['n_weights'])
+                            ws[f'C{mdl_num+1}'] = str(info['input1_shape'])
+                            ws[f'D{mdl_num+1}'] = str(info['input2_shape'])
+                            ws[f'E{mdl_num+1}'] = str(info['x2_chosen_features'])
+            else:
+                ws['F1'] = "min-Max brightness ratio"
+                ws['G1'] = "r_train"
+                ws['H1'] = "# of epochs and patience"
+                ws['I1'] = "train loss"
+                ws['J1'] = "val loss"
+
+                files_name = os.listdir(ey.io_trained_dir)
+                if files_name:
+                    for fn in files_name:
+                        if fn[-7:] == ".pickle":
+                            mdl_num = int(fn[3:-7])
+                            mdl_name = ey.MDL + f"{mdl_num}"
+                            if show_model:
+                                mdl_dir = ey.io_trained_dir + mdl_name + ".h5"
+                                mdl = load_model(mdl_dir)
+                                print(f"<<<<<<<<<<<<<< {mdl_dir} >>>>>>>>>>>>>>")
+                                print(mdl.summary())
+                            info = ey.load(ey.io_trained_dir, [mdl_name])[0]
+
+                            ws[f'A{mdl_num+1}'] = str(mdl_num)
+                            ws[f'B{mdl_num+1}'] = str(info['n_weights'])
+                            ws[f'C{mdl_num+1}'] = str(info['input1_shape'])
+                            ws[f'D{mdl_num+1}'] = str(info['input2_shape'])
+                            ws[f'E{mdl_num+1}'] = str(info['x2_chosen_features'])
+                            ws[f'F{mdl_num+1}'] = str(info['min_max_brightness_ratio'])
+                            ws[f'G{mdl_num+1}'] = str(info['r_train'])
+                            ws[f'H{mdl_num+1}'] = str(info['n_epochs_patience'])
+                            ws[f'I{mdl_num+1}'] = str(info['train_loss'])
+                            ws[f'J{mdl_num+1}'] = str(info['val_loss'])
+
+        else:
+            if raw:
+                files_name = os.listdir(ey.et_raw_dir)
+                if files_name:
+                    for fn in files_name:
+                        if fn[-7:] == ".pickle":
+                            mdl_num = int(fn[3:-7])
+                            mdl_name = ey.MDL + f"{mdl_num}"
+                            if show_model:
+                                mdl_dir = ey.et_raw_dir + mdl_name + ".h5"
+                                mdl = load_model(mdl_dir)
+                                print(f"<<<<<<<<<<<<<< {mdl_dir} >>>>>>>>>>>>>>")
+                                print(mdl.summary())
+                            info = ey.load(ey.et_raw_dir, [mdl_name])[0]
+
+                            ws[f'A{mdl_num+1}'] = str(mdl_num)
+                            ws[f'B{mdl_num+1}'] = str(info['n_weights'])
+                            ws[f'C{mdl_num+1}'] = str(info['input1_shape'])
+                            ws[f'D{mdl_num+1}'] = str(info['input2_shape'])
+                            ws[f'E{mdl_num+1}'] = str(info['x2_chosen_features'])
+
+            else:
+                ws['F1'] = "min-Max brightness ratio"
+                ws['G1'] = "r_train"
+                ws['H1'] = "# of epochs and patience"
+                ws['I1'] = "model-hrz train loss"
+                ws['J1'] = "model-hrz val loss"
+                ws['K1'] = "model-vrt train loss"
+                ws['L1'] = "model-vrt val loss"
+
+                files_name = os.listdir(ey.et_trained_dir)
+                if files_name:
+                    for fn in files_name:
+                        if fn[-7:] == ".pickle":
+                            mdl_num = int(fn[3:-7])
+                            mdl_name = ey.MDL + f"{mdl_num}"
+                            if show_model:
+                                mdl_dir = ey.et_trained_dir + mdl_name + "-hrz.h5"
+                                mdl = load_model(mdl_dir)
+                                print(f"<<<<<<<<<<<<<< {mdl_dir} >>>>>>>>>>>>>>")
+                                print(mdl.summary())
+                            info = ey.load(ey.et_trained_dir, [mdl_name])[0]
+
+                            ws[f'A{mdl_num+1}'] = str(mdl_num)
+                            ws[f'B{mdl_num+1}'] = str(info['n_weights'])
+                            ws[f'C{mdl_num+1}'] = str(info['input1_shape'])
+                            ws[f'D{mdl_num+1}'] = str(info['input2_shape'])
+                            ws[f'E{mdl_num+1}'] = str(info['x2_chosen_features'])
+                            ws[f'F{mdl_num+1}'] = str(info['min_max_brightness_ratio'])
+                            ws[f'G{mdl_num+1}'] = str(info['r_train'])
+                            ws[f'H{mdl_num+1}'] = str(info['n_epochs_patience'])
+                            ws[f'I{mdl_num+1}'] = str(info['hrz_train_loss'])
+                            ws[f'J{mdl_num+1}'] = str(info['hrz_val_loss'])
+                            ws[f'K{mdl_num+1}'] = str(info['vrt_train_loss'])
+                            ws[f'L{mdl_num+1}'] = str(info['vrt_val_loss'])
+
+        if io and raw:
+            info_name = "info_io_raw"
+        elif io and not raw:
+            info_name = "info_io_trained"
+        elif not io and raw:
+            info_name = "info_et_raw"
+        else:
+            info_name = "info_et_trained"
+
+        wb.save(ey.files_dir + info_name + ".xlsx")

codes/eye_track.py

@@ -0,0 +1,701 @@
+"""This module contains the EyeTrack method. In this method, the eye movements will be predicted using the inputs and retrained models.
+Also the fixations will be calculated."""
+
+from tensorflow.keras.models import load_model
+import numpy as np
+from joblib import load as j_load
+from codes.base import eyeing as ey
+from scipy import signal
+from openpyxl import Workbook, load_workbook
+import os
+
+
+class EyeTrack(object):
+    @staticmethod
+    def get_pixels(
+        subjects,
+        models_list=[1],
+        target_fol=ey.SMP,
+        shift_samples=None,
+        blinking_threshold="uo",
+        use_io=False,
+        delete_files=False
+        ):
+        """
+        Predicting the eye movements using the inputs (eyes images and face vectors). This can be done on sampling (smp) data,
+        tsting (acc) data or latency (ltn) data. You can predict outputs for several subjects and with several models, to exactly know
+        which one is working better. In this method, the samples that are during blinking, will be deleted. The values for eye movements
+        are between 0 and 1. It means it is independant to the size of screen.
+
+        Parameters:
+            subjects: list of subjects that we want to predict their eye viewpoints
+            models_list: list of models that we want to use them to predict the eye viewpoints
+            target_fol: the targeted folder that we want to predict its data. smp, acc, or ltn
+            shift_smaples: whether or not shift the inputs
+            blinking_threshold: blinking threshold. It can takes "d" as default, "uo" as user offered and "ao" as app offered.
+            use_io: whether or not use the io model
+            delete_files: whethere or not remove the inputs after prediction. Because of size of the saved images
+        """
+        tfn = 1         # For sampling data
+        if target_fol == ey.ACC:
+            tfn = 2     # For testing data
+        elif target_fol == ey.LTN:
+            tfn = 3     # For latency data
+
+        out_threshold_min = 0.005
+        out_threshold_max = 0.995
+        latency_radius = 0.33
+        median_filter_window_size = 5
+
+        x1_scaler_et, x2_scaler_et, y_scaler = j_load(ey.scalers_dir + "scalers_et_main.bin")
+        if tfn == 1:
+            x1_scaler_io, x2_scaler_io = j_load(ey.scalers_dir + "scalers_io_main.bin")
+            mdl_io = load_model(ey.io_trained_dir + ey.MDL + "1.h5")
+
+        # Going through each subject's folder to predict their eye movement
+        kk = 0
+        for num in subjects:
+            print(f"<<<<<<<<<<<<<<<<<<<<< Subject {num} >>>>>>>>>>>>>>>>>>>>>>>")
+            sbj_dir = ey.create_dir([ey.subjects_dir, f"{num}"])
+            sbj_models_dir = ey.create_dir([sbj_dir, ey.MDL])
+            target_dir = ey.create_dir([sbj_dir, target_fol])
+
+            # Loading the data and shifting the inputs if it's needed
+            if ey.file_existing(target_dir, ey.X1+".pickle"):
+                if tfn == 1:
+                    t_load, sys_time_load, x1_load, x2_load, eyes_ratio = ey.load(target_dir, [ey.T, "sys_time", ey.X1, ey.X2, ey.ER])
+                    if shift_samples:
+                        if shift_samples[kk]:
+                            ii = 0
+                            for (x11, x21, t1, st1, eyr1) in zip(x1_load, x2_load, t_load, sys_time_load, eyes_ratio):
+                                t_load[ii] = t1[:-shift_samples[kk]]
+                                sys_time_load[ii] = st1[:-shift_samples[kk]]
+                                x1_load[ii] = x11[shift_samples[kk]:]
+                                x2_load[ii] = x21[shift_samples[kk]:]
+                                eyes_ratio[ii] = eyr1[shift_samples[kk]:]
+                                ii += 1
+                elif tfn == 2:
+                    t_load, x1_load, x2_load, y_load, eyes_ratio = ey.load(target_dir, [ey.T, ey.X1, ey.X2, ey.Y, ey.ER])
+                    if shift_samples:
+                        if shift_samples[kk]:
+                            ii = 0
+                            for (x11, x21, y1, t1, eyr1) in zip(x1_load, x2_load, y_load, t_load, eyes_ratio):
+                                t_load[ii] = t1[:-shift_samples[kk]]
+                                x1_load[ii] = x11[shift_samples[kk]:]
+                                x2_load[ii] = x21[shift_samples[kk]:]
+                                y_load[ii] = y1[:-shift_samples[kk]]
+                                eyes_ratio[ii] = eyr1[shift_samples[kk]:]
+                                ii += 1
+                else:
+                    t_load, x1_load, x2_load = ey.load(target_dir, [ey.T, ey.X1, ey.X2])
+                    if shift_samples:
+                        if shift_samples[kk]:
+                            ii = 0
+                            for (x11, x21, t1) in zip(x1_load, x2_load, t_load):
+                                t_load[ii] = t1[:-shift_samples[kk]]
+                                x1_load[ii] = x11[shift_samples[kk]:]
+                                x2_load[ii] = x21[shift_samples[kk]:]
+                                ii += 1
+                kk += 1
+
+                # Going through each model to predict the output
+                for mdl_num in models_list:
+                    mdl_et_name = ey.MDL + f"{mdl_num}"
+                    mdl_et_hrz_dir = sbj_models_dir + mdl_et_name + "-hrz.h5"
+                    mdl_et_vrt_dir = sbj_models_dir + mdl_et_name + "-vrt.h5"
+                    if ey.file_existing(sbj_models_dir, mdl_et_name + "-hrz.h5"):
+                        info = ey.load(sbj_models_dir, [mdl_et_name])[0]
+                        x2_chosen_features = info["x2_chosen_features"]
+                        mdl_et_hrz = load_model(mdl_et_hrz_dir)
+                        mdl_et_vrt = load_model(mdl_et_vrt_dir)
+
+                        # x1_load and x2_load are lists of lists. So, we should predict each list
+                        y_prd = []
+                        for (x11, x21) in zip(x1_load, x2_load):
+                            n_smp_vec = x11.shape[0]
+                            x21_new = x21[:, x2_chosen_features]
+                            x11_nrm = x11 / x1_scaler_et
+                            x21_nrm = x2_scaler_et.transform(x21_new)
+                            x0_nrm = [x11_nrm, x21_nrm]
+
+                            y_hrz_prd = np.expand_dims(mdl_et_hrz.predict(x0_nrm).reshape((n_smp_vec,)), 1) / y_scaler
+                            y_vrt_prd = np.expand_dims(mdl_et_vrt.predict(x0_nrm).reshape((n_smp_vec,)), 1) / y_scaler
+
+                            y_prd.append(np.concatenate([y_hrz_prd, y_vrt_prd], 1))
+
+                        # For calculation of latency, it's just needed to see if the subject is looking in left or right, not exact location
+                        if tfn == 3:
+                            t_delay = []
+                            j = 0
+                            for (t1, y1_prd) in zip(t_load, y_prd):
+                                for (t0, y0_prd) in zip(t1, y1_prd):
+                                    if j % 2 == 0:
+                                        d = y0_prd[0] - 0.66
+                                    else:
+                                        d = 0.33 - y0_prd[0]
+                                    if 0 < d < latency_radius:
+                                        t_delay.append(t0 - t1[0])
+                                        break
+                                j += 1
+                            print(t_delay)
+                            t_delay = np.array(t_delay).mean() - ey.LATENCY_WAITING_TIME/1000.0
+                            print(t_delay)
+                            ey.save([t_delay], target_dir, ["t_delay"])
+
+                        else:
+                            # predict the samples that are looking outside of the screen
+                            y_in = y_prd.copy()
+                            if (tfn == 1) and use_io:
+                                for (x11, x21, yi1) in zip(x1_load, x2_load, y_in):
+                                    x1_io = x11 / x1_scaler_io
+                                    x2_io = x2_scaler_io.transform(x21)
+                                    y_io_prd = mdl_io.predict([x1_io, x2_io]).round()
+                                    
+                                    for (et0, yio) in zip(yi1, y_io_prd):
+                                        if yio == 1:
+                                            et0[0] = -1
+                                            et0[1] = -1
+
+                            er_dir = ey.create_dir([sbj_dir, ey.ER])
+
+                            # Removing the samples that are during blinking
+                            blinking_threshold = ey.get_threshold(er_dir, blinking_threshold)
+                            blinking = ey.get_blinking(t_load, eyes_ratio, blinking_threshold)[1]
+                            for (yi1, bl1) in zip(y_in, blinking):
+                                for (yi0, bl0) in zip(yi1, bl1):
+                                    if bl0:
+                                        yi0[0] = -1
+                                        yi0[1] = -1
+
+                            """Putting the values that are consecuitive and are looking inside the screen and they are not blink,
+                            into one list"""
+                            y_prd_mat = []
+                            for yi1 in y_in:
+                                blinking_out = (yi1[:, 0] == -1)
+                                n_smp = yi1.shape[0]
+                                i = 0
+                                while i < (n_smp):
+                                    bo_vec = []
+                                    in_vec = []
+                                    now = blinking_out[i]
+                                    if now:
+                                        bo_vec.append(yi1[i])
+                                    else:
+                                        in_vec.append(yi1[i])
+                                    j = 1
+                                    if (i+j) < n_smp:
+                                        while blinking_out[i+j] == now:
+                                            if now:
+                                                bo_vec.append(yi1[i+j])
+                                            else:
+                                                in_vec.append(yi1[i+j])
+                                            j += 1
+                                            if (i+j) >= n_smp:
+                                                break
+                                    if now:
+                                        y_prd_mat.append(np.array(bo_vec))
+                                    else:
+                                        y_prd_mat.append(np.array(in_vec))
+                                    i += j
+
+                            # Implementing median filter to the predicted values
+                            for y_prd_vec in y_prd_mat:
+                                if y_prd_vec[0, 0] != -1:
+                                    if 3 < y_prd_vec.shape[0] < (median_filter_window_size+2):
+                                        y_prd_vec[:, 0] = signal.medfilt(y_prd_vec[:, 0], 3)
+                                        y_prd_vec[:, 1] = signal.medfilt(y_prd_vec[:, 1], 3)
+                                    elif y_prd_vec.shape[0] >= (median_filter_window_size+2):
+                                        y_prd_vec[:, 0] = signal.medfilt(y_prd_vec[:, 0], median_filter_window_size)
+                                        y_prd_vec[:, 1] = signal.medfilt(y_prd_vec[:, 1], median_filter_window_size)
+
+                            # Concatenating y
+                            y_prd_fnl = y_prd_mat[0]
+                            for (i, y_prd_vec) in enumerate(y_prd_mat):
+                                if i == 0:
+                                    continue
+                                y_prd_fnl = np.concatenate([y_prd_fnl, y_prd_vec], 0)
+
+                            # Saving the data
+                            if tfn == 1:
+                                t = []
+                                sys_time = []
+                                for (t1, st1) in zip(t_load, sys_time_load):
+                                    for (t0, st0) in zip(t1, st1):
+                                        t.append(t0)
+                                        sys_time.append(st0)
+                                t = np.array(t)
+                                wb = Workbook()
+                                ws = wb.active
+                                ws['A1'] = "Time"
+                                ws['A2'] = "sec"
+                                ws['B1'] = "SystemTime"
+                                ws['C1'] = "EyeTrack"
+                                ws['C2'] = "(p_x/scr_w,p_y/scr_h)"
+                                ws['D1'] = "Condition"
+                                ws['D2'] = "{start,stop}"
+                                ws['D3'] = "start"
+                                for i in range(y_prd_fnl.shape[0]):
+                                    ws[f'A{i+3}'] = f"{t[i]}"
+                                    ws[f'B{i+3}'] = sys_time[i]
+                                    ws[f'C{i+3}'] = f"({round(y_prd_fnl[i, 0] * 10000)/10000},{round(y_prd_fnl[i, 1] * 10000)/10000})"
+                                ws[f'D{i+3}'] = "stop"
+                                wb.save(target_dir + "eye_track.xlsx")
+                                ey.save([t, y_prd_fnl], target_dir, ["t_vec", "y_prd"])
+
+                                if delete_files:
+                                    ey.remove(target_dir, [ey.FV])
+                            else:
+                                y_vec = []
+                                for y1 in y_load:
+                                    for y0 in y1:
+                                        y_vec.append(y0)
+                                y_vec = np.array(y_vec)
+
+                                y_vec = y_vec[y_prd_fnl[:, 0] != -1]
+                                y_prd_fnl = y_prd_fnl[y_prd_fnl[:, 0] != -1]
+
+                                losses = np.sum(((y_prd_fnl-y_vec)*y_scaler)**2, 0) / y_vec.shape[0]
+
+                                print(f"Lossess for two hrz and vrt models: {losses}")
+
+                                info["hrz_retrain_test_loss"] = losses[0]
+                                info["vrt_retrain_test_loss"] = losses[1]
+
+                                y_prd_fnl[y_prd_fnl < out_threshold_min] = out_threshold_min
+                                y_prd_fnl[y_prd_fnl > out_threshold_max] = out_threshold_max
+
+                                ey.save([info], sbj_models_dir, [mdl_et_name])
+                                ey.save([y_vec, y_prd_fnl], target_dir, ["y_mdf", "y_prd_mdf"])
+
+                                if delete_files:
+                                    ey.remove(target_dir, [ey.Y])
+                            
+                            if delete_files:
+                                ey.remove(sbj_models_dir)
+                                ey.remove(target_dir, [ey.ER, ey.X1, ey.X2, ey.T])
+                    else:
+                        print(f"Data does not exist in {sbj_models_dir}")
+            else:
+                print(f"Data does not exist in {target_dir}")
+
+
+    @staticmethod
+    def get_fixations(
+        subjects,
+        n_monitors_data=1,
+        t_discard=0.1,
+        x_merge=0.2/2,
+        y_merge=0.25/2,
+        vx_thr=2.5,
+        vy_thr=2.5
+        ):
+        """
+        Compute the fixations using eye movements. IV-T method is implemented for this. Visit README.md for more details.
+        You can do this for all the subjets once.
+
+        Parameters:
+            subjects: subjects list
+            n_monitors_data: The number of monitors while the data is collected.
+            t_discard: fixations lower than this will be removed.
+            x_merge: fixations closer than this value (horizontal direction) will be added together.
+            y_merge: fixations closer than this value (vertical direction) will be added together.
+            vx_thr: This is the threshold for detecting saccades in the x direction.
+            vy_thr: This is the threshold for detecting saccades in the y direction.
+        
+        Returns:
+            None
+        """
+
+        # Going through each subject's folder to compute their fixations
+        for num in subjects:
+            smp_dir = ey.create_dir([ey.subjects_dir, f"{num}", ey.SMP])
+
+            if ey.file_existing(smp_dir, "eye_track.xlsx"):
+                sheet = load_workbook(smp_dir + "eye_track.xlsx")["Sheet"]
+                max_row = sheet.max_row
+                et_xl = []
+                for i in range(3, max_row+1):
+                    et_cell_list = sheet[f"C{i}"].value[1:-1].split(',')
+                    et_xl.append(
+                        [float(sheet[f"A{i}"].value),
+                         sheet[f"B{i}"].value,
+                         float(et_cell_list[0]),
+                         float(et_cell_list[1]),
+                         sheet[f"D{i}"].value]
+                        )
+                n_smp_all = len(et_xl)
+
+                """There is some tims that you don't want to calculate the fixations. you can simply put 'start' and 'stop'
+                in the last column in the eye_track.xlsx file to determine the moments that you want be calculated. So, each
+                series of values that are between 'start' and 'stop' is considered as a vector and in this way, these vectors
+                go to matrices (each matrix contains several vectors). for example, t_mat_seq"""
+                i = 0
+                t_mat_seq = []
+                t_sys_mat_seq = []
+                et_mat_seq = []
+                while i < n_smp_all:
+                    if (et_xl[i][4] == "start") or (et_xl[i][4] == "Start"):
+                        t1 = []
+                        ts1 = []
+                        et1 = []
+                        j = 0
+                        while True:
+                            t1.append([et_xl[i+j][0]])
+                            ts1.append([et_xl[i+j][1]])
+                            et1.append([et_xl[i+j][2], et_xl[i+j][3]])
+                            if et_xl[i+j][4] == "stop" or et_xl[i+j][4] == "Stop":
+                                break
+                            j += 1
+                        t_mat_seq.append(np.array(t1).reshape((len(t1),)))
+                        t_sys_mat_seq.append(ts1)
+                        et_mat_seq.append(np.array(et1))
+                        i += j
+                    i += 1
+
+                # Creating the the vectors for time and eye track
+                t = t_mat_seq[0]
+                t_sys = t_sys_mat_seq[0]
+                et = et_mat_seq[0]
+                for (i, t1) in enumerate(t_mat_seq):
+                    if i == 0:
+                        continue
+                    t = np.concatenate([t, t1])
+                    t_sys += t_sys_mat_seq[i]
+                    et = np.concatenate([et, et_mat_seq[i]])
+
+                # Removing the samples that are during blinking or are looking outside of the screen
+                t_mat = []
+                t_sys_mat = []
+                et_mat = []
+                for (t1, ts1, et1) in zip(t_mat_seq, t_sys_mat_seq, et_mat_seq):
+                    n_smp1 = t1.shape[0]
+                    blinking_out = (et1[:, 0] == -1)
+                    t_mat1 = []
+                    ts_mat1 = []
+                    et_mat1 = []
+                    i = 0
+                    while i < (n_smp1):
+                        t0 = [t1[i]]
+                        ts0 = [ts1[i]]
+                        bo_vec = []
+                        in_vec = []
+                        now = blinking_out[i]
+                        if now:
+                            bo_vec.append(et1[i])
+                        else:
+                            in_vec.append(et1[i])
+                        j = 1
+                        if (i+j) < n_smp1:
+                            while blinking_out[i+j] == now:
+                                t0.append(t1[i+j])
+                                ts0.append(ts1[i+j])
+                                if now:
+                                    bo_vec.append(et1[i+j])
+                                else:
+                                    in_vec.append(et1[i+j])
+                                j += 1
+                                if (i+j) >= n_smp1:
+                                    break
+                        t_mat1.append(np.array(t0))
+                        ts_mat1.append(ts0)
+                        if now:
+                            et_mat1.append(np.array(bo_vec))
+                        else:
+                            et_mat1.append(np.array(in_vec))
+                        i += j
+                    t_mat.append(t_mat1)
+                    t_sys_mat.append(ts_mat1)
+                    et_mat.append(et_mat1)
+
+                # Calculating the saccades
+                saccades = []
+                vet_mat = []
+                for (t2, et2) in zip(t_mat, et_mat):
+                    saccades1 = []
+                    vet_mat1 = []
+                    for (t1, et1) in zip(t2, et2):
+                        if et1[0, 0] != -1:
+                            if et1.shape[0] == 1:
+                                vet1 = np.zeros((1,2))
+                                s1 = [None]
+                            else:
+                                vet1 = et1.copy()
+                                vet1[1:, 0] = (et1[1:, 0] - et1[:-1, 0]) / (t1[1:] - t1[:-1])
+                                vet1[1:, 1] = (et1[1:, 1] - et1[:-1, 1]) / (t1[1:] - t1[:-1])
+                                vet1[0] = vet1[1]
+
+                                s1 = ((vet1[:, 0]>vx_thr)+(vet1[:, 0]<-vx_thr))+((vet1[:, 1]>vy_thr)+(vet1[:, 1]<-vy_thr))
+                        else:
+                            et_shape = et1.shape[0]
+                            vet1 = np.zeros(et1.shape)
+                            s1 = np.array([None] * et_shape)
+                        vet_mat1.append(vet1)
+                        saccades1.append(s1)
+                    saccades.append(saccades1)
+                    vet_mat.append(vet_mat1)
+
+                # Creating a vector of eye movement velocity
+                vet4 = []
+                for vet3 in vet_mat:
+                    vet2 = vet3[0].copy()
+                    for (i, vet1) in enumerate(vet3):
+                        if i == 0:
+                            continue
+                        vet2 = np.concatenate([vet2, vet1], 0)
+                    vet4.append(np.array(vet2))
+                vet = vet4[0]
+                for (i, vet1) in enumerate(vet4):
+                    if i == 0:
+                        continue
+                    vet = np.concatenate([vet, vet1])
+
+                """Separating the time and eye movements based on the saccades. It means we are considering a vector for
+                each series of values that we think they are one fixation."""
+                sac_mat_new = []
+                t_mat_new = []
+                t_sys_mat_new = []
+                et_mat_new = []
+                for (t_mat1, ts_mat1, et_mat1, saccades1) in zip(t_mat, t_sys_mat, et_mat, saccades):
+                    k = 0
+                    sac_mat_new1 = []
+                    t_mat_new1 = []
+                    t_sys_mat_new1 = []
+                    et_mat_new1 = []
+                    for (t1, ts1, et1, sac1) in zip(t_mat1, ts_mat1, et_mat1, saccades1):
+                        if et1[0, 0] != -1:
+                            n_smp = t1.shape[0]
+                            i = 0
+                            while i < (n_smp):
+                                s0 = [sac1[i]]
+                                t0 = [t1[i]]
+                                ts0 = [ts1[i]]
+                                et0 = [et1[i]]
+                                now = sac1[i]
+                                j = 1
+                                if (i+j) < n_smp:
+                                    while sac1[i+j] == now:
+                                        s0.append(sac1[i+j])
+                                        t0.append(t1[i+j])
+                                        ts0.append(ts1[i+j])
+                                        et0.append(et1[i+j])
+                                        j += 1
+                                        if (i+j) >= n_smp:
+                                            break
+                                sac_mat_new1.append(np.array(s0))
+                                t_mat_new1.append(np.array(t0))
+                                t_sys_mat_new1.append(ts0)
+                                et_mat_new1.append(np.array(et0))
+                                i += j
+                        else:
+                            sac_mat_new1.append(sac1)
+                            t_mat_new1.append(t1)
+                            t_sys_mat_new1.append(ts1)
+                            et_mat_new1.append(et1)
+                    sac_mat_new.append(sac_mat_new1)
+                    t_mat_new.append(t_mat_new1)
+                    t_sys_mat_new.append(t_sys_mat_new1)
+                    et_mat_new.append(et_mat_new1)
+
+                """We are turing each vector of fixations to a list of some information, like the number of values that it contains,
+                The start time, mean of the eye movements, and sys mean time."""
+                fix = []
+                k = 0
+                for (sac_mat_new1, t_mat_new1, t_sys_mat_new1, et_mat_new1) in zip(sac_mat_new, t_mat_new, t_sys_mat_new, et_mat_new):
+                    fix1 = []
+                    for (s1, t1, ts1, et1) in zip(sac_mat_new1, t_mat_new1, t_sys_mat_new1, et_mat_new1):
+                        sac_shp = s1.shape
+                        if s1[0] == False:
+                            if not s1[0]:
+                                fix1.append([k,
+                                             sac_shp[0],
+                                             t1[0],
+                                             round(t1[-1]-t1[0], 2),
+                                             round(et1[:, 0].mean(), 4),
+                                             round(et1[:, 1].mean(), 4),
+                                             ts1[0]])
+                        k += sac_shp[0]
+                    fix.append(fix1)
+
+                # Merging the fixations that are near together
+                fix_mrg_one = []
+                for fix1 in fix:
+                    fix_mrg1 = []
+                    n_fix = len(fix1)
+                    i = 0
+                    while i < n_fix:
+                        f_new = fix1[i]
+                        j = 1
+                        while (i+j) < n_fix:
+                            fj = fix1[i+j]
+                            fj_d = ((fj[4]-f_new[4])/(x_merge/n_monitors_data))**2+((fj[5]-f_new[5])/(y_merge))**2
+                            if fj_d < 1:
+                                f_new = [f_new[0],
+                                         f_new[1] + fj[1],
+                                         f_new[2],
+                                         round(f_new[3] + fj[3], 2),
+                                         round((f_new[4]*f_new[1]+fj[4]*fj[1])/(f_new[1]+fj[1]), 4),
+                                         round((f_new[5]*f_new[1]+fj[5]*fj[1])/(f_new[1]+fj[1]), 4),
+                                         f_new[-1]]
+                                if (i+j) == n_fix-1:
+                                    fix_mrg1.append(f_new)
+                                not_joined = False
+                            else:
+                                fix_mrg1.append(f_new)
+                                not_joined = True
+                                break
+                            j += 1
+                        i += j
+                    if not_joined:
+                        fix_mrg1.append(fix1[-1])
+                        
+                    fix_mrg_one.append(fix_mrg1)
+
+                # Removing the fixations that are short
+                fix_dcd = []
+                for fix_mrg1 in fix_mrg_one:
+                    fix_dcd1 = []
+                    for f in fix_mrg1:
+                        if f[3] >= t_discard:
+                            fix_dcd1.append(f)
+                    fix_dcd.append(fix_dcd1)
+
+                # Merging the fixations that are near together
+                fix_mrg_two = []
+                for fix1 in fix_dcd:
+                    fix_mrg1 = []
+                    n_fix = len(fix1)
+                    i = 0
+                    while i < n_fix:
+                        f_new = fix1[i]
+                        j = 1
+                        while (i+j) < n_fix:
+                            fj = fix1[i+j]
+                            fj_d = ((fj[4]-f_new[4])/(x_merge/n_monitors_data))**2+((fj[5]-f_new[5])/(y_merge))**2
+                            if fj_d < 1:
+                                f_new = [f_new[0],
+                                         f_new[1] + fj[1],
+                                         f_new[2],
+                                         round(f_new[3] + fj[3], 2),
+                                         round((f_new[4]*f_new[1]+fj[4]*fj[1])/(f_new[1]+fj[1]), 4),
+                                         round((f_new[5]*f_new[1]+fj[5]*fj[1])/(f_new[1]+fj[1]), 4),
+                                         f_new[-1]]
+                                if (i+j) == n_fix-1:
+                                    fix_mrg1.append(f_new)
+                                not_joined = False
+                            else:
+                                fix_mrg1.append(f_new)
+                                not_joined = True
+                                break
+                            j += 1
+                #             if (i+j) >= n_fix:
+                #                 break
+                        i += j
+                    if not_joined:
+                        fix_mrg1.append(fix1[-1])
+                        
+                    fix_mrg_two.append(fix_mrg1)
+
+                # Saving the fixations into the fixations.xlsx
+                wb = Workbook()
+                ws = wb.active
+                ws['A1'] = "FixationTime"
+                ws['A2'] = "sec"
+                ws['B1'] = "FixationSystemTime"
+                ws['C1'] = "FixationDuration"
+                ws['C2'] = "sec"
+                ws['D1'] = "FixationLocation"
+                ws['D2'] = "(p_x/scr_w,p_y/scr_h)"
+                i = 0
+                for f_seq in fix_mrg_two:
+                    for f in f_seq:
+                        ws[f'A{i+3}'] = f"{f[2]}"
+                        ws[f'B{i+3}'] = f[6][0]
+                        ws[f'C{i+3}'] = f"{f[3]}"
+                        ws[f'D{i+3}'] = f"({f[4]},{f[5]})"
+                        i += 1
+                
+                wb.save(smp_dir + "fixations.xlsx")
+            else:
+                print(f"Data does not exist in {smp_dir}")
+
+    @staticmethod
+    def get_models_information(show_model=False):
+        """Writing the NN models' information in the xlsx files.
+        
+        Parameters:
+            show_model: Whether or not show the model
+        
+        Returns:
+            None
+        """
+        
+        wb = Workbook()
+        ws = wb.active
+        ws['A1'] = "subject"
+        ws['B1'] = "model"
+        ws['C1'] = "trained model"
+        ws['D1'] = "weights"
+        ws['E1'] = "input 1 shape"
+        ws['F1'] = "input 2 shape"
+        ws['G1'] = "x2 chosen features"
+        ws['H1'] = "min-Max brightness ratio"
+        ws['I1'] = "r_train"
+        ws['J1'] = "epochs and patience"
+        ws['K1'] = "model-hrz train loss"
+        ws['L1'] = "model-hrz val loss"
+        ws['M1'] = "model-vrt train loss"
+        ws['N1'] = "model-vrt val loss"
+        ws['O1'] = "r_retrain"
+        ws['P1'] = "epochs and patience-retrain"
+        ws['Q1'] = "trainable layers"
+        ws['R1'] = "model-hrz-retrain train loss"
+        ws['S1'] = "model-hrz-retrain val loss"
+        ws['T1'] = "model-vrt-retrain train loss"
+        ws['U1'] = "model-vrt-retrain val loss"
+        ws['V1'] = "model-hrz-retrain test loss"
+        ws['W1'] = "model-vrt-retrain test loss"
+
+
+        subjects = os.listdir(ey.subjects_dir)
+        i = 2
+        for sbj in subjects:
+            sbj = int(sbj)
+            models_dir = ey.create_dir([ey.subjects_dir, f"{sbj}", ey.MDL])
+            files_name = os.listdir(models_dir)
+            if files_name:
+                for fn in files_name:
+                    if fn[-7:] == ".pickle":
+                        mdl_num = int(fn[3:-7])
+                        mdl_name = ey.MDL + f"{mdl_num}"
+                        if show_model:
+                            mdl = load_model(models_dir + mdl_name + "-hrz.h5")
+                            print(mdl.summary())
+                        info = ey.load(models_dir, [ey.MDL + f"{mdl_num}"])[0]
+
+                        ws[f'A{i}'] = str(sbj)
+                        ws[f'B{i}'] = str(mdl_num)
+                        ws[f'C{i}'] = str(info['trained_mdl_num'])
+                        ws[f'D{i}'] = str(info['n_weights'])
+                        ws[f'E{i}'] = str(info['input1_shape'])
+                        ws[f'F{i}'] = str(info['input2_shape'])
+                        ws[f'G{i}'] = str(info['x2_chosen_features'])
+                        ws[f'H{i}'] = str(info['min_max_brightness_ratio'])
+                        ws[f'I{i}'] = str(info['r_train'])
+                        ws[f'J{i}'] = str(info['n_epochs_patience'])
+                        ws[f'K{i}'] = str(info['hrz_train_loss'])
+                        ws[f'L{i}'] = str(info['hrz_val_loss'])
+                        ws[f'M{i}'] = str(info['vrt_train_loss'])
+                        ws[f'N{i}'] = str(info['vrt_val_loss'])
+                        ws[f'O{i}'] = str(info['r_retrain'])
+                        ws[f'P{i}'] = str(info['n_epochs_patience_retrain'])
+                        ws[f'Q{i}'] = str(info['trainable_layers'])
+                        ws[f'R{i}'] = str(info['hrz_retrain_train_loss'])
+                        ws[f'S{i}'] = str(info['hrz_retrain_val_loss'])
+                        ws[f'T{i}'] = str(info['vrt_retrain_train_loss'])
+                        ws[f'U{i}'] = str(info['vrt_retrain_val_loss'])
+                        ws[f'V{i}'] = str(info['hrz_retrain_test_loss'])
+                        ws[f'W{i}'] = str(info['vrt_retrain_test_loss'])
+
+                        i += 1
+
+        wb.save(ey.files_dir + "info_et_retrains.xlsx")

codes/jupyter_notebook/check_boi_model_on_et.ipynb

@@ -0,0 +1,178 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "fecbac60",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from tensorflow.keras.models import load_model\n",
+    "from joblib import load as j_load\n",
+    "import cv2\n",
+    "import pickle"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "c0f85582",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Uncomment this box if the file is in your local directory\n",
+    "path2root = \"../../\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 22,
+   "id": "bf26ac93",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "subjects_fol = \"subjects/\"\n",
+    "sbj_scalers_boi_name = \"scalers-boi.bin\"\n",
+    "sbj_model_boi_name = \"model-boi2.h5\"\n",
+    "sbj = 3\n",
+    "data_et_fol = \"data-et-clb/\"\n",
+    "chosen_inputs = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 23,
+   "id": "2787fe8b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "sbj_dir = path2root + subjects_fol + f\"{sbj}/\"\n",
+    "model_boi_dir = sbj_dir + sbj_model_boi_name\n",
+    "scalers_boi_dir = sbj_dir + sbj_scalers_boi_name\n",
+    "data_et_dir = sbj_dir + data_et_fol\n",
+    "    \n",
+    "with open(data_et_dir + \"x1.pickle\", \"rb\") as f:\n",
+    "    x1_load = pickle.load(f)\n",
+    "with open(data_et_dir + \"x2.pickle\", \"rb\") as f:\n",
+    "    x2_load = pickle.load(f)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 26,
+   "id": "eff56379",
+   "metadata": {},
+   "outputs": [
+    {
+     "ename": "FileNotFoundError",
+     "evalue": "[Errno 2] No such file or directory: '../../subjects/3/scalers-boi.bin'",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[1;31mFileNotFoundError\u001b[0m                         Traceback (most recent call last)",
+      "\u001b[1;32m~\\AppData\\Local\\Temp/ipykernel_5904/1123430948.py\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[0mx2_chs_inp\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mx2_load\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mchosen_inputs\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 2\u001b[1;33m \u001b[0mscalers_boi\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mj_load\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mscalers_boi_dir\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      3\u001b[0m \u001b[0mx1_scaler_boi\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mx2_scaler_boi\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mscalers_boi\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      4\u001b[0m \u001b[0mx1\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mx1_load\u001b[0m \u001b[1;33m/\u001b[0m \u001b[0mx1_scaler_boi\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      5\u001b[0m \u001b[0mx2\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mx2_scaler_boi\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtransform\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mx2_chs_inp\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
+      "\u001b[1;32mF:\\Education_Study\\University\\MSC\\Thesis\\EyeTracker\\venv\\lib\\site-packages\\joblib\\numpy_pickle.py\u001b[0m in \u001b[0;36mload\u001b[1;34m(filename, mmap_mode)\u001b[0m\n\u001b[0;32m    577\u001b[0m             \u001b[0mobj\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0m_unpickle\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mfobj\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    578\u001b[0m     \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 579\u001b[1;33m         \u001b[1;32mwith\u001b[0m \u001b[0mopen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mfilename\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'rb'\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0mf\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    580\u001b[0m             \u001b[1;32mwith\u001b[0m \u001b[0m_read_fileobject\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mf\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfilename\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmmap_mode\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0mfobj\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    581\u001b[0m                 \u001b[1;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mfobj\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mstr\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
+      "\u001b[1;31mFileNotFoundError\u001b[0m: [Errno 2] No such file or directory: '../../subjects/3/scalers-boi.bin'"
+     ]
+    }
+   ],
+   "source": [
+    "x2_chs_inp = x2_load[:, chosen_inputs]\n",
+    "scalers_boi = j_load(scalers_boi_dir)\n",
+    "x1_scaler_boi, x2_scaler_boi = scalers_boi\n",
+    "x1 = x1_load / x1_scaler_boi\n",
+    "x2 = x2_scaler_boi.transform(x2_chs_inp)\n",
+    "    \n",
+    "# model_boi = load_model(model_boi_dir)\n",
+    "    \n",
+    "# y_prd_boi = model_boi.predict([x1, x2]) #  .argmax(1)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "id": "259cacf4",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "16\n"
+     ]
+    }
+   ],
+   "source": [
+    "print((y_prd_boi.round()==1).sum())"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 25,
+   "id": "52ffe7c6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "for (i, img) in enumerate(x1):\n",
+    "    if True: # i % 10 == 0:\n",
+    "#         cv2.putText(img, str(round(y_prd_boi[i, 0])), (40, 80), cv2.FONT_HERSHEY_SIMPLEX, 1.5, (0, 0, 255), 1)\n",
+    "        cv2.imshow(\"Eyes\", img)\n",
+    "        q = cv2.waitKey(5)\n",
+    "        if q == ord('q'):\n",
+    "            break\n",
+    "cv2.destroyAllWindows()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "2304c430",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "smp_num = 20\n",
+    "print(round(y_prd_boi[smp_num, 0]))\n",
+    "cv2.imshow(\"Eyes\", x1[smp_num])\n",
+    "cv2.waitKey(0)\n",
+    "cv2.destroyAllWindows()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0254f097",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7fa3edcb",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "venv",
+   "language": "python",
+   "name": "venv"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

codes/jupyter_notebook/crt_emp_2mdl_et.ipynb

@@ -0,0 +1,317 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "executionInfo": {
+     "elapsed": 3227,
+     "status": "ok",
+     "timestamp": 1635187414445,
+     "user": {
+      "displayName": "mostafa lotfi",
+      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj4ilTay6rvtZq-saddDY_b1Sar82sct4P4Z2d_Wg=s64",
+      "userId": "11325640182532482321"
+     },
+     "user_tz": -210
+    },
+    "id": "gdpcJ0ywSIY8"
+   },
+   "outputs": [],
+   "source": [
+    "from tensorflow.keras.layers import (Input, Conv2D, Flatten, MaxPooling2D,\n",
+    "                                     Dense, Dropout, Concatenate)\n",
+    "from tensorflow.keras.models import Model\n",
+    "from tensorflow.keras.utils import plot_model\n",
+    "import numpy as np\n",
+    "import os\n",
+    "import pickle"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# # Uncomment this box if the file is in google colab\n",
+    "# path2root = \"/content/drive/MyDrive/Projects/EyeTracker/\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Uncomment this box if the file is in your local directory\n",
+    "path2root = \"../../\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "executionInfo": {
+     "elapsed": 23,
+     "status": "ok",
+     "timestamp": 1635187414448,
+     "user": {
+      "displayName": "mostafa lotfi",
+      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj4ilTay6rvtZq-saddDY_b1Sar82sct4P4Z2d_Wg=s64",
+      "userId": "11325640182532482321"
+     },
+     "user_tz": -210
+    },
+    "id": "YGlf78zqN8xb"
+   },
+   "outputs": [],
+   "source": [
+    "subjects_fol = \"subjects/\"\n",
+    "data_et_fol = \"data-et-clb/\"\n",
+    "models_fol = \"models/\"\n",
+    "models_et_fol = \"et/\"\n",
+    "raw_fol = \"raw/\"\n",
+    "chosen_inputs = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "executionInfo": {
+     "elapsed": 1441,
+     "status": "ok",
+     "timestamp": 1635187415872,
+     "user": {
+      "displayName": "mostafa lotfi",
+      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj4ilTay6rvtZq-saddDY_b1Sar82sct4P4Z2d_Wg=s64",
+      "userId": "11325640182532482321"
+     },
+     "user_tz": -210
+    },
+    "id": "wIrwyDFyQpof"
+   },
+   "outputs": [],
+   "source": [
+    "data_et_dir = path2root + subjects_fol + f\"{3}/\" + data_et_fol\n",
+    "\n",
+    "with open(data_et_dir + \"x1.pickle\", \"rb\") as f:\n",
+    "    x1 = pickle.load(f)\n",
+    "with open(data_et_dir + \"x2.pickle\", \"rb\") as f:\n",
+    "    x2 = pickle.load(f)\n",
+    "with open(data_et_dir + \"y.pickle\", \"rb\") as f:\n",
+    "    y = pickle.load(f)\n",
+    "\n",
+    "x2_chs_inp = x2[:, chosen_inputs]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "executionInfo": {
+     "elapsed": 944,
+     "status": "ok",
+     "timestamp": 1635187416811,
+     "user": {
+      "displayName": "mostafa lotfi",
+      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj4ilTay6rvtZq-saddDY_b1Sar82sct4P4Z2d_Wg=s64",
+      "userId": "11325640182532482321"
+     },
+     "user_tz": -210
+    },
+    "id": "4PX0-j2gRhWL",
+    "outputId": "7bc4570b-7576-48c6-9455-845aa581d904"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Model: \"model_4\"\n",
+      "__________________________________________________________________________________________________\n",
+      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
+      "==================================================================================================\n",
+      "input_10 (InputLayer)           [(None, 100, 100, 1) 0                                            \n",
+      "__________________________________________________________________________________________________\n",
+      "conv2d_15 (Conv2D)              (None, 50, 50, 16)   800         input_10[0][0]                   \n",
+      "__________________________________________________________________________________________________\n",
+      "conv2d_16 (Conv2D)              (None, 25, 25, 32)   12832       conv2d_15[0][0]                  \n",
+      "__________________________________________________________________________________________________\n",
+      "conv2d_17 (Conv2D)              (None, 12, 12, 64)   18496       conv2d_16[0][0]                  \n",
+      "__________________________________________________________________________________________________\n",
+      "flatten_4 (Flatten)             (None, 9216)         0           conv2d_17[0][0]                  \n",
+      "__________________________________________________________________________________________________\n",
+      "input_11 (InputLayer)           [(None, 10)]         0                                            \n",
+      "__________________________________________________________________________________________________\n",
+      "concatenate_4 (Concatenate)     (None, 9226)         0           flatten_4[0][0]                  \n",
+      "                                                                 input_11[0][0]                   \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_20 (Dense)                (None, 256)          2362112     concatenate_4[0][0]              \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_21 (Dense)                (None, 128)          32896       dense_20[0][0]                   \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_22 (Dense)                (None, 32)           4128        dense_21[0][0]                   \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_23 (Dense)                (None, 5)            165         dense_22[0][0]                   \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_24 (Dense)                (None, 1)            6           dense_23[0][0]                   \n",
+      "==================================================================================================\n",
+      "Total params: 2,431,435\n",
+      "Trainable params: 2,431,435\n",
+      "Non-trainable params: 0\n",
+      "__________________________________________________________________________________________________\n",
+      "None\n"
+     ]
+    }
+   ],
+   "source": [
+    "input1 = Input(x1.shape[1:])\n",
+    "layer = Conv2D(16, (7, 7), (2, 2), 'same', activation='relu')(input1)\n",
+    "# layer = MaxPooling2D((2, 2), (2, 2))(layer)\n",
+    "\n",
+    "# layer = Conv2D(32, (7, 7), (1, 1), 'same', activation='relu')(layer)\n",
+    "# layer = MaxPooling2D((2, 2), (2, 2))(layer)\n",
+    "\n",
+    "layer = Conv2D(32, (5, 5), (2, 2), 'same', activation='relu')(layer)\n",
+    "# layer = MaxPooling2D((2, 2), (2, 2))(layer)\n",
+    "\n",
+    "layer = Conv2D(64, (3, 3), (2, 2), activation='relu')(layer)\n",
+    "# layer = MaxPooling2D((2, 2), (2, 2))(layer)\n",
+    "\n",
+    "layer = Flatten()(layer)\n",
+    "\n",
+    "input2 = Input(x2_chs_inp.shape[1:])\n",
+    "layer = Concatenate()([layer, input2])\n",
+    "\n",
+    "layer = Dense(256, 'relu')(layer)\n",
+    "\n",
+    "layer = Dense(128, 'relu')(layer)\n",
+    "\n",
+    "layer = Dense(32, 'relu')(layer)\n",
+    "\n",
+    "layer = Dense(5, 'relu')(layer)\n",
+    "\n",
+    "out = Dense(1, 'linear')(layer)\n",
+    "\n",
+    "input_layers = [input1, input2]\n",
+    "\n",
+    "model = Model(inputs=input_layers, outputs=out)\n",
+    "\n",
+    "model.compile(optimizer='adam', loss='mse')\n",
+    "\n",
+    "print(model.summary())"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {
+    "executionInfo": {
+     "elapsed": 27,
+     "status": "ok",
+     "timestamp": 1635187416814,
+     "user": {
+      "displayName": "mostafa lotfi",
+      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj4ilTay6rvtZq-saddDY_b1Sar82sct4P4Z2d_Wg=s64",
+      "userId": "11325640182532482321"
+     },
+     "user_tz": -210
+    },
+    "id": "woulgNfGVvMO"
+   },
+   "outputs": [],
+   "source": [
+    "# plot_model(model, show_shapes=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "models_dir = path2root + models_fol\n",
+    "if not os.path.exists(models_dir):\n",
+    "    os.mkdir(models_dir)\n",
+    "\n",
+    "models_et_dir = models_dir + models_et_fol\n",
+    "if not os.path.exists(models_et_dir):\n",
+    "    os.mkdir(models_et_dir)\n",
+    "\n",
+    "raw_dir = models_et_dir + raw_fol\n",
+    "if not os.path.exists(raw_dir):\n",
+    "    os.mkdir(raw_dir)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "executionInfo": {
+     "elapsed": 3087,
+     "status": "ok",
+     "timestamp": 1635187419881,
+     "user": {
+      "displayName": "mostafa lotfi",
+      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj4ilTay6rvtZq-saddDY_b1Sar82sct4P4Z2d_Wg=s64",
+      "userId": "11325640182532482321"
+     },
+     "user_tz": -210
+    },
+    "id": "Oa3y4Mfyg7g7",
+    "outputId": "b083e78a-bb99-4410-b237-70205bdbc05e"
+   },
+   "outputs": [],
+   "source": [
+    "models_numbers = []\n",
+    "models_name = os.listdir(raw_dir)\n",
+    "if models_name:\n",
+    "    for model_name in models_name:\n",
+    "        model_num = int(model_name[5:6])\n",
+    "        models_numbers.append(model_num)\n",
+    "    max_num = max(models_numbers)\n",
+    "else:\n",
+    "    max_num = 0\n",
+    "\n",
+    "model.save(raw_dir + f\"model{max_num+1}-hrz.h5\")\n",
+    "model.save(raw_dir + f\"model{max_num+1}-vrt.h5\")"
+   ]
+  }
+ ],
+ "metadata": {
+  "colab": {
+   "authorship_tag": "ABX9TyNJ55D+JOVdA6fiv2ayAPT8",
+   "collapsed_sections": [],
+   "mount_file_id": "1mEGliX-rV2YkSx8ASgAVoGgB28BGXYz1",
+   "name": "crt_emp_2mdl_et.ipynb",
+   "provenance": []
+  },
+  "kernelspec": {
+   "display_name": "venv",
+   "language": "python",
+   "name": "venv"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.10"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 1
+}

codes/jupyter_notebook/crt_emp_mdl_io.ipynb

@@ -0,0 +1,334 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "executionInfo": {
+     "elapsed": 3115,
+     "status": "ok",
+     "timestamp": 1634906898989,
+     "user": {
+      "displayName": "mostafa lotfi",
+      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj4ilTay6rvtZq-saddDY_b1Sar82sct4P4Z2d_Wg=s64",
+      "userId": "11325640182532482321"
+     },
+     "user_tz": -210
+    },
+    "id": "gdpcJ0ywSIY8"
+   },
+   "outputs": [],
+   "source": [
+    "from tensorflow.keras.layers import (Input, Conv2D, Flatten, MaxPooling2D,\n",
+    "                                     Dense, Dropout, Concatenate)\n",
+    "from tensorflow.keras.models import Model\n",
+    "from tensorflow.keras.utils import plot_model\n",
+    "import numpy as np\n",
+    "import os\n",
+    "import pickle"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "executionInfo": {
+     "elapsed": 28,
+     "status": "ok",
+     "timestamp": 1634906898992,
+     "user": {
+      "displayName": "mostafa lotfi",
+      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj4ilTay6rvtZq-saddDY_b1Sar82sct4P4Z2d_Wg=s64",
+      "userId": "11325640182532482321"
+     },
+     "user_tz": -210
+    },
+    "id": "YGlf78zqN8xb"
+   },
+   "outputs": [],
+   "source": [
+    "# # Uncomment this box if the file is in google colab\n",
+    "# path2root = \"/content/drive/MyDrive/Projects/EyeTracker/\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Uncomment this box if the file is in your local directory\n",
+    "path2root = \"../../\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "subjects_fol = \"subjects/\"\n",
+    "data_boi_fol = \"data-boi/\"\n",
+    "models_fol = \"models/\"\n",
+    "models_boi_fol = \"boi/\"\n",
+    "raw_fol = \"raw/\"\n",
+    "# chosen_inputs = [0, 1, 2, 6, 7, 8, 9]\n",
+    "chosen_inputs = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "executionInfo": {
+     "elapsed": 1534,
+     "status": "ok",
+     "timestamp": 1634906900506,
+     "user": {
+      "displayName": "mostafa lotfi",
+      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj4ilTay6rvtZq-saddDY_b1Sar82sct4P4Z2d_Wg=s64",
+      "userId": "11325640182532482321"
+     },
+     "user_tz": -210
+    },
+    "id": "BBr2Jp0jRp-N"
+   },
+   "outputs": [],
+   "source": [
+    "data_boi_dir = path2root + subjects_fol + f\"{3}/\" + data_boi_fol\n",
+    "\n",
+    "with open(data_boi_dir + \"x1.pickle\", \"rb\") as f:\n",
+    "    x1 = pickle.load(f)\n",
+    "with open(data_boi_dir + \"x2.pickle\", \"rb\") as f:\n",
+    "    x2 = pickle.load(f)\n",
+    "with open(data_boi_dir + \"y.pickle\", \"rb\") as f:\n",
+    "    y = pickle.load(f)\n",
+    "\n",
+    "x2_chs_inp = x2[:, chosen_inputs]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "executionInfo": {
+     "elapsed": 1366,
+     "status": "ok",
+     "timestamp": 1634906901865,
+     "user": {
+      "displayName": "mostafa lotfi",
+      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj4ilTay6rvtZq-saddDY_b1Sar82sct4P4Z2d_Wg=s64",
+      "userId": "11325640182532482321"
+     },
+     "user_tz": -210
+    },
+    "id": "4PX0-j2gRhWL",
+    "outputId": "fa4469a0-3fe3-4b6b-93b7-e3b150335149"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Model: \"model_2\"\n",
+      "__________________________________________________________________________________________________\n",
+      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
+      "==================================================================================================\n",
+      "input_5 (InputLayer)            [(None, 100, 100, 1) 0                                            \n",
+      "__________________________________________________________________________________________________\n",
+      "conv2d_8 (Conv2D)               (None, 100, 100, 16) 1952        input_5[0][0]                    \n",
+      "__________________________________________________________________________________________________\n",
+      "max_pooling2d_8 (MaxPooling2D)  (None, 50, 50, 16)   0           conv2d_8[0][0]                   \n",
+      "__________________________________________________________________________________________________\n",
+      "conv2d_9 (Conv2D)               (None, 50, 50, 32)   25120       max_pooling2d_8[0][0]            \n",
+      "__________________________________________________________________________________________________\n",
+      "max_pooling2d_9 (MaxPooling2D)  (None, 25, 25, 32)   0           conv2d_9[0][0]                   \n",
+      "__________________________________________________________________________________________________\n",
+      "conv2d_10 (Conv2D)              (None, 25, 25, 64)   51264       max_pooling2d_9[0][0]            \n",
+      "__________________________________________________________________________________________________\n",
+      "max_pooling2d_10 (MaxPooling2D) (None, 12, 12, 64)   0           conv2d_10[0][0]                  \n",
+      "__________________________________________________________________________________________________\n",
+      "conv2d_11 (Conv2D)              (None, 12, 12, 128)  73856       max_pooling2d_10[0][0]           \n",
+      "__________________________________________________________________________________________________\n",
+      "max_pooling2d_11 (MaxPooling2D) (None, 6, 6, 128)    0           conv2d_11[0][0]                  \n",
+      "__________________________________________________________________________________________________\n",
+      "flatten_2 (Flatten)             (None, 4608)         0           max_pooling2d_11[0][0]           \n",
+      "__________________________________________________________________________________________________\n",
+      "input_6 (InputLayer)            [(None, 10)]         0                                            \n",
+      "__________________________________________________________________________________________________\n",
+      "concatenate_2 (Concatenate)     (None, 4618)         0           flatten_2[0][0]                  \n",
+      "                                                                 input_6[0][0]                    \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_11 (Dense)                (None, 256)          1182464     concatenate_2[0][0]              \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_12 (Dense)                (None, 128)          32896       dense_11[0][0]                   \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_13 (Dense)                (None, 32)           4128        dense_12[0][0]                   \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_14 (Dense)                (None, 8)            264         dense_13[0][0]                   \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_15 (Dense)                (None, 1)            9           dense_14[0][0]                   \n",
+      "==================================================================================================\n",
+      "Total params: 1,371,953\n",
+      "Trainable params: 1,371,953\n",
+      "Non-trainable params: 0\n",
+      "__________________________________________________________________________________________________\n",
+      "None\n"
+     ]
+    }
+   ],
+   "source": [
+    "input1 = Input(x1.shape[1:])\n",
+    "layer = Conv2D(16, (11, 11), (1, 1), \"same\", activation=\"relu\")(input1)\n",
+    "layer = MaxPooling2D((2, 2), (2, 2))(layer)\n",
+    "\n",
+    "layer = Conv2D(32, (7, 7), (1, 1), \"same\", activation=\"relu\")(layer)\n",
+    "layer = MaxPooling2D((2, 2), (2, 2))(layer)\n",
+    "\n",
+    "layer = Conv2D(64, (5, 5), (1, 1), \"same\", activation=\"relu\")(layer)\n",
+    "layer = MaxPooling2D((2, 2), (2, 2))(layer)\n",
+    "\n",
+    "layer = Conv2D(128, (3, 3), (1, 1), \"same\", activation=\"relu\")(layer)\n",
+    "layer = MaxPooling2D((2, 2), (2, 2))(layer)\n",
+    "\n",
+    "layer = Flatten()(layer)\n",
+    "\n",
+    "layer = Dense(256, \"relu\")(layer)\n",
+    "\n",
+    "input2 = Input(x2_chs_inp.shape[1:])\n",
+    "layer = Concatenate()([layer, input2])\n",
+    "\n",
+    "layer = Dense(128, \"relu\")(layer)\n",
+    "\n",
+    "# layer = Dense(64, \"relu\")(layer)\n",
+    "\n",
+    "layer = Dense(32, \"relu\")(layer)\n",
+    "\n",
+    "layer = Dense(8, \"relu\")(layer)\n",
+    "\n",
+    "output_layer = Dense(1, \"sigmoid\")(layer)\n",
+    "\n",
+    "input_layers = [input1, input2]\n",
+    "\n",
+    "model = Model(inputs=input_layers, outputs=output_layer)\n",
+    "\n",
+    "model.compile(optimizer=\"adam\", loss=\"binary_crossentropy\")\n",
+    "\n",
+    "print(model.summary())"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/",
+     "height": 1000
+    },
+    "executionInfo": {
+     "elapsed": 32,
+     "status": "ok",
+     "timestamp": 1634906901869,
+     "user": {
+      "displayName": "mostafa lotfi",
+      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj4ilTay6rvtZq-saddDY_b1Sar82sct4P4Z2d_Wg=s64",
+      "userId": "11325640182532482321"
+     },
+     "user_tz": -210
+    },
+    "id": "woulgNfGVvMO",
+    "outputId": "1357e75d-a130-47b9-8b8a-a283a28ef9ee"
+   },
+   "outputs": [],
+   "source": [
+    "# plot_model(model, show_shapes=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "executionInfo": {
+     "elapsed": 1600,
+     "status": "ok",
+     "timestamp": 1634906903452,
+     "user": {
+      "displayName": "mostafa lotfi",
+      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj4ilTay6rvtZq-saddDY_b1Sar82sct4P4Z2d_Wg=s64",
+      "userId": "11325640182532482321"
+     },
+     "user_tz": -210
+    },
+    "id": "Oa3y4Mfyg7g7",
+    "outputId": "cf277369-f69c-4b36-9140-c5c0480876e0"
+   },
+   "outputs": [],
+   "source": [
+    "models_dir = path2root + models_fol\n",
+    "if not os.path.exists(models_dir):\n",
+    "    os.mkdir(models_dir)\n",
+    "\n",
+    "models_boi_dir = models_dir + models_boi_fol\n",
+    "if not os.path.exists(models_boi_dir):\n",
+    "    os.mkdir(models_boi_dir)\n",
+    "\n",
+    "raw_dir = models_boi_dir + raw_fol\n",
+    "if not os.path.exists(raw_dir):\n",
+    "    os.mkdir(raw_dir)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "models_numbers = []\n",
+    "models_name = os.listdir(raw_dir)\n",
+    "if models_name:\n",
+    "    for model_name in models_name:\n",
+    "        model_num = int(model_name[5:-3])\n",
+    "        models_numbers.append(model_num)\n",
+    "    max_num = max(models_numbers)\n",
+    "else:\n",
+    "    max_num = 0\n",
+    "\n",
+    "model.save(raw_dir + f\"model{max_num+1}.h5\")"
+   ]
+  }
+ ],
+ "metadata": {
+  "colab": {
+   "authorship_tag": "ABX9TyO+imeiHVHbkQtIcxwp1CO/",
+   "collapsed_sections": [],
+   "mount_file_id": "1uCQ0LdEXc_1zKNjksNB7MFXCz9JZIiOY",
+   "name": "crt_mdl_in_blink_out.ipynb",
+   "provenance": []
+  },
+  "kernelspec": {
+   "display_name": "venv",
+   "language": "python",
+   "name": "venv"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 1
+}

codes/jupyter_notebook/detect_blink.ipynb

@@ -0,0 +1,333 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "da79030b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import mediapipe as mp\n",
+    "import cv2\n",
+    "import numpy as np\n",
+    "import time"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "cd77dedc",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "6\n"
+     ]
+    }
+   ],
+   "source": [
+    "th1 = 10\n",
+    "th2 = 7\n",
+    "blinking_time = 0.4\n",
+    "sampling_period = 1/15\n",
+    "n_smp_blink = round(blinking_time/sampling_period)\n",
+    "print(n_smp_blink)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "0ee09c2b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "cap = cv2.VideoCapture(2)\n",
+    "face_mesh = mp.solutions.face_mesh.FaceMesh(\n",
+    "    static_image_mode=False,\n",
+    "    min_tracking_confidence=0.5,\n",
+    "    min_detection_confidence=0.5\n",
+    ")\n",
+    "\n",
+    "fw = cap.get(cv2.CAP_PROP_FRAME_WIDTH)\n",
+    "fh = cap.get(cv2.CAP_PROP_FRAME_HEIGHT)\n",
+    "fs = fw, fh\n",
+    "t_vec = []\n",
+    "eye_ratio = []\n",
+    "t0 = time.perf_counter()\n",
+    "i = 0\n",
+    "bv = 0\n",
+    "while True:\n",
+    "    success, frame = cap.read()\n",
+    "    frame = cv2.flip(frame,1)\n",
+    "    frame_rgb = cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)\n",
+    "    if success:\n",
+    "        results = face_mesh.process(frame_rgb)\n",
+    "        mfl = results.multi_face_landmarks\n",
+    "        if mfl:\n",
+    "            all_landmarks = np.array([(lm.x, lm.y) for lm in mfl[0].landmark])\n",
+    "            wl = np.sqrt(((all_landmarks[33]-all_landmarks[133])**2).sum())\n",
+    "            hl1 = np.sqrt(((all_landmarks[159]-all_landmarks[145])**2).sum())\n",
+    "            hl2 = np.sqrt(((all_landmarks[158]-all_landmarks[153])**2).sum())\n",
+    "            hl = (hl1 + hl2) / 2\n",
+    "            \n",
+    "            wr = np.sqrt(((all_landmarks[362]-all_landmarks[263])**2).sum())\n",
+    "            hr1 = np.sqrt(((all_landmarks[385]-all_landmarks[380])**2).sum())\n",
+    "            hr2 = np.sqrt(((all_landmarks[386]-all_landmarks[374])**2).sum())\n",
+    "            hr = (hr1 + hr2) / 2\n",
+    "            \n",
+    "            t_vec.append(round(time.perf_counter() - t0, 3))\n",
+    "            eye_ratio.append((wl / hl + wr / hr) / 2)\n",
+    "            \n",
+    "            if i>=1:\n",
+    "                bv = (eye_ratio[-1]-eye_ratio[-2]) / (t_vec[-1]-t_vec[-2])\n",
+    "#             print((all_landmarks[33]-all_landmarks[133])**2)\n",
+    "#             vec.append(np.abs(all_landmarks-all_landmarks[0]).sum().sum()/all_landmarks.shape[0])\n",
+    "            all_landmarks_pix = np.array(all_landmarks * fs, np.uint32)\n",
+    "            \n",
+    "#             for pix in all_landmarks_pix:\n",
+    "#                 cv2.circle(frame, pix, 2, (0, 0, 255), cv2.FILLED)\n",
+    "#             print(all_landmarks[0].shape)\n",
+    "#             break\n",
+    "#             cv2.circle(frame, int(all_landmarks[0, 0]), int(all_landmarks[0, 1]), 5, (0, 0, 255), cv2.FILLED)\n",
+    "#             print(all_landmarks[0])\n",
+    "            if bv >= th1 or bv <= -th1:\n",
+    "                cv2.imshow(\"Webcam\", frame[:2])\n",
+    "            else:\n",
+    "                cv2.imshow(\"Webcam\", frame)\n",
+    "            q = cv2.waitKey(1)\n",
+    "            if q == ord('q'):\n",
+    "                break\n",
+    "    i += 1\n",
+    "\n",
+    "t_vec = np.array(t_vec)\n",
+    "eye_ratio = np.array(eye_ratio)\n",
+    "cv2.destroyAllWindows()\n",
+    "cap.release()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "7c94eeb0",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "30.706434901829933\n"
+     ]
+    }
+   ],
+   "source": [
+    "dt = 1 / (t_vec[1:] - t_vec[:-1])\n",
+    "print(dt.mean())\n",
+    "# print(t_vec)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "221c52c9",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "eye_ratio_v = eye_ratio.copy()\n",
+    "eye_ratio_v[1:] = (eye_ratio[1:] - eye_ratio[:-1]) / (t_vec[1:] - t_vec[:-1])\n",
+    "eye_ratio_v[0] = eye_ratio_v[1]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "d9cce07e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import matplotlib.pyplot as plt"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "dfb997ea",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[<matplotlib.lines.Line2D at 0x1d80031ee20>]"
+      ]
+     },
+     "execution_count": 8,
+     "metadata": {},
+     "output_type": "execute_result"
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAABUq0lEQVR4nO2deZgcVbn/v29v07NnmclCEjIhCQkhkECGfQ+yC+gF9V4RBfVyuRcVrrjgBor6U0TwLiqICiiXRTYFFBSUsENCErKQlex7MplkMvtML+f3R9WpOlVd1XWquqZ7pud8nidPd7pPd5/pPvXWW9/zLsQYg0KhUCiGPpFST0ChUCgU4aAMukKhUJQJyqArFApFmaAMukKhUJQJyqArFApFmRAr1Qc3NDSwpqamUn28QqFQDEmWLFmynzHW6PRcyQx6U1MTFi9eXKqPVygUiiEJEW11e05JLgqFQlEmKIOuUCgUZYIy6AqFQlEmKIOuUCgUZYIy6AqFQlEmKIOuUCgUZYIy6AqFQlEmKIOuUAxhtrV247X1LaWehmKQULLEIoVCUThn/XQBGAO2/PiSUk9FMQhQHrpCMYRR/WkUIsqgKxQKRZmgDLpCoVCUCcqgKxRlQDartBeFMugKRVmQymZLPQXFIEAZdIWiDEhllIeukDDoRJQkokVEtJyIVhHR9/KMvYKIGBE1hztNhUKRj3RGeegKuTj0PgDzGWOdRBQH8AYRvcAYe0ccRES1AG4EsHAA5qlQKPKgPHQFIOGhM41O/b9x/Z/T6vk+gDsA9IY3PYVCIUNaaegKSGroRBQlomUA9gF4iTG20Pb88QAmMcb+4vE+1xHRYiJa3NKi0pUVikIh0m5TaeWhF0JvKlPqKYSClEFnjGUYY3MBTARwIhHN5s8RUQTA3QBulnif+xhjzYyx5sZGxx6nCoXCB7GIZtFVlEtw1u3pwMzv/BXPr9xd6qkUjK8oF8ZYG4AFAC4UHq4FMBvAK0S0BcDJAJ5VG6MKxcAT1Q16WmnogVm0uRUA8MaG/SWeSeHIRLk0EtEI/X4lgPMArOXPM8YOMcYaGGNNjLEmAO8AuIwxtnhgpqxQKDixiHYIpySiXNbv7cCCdfsGekpDjkM9KQDAiMp4iWdSODJRLuMB/I6IotBOAI8zxv5MRLcDWMwYe3ZAZ6hQKFzhHrqMQT//Z68BUJUZ7bR16wa9ahgYdMbYCgDHOTx+q8v4swuflkKhkCEe1SUXlfofmDbdQ6+uGPrVxFWmqEIxhPHjoSuc4R56pgxOisqgKxRDGK6hq03R4Bzq6QdQHt+hMugh8sCbm/HlPywr9TQUwwjloRcO3xQNKzlrb3svLv3fN7DnUPFzLJVBD5HvPbcaT7+3s9TTUAwjYlFu0Ie+d1kquNQS1nf4+LvbsXLnITz0zpZQ3s8PyqCHBFO9wBQlgCcWqdT/4ET0dNuwNPQR1QkAwEFdmy8myqCHREtHX6jv97OX1uP6h5aE+p6K8iMaQENXzTCs8PIJYVWsHFWlG/Su/lDezw9DP05nkNATci2I//7HBwA0r4HrpAqFHe6h9/swRhnGEIFaUxx+cR009PPtja2YdVgd6vXEpOqKKADgQAkMuvLQQ2KgNqU27Ov0HqQYtgRJ/S+HaI4wyegWPYhB7+pL419+/Q7+7SEzMZ6/S5uSXIYufWnToIehpzfUVAAAtrR2FfxeivIliIau9HYr/HAN4pTx43759kPC+2lveLBbeehDln7BoIchUfIMwHJIdlAMHGbYovw68bumdrb1GKF95UhWN8BBjjV+3IsnA36C6OhNFz45nyiDHhKiQQ/DA+I77yqlW5EPbtAzPtac3/C80378Ms696xVfrxlKcIMeJGyxL63tnYnHqanJF/9KSBn0kBA3pcLwqvXgBV8HqmL4YYbcyb8myPrc31l8+aBY8EMsSJSL6Mhx+LdbitwAZdBDQvxhw/gho6TqXCu8iQTw0JWGboUVILn0ORn0EuakKIMeEqJBD8dDVxq6whse0erHuVROghV+iKUCGfTccOVSHrLKoIeEKLmE4QFFlYaukIBHk2d8eIVqTVnhGnoQyaUv5fQa5aEPefpC9tCjykNXSMBXhx/JRa0pK1ljEzOAh+5wEihlFRBl0EPCEuUSwiWtinJRyMCNhy/JRWnoFljIHnopj1iZnqJJIlpERMuJaBURfc9hzJeJaDURrSCifxDR5IGZ7uDFGrYYpoeuDj6FO0E8dKWhW8kWkCkqaujcBgx2D70PwHzG2BwAcwFcSEQn28a8B6CZMXYsgCcB/CTUWQ4BrGGLYcSha7fKQ1fkw4zQkH9N0DXlFKJXDnAJKsiJTpRau/q0RCI2mDV0psELisT1f8w2ZgFjrFv/7zsAJoY6yyFA2B66EeWivCmFBF5OhBhKF1RD7+4vfuZjMSgkEajP4bgXv95i71dIaehEFCWiZQD2AXiJMbYwz/DPAXjB5X2uI6LFRLS4paXF92QHM2Fr6CrKRSGDoaF7XOeL6yhomdiwK4oOFgqRXJzClcWTZ7E7SUkZdMZYhjE2F5rnfSIRzXYaR0SfAtAM4E6X97mPMdbMGGtubGwMOOXisWrXISzb3iY11hq2WLgRNrVRZdAV7vDLey+7IToZQddnua5FI8qlgNR/wPmk6pR4NJD4inJhjLUBWADgQvtzRPQhAN8CcBljLNxuDyUglcnikv95Ax/5xZtS461n6sJ/RFaA16AYPphRLvnXnCgnBDXM5dqUy6zlUliUS9bw0M3ni73vIBPl0khEI/T7lQDOA7DWNuY4AL+CZsz3DcA8i866PR2+xls89BAklyDRC4rhh2zYorgmg8oA2TK16OZ3WNimKHe+xE3RYksuMh2LxgP4HRFFoZ0AHmeM/ZmIbgewmDH2LDSJpQbAE6Rpv9sYY5cN1KSLgd8fIhPCJa1IIckOiuGDKbl4eeiFb4qWr+QSTtgi/37En6LYHrqnQWeMrQBwnMPjtwr3PxTyvEqOX29EHB+KES6gYJBi+MCXh9dFobiOgq7Pcl2KhUguGYcTpfg1+WkNGAYqU9QFv7+DNVQpBA1dv1UeuiIvkhp6NoSwxXKVXIyTYoDvhVmO+9wol0GnoQ9X/P64om4WiobOF5mKQ1fkwZRc8q+TbAihdOVo0K0hhgEMunDc8+9HeeiDEL81jcXh4YQtqigXhTeym6JOnqRfynF/3ipF+f8DHY/7wRzlMlzxU44UsJ4AQjHokpfSiuGNbDRUGA5HOXroFqk0gIfulBUahrwVFGXQXRB/CBlvPWwNXUW5KGQwarl4LBOrJKgkF45Figp03ObfFC328asMugt+z7IMQCKqfZ3haOgqykXhjayHng3ooYdRA2Yww/+8aIQCHbfOm6LmY0FPnkFRBt0F8XeQ2djIMoZ4NPz6K8pDV+RDNikmqGEWjVM5LkXuuMWjhHSWFbR3ZnroIYcw+0AZdBcsl2JpiR+FAfGY7qGHqqGX4VGkCA3TQ/cw6MJ9P+tTPA5K2fx4oOB/H7+69nu8Wa7kmZOHrgz6oCAr/LCyHnosoi+KEC6zVJSLQgpJac6yae/DyIgjy9G54H9SIhYF4P94E0ebtVwKi5wpBGXQXRCjXGQNeiJEyUVFuShkMDx0r03RgJv2ogdahvbcML5Bj12n6KFSngSVQXdB/CFSErGkLGTJxexEXoZHkSI0ZE/8wTdFxfvltxZND50HNPhzoJz2JpTkMgjxm1mXZUA8oA7nhKw2qhjeyNZDD7pRZ/Hsy9Kg6x56QGfMyRsPOyfFD8qguyA6PHJF6plh0EM5K6s4dIUE0h668LQ/Db28JRczyiXYscucNkWF55WGPkjIBPDQoxGAKJwfUXnoChmkwxYtCTDy69MStliGa5F/FdxD91vnxmlT1CJvKcllcCAuXpmiPYwxRIgQi1BIm6IqykXhjXTYovB0KmDYYjlnigYPW0RO/kkpk7GUQXfBEuUiIblkGUAAYpFIqBp6OXpFivAwU//lDbqfmiUWD9TnUuxLZ3D9Q0vwy1c2+HthEcnV0P1vikYjmkF3OlaDlRMIjjLoLlg9dIkoFwDEPfQQy+cWW4NTDE28lknQBixMeF+/jsriLQfx11V7cNeL6329rpjwr6XCkFz8b4ry/BPTQzefL3b5a5meokkiWkREy4loFRF9z2FMBRH9gYg2ENFCImoakNkWEXHtysShM8ZABESjFIoR5gegctAV+TDbp3lUWxTu+1mfovbuN2yRHzeDeR/I7qH7nisDYrrkYm6KCs7gIJRc+gDMZ4zNATAXwIVEdLJtzOcAHGSMTQPwMwB3hDrLEiD+sDKSC2MIWUPPnYdCYUd2nQT10C1VRP22ZRwCa5dPkUe5+N0UFTPEs04e+mCTXJhGp/7fuP7P/ktdDuB3+v0nAZxLerfooYr/OHRmaughXmYpg67IB18dXkY6sIZeQKboUNjQt2+KBskUtW+KDvooFyKKEtEyAPsAvMQYW2gbMgHAdgBgjKUBHAIw2uF9riOixUS0uKWlpaCJDzR+PfSsHuUSDTnKpRwjCxTevLa+BV/+wzLPcXydeGczix66n9T/3M+Sxa8zsnl/F1bsaPP1mkJh9k1R3xp67qYol1zCulr3g5RBZ4xlGGNzAUwEcCIRzQ7yYYyx+xhjzYyx5sbGxiBvUTQscegy9dCZFoMeC0lDV3How5tP378IT7+3E9396bzj+Orw2swLnPrv0MBBFr/G7JyfvoLLfv6mr9cUil1y8R/lIr7WKrnEojS4wxYZY20AFgC40PbUTgCTAICIYgDqAbSGML+S4bdIPTfo4Xno2q3y0IcnDTUJAMCutt78A/Xl0Z/J5vWgg/YULaQe+lAoLMePr4qAHnqWaZ64+F6ceDQSuCF3UGSiXBqJaIR+vxLAeQDW2oY9C+Az+v0rAbzMhnglH0txLqmwRU1yiYekofPFoTz04cn4+koAwK62nrzjrNEr7mvFsinqS0N3fg8ZxMNmsJoDfnzZvWx5TMnFXsslEQ0nJ8UPMh76eAALiGgFgHehaeh/JqLbiegyfcxvAYwmog0AvgzgloGZbvGwGnTvHyUbtoeu3w6FjSVF+IyrTwIAdnoZdMnNez7Mb2kKS6aoz7Uoeuh+47uLBf/zgldbhGHQ7ZuisSgV/e+OeQ1gjK0AcJzD47cK93sBfCzcqZUWvx6NkfoflobOJRdl0IclIyrjAIC27lTeceLqSKUZkHAZJxSh8qehmxQS5dKbzhhG04tM1vR6Bxp7HLrfuHEGM1zZHrYYj0YGX9jicCWrp/RGSL44F6CdrcO5zNIll0F6qaoYWGJROY9RXB75EuD4ML8ygOhQ+F2L4uf0peQNW08q4+tzCsG+KerXgcrqCYUR4cqcbyT7PXmGgTLoLmSyQJQIsWhEqh4DP1PHI5FQU/+HwL6SYgDgWRxeWcqWrMQ8Y8VmyEFlgELCFvvS8kbaK7InTMTvRfy/LIxpNZyiRMZrTQ89nDIgfvCUXIYrWcYQiWiJQrKSC9fQQ21woTz0YQk3np4GXVgeMhp6zKcMUEj5XPE46PXjofcXz0Pn37N9Y1P69QBgq+HE3yEWUR76oCGTZbqHTnJRLgyGhh5GhTUV5TK84Usolc7/+zNmZjnKeOgJnzKAtau99MsAWDV0fx56CSQXPX3fr/+k7Z1pNZz4iZI7d2Htp/lBGXQXMlmGSIT0WFKZKBf9hw3LQy/AM1IMfbghlXEm+IZefx7jb3iNPmUAcWRhkssg1dCzNg89oOQSj0aMDVVDhgltP00eZdBd4FEr8Yich679biQt0ch8PkeFLg4/+E/utfYYY3LddiyRFwE99IIkF28jzf+OYkouYoih9n+/kgsDcTuhn7TMxwZhYtFwJaNHucSiEclMUc1Dj4WsoQMqW3Q4Iq2hw5/kokVeBNTQC5JcvD8zqRv0Ykou/Hs2Kib6llyACAHxmGm8Da89VvxNUWXQXchk9aiVKPmq5RINSUMXLbrS0YcfpuQioaFzyUViUzThs76ItdqiXw/dnE+fhIdeEY8CKK7kwiWWfF2H8qFVWSWr5ALNFgzK1P/hSjbLEI3oP4pMPXSYPUXD9tBVpMvww5BcPNYeA5PqtiN66H7CFi1XigUU55Lx0PmVhkx107Aw49CDhy1CvzLnv5UWm64Z+X7loQ8OMkyLcpFNDuCp/2Fp6Frh/GBeg2LokzE89PzGLSt46PmMP19BhWjofpdh1qdB50bVT0RMofC/jydyBQlbJGi/gfFb6ZJLQnnogwd+lpUPW2RGT9GwolyCxsYqhj5+4tBlNkUNrdhnKF0hxblER0jGyYmXwEM3SiLox5rvi2HGpdmIkCnKJRc52xEmyqC7kNXrScjuVBuhSqHVQzc9dCW5DD+MOHTPtcdMqUJKQ/d3BZktSEM3x8scQ9yg+wlxLBT+PQcNW+Sp/7EIGSciJurqRfxbAGXQXcnoHrLsTjWDdqZO+NQoXd+PiRs1Bb+dYogRZFM0v4au3XJPUjamvFAPXSqkUiceK4WGbpVc/Ictat54wh7loke+yDSYDxNl0F3IZnkYopyHzs/UYV1mMQTvoqIY+nAD7GXcGCBsiubT0HVpwWd3e2tjDKmXmOMz5oatzD4UL7BYik3RWEDJRfTGRcmFO3fF/FsAZdBd4SU8ZaMCzNT/kDZClIc+rDF6hYaUWGSP5pBNVhOLf/nOFGUMST0UUbbrF1DcTdEwarnYJRfeMF7z2lWUy6CAN32OS2ripoeu/YiFdmgRo1yUhj784Jf+/hKL3NcJX48VEjHr1nnkzkmWTJYhGeefJ+MU6X9zScIWg0kuWnQb5SQWQW2KDi64QY/58NAJhIRPD8j1/RA8lEox9OFLzttDl4ty4VTENI9ZdrNOdEz82qZ0liEeieiVCOVKUANF3hQ1PHT9/wFSRXmIonjM8/ou6SwratixTE/RSUS0gIhWE9EqIrrRYUw9ET1HRMv1MdcOzHSLhym5yIctRsg0woWemZkYh6489GGHIbl4VltkUnHo9mbIslJAYR56FpEI7+Ilk8tRCg9d+0wiQjRC/lP/IYQoilEueigjgHAyxyWR8dDTAG5mjM0CcDKAG4holm3MDQBWM8bmADgbwF1E5NIMa2iQYVoXEtmGFTyxyPgRPQ5ELxhUHPpwRrbaIoO2cU8enbW4LZbZQM39BP4e/iWXmM/QX6A0HnqEtO5kQaot8iv5fqEeeoTkpLCw8TTojLHdjLGl+v0OAGsATLAPA1BLRASgBsABaCeCIQv3uOMxSQ9dT/3nkkuhZ2WVWDS84cvHU+sWHIl8OjVfQjJ1XyxvL0a5BDDoWuivbKSYdluKOPQIaV66fw1dlFzsqf+6LSji3+NLQyeiJmgNoxfanvo5gKMA7AKwEsCNjLGcv4KIriOixUS0uKWlJdiMi4Rx5pUOW7R56CFshvCSnsqgDz/8eOja3k3+dWpuikal3tech/N9GdK6hy5288kHn2PQKJe3N7bi4v9+XapUL0f00KNEAcIWnSQXXm0xPFsgi7RBJ6IaAE8BuIkx1m57+gIAywAcBmAugJ8TUZ39PRhj9zHGmhljzY2NjYEnXQx4wwrZHoxM3+2OhSC52Et6qiiX4Qf/yb0Ti+TyH5jNQ5ddn5ZqiwHqoftpEsM/KqiG/q0/rsTq3e3Y0tol/Rr+mRG9IXygFnSwBk8YyUYlyHyVMuhEFIdmzB9mjD3tMORaAE8zjQ0ANgOYGd40i49YMU0mbJHpl178MquQDLGcCnDKQx92iC0I8xkZPULOs1QrjycvdthizNgUlZMtgeAGkG+8dvXJq72mh64Zdf/VFrnjpzWTZ4zpJwnyFX0UFjJRLgTgtwDWMMbudhm2DcC5+vixAGYA2BTWJEtBVr9siknGlWsbIWSclQvJ7iw02UEx9BENi5fnbWjoebxuu4Yua2TExKIgkkvUl+Si3Qb10Plxsr+zX/o1/G/SNkXJt/PEv/9EVJNrtDmYRh4o7qZoTGLMaQCuBrCSiJbpj30TwOEAwBi7F8D3ATxIRCuh2cGvM8b2hz/d4sEYQzQSscSVc4/ZCaNITxiSi36rJJfhi2gD+jNZI+PSDm93lvDYeDSjXPxp6NbUf/8eejwe0Tds5ZLzgOBXt3x+B7r8GHQetoiAYYta6n9MMN7ZbPj7abJ4GnTG2BvQjHS+MbsAnB/WpAYDjAGRiGmg0xkGl2MKAK/9QqFILvwgUqn/wxfxijBvnXO+AeehoXPD5dtDFwyc32Q5M9taso2jfivroe/v7MOoqgQiEe50aa/zY9CZIbloGnqQBheRCCwx59zIh2EL/KIyRV0wMkUjcmGI9o2QQs7K/DJXpf4PX6ySSx4NXd+M99bQNYywRckrSGuTaH9rOmvEaMslFvGPkjH+B7v60fyDv+Oul9bpr2U41JMCoBl6P3MEtHkGD1sUwpXTWUGG8U74Chtl0F3gNRpksvAAM/WfhyoV0rWIrykzbFG56MMN8Sf32uyUiUPni8pvYpH4jn7XtLFhKJ1YxCUX78/Z16EZ7eeW7wYAHOxOGSe+oJuiUSLfV8P6/qdFcjE2qn1m5YaBMugu8KgVrmN7eRhG6j/36EOQXMxaLoHfSjFEET1Fr8YVRruzvKn/2q1fycXqoQeUXGJyob/87WUCCrissu1ANxas24d9Hb3Gc36kIf43UcBMUd5uTtTLxasm/lixUAbdBS1qxfSSvXQ9e2JRQRq6XXJRUS7DDtGueEkpJJHRbK+2KG1kxHn4NehZPTcjIquha+8vcyXQ2mXKKj/56zrsbTf/7+d4MeLQg4YtApa9M82gM4vkojT0QQD3LswwRK+wRX18iJJLKc7wisFBljFBg82noQvtziTqoZtx6P409EQ0EkBD95ecJ992D9jfYRrwptFV2Nfea8zTj4dureXiP2xRLJsNCJILAYmYnDMYJsqgu2B4FzxsUaL7OsKSXPRb/4WUFOVCljGpJCDDQ/fQ0M015bd8rnYbj8rFkttfy8tn+MnLkFnvrUIky9i6pLEhOromgYyvnqnabTQSsNqio+RinmT5Y8VCGXQX7GdemSJJEUE3KyxsMViImaJ8yDKgIu4dM27R0CUkl6AaeiwaCaahR6AX55LR0Jl+650dLUaypLNZo35LbTIWyEMn0v75l1xshbgyWctJlj9WLGQSi4Yt/HIR8JZQeNW1Cr1Di58CQXbs3pTs5bGifMgys9uPp0Eg77r9Qcvn8pUXlww9FDHKZ0RkewqY91PZLCoi7okf+zv7MXNcLdp7Uvi/d7ahaXQVYhFCMh71JQ0xu+QSIA49R3Kxee3FPH6Vh+6CGYcul8rPN0dGVGpl4Nu6U4E/m9epNC65i9xoVlF6RMnFzRhyY2TUcpFocGEYGd+Siz/ZhL/WiEP34aED3qF+vakMqhJRIxJsS2s3KhNRrcyALw9du+XVFn2HLfJwZcEbt++/qTj0QQBPijAPAG8PPULaJW1tMuYrW80O3+1XksvwJZsVrtBc1h63f1qUi5yGHiFCRSyCPulMUfNE4FdD58dETLbAnXDfa88qlckiHo1YSu1WxqOIRfxJQ2IcOgVqcGFWu+Tz4rHp8Vjh+2l+UQbdhazth/L00LlwBmBUdQIHuwsw6PbLY+WhDzsYY4Z857Yfw00Pr4eer464qBUnYhH0pfxLLv41dLNgncwVAfPhoacyWuu9XuHvqExEEQ3oofMWdH67MhlJRGIhLoeN0mKhDLoLfpIDmHCWB4CRVYkCPXSNWIQ8W4spypMs8z6hM8FIV8TyG03RTlXEotIlakWpJsimKOmRX7Kp/6Knm49UJotYhNCTsnnoPk88PCEQ0E4+vuuh267kNQ+d6fH32vGrNPRBgNn0mS8w9x/FOMsjLA/dzF6LR+UvjxXlg6ah549yMT103aDrIXOOY4XNPy/jb32ddhv3Gd/NX8v7bcpq6Ebeh8f4/nRWLxlcqIeu6d0A9OJc0i81Xp8juegeukyNnbBRBt0Fu4aeb4E5eegHuwrYFNVvIwRURCMFN5xWDD0sHrrrpqh2S6SFODLm7nhYx+aXZ6zz4B46SWV72l9rJBZl3U82xhwh1D/xkDhTmawxlqNp6OQryiWThWnQA2aKOkW58PeUlZvCQhl0F3IzwLyz8PTfEKOq4wVJLlnh6JNtsKsoL7KMob4yjniUsKOtx3EM3zwnIaLCzVBbN0XlJRdOEA9dLJ/LmHdKfjYr76GnMuZYThX30H1IHHxTE0DgsEU4RLnw9/QKJw0bZdBd4OFIMpmf4oEFACOqEuhJZYLHohsSTvEXhGJwkGUMVYkopo2pxYa9nY5jLLp4PH//SmNTFJrnXxwN3ZZt7Vk+Qz6yS4tysbZpSAaMcuF9B4KELZrdicyMcvHTleQySOAaulGbJV9fR5uHXpvU8rU6fZTxtLwfzPez64SK4UE2q3cikkjo4ZuigHt8uUVyiUXQJ+lsWFL/fWvoZvlcwNtIM+bXoEdw/zXNxmMHu/sRjfjb2OTSKhAsbJEX5ePx8P08sUh/T6/WgGEj01N0EhEtIKLVRLSKiG50GXc2ES3Tx7wa/lSLixhyBSDvAWBWbNN+xKqEZtB7+oN56MbBpzeaLWa1NsXggOuw+XRd+zoB3D10caM94ctD125jkl2H7K+NiB66hxTCxE1RD6PMN0XnzxyLld89H6ccMRqXz50QILHIlEcChS0yZoSNAvxExIwWb16tAcNGJvU/DeBmxthSIqoFsISIXmKMreYDiGgEgF8CuJAxto2IxgzMdIsHr0NRmdAiDXryxO2Kl7MAUK2/pqs/qIcuxAwX+ZJNMTgwknLyaMJiUgyPiMmnofNN+4pYFK1puT0eowZMYA1dbP7gXeBOtqFMOsuMsbXJOB697mQAwLubD/gun2tGuQTpKWovEZI12tIBg1BDZ4ztZowt1e93AFgDYIJt2CcBPM0Y26aP2xf2RItNVt/s4JeyPfk8dP2WLwx+EujqK9RD5xqcinIZbmSZFnURIXKVAURpzkty4XVVAH9RLqLk4ltDF2QjwLueOoNQMthjrJOGDkBvdydvQPlJB9All4B/I9fh+zPMaEsHFN9D96WhE1ETgOMALLQ9dSSAkUT0ChEtIaJPu7z+OiJaTESLW1paAk24WHD9j4hQGY/m3eAUs/AAoLqiMMlFrNGc8BEzrCgfuBQQjbjX6DZruchILqKHLi+58KtFHuXiR5IQy+cC3un8WmKR91jGGFIZZowV8a+hm3HoQTNFATPSSKy2CEi0BgwZaYNORDUAngJwE2Os3fZ0DMA8AJcAuADAd4joSPt7MMbuY4w1M8aaGxsbC5j2wKNdSpkedz7jzItpkaGhFyi5CBkj8SgpDX0Ywg1NNCLroeuSi4s0mGVm4pu/TFHtlssmfhxYU3LxTs4D5DdF+fs4GfRYxN8VLY/EAZD3asgVJhpvMiQXfu3gVTQtbKQMOhHFoRnzhxljTzsM2QHgb4yxLsbYfgCvAZgT3jSLj3gpVhmPekgu1sSian1TtDugQeeYkosy6MONrO5R5/M4ren8PMrCTUM3N/+CRrkA/spQcNlItqYJA0NlnO8F5DPo2nP2OHTAv4duT/0P0iSaO34xXR7V1FohsWgwSS6knb5+C2ANY+xul2HPADidiGJEVAXgJGha+5CFa2MAkIxH8hp0M/Vfg3vojy7aHuizzRAzUpuiw5QsY4jqJV1dDZS4Trjk4uKhM8GTrIjLGxl72V2/coaYnOfdU8CUK/PtP/HjwVFDj/jT0DNZe+q//41fPouKWAS9qYzlJFHsTVGZKJfTAFwNYCURLdMf+yaAwwGAMXYvY2wNEf0VwAoAWQC/YYy9PwDzLRriAVCZiKI3n+TCNW/9V6zSF+WizQdwqDuF+qq4v8/mUS6A0tCHIYyZscyRfB66sE4qJMIWDcklqmnoTNgodZ2LfhuXDCe0fqY1bNErnZ8xZuRw5Lu65Scje+o/EERDN6+so0FS/wU7UVcZR0dvOkdyKebx62nQGWNvwJxfvnF3ArgzjEkNBiwauofkYvfQ+WUjAHT0BTDoQhKIinIZfoh5DdE86ej2Wi5A/sQibrhqk3EwBmw70I3Jo6ulJsMjVfxvOJqJRXIeuneEWF4NPUCTaH5S29/Zh/V7O7FuTwdmjKuVej2vrAgAdckY2ntTlseKXbpDZYq6IGroSUkNnf+IPIQJCBa6KIZBqkzR4YcYXx7Nk6Ep7J171nIRN/8um3sYohHCk0t2SMxFu40ZHrrfkEDBQ5coQR2LRFARi6A75e6h801GN8mFMe+epObnmsfru1sOAgD+vGKX1Gv560UPvb0nZfHQE0V2yJRBd0EMZ6qMe0S5CJ4S52sXzgAAdPb5r7pobUagarkMN4y2aBHeFs0jbJHIs5aLuCk6ti6J2mQMh3q816bYsQjw9rLtf4d8TwHtlkjT0bulNHRnyQWQl4ZEx+3SOYcBAMbVJ6VeC5g1nwCgLhlHO5dcDGcw//5b2CiD7kKWwTjNViU8PHThEplz0pTRAICO3jRuf2417n11o/Rni1fYxd4lV5Qe8YQuH7boHYcu+rPJWP7cCnMu2q1ss3Tz88yrDJnXilelVYlo3pDf/jwGnRfTk5WGxFouP7h8NgBYuiB5IZ4o6ypjmocOZjXyEifOsJDZFB2eMPk4dHvqP2At0HX/m5sBANefNVX+wyEUyFeSy7BCTCyLRggZrxrnECUX903RiCAFVsQjUoaLf7Kh0UtHx2i3so3WxWOoOuHloXNd391DT2WzqEQ053mnzxWjfwD4qpIqnijrknEc6kkZBbsATYbpS2fRm8ogGfeeT6EoD90Fu4be1Zd27waj34oeeo0e6fKFR97z/dmWAzVmRiQohgemMdRKurp76KZOYRbdyqOhC/9PxqJS6f983SVj+TX63M/L9dDzacnGVW6EUOnhoXODyw2wiF9piAnSakUsAiKfBh3mcV9XGUc6y9Ddn7YYdABo7y2Ol64Muguihj55VBW6+jNo6ehzHps1L5E5NcngFz/ipfTYuiTSWYb9ncEbZiiGFqKHroUtugy0RVflay3HwCwOR1LWQ2d8fP4oGjtZZr3KBLyaxJgGuLoiiu48V8Tc4Dp5vAmPmjY5nyt0LCIiI5ZcFtHD51fl7T1pQ3Kp5wa9SLKLMuguiB7NUePrAACrdtsrHmiIiUAcni3KqXCImXVD1OQnjqwEAOw42C39esXQhpeS0CQX92QX7rlzmSFfjRZRBtDGymrouocukcFp+RuENWxKGd6v1TT0GLry9BLg75OMORj0qE+DLhhkAHrdJh8aumAnaoykKMFD1438oZ7CssZlUQbdATF6AABm6gZ9/Z4O5/G21H/APMiuPnky/uPsqb4KG4l64sSRVQCAHQed25Apyg9RrohF3OuQc6PFjVhFLOqRKWrT0CWMHl+xlQE99AiZXmq+qBpxI7jaIwiByz5JB8kl4VECwelzrVcu5omuuz+NTS3O3aIs6K/nBv1Ad78R5llsD11tijpgj1qpr4yjKhHFPjfJxSFsEQA2/+hiAMC9r25CJstwwyNLcVh9Jb794VlSn08EHDZCC6Ha5dJXUlF+GMZQL5/rFrBhGPQYN+juEVFMSFEHNMPlJiE6zSXpERaZ+zrtNqJXK03EImjrdpcNzWNOy7TOl78hI7n4mWdEOC8k41G8t70Nv35tExZvPYC/rdqLtd+/0PGz7M3hedkCxkz5RWnog4Asy/W4R9ck0NqpHQAvr92LdYK3zgTNU4T0zSr+4z6/cg9+88Zmz89nQsqIWeireLGsitKSFSS8aEQLwTvjJy/jO3+yVtPosxn0RJ6iWyxHcpEroWvX0P1uihJpf8fIqjjauiU8dBCqE1E5ycXJoAeQXMTjtiIWwYZ9nfjh82vw5oZWAMDWVme508wQt3rogCm11CWVhl5yxMQOzujqCrR2aR7GZx9cjAv+67Wc8W7U2jZIvaQX0UOPRLQCXX67tCuGLqLnF9Hj0Lcf6MFD72y1jONrokLw0PMlFjlJC5taOvGtP6501dOZzUOXNZTiPgAAjKhM4GA+D12/JV2i6UllXE8epofuLrnIZmeKcejae0aF+9p7bdjn1qTbGgxRLRj0Wt2QcyPfGbDZjV+UQXfAaROqoSaB1z/Yj30dvQ6vcPbQOfwszZGNWBGjF2Q9I8XQJyNc8VXkKf2QK7lE83QssoUtxjXjP/+uV/Hwwm15jBaM9wb8SBnWq9wRVXG05fFSxZ4C9VUJAO6aO9fXHTdFfUa5iJURAetJgu+DbWzpxBOLt2PR5gPW1+q39k1RAKjV7yfjEUQI2H2oB197cjle/2BgG/soDT0PooHe064Z8jteWGc81pfOoCIWddXQOXYPfUtrFxprK1w/1x41UxGXb0igGPrw9RQVjJsTXC+vECQXtyqF9k1Re6aom/aetUsukiF94j4AAIysSmDTfvcNRjGwYATfRO1OYUxtbhp+byqLRDRiuYLmGJJLwE3RLftNeYU7XtsPdOPul9Zrz//4EstrAfNvtBh0/ZgnfQ/k929rV1edfWmcMX3gmvsoD90BJw39hrOnAbCGD25q6QLgnPovUmvz0L1CEHmZUV7USGtIoAz6cEHMa+DGzQkzykUztvkkl0w2a6wnQItyESNJ8sWvA4Lk4jNTlITAgi37u7G1tSv/eGjePABXj743lXFMKgIKi0MHTMcNMMsHuF0p2C/kRe++Jun8u9VW+Ku86hdl0B3IOhjoi44Zj7NnNGK1EIu+fm+HPj439V/E7qG3e8Sk8sVYwcPR4hH0Ksll2CA6CCNspZfFQl05kkvcXZ5JZZmlCmgyFrUYpHxldwHvFne5r8uNAOnPZHHWna/kHU9EGFGpXZW4baL2pd3T6P1HuVjj0O+/phkXHzPOMmavYOR3OkSb8deLV0D2Y57TPcCFupRBd8De9JkzfUwNOnpNY5xj0F09dOuP61Xljh9ccUEbVR768MG8lEeOQX9z437jPpcVjCiXPJvn6UzWqEsOaPWJRPrSWew42G2saY5ZbZEQoWBhi4BZ59x7vOChu2yi9qayjhuigP8oF96EgzN/5lh8cf50y5itB8wr6tN+/LJx/yFdRhFf31CjSalOBr2mIha4cbwsyqA74JT5CQBnHTnG8v/dh3ot4x0kPQC5WaNeIUz2nolqU3R4Iab+11daNfRHFm4z7jttirqtk3SGWSSXOpuU05/O4vQ7FuD8n71meZzBDD1M5Ilzd/8btP9XCceAU5SXWJem3jDo7pKL04YoICYW+QhbtFlBe/ih0zy2H+jGD5/XumyKh/3fbjoDXz7vSBx/+Mic10wcWemrrEAQZHqKTiKiBUS0mohWEdGNecaeQERpIroy3GkWF/vlIuekI0ZZ/n9I/6HdTgAccfNmXF1S2kPni5NHJCiGB6L+LHroY+sqLMlARhx6VE5yiQnVCe3avNsmYirDjFos2pWi3zj0XA/dMQtUcIpqEmalUid6UpmcKwxOoXHogNWgzzqsLuc1mSyzzE18+eiaCnzp3OmOktDomkTBjeO9kPHQ0wBuZozNAnAygBuIKCfVkYiiAO4A8GK4Uyw+Tho6YK2/LBpmp9R/N+or455ZY9y7sHpeyqAPF0SHYqQQ5TJ9TC1W7DiE+9/YjIfe2YrfvK4lqfENQm/JxVyg9Q4euhOpTNYwkn48dHuggHiV2tmba9TEJJ1IRKuJ7mb8uvrSFqMr4ndTNMNyHTGxsB6v4yRysLsfrULoMbnunmmcNEVzBCvjMfQMsHTqadAZY7sZY0v1+x0A1gCY4DD0iwCeArAv1Bn6IJXJ4qbH3rNkcQbBTUMHgH89YwoAoKmhCm09KaQyWUMi8ei3C0Argu/loed4XnkyABXlhxiHHo0QFn3zXPzyquMxc1wt+jNZ3P7n1fjOn97HNl3bFT10V4Nu2xQdYQuHFF8nSiKpjBkdk4xHpDVgHiFiboqKfXZzDbXdKapKxNDl8lkdvd4GXbbLlz0OHdAct386fgJGVsVxRIPZc/WWi2YCAK7+7SJ86rcLjce9jvv/+/xJWHP7hahKRLFmdzt+//YWqbkFwZeGTkRNAI4DsND2+AQAHwVwj8frryOixUS0uKUl/AD7ra3d+NOyXbjy3rcKep98m5zfvPgovP2N+ZjSUI1DPSmcfsfLuOKet13Hc175ytl45xvnah66bJSLEb2gPPThBO8DwQ3NmLokLj5mvGVDXkQszpXJMsdiXulM1nKFad9sFT1aca2lhNc11lSgpdO7/gtg3QcArBp6Xg9dOAF0u0gunX1p1/LUsQiByJ/kEnU4bu/++Fws+taHcMrUBgDA8YePQPNkTRdfI0S6nTG9AfNnjsl5vUg8GkFlImokIt36zCq8vbFVan5+kTboRFQDzQO/iTFmryP7XwC+zhjL+y0yxu5jjDUzxpobG8MPrue74m4LX5o8m5xEhPH1laivTOBQdwp7280Fnu9E3dRQjXH1SdR7pEADDhq68tCHFbyzT8y2W3f2DOdjhu/R5AvZS2etm6J2yUV8TWdfGjsOdiOTZUhlmHHCGF9faQQCeGE30Hxu/P3t2Cuc5vPQO/NILkR6qQxJDz2Tdfew49EIpo2pwdvfmI+HPncSjp04ImfMPZ+ahyMaa6Q+60CXaSve2DAwGaNSBp2I4tCM+cOMsacdhjQDeIyItgC4EsAviegjYU1SFl5rpVDcNHSREVXxHD0x33hOQ20CrV39eeu55GjoalN0WMEbHMdsXe0vOmY8PvjhRa6v40bOyaFJZZjlBJGMR1FfGcdXL9CamYvra39nH06/YwGueWCR7qFr8xhXn8Smli48uWSH599gL1gnLnen+YldugCthK6Ths4Yy6uhA7rWL1V4jGH3oR40OmSjioyvr0R1RQyJWAQLv3kuXv/aOcZz1S6bs05w3f2Ba0/AVy+YKf06P8hEuRCA3wJYwxi722kMY2wKY6yJMdYE4EkA/8EY+1OYE5XhoGDQ3Tqly+CUKWqHN54QkdHQG6or0J/OOuqIHMc610PAoD+ycBs+/7t3Sz2NIQ9vn2b30AHnxsgcvoHqdAWYFgwzZ/lt5+OGc6ZpHq1wBbi7TfPCX/9gP3pTGSHKRbv9yhPLPf8Gu1M0vt40ms4eunW8WwndvnQWqQzL2xEskaf+jUhLZx/aulM4cqychw1oHcQmjarCl+ZPwxEN1XllVjt87FHjcjdaw0LGQz8NwNUA5hPRMv3fxUR0PRFdP2AzC4DooefrSeiFWMrTjVOOGJ3zmIyHPrpGO+g+8vM3XTuT96ezRsd3QLs87upPD3jIU6F8848r8fc1JdsTLxvSttIPdhZ981y8/Y35OY+P1HVxJ4OesW2KiiRiEctGvZgZuaut1zDo5x41FoBm2BljeHPDflfHye4UNTVU45kbTgMAdDpEedkDEariUSzb3ob3dx6yjOMng9oQPPRVOzXl+MixtZ5j7Xz5/Bl4+Stn+3rNPVcdj29cNBNj69zrOBWKTJTLG4wxYowdyxibq/97njF2L2PsXofx1zDGnhyY6ebngGDQ3WJYZWA2/c+J0TUVmDNphOUxmZP1aD2TbNP+Ltyv10a3Rw7066Fi/Iw+Y1wtGEPB0TuKoYHpoTsvqDF1SYyvz71C5JErhxwSYVLZrKt3n4hFLCcBsZ7JltYuI2N53uSR+KfjJqCxtgLPrdiNq36zEI8v3u74nvbCVQAwc7xmOLk23tWXNhKh+GmBO0XcIROjSQAzKa/ay6BLaOiPvbsNI6vimDc5NwloIGhqqMa/nTXVl1fvl7KqtigmXeQrkO+FV7Etzu+vPRGpbBbNP/g7AO94VACoEcK3fvj8GiPb7LZLZ+Gjx01AMq6VQBU3kWbpsbDPLNuFmgpts2hKQzUYYznhZ4qhj+mh+0vkHlnNPfRcg57OMNcTRCIasbxmj7Dx2dGbtsSvJxNalcZFm7UoDbcCWk7HUEUsiuqE2Snp6Nv+hqPG1+GFG8/I8dB5hy57lubNutyTV0OXlFzW7unA6dMbXevCDEXKyqBvESq5FVJQXqylkY96W+iXTGLRsRNH4HOnT8Hp0xtw7QOm3vy951bje8+tBgCMrk5YmkpPHFmJM6Y34MG3tuDBt7ZY3u/Ba0/A2TPyh00Vm3yX9wpvvDx0N/Jp6KkMcz1BVMSt7eHsBahEz74qHkV3fwbbD2hj3GRDt32oaWNr8cE+80qThwDagwT2CdFjYugk3yc7aUqu5MlJSHZjautOGTJVuVA2tVweXbQNK3YcwgxdD+vqS4Mx+cbMIjIausixE+sByIVLxqMRfOfDs3DOjDG46UPTHce0dvUbG6KAtply9cmTHceGFc+6cschbHNpteUX2aQOhTNuUS52nvr3U/D7z55o/D8ZjyKpG+f3th1E0y1/MRpXpLO5m6KciljEcnVr718bF5yLSr2BMzfEbn1J3SLFZoytwZsbWrH9gHWt8drjo6s1SfJXV88znJrnV+7GA29uRjqTRWUihvNmjc1xpkQmjKj0bO6czTK096byliceipSNQf/G0ysBmLUX2rpT+OyD7+KEH/7Dd0Ecr4YVdm6/fDZqkzHMnlDv63OuP2sqAK2KI+fL5x0JIHcP4JiJzu/t1F0pCJf+/A2ceecCx+fuf2Mztux3rmPthGx6uMIZw6B7XCLOmzwKZx5pjU2vS8bR2ZfGH97VtO3X1mvxzumM+1XT1MYai+SyS49y4d5rQjgRVCa0sru8YbpbqLBbtvVp07REnWseWGQ8lspkDWfi8FFVAIBTpzVg+W3nIxGN4MbHluF7z63Gva9uRHd/2jNUsLlpJLa0drt0F9Po6E2DsdwiZUOdsjHo4+uTqK+M43Ona6n5NzyyFAvWtWB/Zx82+zBGGvlbytmZO2kEVn73Aoyrzx/PaicZj+LNW+bjuS+eDgC4aPY4nDJVu5Rst3n74+srcccVx6BpdJXl8bBi793o7Evj9j+vxtk/fSXnOaeMRABIDYEQy8EM/179Si6AtlnY1ZcxCmDxyKh0nk3RE5qsRef4a3n0h/i6SkFvHlkVR2tnH5ZsPZjjNLk1Tr987gT8+9lTsbHFPCavvOctbDvQjWiEMH6EeQwl41EcLqz397a1oasvjao8+jkAnKxHoL2+fr/rmLYe7bgptz2osjHoB7v78fHmiZg2Jjem9AOXfoluyCQWhcWEEZVIxqNYffsF+J9/OQ5HO1R343zihMPxwLUnYlS1uQiXbD2I7z67yij49aMX1uDk//ePgufFGMOh7pQlAkeUr97dcgDTvvUClmw9kPNa5aEXhqGhe0guTlTGtYQcLmls0T3ffJuiUxrNeiWTRpnRM/xYsmjounc8vj6JE5pGYfP+Llxxz1v41G+s0Sj5jqFTp1r17+U7DmHbgW4cNiKZc9IRa6l092fQ1ZfJuyEKAMdMqMf4+iReeH+P6xgepmnPmB3qlIVB7+5PozeVxajqCscd6w17/YX75SvONVBUJWKIRyOWmhdOTGmoxptfN2OQt7Z248G3tuDY776Iv6/ei1+9ugl72nuxX7Lmhhv3vroJc25/EduFdnk9qQze2dSKnW09hnbvFHeeSocjAw1XZCUXJ6oroujqy2D9Xs2J2dXWA8aYnvrv/H6NNWZc9GF6OGR1Imr08xSPA358ja1LYnRNhVEKYPHWg5b3zGadN0UBM2pLZGNLJyaPqs55XEyrb+3qQ08qY5xU3CAiXHD0OLz2QQt+9Pwa3PDI0pwxPHrGXtNmqFMWBp3Hn4/Sw7bG2Bow723vw89f/gCPv+scM2vHXhyp2Dzyryfh4c+f5Pp8ZSKK7146C9+91FrF+PO/X2zcX7XLXm5H87B/99aWvOV7+eX+Y+9qjRT2CiFs7T1p/PN97+Dcu14xDiqnAkrKQy8Mr8SifFQlYvhgX6exB3OoJ2WcIOJuce3C8cINXE0yZoRBiiHA3IMeXZ3I8W7FzXB7T1GR0TUV+PzpU4xgAkBbr5NGVeWMFa+4dx7UNmvtDWOcmD9zDPrTWfzqtU34y4rdWLT5AFbsaDOeb9XrqoxUksvgwzTo2sJ89gunG30BpzRU4+V1+/DTF9fja0+tkHo/XspzIBMA8nHq1AZj88iNa06bgo81T3J9ftWuQzmPvf7Bftz27Cr86Pm1lsfF0DO+OXZAjzoQSxTwE0FvKmsYbafwUNlKdwpnuOQSD+ChVyWixtXZpFGVaO9NCRKO8/uJEh430tUVMUNfFjfouVQxqjqBukqrYT3Q1Y9HF23D8u1trk1iON/+8Cw8c8NpuP+aZuOxwx0M+hzB6POEpHxJRZzJtr2mj//qbVz28zeN/29t7QaRcwmPoUxZGPRWm4c+rj6JX141D1t+fAnG1SWN0CpZvUw2sajU5FvYBzpzN0v5ia/D5qGLBpjHMHNDLmYd7hdC1Dbpm1qtXX1IZbIWL457ajsOduNzD75bUNbucIR76NGAHjqghSKe0DQK7T1poXqj8/uJhp4fI6OqEhilG3SxpspcPTv6o8dPyDmeWjr68I2nV+LyX7xpauh5LnOJCPNnjsVUXcOf4GBcpzpUMvTqTwrAEqDgpLlva+3GuLpkWSUVAUPcoN/w8FLc/eI6I9mAe+gi/LIRMC8X+9NZfPtPKy01K0RkinMNZhpqEo61bPhjlbZFLPahtDffEJNUdgnyC6+419LRh5+9tB6X/8L0fi7/xZtIZbL4yV/X4R9r9+FveTanFLmYGrr/BcgzjD9zahMmjtA89JTkJmtdMmYY6ZpkzJBfxBPyUePrsOXHl+DUqQ05Bl1sMM1b2sn8DZ/ScyymjM7V0CMRwpPXn4JvXmxWJ/TaZwK0rFSOKO3wtb71QHeOF18ODOlM0b+s3A0ARjKReOnIETWyHt2gvblxP/7vnW3Y196H+z7dnPOa3lSw1OtScMmxWuODK46fgBsfWwYAqE3GHaWQ/R2acbZ7JWJWXVt3yhLN8stXNhr39xyyJpwAmte/fm+HkcDC2draZTbqcOnQrnAmaKYoYNY6mdZYg/beFBgzT8r51vOSb38I8VgEzyzbpX92BCOrcyUXkbqk1aCv2GHKfK/pIYONtd6FqK45tQlnzxiDKQ25Bh0AmptGYe6kEXh66U4c6kkZx7ssXzp3Ot7dcgCpDMO+9j4s296GJVsP4hN5JMuhypAz6Mu3t+HhhVst9YTX6Z5BnUNJTdGgd/Vn0N2fRoW+sN26r/AsOD9lNUvFLz55PABzzjUVMdduL3v1RAt7k94+oc/hoZ4U7n5pveNnOTU3aO3qtxRz4mzY12no7G7p4Qpn+MZ0kPIJfLNPlBy41Oa2KQqYReP4MRSLkJFF6ZYeb/fQ391ihrDyEhVjPGqNA5r04mbMObFoBC/ceIYxXoa/f/ksxCKEpoZq3PXxufjSo+/hjJ+YyXOHKw+99Ozv7MPji3fAyUY4/dD2rt3n/+w13Hbp0QCcq9IBmqfRUFOBcXX+EoVKCY9UuHTOeGxq6XL0qvbphteuodsll/99eYPjZ2w/aHrot19+NPa29+IXCzbi/Z25ETXr93YaHnrBHaSGGeks01upBZFctKuvMXUV6NY3Eb+uBwP4ueKMRgnVFTH87BNzXOumiFmW4+uTOZFVo6sTliJzheL3+xAjZJz2z5ocJJ6hzpC7Fj5shLZxItM1BdCyL2+5aCZO1LPhdhzswQNvamVr3SrFrdp1CLMn1JUsyiUIo2sq8OJ/nonbL5+tZQs6aOjck27vSWPtnnb8YoFmuEXJZa3u6X/2tCk5r/9A0EgnjazKe0As3nrQYtA3tXSi+Qd/x1sb3LP3FBr2dnF+uPPKY3HbpbMwY2ytUS6Cb2BXSBhXPqZBl1s+etxE45izM64uiabRVfjWxUdhjIPz4/RYqXAy6OWooQ9Zg27nro/NcXyciHD9WVNx8/lHGo+9pSfFHOjqd6zzcqCrf0h555wjx9YiHo2guiKGbl1Dz2YZ3tq4H4wxo//p25taceF/vY47/7YO7b0pi4f+xJIdqIhFcNXJh1vee2pjtUVyqYhH8kbZLNzUiv365X97bwo/+es67O/sw88XbHAtGaDQSGdYoJBFQEv4ufa0KSAiTBhRib986XTjOS9ZAwDOm6U5QF+90LtFWmUiile+eg7+9cwjHCW+gWzk4Jcah8iYcpRchpxBF8+0H/zwIrz+tXOw8f9djCvmTcz7upOOGI23bpmfs3H69qZWMMbwhUeWYsE6LesxXxPaoUBNRRSb9nfhN69vwh/f24lP/nohHl203TF7dOfBHouGDgBXnzwZUxtrLCfJsbYT3JjapKWtmJ2+dNbwDPd39OGV9dp3+9bGVsfMPYVJOpsNFLLohJiV6RQCaCca0Rwgv+v/omPGA7Aa8bES+nmxmNpYgzuvPBZfOnc6Hv78SXjk8yflbOqWAzI9RScR0QIiWk1Eq4joRocxVxHRCiJaSURvEZGzuxwy8WgEk0ZVSW8eHTaiEi/955nG/4m0srEdfWn8ecVuXPvAu8hkGbr7M1LJC4MVXn/lB39Zg7V7NAnl4YVbwRjwg4/MtlR3/L93tlrS+wFgpm4ExJPkWXpVv/NmjcVT/34Kpo2pwXGHj8TT/3Eqzps1FidOMQs8JW1RLU8s2YHeVBa/unoeLp1zGBZtzq3/ojDRNPRwfC0iMmLHK300NPbLjedOx6JvnYtawUiOGUQeOhHhY82T8OXzjsRp0xpwqkfi3lBFxmqlAdzMGFtKRLUAlhDRS4yx1cKYzQDOYowdJKKLANwHwD13vUD+6xNzA2uMo4W6FZNHVeHul9Zbojq49lybpwntYEfcvPz169p+Ad+wGluXxLj6pFGw7OGF2/CwXlfp/Flj8eLqvZbNpKMPq8OqXe247swj8JlTm3JCHo8/fCR+/elmfO3J5YahPn1aI267dBZeXrsPv3t7i+Gpz55QjzW72/Hc8l2WpgUKK+lMNlDIohuPXXfygNeoj0YIY2qTOKFppBHCKhMvrggXz2+cMbYbwG79fgcRrQEwAcBqYcxbwkveAZBf/yiQjxw3IZT3cYrV5hmPQ1lyOXJsDZbYiiVxpjZWuxY3+s6HZ+HfzjrC8OgAzRi0dPSBiPJm1X3q5Ml4fPEO/PRjc3DxMeNQlYjhM6c24fK5h2Hu7S8B0DbRGvQT6oGu/hwZR6Hx+GK5DX9ZtMYXxcmI/O5lR2NTSxcWbj4QaoSLQg5f3zgRNQE4DsDCPMM+B+AFl9dfR0SLiWhxS0uLn48Olbs+NgdfvWAGvn7hjJznOvUQu6Esudz64aMt3Wl4CdKPN0/EEY01RqboP58wCY8IRcAmjqzEvMnW2ti1ybil4p0bx04cgS0/vgRXzpto8czEetPRCBkG3a3TTdgsWLsP33h6pbE/MhjYsK8T97660bGbFs8nGKpUxKK47swjAAAnNBWn+bLCRNqgE1ENgKcA3MQYc1x1RHQONIP+dafnGWP3McaaGWPNjY2NTkOKwhXzJuKGc6Y5FrfiBahqhrDkUpmIYuJIcwd/xjgts44nefC06MbaCjTr4ZzzZ44ZsDDNaISMDdTGWs3AuyV1FcKKHW05XeivffBdPLpom6V/a6n5yhPL8eMX1uY0XvnHmr246L9fL9GswuPco8Zi9e0X4NiJI0o9lWGHlNUiojg0Y/4wY+xplzHHAvgNgIsYY+E0uiwiJ04ZhUWbD2DjPu0gqx3CHjqgJf5c/9ASfObUJiMJi8cYcz11RJWW+PHG188xPOeBYMVt5xuFzvjn7B8AD/1j976NvnQWpxwxGlnG8HOXBKlisW5PB/7r7+vxo386xnKlwvdn3tiw33L18+gi82TEWxEOVZR+Xho8v3XS3LbfAljDGLvbZczhAJ4GcDVjzDlvfJDymVMm43dvb8XXL5yBK+552wivG8qSCwCcMb0Rq26/EACwZOsB3PvqRpwzcwwA4Mp5E7Fub4dxSSx68wOB+F0aBt2hGmRQ3tt2EKt3txsJUmJ6d7HpTWXwwd5OzJ5Qh0t//gb601lccfxEfGjWWGMMD721N0refqAb584cg+vOPCKnLZxCIYOM1ToNwNUAVhLRMv2xbwI4HAAYY/cCuBXAaAC/1C/b04yx3KpXg5DbLj0at1x0FKIRQkUsgudX7sHo6oRUEsZQYd7kUdjy40uM/586rQF/+dIZJZlLdUXMUrM7DK68922pejGZrHujZADYsK8Dn31wMZ64/hRjw/YvK3YjGiEs2nwAt9oaijhx2zOr8Aeb7LP9YDcYY3hq6U5cOHuc0S1H3JRnjGFnWw9OmToaJx3hnGqvUHghE+XyBoC84ipj7PMAPh/WpIpJJEJGfO5R4+uwbHsb/un4CWVXJ3kw0VBTEeqmqGjMP3nS4XhkodZtacbYWpwzcwzufVWrGLlsexvmTXbfqHt44TZsO9CNJ5fswA3nTENLR58lCeqL86dhY0unse/gxJJtudFF33tuNSaPrsJXnliO51fuxm69amW3UJ6hvSeNzr40JrhkQisUMqi4IgFeOH/2hHqPkYpCaKhJhOqh86up+66eh+9fPht3XnksAGDbgW4j4gIArrjnLcfXc7gx3drahWeW7TSSsjhfeHQprrz3bSzZegBffWI52rpzZSN7rXnOS6s1Ke/ltfuMjvc8RHZnWw/m3/WKNocy66CjKC7KoAt8+5JZOLFpFM6eMabUUylrGmoqQjXo3f1pfGzeRJx/9DhEI4Qr503EFcdPxB1XHotR1Qmj7CrgXtsbMHuhPr54B258bBlueWql5fk3N2h7/bc9uwpPLNmB/3tnKwAtEYhn54oJbycfMQrf/8hsAJqcY6ezL43dh3rw99V7ja5b5dYSTVFclEEXOGp8HR6//hTpVnWKYIyoiud0RgpCJstw8+PLsbe9Dw1CIwUiwl0fn4PL5hwGQPtded14XjHyUE8qp0hYp63M78623IYeAIxywY8u2o7u/jS++Oh7OOrWv+JQTwrvbWszxt1z1Tx88sTDESE4lhju7s/glB+9jNueXQUAuOlD0zH7MHV1qAiOMuiKolOViBm1ugth6baDeGqpllWZ9dgUnT1Bq0+zfm8HevozmPO9F/HjF9bixVV7hDrx+eu2//Cjsy3/39nWg+//eQ1e0FvsXfPAIgDAV84/Es/ccBpGVicQjRAaayssTUXmTKw3Ok2JXH/W1Lw9OBUKL5RBVxSdykTUkCgKoVUPfZw9oQ4fPyF/O7FJI6uQjEewfm8n7tfr4f911R5c99ASI8yxsy+NiSMr8eT1p+S8fsFXzsZVJ03Gtac1WR5/4f3dxv33trXhqpMOxxfmT8ccoXzC+HpNRuFFy646aTJqEjFLYlEiFlEb8YqCUQZdUXSq4lGks8xogBEU3nnpl5+c51kaNhIhTB9Ti1fXt+DOv60DYNbW70tn8fLavfjjeztRm4yjuWkUvjR/Gk6aMgq/unoePjZvIpr02tnfucQauthm63r1Lyda68gDwDn6nsx1ZxyBdT+4EB9rnpiT56Da9CnCYGhnzyiGJDxMtKc/U1ABp3ZdsqirlFvG08fW4OmlO43/7xN6oX72wcUAgD5dGvny+WadnwuOHmfcj0QIf/7i6fjw/75hee8vnDMNW1q7cLSt5SEA/OuZU5CMR/DpU5qMsgu9aesVijLoijBQBl1RdHhaeHcqjXoE34DmHrpsZcwjhW7xcybW5/TABLQa+V44hbXefP6RrrVwqhIx/NtZUy2PTdOvKGorYujIE3mjUPhBGXRF0eHlewvdGG3vSaM6EZVufnzkWFOWmTGuFst3HAKgFSnjiU7fvsQ7GxTQKgl29KZx04emo6M37buwGS8tPKo6gSnfeN7XaxUKN5RBVxQdUXIphI7elKVDjhfHThyBEVVxXHzMeEv7sYR+Qvj+R2Yb9W68eOL6U/1N1kY0QkazlWtObcLUMd4lihUKL5RBVxSd0Dz03pS0fg5oCU3Lbj0fALB2T7tREuCyuYfhnlc24sQSFcT67mVHl+RzFeWHMuiKosPT48XY7CAU0sx75rg6HNFYjU0tXfjI3Am44ZxpQ7pLlUIBqLBFRQkwJZfCNgMLbeb9nx/Sao6Pq08qY64oC9QqVhQdI8qlQMmluy+DMbXBG3NcOucwXKqXB1AoygHloSuKTlgaeld/WnXGUSgElEFXFJ2wolx6+jPGyUGhUCiDrigB1YkYEtFIwSV0NQ9dGXSFguNp0IloEhEtIKLVRLSKiG50GENE9D9EtIGIVhDR8QMzXUU5EI0QJo2qxJbWLu/BLmSyDL2prJJcFAoBmaMhDeBmxthSIqoFsISIXmKMrRbGXARguv7vJAD36LcKhSNNo6uxtbXbe6ALPOSRd5lSKBQSHjpjbDdjbKl+vwPAGgATbMMuB/B7pvEOgBFEND702SrKhqaGamxp7QJj1qJU+9p7ccn/vI71e3M7/ADAoe4Unlm2E6+vbwEAVCoPXaEw8HU0EFETgOMALLQ9NQGA2Op8h/7YbnEQEV0H4DoAOPzw3DKjiuHDuLokelNZdPSl8df392Djvk6ce9RYLNl6EKt2teMXCzbg6xfOxIZ9nTjzyEbjddc+uAhLha5A1UpDVygMpA06EdUAeArATYyx3DJ1EjDG7gNwHwA0NzereqHDmFHVCQBak4qvPbkCAPDbNzZjll5+9pllu/DMsl0AgOW3no/6qjjae1MWYw7IVUdUKIYLUgadiOLQjPnDjLGnHYbsBCC2jJmoP6ZQODK6RjPoa3ZrvsFlcw7Ds8t3YcWOQxhdnTCaJgPAlx57D8l4xOjLecmx49GfzuKl1XsRi6hALYWC42nQSasL+lsAaxhjd7sMexbAF4joMWiboYcYY7tdxioUaNArDb675QAA4CPHaQYdAH740WPwnWfeN0ravqrr5QAwa3wd/vsTcxGNEF5d34IzpzdCoVBoyHjopwG4GsBKIlqmP/ZNAIcDAGPsXgDPA7gYwAYA3QCuDX2mirKCSy4PvLkFgFYs69I5h+FgVz/OntGIaY01aOnowxfnT8MnTpiE0+9YgNpkDI9ed7JR//zsGXKlbhWK4YKnQWeMvQEgr1LJtFCFG8KalKL84QYdAD563ASMr0/if//lOOOxi44Zh7c3taKhpgITR1bh/muaMXfSSNRXBu9wpFCUOyrmS1ESkvEobj7vSJw6rQHzJo/Mef5TJ01GQ00F5usNJ+bPHFvsKSoUQw5l0BUl44vnTnd9LhIhXHyMSmVQKPygQgQUCoWiTFAGXaFQKMoEZdAVCoWiTFAGXaFQKMoEZdAVCoWiTFAGXaFQKMoEZdAVCoWiTFAGXaFQKMoEsjcYKNoHE3UAWFeSD5ejAcD+Uk/ChcE8N2Bwz0/NLTiDeX6DeW5AuPObzBhzrEpXykzRdYyx5hJ+fl6IaPFgnd9gnhswuOen5hacwTy/wTw3oHjzU5KLQqFQlAnKoCsUCkWZUEqDfl8JP1uGwTy/wTw3YHDPT80tOIN5foN5bkCR5leyTVGFQqFQhIuSXBQKhaJMUAZdoVAoyoQBNehENImIFhDRaiJaRUQ3Oow5m4gOEdEy/d+tAzknh8/fQkQr9c9e7PA8EdH/ENEGIlpBRMcXaV4zhO9kGRG1E9FNtjFF/e6I6H4i2kdE7wuPjSKil4joA/02t/2QNu4z+pgPiOgzRZrbnUS0Vv/d/khEI1xem3cNDNDcvktEO4Xf7mKX115IROv09XdL2HPLM78/CHPbIvQTtr92oL87RxsyGNZdnrmVbt0xxgbsH4DxAI7X79cCWA9glm3M2QD+PJDz8JjjFgANeZ6/GMAL0PqqngxgYQnmGAWwB1pCQcm+OwBnAjgewPvCYz8BcIt+/xYAdzi8bhSATfrtSP3+yCLM7XwAMf3+HU5zk1kDAzS37wL4isTvvhHAEQASAJbbj5+Bmp/t+bsA3Fqi787RhgyGdZdnbiVbdwPqoTPGdjPGlur3OwCsATBhID9zALgcwO+ZxjsARhBRsXujnQtgI2Nsa5E/1wJj7DUAB2wPXw7gd/r93wH4iMNLLwDwEmPsAGPsIICXAFw40HNjjL3IGEvr/30HwMQwP1MWl+9NhhMBbGCMbWKM9QN4DNr3HSr55kdEBODjAB4N+3NlyGNDSr7u3OZWynVXNA2diJoAHAdgocPTpxDRciJ6gYiOLtacdBiAF4loCRFd5/D8BADbhf/vQPFPSv8M9wOqlN8dAIxljO3W7+8B4NTNeTB8h5+FdqXlhNcaGCi+oF+W3+8iGQyG7+0MAHsZYx+4PF+0785mQwbVustj34q67oqS+k9ENQCeAnATY6zd9vRSaFJCp64j/gmAe/fg8DmdMbaTiMYAeImI1uoey6CAiBIALgPwDYenS/3dWWCMMSIadHGwRPQtAGkAD7sMKcUauAfA96Ed1N+HJmt8doA/Mwj/gvzeeVG+O7sN0S4cNEq97tzsWynW3YB76EQUh/bHPswYe9r+PGOsnTHWqd9/HkCciBoGel7C5+/Ub/cB+CO0y1yRnQAmCf+fqD9WLC4CsJQxttf+RKm/O529XILSb/c5jCnZd0hE1wD4MICrmC5c2pFYA6HDGNvLGMswxrIAfu3ymSVde0QUA/BPAP7gNqYY352LDRkU687NvpVq3Q10lAsB+C2ANYyxu13GjNPHgYhO1OfUOpDzEj67mohq+X1omxnv24Y9C+DTpHEygEPCpV4xcPWQSvndCTwLgEcPfAbAMw5j/gbgfCIaqUsL5+uPDShEdCGArwG4jDHW7TJGZg0MxNzEfZiPunzmuwCmE9EU/Urtn6F938XiQwDWMsZ2OD1ZjO8ujw0p+bpzm1tJ112YO6wOu7inQ7ukXAFgmf7vYgDXA7heH/MFAKug7eC/A+DUgZyTbX5H6J+7XJ/Dt/THxfkRgF9AizZYCaC5iPOrhmag64XHSvbdQTux7AaQgqZHfg7AaAD/APABgL8DGKWPbQbwG+G1nwWwQf93bZHmtgGahsrX3r362MMAPJ9vDRRhbg/p62kFNOM03j43/f8XQ4ue2DgQc3Obn/74g3ytCWOL/d252ZCSr7s8cyvZulOp/wqFQlEmqExRhUKhKBOUQVcoFIoyQRl0hUKhKBOUQVcoFIoyQRl0hUKhKBOUQVcoFIoyQRl0hUKhKBP+P0JmoQzXLuTyAAAAAElFTkSuQmCC\n",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "plt.plot(t_vec, eye_ratio)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "ae7b8769",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[<matplotlib.lines.Line2D at 0x1d80048f460>]"
+      ]
+     },
+     "execution_count": 9,
+     "metadata": {},
+     "output_type": "execute_result"
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD4CAYAAAAJmJb0AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAABEIUlEQVR4nO2dd5gcxbX235q0u1qt4q5yWCQkoQASsBaSyEEgBCYZDNgmGGyifcH4fhhjG2ODL9jYxtj42mCCscmXDCJKBAkZlEAo55zz5jChvj+6q6e6p3umu6dnpnfm/J5nn53p6VDTU/3WqXNOVTHOOQiCIIjiJFDoAhAEQRC5g0SeIAiiiCGRJwiCKGJI5AmCIIoYEnmCIIgiJlToAshUV1fz2traQheDIAiiU7Fo0aJ9nPMas898JfK1tbVYuHBhoYtBEATRqWCMbbb6jNw1BEEQRQyJPEEQRBGTtcgzxsoZY/MZY18xxpYzxn6lbj+MMTaPMbaOMfYCYyySfXEJgiAIJ3hhybcDOI1zPh7ABADTGGOTAPwWwIOc88MBHARwrQfXIgiCIByQtchzhSb1bVj94wBOA/CSuv0pABdkey2CIAjCGZ745BljQcbYYgB7AHwAYD2AQ5zzmLrLNgADLY69jjG2kDG2cO/evV4UhyAIglDxROQ553HO+QQAgwBMBHCEg2Mf5ZzXcc7rampM0zwJgiAIl3iaXcM5PwTgIwCTAfRgjIk8/EEAtnt5LYIgzNmyvwWz11CvmFDwIrumhjHWQ31dAWAqgJVQxP5idberALye7bUIgsjMSQ98hCufmF/oYhA+wYsRr/0BPMUYC0JpNF7knL/FGFsB4HnG2L0AvgTwuAfXIgiCIByQtchzzpcAONpk+wYo/nmCIAiiQNCIV4IgiCKGRJ4gCKKIIZEnCIIoYkjkCYIgihgSeYIgiCKGRJ4gCKKIIZEnCIIoYkjkCYIgihgS+U5ERyyB9li80MUgCKITQSLfiTj19x9j1M/fLXQxCILoRJDIdyK2H2otdBEIguhkkMgTBEEUMSTyBEEQOWbFjgbU3jED6/Y05v3aJPIEQRA55s0lOwAA7y3fnfdrk8gTBEEUMSTyBEEQRQyJPEEQRBFDIk8QBJFjOC/ctUnkCYIgPCSe4GiLmo9MZyzPhQGJPEEQhKdc96+FOOIX/hmZTiJPEAThIbNW7Sl0EXSQyBMEQRQxJPIEQRBFDIk8QRQpvJApHYRvIJEniCKFNN4/cBTuxyCRJ4gihTTefzDkP4eSRJ4gihRy1xAAiTxBEERRQyJPEEUK2fEEQCJPEEULeWsIgESeIIqWQmZ0EAZogjKCILyGLHn/QROUEQRBFAl+yW4ikScIgsgBPtF4EnmCKFYyiUxHLIGrn5yP5TvqHZ97/sYD2Ly/2WXJSgOfaDyJPEEUK5kCr8t21OPj1Xtx56vLHJ/7m498hpMf+NhlyeyzeX8zlm133gj5Ab+4a0KFLgBBELnBJxqTFaIh2XT/OYUtiAsSPrn/ZMkTRIlSgESPkkLuSRVS77MWecbYYMbYR4yxFYyx5YyxW9TtvRhjHzDG1qr/e2ZfXIIg7OITQ7JkMetJFaJh9cKSjwH4Med8DIBJAG5mjI0BcAeAWZzzEQBmqe8JgsgTfvEJlyp+uf1ZizznfCfn/Av1dSOAlQAGAjgfwFPqbk8BuCDba3U2XvtyO+563XlQiyC8wCcaU7IkfKLynvrkGWO1AI4GMA9AX875TvWjXQD6WhxzHWNsIWNs4d69e70sTsG59YXF+NdnmwtdDKJE8YnGlCx+uf2eiTxjrCuAlwHcyjlvkD/jSr/R9Dtzzh/lnNdxzutqamq8Kg5BEERB8Yu7zBORZ4yFoQj8M5zzV9TNuxlj/dXP+wPY48W1CIKwSQaN8YcEFS9yCmUhBd+L7BoG4HEAKznnf5Q+egPAVerrqwC8nu21CIKwj91ZKCmVMkeYZdcU4GZ7MRjqeABXAFjKGFusbrsTwP0AXmSMXQtgM4BvenAtgiBs4hNvQcnil8Br1iLPOf8U1sbA6dmenyAId/hDYkoXv9x/GvFKEEWKXwJ/pYpfLHkSeYIgiBzgE40nkSeIYsWuxjjVIuoh2MMvyy+SyBNEkZJJi91qtV9mV/Q7nJu/zjck8g441NKBA80dhS4GQdgikyXp1iL3i6/Z75hPUJb/HEqaT94BE379AYDOObc1UYJk0GJhkTuVHRJ5e/jlPpElTxAlilsR8ol2+R6/3CYSeYIoUjKJjFuRj5NT3hYJn9wnEnmCKFJyF3j1h3gR9iCRzwOUckYUgkyBV7di7RMD1ffosmsKVwwS+XxAGk8Ugkz1zq1YF7vR0twew09eWoL61mhW5zFrRAsxQRmJfB6g7i3hR8iSN+e5+VvwwsKt+MustVmdxy+3iUQ+D3j9UBS7JUV4Q6ZaQnny5gRUczuW5YPrl/tEIp8HvB7e7JO6Q/icTCKeSCj/nboQ/JI1kitCQSHyiazO45fnlEQ+D3j9Y/uk7hA+J7NPntvaL/U4lwUCsK+pHfub2t2fIA8EA4rIZ5sq6pceN414zQNed9uUykPr+RDZ4VbDsqnPdffOBODvUePhgGL7RuNZirwXhfEAsuTzgOc+eW9PR5Qs5JM3wztL3vx1viGRzwPeW/Keno4oUnKXQunuuM5C0idPgdeSxWngyXufvD8qD5Ff6lujWL2r0fb+uRoMVezTGoRUd00sToHXksVpC+91AMYvlYfIL5c+8hnO+tNs2/vnypL3i4WaK4S7JheWfCFuHYm8C5xaMkVu+BB5YpUDKx6wnyfvOIWyyOuzqvFZW/JmFKIXTiLvgqjD/FnyyRN+xP1Uw8VdAcW3I0u+hIk7TK0inzxRCOwOhnJKsVvy4r55ml2jPrOFuHUk8i5wasmTT54oBLmaT77YffJC22M5yJMnS76TUGiffHE/YoRX5Go++WLPrhGNWLbTGpi6a8gn3zlw2sJ7YfnIvYFi94kS6bH/++cmhbLYq59myXvorkm3LdeQyLvA6Y/vjchLr7M+G9GZsVv9KIXSHaIRzdpd45P7RCLvgrhjn3z210zoLPnsz0d0XrwS2c7kk8+nYCa8CryabSvAvSORd4HTiYu8+F255Rui1LBbnzLmybu8fiFc8vnURmHD5WKqYXLXdBKcB169dteQypcydutT5sBrJ7Lk83gtbQpmj84DJH8LSqHsJEQdjoTzROSRWmGI0sS+JZ8pT96lyBfAlM9nw6JdKstLkiXfiSlECiUFXgmB3Z5c7gKv7o7Lhry6azyy5M16SpRC2UlwvpiAx+4aMuVLGq9EtjNNa5BPcYyr3y/b3gMNhurEFMSSl9012Z+O6MR455N3e313x2VDfi15b65pPhgq/5DIu8Bp1N3zwCupfEnDbVa/XM0n7/XgPr/BNXeNd4OhtO9LKZSdA8cjXj2YsVQXqSdbvqQptE8+7oFQOe0N5zeFkntyTW7ymiz5ToJTS8YLUaY8eULgtU/e6ZLwXljhhRg1bv9ayn8v3TVaMLez+uQZY08wxvYwxpZJ23oxxj5gjK1V//f04lp+wLHIU3YN4SFeCZ7rPHmPe6Z2KEievIemfDJPvvO6a/4JYJph2x0AZnHORwCYpb4vCpxaUp48lOSTJ1S8CrwWcu6aQi+hmf5a6v8sz6N3serPnU88EXnO+WwABwybzwfwlPr6KQAXeHEtP1CI7BryyRMaXg2Gch14dXWYDscL72R/Sdt45VoxS5YoNp98X875TvX1LgB9c3itvOL04fBmxKv0mjS+pPF6Fkqn1cmL+uw0eFuIFMpsv6f++E7uk88EV/papl+PMXYdY2whY2zh3r1781GcrCmMT57y5AkF2+6ajDuIQT+5uX46HM/wmFeRFymU2WFmmHVmn7wZuxlj/QFA/b/HbCfO+aOc8zrOeV1NTU0Oi+MdTgNPXvgT9RWGZL6U8erXT2aR5N/9WIhJ/uzCc+iuKYSFlkuRfwPAVerrqwC8nsNr5RWnXU3PffKk8SWN3QnCMi7k7XL4vhdGhuM8+ayvaB+3jZ8R+Xivegdu8CqF8jkAnwEYxRjbxhi7FsD9AKYyxtYCOEN9XxQ4t3y8za4hSpt01cmJW89tPnixZ9fkxF0j/hfAQgt5cRLO+eUWH53uxfn9hsOZhj1fNIQs+dLDzCo038/8dbpzOvbJu8yTl79DZ7DkvQy8FnBWAxrx6oZCdG/1g6FI5UsN2fJN9+vzNO+MuB3043ZaA/kwP09rIO6H0zTP1PNIr0V2TVZndAeJvAsKEagin3xpI8+XlN6St1853FqsXqwo5dySz7+7Jts5ehJ6lVf+kSXfOSjI8n8Wr4nSQJ75NJ3IOnHrJdy6a9T9mcNJb+TL+NmSF0XLdiFvmWRyTXGlUBYthcgr1gXUyJQvOWRLPn3gVXqd4ZxiX7eD+5xObCZfx+l03fkVedWSz1Lk5TJ3+gnKSgG7gS+z/b34XWmCstJG9smn0x4nawG7FZ6kJe9M5s1Ez/axeaz1omheumsKaZeRyNvEaQXVZzl4HHgllS85ZMvXfnZNbgKvIk8/K0ve6dw1+bTkpfnks1m03MwwK0QvnETeJnp/Yub9dXNJezA1KyevfEljP/Aqvc5wzmTg1VlZNHeNQ5WXr9MZUiiB7Kx5vSVP2TW+x2wBgPT7mx/rFrLkSxtdCqVNd00mknny7rLFmENbXpdd49TlWYDBUEB2fnkzs4x88j5G566x8cN7vfA22fGlTSwuZ9dY7+dsMJS9/VKP01Te2XFSj9bOiNdEgQwbz0TeLIWSsmv8i96Sz7y/9z55fwRxiMKgD7ymcdfoXtvzyRciu8aWoVSgOu+0x2GF6WAosuQ7B3Z+eKeNQubzJV/TiNfSI25X5B045d0OhnKbJ69PobTTG5Zf59NdI732yF0j4nLkk/cxZkGU9PubH+sesuRLGVnk0/38Ttx64pxOdUwcF3Co8k4Dr4VKQeQOGyMrzFZzI0vex2RTQb2w5CnwWtroA6/2smsyEVX9/E7didylu8bpBGWFGhsiZ8N5lkKpvSafvG/hDkXbc5+87jWpfKmhd9dY7+fEjy3SMt0OhnJKVimUhcquyconn9r7Ikvexzh1v3gdNPLL6DmiMOjcNXazazIYA2KAlevAq2N3jTNLPmEikvlALprTQVsyOsOM6//nExJ5uzgMxuQyT54oPWwHXuXXGepMNO7OJ+/eks/CXZPHB8DpFCZW6HVC7TWRu8a/OO3CkU9eYcaSnfj5a0sLXYxOT9ym8DiZM8m1Ja8N+3fqcpGv7TC7xsalfvXmctz6/JeOymRGLgZDuV2FywtI5G3i1F3ivU8+NVLfGbj52S/w9OdbCl2MTk88YXMwlINzRl365EWD47QWZmMo2bnWk3M34bXFOxyWyuy6ydfZDYaSX7u7Z15AIm8T+cex19X01ofeWS15whvk+ZLsj3jN4JOPu7Pk4wl3jYNOPG1MAFWoOu9V4NWskSJL3sf4au6arM9GdDbidmehdDCdhnCZOK2fWlaOw5qoF08bBzgIInuJfDu8WjhEC7yST97H6EQ78+5OR/c5OR8tGlJ62J7WIDXWZ4nbwGtc8+U7O06fJ5/Zki9URplXPnlTdxNZ8v7F6VBn+QeO2pmbOAPyOUjiSw+7I171C37bc9c4rVDaNbJw1zgNvGbqbXhp+Hg3QZn8mnzyvkd+YJwuGtIRy17kO2zOQugln6zZi+U76vNzMSIt+jx56woQc2BQuHXXaD75LNw1Xk9Q1tIRd3TudHgVeDU7lBYN8TFOFxLwWuT1gzLyU1GuemI+zvnzp3m5FpEeXZ58muoUdWAMRF0GXmNuA69ZTDWciYa2qPa6LRZPs2dmnE6/YIVZHIUseR+TSNi3KgD9g9PhtbumwP4azjke/nAtNu5rdnQM4R67g6E6Yvbracy1T96dYDkeDOVgUr7Gtpj2urUjO5HPdmUoMRC4w2Txdcqu6SQ4HZLd7oEl7yef/KGWKH7//hp8+x+f2z7GoySFkkUWm3S30kk9iely7+3/QElL3tmP6jhrxUF2zaEW2ZLP7nnL1icvDjdb6IUseR/jNIVS3sMbn3xhMg3MEJdvdmAxeZWKVqrY9sk7EO6obt1YJ2VRZ6+0f4h6DWfi6WRlqH1N7dprYcnvb2rHiwu2OiskvPPJ6xtcdw2jF4TyfsVOivzbOJ2gzAtLXm8VFFYw3QyH92ZO/dIlZlOQZXdN5nPqfcZBm5MHu5+9Mgt3TYZ990si3xZVRP7WFxZjztp9mHhYL9RWV9ouJ+ccwQBDPMEd11v52YyauWscnc0byJK3iT4zwM7+yddeWPL5dteky9IwDodvj8UzPrT5suTvfHUprn5yfl6ulU/s9iSd1JOoDT//5Y9+jsn3zdJts9urMOI4hdKmYVXfEsXaPU3aeyHy+5o6AADNHTHT46zLyREOKg2e01ko9fPzmPwW5JP3L3Kr7HTeDS8Cr/l216Qrs3E4/Kifv4sbnl6U9nzZDA93wrPztuDj1XtdH885x6ifv4N/zt3oYamyJ2Yz8O8kQB+LJxAMsLT7frZhP3bWt4FzjodmKsF2u2Ux4jRrxe5gqPG/fh//+myz9r5VFflISJE3pz3peIIjHAiklMEO8t7RWGpjSCNeC0g8wfGXWWtR3xo1/fysP83WXptZL+8u26XLKZet/fZodtF+AIjGUv17uaQ9mtmSlx+AD1bsTns+nn0754jGNvPfMROxBEd7LIG731wBADjU0oH73l7pyYC2bLCbXRO1mWqbSHAkOBAJKhKQScs272/BgzPX4Jbnv7Q9MCvlmiZZK3PX7cO/P99sur++TPav1KbW3YhqjTe1ObXkgbDaQDj92XXuGpNJ5Si7poB8tGoP/vDBGtz71oqM+5pZITc8vUiXUy4LsdcplPkwBtJZP8ll4+wPPMmXJS/YsNd+eqeMUcx/++4qPDJ7A95eutOLYtkmnuD4z7p9uvcCcSun/Wk27ntnpe44u5a8ECBh7Zo1HLLLbkd9KwAgFGC2grtvL92JQy0dum06n7zaGH37sXn4xWvLrAuqXSfjLujZJQwg1ZJvdCjynHOE1B5OzI5vVj5Weq0PbLuLY3gBibyKEGI7/juny/95MhjKpfXklvY0A0rkwJvcGCQSHO8u22kq/PnOrpGzLZwQNQQuRaZGNIsVguxQ3xLFvA37tff/mLMB33psHj5Zo7ie9Naz8nrVrkY88skG3Xns+uTFb5hO5Hc3Ju/hlv0tAICaqrKMlvyOQ6246Zkv8MPnvtRt16VQOnR52rn7vSojAIC2jjj2NbVr12tw0Kt7bM4GLNh0EOFgZndN3b0z8ZOXliTLyDn+8P4a7b3cSIoRueSu8ZD9Te248on5WLmzAa8v3m7phhGI35KZZBgYBcr4w5tZMzqffBqRf3vpTny6dp/l52bnyJU18MKCLTj+/g/BOU9ryXdIPvlWyRX1w+e+xA1Pf4FXvtyuljNZ0Gl/mu1oyH22OLXeBMZel9Ml7txy7VMLcOmjn2uN63o1kLjjkGJBG0e8WlnQdt1KmshrYpb8bMm2Q9i4rxl7Gtq0bZtUke9REcnokxeCtvVAi7btH7M3YP7GA9p75ytD6T9bt6cJ33tqoRZkBYDeXcsAKCNe6+6dif+sVxpNJ667e2coPSMReDXezmXb6/HgB4qQ72tqxwsLkyma6/c24e+frNfeR81Eniz57Pnn3I14f/kurNrViNlr9uLsh+bglucXp7hhOOdYsaMh+T5NC2u07o0ib+aO0WXXWDx4nHPc9MwX+M7j8yyvLTDLuU2HHUFds7sRW/a3YMaSnWhuj+EnLy/F9kOtaI8l0vrk5cBri3RvZqguDWFFy2Kwv7kDB5o7sGpXA077/cc42KzvynvNwx+tw5VPWGfZrNvTpFmnMvn2vX+6dh/aY3Es2a7Ec5rbFTEQAVFR14wrQ1k1wmZpe6b7Gdw1cqNx3sNzcervP9bNB7NdbWzaDJlUZnVR1IlAINlA/ubtlXhwpmTlWoj8/qZ2NLXH1HMnMT5zv3xjGWau3I0Fm5INR3VXxZI3jnj9n7dX2XIryt9LWPLG2TLP/cuneGjWWl3jkjxe/17+LZoN36m+NYraO2bgo1V7MpYrW4pG5OMJRbTvfnMFrvv3Il0FBZIt6bo9jahvieKf/9mE6X+eo1USrQ4xpcLf/OwXOOIX7+D95bu0H0i+loyxUr27bBdmrVQCkZFQwNKSdzItgF1f61dbD2HR5oM4/Gfv4D/r0/cQznxwNk564CPc/OwXeOOr5Io6rR1xU3dNLJ7Asu31uilqzSp7KMBw56tLMfLn7+i2t8cS+MP7a7BhXzM+k1wTXiLEcd2eJsxeY51lc8YfP8FJD3yUst1K5O2I/9rdjai9Y4bOlw4ovbVxv3wv5V4t3VaP7zw+D797d7X29IsgoehBtHbE0dIRw9NS9gjn0IQwXTnTGQNWlrzcaMvPUIt6vZaOuC6tUNTFaDyh1RnxHYLqdzBOchcKsBTRFYbDsffOxPSH5qhlsm6wgmr2i1yWqjLFJ292b/Y0Znbf7WlM9lxCwfSB1131bSnbjHVE9ucLQ1F8j03qs/+HD1ZnLFe2FI3IP/35Zkz/8xztfYvB+u7TTenKnfHH2bjwb3OxVLWcNuxVusXiB2JQKvKMJTvRFk3gun8vShF5o1FgbFBueHoR/vLhOgBARThoaXWtdxAc1FloJp9Pf2gOznpwNs7/61zc+YqypursNaki39weM7Xy5YeoJRo3LfNdbyzHuX/5FFukbnhrR+p+4WAAz87bkvJgtkXjOKBa8AxKTrtVV7qhLWragOyqb8PcdeaNF+c8xY0hN8jvLtuJ2jtmpPhoOed4bv4W1LdGLcXcWAcem7MBk/5nlna91o44Lvzf/wAAXlu8Xbfvr95cjqb2GM79y6d4Uerei2Dm459u1Hp7Te0xLN1Wr7k67p2xEmPueg+N0vU5eEp5BFbGwAV/nWvqSgiHRAqlsvP6Pck6KT9Doj60dsQtg8BH/vJ9AECDKvIBVeTlhITJw3pjeE3XlIBmeyyhlUHUryelNFZjgyUCo7KgBwIMkWDA1DUr3Iq7G9pw6SOfYbfkiorGE9h+qBWT7/tQ2yYyc+QelHw/RM/G7BraedX4Tiye0DJ+xNNbFlakt7k9jiPvfg9//Whdyvm8omhEfqehZb3l+cW69/EEx+b9SgXesLdZs2BE5RU/AmMsxT3T1K7/8YxCYhR5mfJw0pKPJ7jmpnj4w7V4/NMNpsfMXLEb1/5zAVo74nh98XZwzg0Pb6rMr9jZgNW7GwEAm9Tv+fdP1uOP76/G9f9eiI37mvHUfzZh7C/fww+f+zLFEpEt99aOmO79/368Dq8v3o63VGv/zleTC3MbKzaQFC8jTe0xbWTis/O34Nl5W3TCI3PU3e/jChM31nkPf4pvPzbP9B5E4zylAW5si2pWo2h4jZk3S7fX46evLMXPXl2qGzF6sLlDC4Y2t8dxqKUDTe0x1N4xA/fOWIldDW1a/Xli7kZNcFqjCSzdVq/91sleZBNuf2mJ5ps2ixs0tcfw9Yc/xacWDRmgGBnGY3fWt2L6Q3OweldyUJB8KxZvPYT731mFG59ehGg8oblLjJb8hn3J4+UeqngmWjpihkE+auOwtxkd8QRW7GjQGu5AgKWIYe+uEXU0qf47tccSuvlnPlq9B5+tl3p7aiZX7R0z8LeP12s9NlmsA0wRTzln3vhdXlq0DfM2HsCjszdgT2MbVu5swI9eWIzj7/8w5RgguUzhCwu2aD0MANh+MLWOG3VAuMRapGdE6/mo9ayxLYbGthgeeC93Fn3OpzVgjE0D8BCAIIDHOOf35+I6wh9nRX1rFCc/8LFULuX/zJV7cNxhvXVi1WIQ9Uv+/h/d+3TuGqOboywU1H78/3l7JR7/dCMiwUCKn/43M1Zg4eaDePWm4/G9fy0EANz+8hK8+dUO9O1WnjZrwtj1LZd6D39WhW3LgVas3KnEIN5ZtgvvLNulO0a23Fs64jqf/O/eta6AZiJvzPiQz7tfFb6yUBAAsFyKiwiEWC7YdBCA4gb528fr8dPpo7Vud2N7DN3Kle75W0t2YHhNVwzsWZFyrhueXoSvttbjv88apY2KvOCvc3X7iBjCW0t24uJjB2nbv/XYPOxQG8O56/bhwZlrcM6R/XXHrtjZgGfnbUF5OGkvbT3Qgq8//CmqykNY9POpKYL8zUc+w6b7z9EJlMCYdmhGgnOdBTtvw358tHovVuxswIqdqfdT5p1lu/DF5oOaL15kpIhGU7aC5XILF0yLhSUvmP7nObhy8lAAiugaxbMiHEQoyFJ83e2xZC8PAL775AIMq65ETVUZ9ja2gyNZR3/77iqcPa4fAL3bJMBYyr0OBxmi8WSCQE2V0qPfeqAFk+/70DIA3K1CqVtxrvQqf/LyUt3n26TGa8GmAzhyYHe0GoxD8cw263phye8LAPWtuY1NATkWecZYEMBfAUwFsA3AAsbYG5zzzMnoDjHr2stsMgTZDjYrlXn2mr2YvWYvrjn+MO2zW57Xp34JV8kTV9fhn//ZjAbpQfhgxW5dRdlr8P2FgkzzLb61RLGEzQKx/5izMeV7fKwGZRZsPGAIqCVf72tqx7cMs0GKzACZVbvSP/zyda/71yIM6dUl7f6CFgu3gRnN7THtITyoitnHq/diT0Mb+nQr18rxwYpkA8Q5xzcf+QwHW6KYcni1tn1/Uwfmrt2HG5/5Qts2sbZXyjU/36BYzfdYjH/YuK9ZZ5XJPcCVkmDOV2M3Mwz58hepLprhNcm5UYTQNrbFcNz/zDS97lVPzNfyumWenrfFdH8ZzvUDfC591Hw2UFFPjO65gy1Rzd0hRE9UYVkkZSs82UsxBl6hC34CwLwN+vcyFZEgAoylBF7bownsPKRv9HY3tGFYTVdF5LnegBKuoJ1SQ2mWCNWjSwR7G9s1Q0wMTNx6sNVU4Af3qsDWA60YM6Ab5qzdh3V7GnHU3e+n7CfXmUv+/hkAYOqYvrp9RLyguV225JVtonef69RcIPeW/EQA6zjnGwCAMfY8gPMBeC7y6VwmALBOdWUI5CALoHS3ASW6/9U289WQJg+rxtOfb9G6q/WtUXxftboFRussEgxolcksPdOIbGUKP+wfPliDAd3Lte1C4xMJjhcWbMWa3U26c5gFnjKlbs1cmRyxuquhDbtMrEwzZJHNxG/eTg7cWbT5oPb6T7PW4saTh2Nwry54Yu5GXc/h3hkrcVDtxv/8taQ19YvXlqW4NOZvshYXK079/ce4/qRh2vtMqbZWyPEVOdB+sMX8fJ9YBIXTBYsF3GDJZ8IYX/nZq0s1ka2pUuqVMERkq1Mn8pIlH5KyZjjnmsgJth1UDCqzYGdFJIiQOvmXzKxVe1Iyrpo74ujXvRxLt9enZBQJ//hOqYwBE5XvURFWRF4Vd/FMrd/TlLIvANx48uFYvPUgvnfCMDzyyQass9jvq22HUrYZR313pLXk85fFlWuf/EAA8lyf29RtGoyx6xhjCxljC/fudT/nSCaRN06Lu3pXo+l+mywyXv506QRURIKIBAOaP63BRBBW7tSfV+maqiJvI+V6lUW5dkjdUiHY98xYYerLa0uT/mjFsu3pLX0v2GySsggo882IdMcdBh/u458mg2/y90rnsz7MwYyDgLMsJzf061aeeac0BAz1JsGhC8RaIeqJaHQurRsMQElnFY1Z0pJXstO2SRZqvdRAieu1dsQtB+ZNG6u4UMSzZuzVAoq7xizj7J63VuBhk+CjuHcc+qk2hHX8xZZD2rZhNam/e88ualqlKvJCcK3Smmuru+B3F49HtwrF/rUaJ2El/jKaJS+5cZKT+qVe30qTsqXggVfO+aOc8zrOeV1NTY3r87R0xFDdNYJJw/Rd9vsvOhInjqhO2d9qLvQdJqlRANBXrWzhUEDztZlZfQsN1mQoIFvy3iAerCfnbrK1/+F9umZ9zSev/hrGD+6R9XmsEELbEUugX7dybLxvOh694lhbx77/o5N07y86OmlHVISDGY9fsOkAThxRjSP6VTkosX3eu/Uky8/Sla8yEsRZY/tizk9O021ft6cJS00sSSMiKCoE7ajB3VNceeL6nCv+dDmV1iyDpKk9pnPRyXMSVZWHdLEJMyrCQZSHg7aX6KtWBzgpA/SSx8i9tqlj+uLZ7x+HKyYNTT2+KjkK9r63V+KvH5kH+gUizhNSUzSNDawV4wd1N5Q7oumEcIGFAkx7ds0amefmZ3bVuSHXIr8dwGDp/SB1m+e0dMTRrTyM66SuNwBcNnEIBtv0L1vBmNLCA4q/W7TCr3yR+lVkywJQrMqkJZ+sMd86bkjKsVdMGooPJMEKBhj+cWUdnr9ukm4/p/PJ1/a2/v53nTsGfVRrLh19u5Xjv047PON+NRnOddvUkbr3D3/raJSpQcCWjhjqW6PoXhEGYwynHtFH22+YwTr/1XljceKIavzzu1/DyL5VuPWMETh2aE8AQP8eFZh35+kAgLPH9cPt00Zpx714/WRcPnGw7lwHW6IY1LMLKsu88V5WRoK4ekotAEXUKsushbyuVi2z5I4TSQRjB3THI1fUYWAPfUD5359vxosLt9kuj7CAy0JBdK9IJigcNai7ZniYDd83irwoo2wgHZQCxZFQAN0rUuMMMhWRIMrDAdu9zfGDFfGUA69GhtVUYsrwalOru0ZtJBZsOoBHZpsnBMiI8gtxD5qovNm2ruX6ujOsuqvmbxci370irD27xkkLb5s6EnedOyZj+dyQa5FfAGAEY+wwxlgEwGUA3sjFhVo64qiIBHWBDGF5VmX58H5r4hD07648aGWqJd/SEdP8+ADwvROUwO2WAy3oql5vYI8KDOxRYTpPR3ko9cHvWh7CiL5VuH3aKLxzy4lY95uzMXVMX0wa1ht3nTsG35mkNAwJnjoOIB1mvkrRtY6EApY51/Jh1WrqmxnC2nrsyjrtu1tx0ynDde/PPWoA7r1gHADgyy2H8N7y3eiuBiTDwQB+PHUk/nDJeLz5wxMwSM2e2XT/ObhqSi3+fe1xOGWU0hDcesZIvHj9ZDx02QRcMGEA+nYrx+s3H4/7vnEkbjrlcMy783Tcf9GR+FptT81KkwkHmZZOCEAXA6kb2lPL5rDDsl+dhbvOHYMrJg3FKzdN0QbWAMDvLxmv23e6mq1z6hF9tPstMl6y7fpp7pq4IihloQB6qPf2hpOH46UbpkDcCjuDvUb376a9Fn55OXYTCQXQoyJ9llswwFAeCpomShgbs1tOH5FsNLj1fEq9K62vKQL6Iq1Y8PhVdThyYPeU/avKk26aADOPo5n9LOWhICYP6629P6y6UovdCbeu3AAaLfmRfat0I4S9JKcizzmPAfgBgPcArATwIud8udfXmbdhPz5ctQeVkRCOO6wXBveqwFs/PAGv33w8gOQPJxDWZjd1+z0XjMOrN01JOe9ZY5VouVwZwmr6oxz4uuTYQbhz+mjt4ewSUQS8rrYnAmqQyWh9V0SUWy8HsYRA3nTK4Rjdv5vOMrnmhMNwxaRaAEplH3PXewCAX58/Fu//6CQMr6nUpf/JGNPK3vjB8ehZmaxwRtdVMMCw6p5pmvgCQM9Ka5EXFl4kFNAswmHVlThv/ACdP3ruHafpBE+4VarV3+Pbjyl58fJVfnj6CHzj2EGoLAth5m0nY9mvzjItgyj3+RMGatcYP7iHlqrZt1s5Lps4BIwxbcg6AAxVezlThlejsT3pfps2ThHfV26agpdunIK/fcfaddS7MoILJRcRYwyBAMM9F4zThLFnlzCumDQ05Te66JiBuHLyUNx48nCtMe6jBkOFDx0A5tx+Kl66YbJlGcwQVU6bejcU0HorVeUhREIB7ZryoLZeFqI5un/SnRUKpuapR0IBrYGW3VDCCBBlKo+Yi/zcO07T7s9Rg7rjplOHa88AB9d6JPdfdCQurRuMCaoR16vSuvdY3TUCxlJdq6eP7ov/On2E9v6/zxyJuqE9NXcNoLhszHo4cv0RlIeDeOqaifjRGSMxvKYSVeUhbXpwMfiuqiKc9MkbejKiN5sLcu6T55y/zTkfyTkfzjn/TS6uEUtwHD2kB74+vj96dIlgzu2nYZwkzEI8D+/TFZvuP0dLdxOVLxRgOHpIT7z1wxPw1DUT8ez3j8M36wZhgGpZHCFZMOFgANFYQpdL/+vzxyEQYFpLffnEIRjVtwo/OmOkJuIJrreMhSUv53ZnsoKFyMqiXRYKYGTfKsz68Sn4/SXjdUIgGjdZvADlIRYPt7Hx+fr4AXjnlhNRHg7qLNtwMGAp8sIPGw4GtHt6wynD8efLj8YfL01arkZL7Y+XTgCQ7FILjAPbktcJZrxHdpDv+Vlj+2HVPdMwbVw/nRBcc0ItvrrrTBwzpKe2bc7tp+r8/2eNVXzBi34xFQ+q38WKL+86E/eojeZXvzxT214WCuLX54/D4F5dtGkALp84BC/fOAXfkBqEwb264MhBqZan4BvHDMLvLj5Kt83o/y0LBdBVdR3JFiugH+9gltoJAL0ry/D18QOUc6snl4OrZcGku2Z4n6R7TY4DcEC15M17DmLMxxWThqIsFNTcJvKMp6P7d8NvLz5Kq5/pLPmq8jAqwkHtt60qC+GM0UrvTxhj4wZ2ww9OG4GXbpyis6YDAfOVoYyuGUAZhBUJBXDLGSMw68enIBQMaCtvNbTGUKlmFRnjJIJIDkW+KNZ4Pf7wahx/eGpwVdC1XO9nG9KrCz7fcADVXcuwYV+zVmHlhmHK8Gq0dMRw0sgazWIAlB8jGue6iLkx2DRhcA/8SPU9C2Fs6YjpMhfKVUtHbsEzCZhoMGQrqNwQuKur7YUPf3wyGGOo7hrBkXe/j2tPOAw/euErbZ/elWWayBtT2a47cRhG9lUsNmGxaP5Ji0wDcf+CAYbxg3pg0eaD2gAjM1eRkeE1XdGrMqINhsk05iFbrp5Si9cXb8eSbfXoXRnR7qHIJnni6joM6pkaxxCxHVHWv3/nWFezVHavCOOZ7x2XYr2JU1WVh7T4gkxZKKgN7jHyy/PGoFt5GLcbpr4FklZjJBRAZUSpY+K/+G3le96rMmI65UZ5OIizx/XDm1/t0E1zIAgEGHqoIj+iTxWWbW/A2eP64auth5In4Vz1ycdNJw0TlrN4boS7JMGBqHA7qc+bcIfIsZSHLpugG+vQtSyEinBQS2V959YTtd9W/O5W9TrImG7hD/mcexvbcdSg7liipluLGIwgEmRS4DWKbhVhMMDSkjfrHXhFwbNr8oEQTyE4w2q6IhhgqFHns7GayKlLJIRTR/XRbRPuGpHCddMpw1MedDn4KCrrzw0LI4iGQfb5jeibPgsm2WAkHyyzbt6wmq44rLoSVeVhbLr/HFx4dNIifPMHJ6AiEpRmOdQfWyY1WMLiFfuETAZZAUmLkTHg5lOH49RRNbhgwkBdmdNREQlimuTzfuyquozHZEMwwPDi9ZNx29SRuPr4Wm37lOGKoXDC4emzvF6/+Xj877ePyWoa4uMPr0adYfCWqJ/pArWyoAnrb87tp2puhi9+MRUv36h36/z4xcUAlEZCPAsiThQwteSTlvG/rpmovS4PB3QGj/G4BAeGq5lclWVBvHzjZDx46QTdferXvQLl4SBiCY4PTWZg7KFeW3xPplnyHO+qo7SFC25Enyq1vMmex/kTdBnamltKpGzqLWZ9g2IkGGCmsQpxD3969mi8eP1kbLr/HBw1qIdun1AwAM4VI6qhLYpu5WEEAslBYB3xOIIBphluZMlnieiaigp9xaSh+FptL3y8WqlkTlwAQlRF9+8MaZTbtHH98LeP16OfFLQTVoJxUq0y1YqQdUIObJkhRFYn8jZSBAHFYktwqXFRr2v0OcqNhnigI5pFnyzsw986Gj949kswlrQYGZQ5vZ/8blIY7FjyANBX9UMfM6RHygOTC8rDQZ1PFgAevHQCftLQlvGBG9yri2nGVrfykKNBSkaE1lSEretjZSSkzfFyy+kj8K2JQ9BTclf0qoyguT2ZWw4k04LLJJ+8CLabWfKyyFdEkvWrIhzEgB4V+MMl47FubxP+9vF6/fgUzjFKTUPd3dCOY4fqG7F7zh+LM0b30caifM8wkBAAbp82CkN7d8HU0frRo41tMby2eIf2PQDg7vPG4tzx/TGir3Xqa1V5WCfiZcHk9xENytGSS04mGGCm7pqLjx2EpdvrMaR3lxQXpEBY5rsb2vDe8t2oG9oT5arbKBpPaKmcyrPFda5RrykJkRe+N5FJUFmmdIfHDeyGft3L8fWjBtg+l/AvitQx0e0FgP8+cxSumlyrCzSJClZZFtJWj5fPM6BHBW6bOhJbDrRk7LKJc8lzZNgN2ERCStqa6KJefOwgPDl3E04f3RezVu7Rpv4tk7J+wsEAnvv+JC2lT1x/QPdybQj3d6cchi+3KqNXzfRc63Zn0HqRz+zFUoluqYgEHQ+kkpn/szOyur5oELtErBtu2SAJB5lO4AXaveb60bdloQBuOnU4Nuxr0npawsrWibx0TrlOipTWbxw7CM/M25xyXIIDJxxeje9MGoKrpySnCRHlOXlkHzDG0ubSd4mE8N3jU4/dejA5kE7U+YpIECeOSN/rqioP6b6D3IAPr+mKN35wvKVxFQwEUiz5AAOunDwUFx4zUBekNSKe73tnKIP7V+9uxKRhvdEejesHBbLUcnlNSYi8cDcYfW9loSC+fVzqAIp0iAoj/LfyAxkMMJ0VL7YB+sagW3lIm/tiWHUlzhxrLz1PpP7J1pPRJ29FJKiIvHDHjB3QHZvuPwcA8PT3jsPwO99W9jNUtsnDk2lh4ruEggGUhYJYeveZqIyEsGJnA+55awXGDkgNDIp7Llv08+48PaVBE8FXN6N1/YLd38IKcYvSirwU9AuapIIq50lmpMhTOUdCAfTvXoFnvpccdyF+Fdm33kvKvBLbJwzuoft+ws2oF3mOcDCAey840vR7CbeoWe/TasCeuM7WA8l4lt3eK6CIvGzJG+t3ul5jMJA6t0xVuTKGI53AA0mdEEZTY1tMmzhQjoOJonXq7Bo/MKZ/N4zu3w13fT37wQaikhxSZ4/LNIAmacknK+aM/zoRFx4zCGP6d8M1JxxmdWjqudSnRZ661Czf3oyJhyldZ7m7aiwjkL6yJUVe+V+l+hnHDeyOF66fbCpyQofkBrZvt/KUND2RRmlcgKWU+Jrqo69II/JyfbMIkWjCzbk+E8uspyga3zbJ4pfdNSP7dkV5OICfnn2E/hrqRWSDw2qI3k/PHo2yUEAbNW4c5VvdNYL/u948PVTUHzNL3g4V4aDm9w4GmK0YkSDImG5ytx9PHYn/s5nGKq4j0rTvu+hIlIeUgLNoGEf06ardf7Lks6QiEsQ7t5zoybnEg3LQxJI3Q1Qw8WMeNai75s9922GZgupT3abzydurHA9ddjRW7WrU8pitsPMAOfEf2nXXCEs+3QLixc6fLz8aa/c0oSqNldhVMhaCFr9D0nLWi7zZFApCRI3ZNYLeXcuw6p6zU44TjXarwZI3Y/qR/bVBX0CyxyPiRM9fN9nU7QQkLXl5TqOQA6FmLCnsTv3ewSDT9Sx/aIjhpEOUUTSCl31tMFbsaEBbNK7ds1+dPxbff0qJS1B2jY+ISO6aYIBlFMWA9GMzBvz72uNcX9tYcQD7LgIRh8h4jTSVTQShnFTIgIm7xgwRx8jkYy1mKstCKdkrKftEZEs+s9gJd80DFx9l4b9PFetuGaYmUI5T/usseZuzbQiffIIDJ46oTju3ktDzxrYYelVGsPG+6bazmub/TJnawm0GS5CZZ9fYOlY8q9E4IqEAGFO0oi2a0BrU8nBQey7s/JZuIZF3iOyu6RIJZqxwooI1t8dw5MDuGef2SIdccQRe+fKMmSZmiKCoVSqlGVolzmB9VUSC+OT/nYL7Ljoy7X6ljuyusbJohfXLeXIpvjEDzIOLmrtGEms72WZBqV4P6F6Or48fgBtOHp7hKAXZMCnL4G4Uz0t9axQV4czPm4wYORx0KfIBi+waO4SkXneZahQpPvmku6YiHMS/rp2IS44dpM16mQtKwl3jJZq7pjmqs6qs0B6Gjpgu68YNWhdZNxAru2Cf4LapI1MmDzPSS/XVCv++Hey6awBgaG/3mS2lgizAVnOdyIFOYclbBQrFKWRL3km9bo3GUV1Vhr9cfnTGYwQVOpFPL7xy7yOTa1Qwqm+VthQmkExYcGoQhQLMdbZXUEqSEI1LeTiAhNTwloeDGN2/m2UKp1eQyDskIuXJd0kzaEWQtHjitv3nVgQCyqRJmQZD5Yra6kq8d+tJulWQMmGWXUO4x54lryBb8sb5mwSaJS/5nu1YvMKibumIa2Mc7CKnUGaqv+GgMqnaoZaobZH/vxsnY7+UruzakpcCr1OkLDM7hAzuGiBpkIklHu1Mg+0F5K5xiJwnb8viYUlL3gurOxQI6DJQchmwMWNUv6q0fnsjIrCXown2Sg458Go5a6EUeN2tDu6ychMygyV/w8nD0bdb5h6nlunVHnNsvMguGjvCK3rAdp+fbuVh3XgH4TpxHHgNMC39+s7pox0dG5B63eI7lmkir/SuMs297xUk8g4RFaWlI27LshBWBOfeuFYCgeRKM/I86X5FfH8nqWuENV0i9n3ygDIn/MAeFZa+bGO++4+mjrDl9xZ62dwRz+hXNyKniNrpifY2zO7qFFH33LhrBE57oiHJnRUJ6t1FBzWRJ0vel8iWh51FJmRxq/Cg5Q4FkgsuOB3IVQjEw5HNPC9EElnoMgoP59h+sNVy6L1yDuX/vqYOVEaCtgVb/j2dWqSyuNmx5HurI67TjR9Ih9vsGuOMlE4Q6c6tHanumvrWDjCWP1cr+eQdIrtHnFjygDctt92BS34jlylipUS5VOcsLXl1czzBseVAC04/oo/pfkBSyPY0tOkm1rvk2EGmM3EKgsx9vS4PyT75zMcKV1O6OX3S4dYnL39Ht5a8LvAqMvNaoigPOcsUygYSeYfIIu8kCwHwRuTlBzuXkxp5RUKb7bDABSkS5GCdlQtMbH1ryU4caO7QLaNoRJxif3OHbiHsBwwrWBnR12uHbpBgQJ3Yzp7wisFhbt01yewaZ8cHde4aZ9eUZ4wNB/WW/MGWaN788QC5axwjW89OsmsAvQXjFnG+cJDlbLkwLxEWULWNdWSJzNgSefWeL9x8ENVdy7SlHtPtC8BRim9AV6+di6+oF3Z6o2JqALdxHXGcU3ePfD2nVre8xKQ8oRqg9Jq62DAQvYIseYfIFcXWoBGpcjiZWMkKzb/YCax4QJmn5jcXjsMZhqljCXdUROxb8gAwpFdFWmNAdkNkWoRdf1zytZseaoABcdiz5EXsy7jAjV3EM1OZhcg7ddeYHSt6IvubO3Bamt6V15DIO0RORbPTGsujQz3xyYt0sE7kj+8MAeLOgpwaaEd4xAL0Vshi3TvNWqlGdMaLi7qoZPVwWy4U4e6ImazSZAchuE6t52zcNbJbVSwiLl/fbBHxXNF5lMIn6HzyNtw1gSyyEMwQD1dnEnnCO8LBgDaBmNX0ErL29++efqCSnG7Z3cHQ+kCWsabk0n2ZjxWCabbsoR3EfXLsrski8Co3EGKxc7knMczBgMJsIaXIArPVgYzIvjkvFqF2mylAFA/csHSfEVm4xfS+VsinyDRDqUy2xovwvBgXcTdD1Hm37hrRAcjGXeM0EUY+VqQ8d5Gefye9pmwhpciCkWmWHRPI+bVWQ8udIBqNzuKTJ7xHW3M3w4hXIPNYDlmsMy2EISNXv2xiTXbiAMKYcjKdhoyYf6YiK3eNuxRKADh/grLynBw071npfqJCp5BPPgsGZOgKA0ZLPvsfNmnJ52e0HOE/RFqqVeA1LLlxMk+FnXztZIbUQBZ58jJ2RH7SsN548frJtqbKNkMsgeg0BdOrwOufLp2Qss24aE4uIZF3wdVTarFmd6Oj4d+AN+6aUCcMvBLeIuZttxJ5OZiZaV4ZnSXvVuSzqIt2GxYnM58aEQvROBX5HpL7ynngNXlPzHRCXn0r15DIu+Du88ba3ldeh9MLd402Dwe5a0oWnsGS14/NSC9s8imcWPK6kddZWPL5GPXZrlnyzp6/mq7JnrrTcgYzrLmQr3lrABL5nCNH6D2x5CnwWvIkMljyMpkseVm8XLtrXNTFT/7fKTjQ3JF5Rw8Qqzs5DbzKrqRsUigLDYl8jpFbdDsTmmUiHwv/Ev5G88nbsC4z5aG79a3Lvnw3VunQ3pV5WyRG+OSd9jj0Iu/eJ19oSORzjPwgeiHMwiffmSYnI7xFJBLasuQz1BO3UpTNBGX5RrhrnD4zOpF3KNpWlvyvzx/rOhXULSTyOaZaTJPq0YMgfPxkyZcumXzyMpkE2O2KXbLo+d3g6HAp8rL7yu0EZUaunFzr7EQeQCKfY0LBAJb/6izP/I+dbe4awnsy5slLZLTk1VP0dDAQCtBb8mGfi3xy0RBnhlY2i4aQu6bEqCwLeeKPB5I50GTJly7a9M0eBF7FuSYNc7aGqSx6YR8Jmhl//86xeOWLbRjcK/08PkbkKUxI5Im8IeawJ5EvXbgDSz5TCuXQ3pV49IpjceKIGkdlkAOvTtb8LQS11ZW47UznS2XKcwM59WqFnC4llUNI5DsZokdAIk94YckDwJlp5pu3QrZUrSZK6+yEA+4teR8Z8jR3TWdDLFRCg6EIO5Z8rmI3endNcdZFufFyKtp+WtO4OH+dIkYLHvmoEhGFwY51mStXinztYrXk9SLfeb8jiXwnQwS5YnF3CygQxUMhR1Xq3DV+8k14SFg3/0wBC5IlJPKdDJGuFsvzgArCf6TL4DiiX+ZpsLNBvrSfXBNeIsc83H5Hp6mpuYACr50MkdYlBngQpUs64Xn5xiloao/l7NqdYRH5QvPKTVMwqIeztM1ckJUlzxi7hDG2nDGWYIzVGT77KWNsHWNsNWPsrOyKSQgiQbEUGol8qfJfp4/IuE9lWSjjqlDZYGfenFLnmCE90SeHv4FdsrXklwG4CMAj8kbG2BgAlwEYC2AAgJmMsZGc83iW1yt5hCVPIl+63DZ1JG6bOrKgZfDTYB8iPVlZ8pzzlZzz1SYfnQ/gec55O+d8I4B1ACZmcy1CYfzgHgCAU0b1KWxBiJKGDPnOQ6588gMBfC6936ZuS4Exdh2A6wBgyJAhOSpO8TC6fzes/PU0xyvPE4SXkLum85BR5BljMwGYDYn7Gef89WwLwDl/FMCjAFBXV0cpIzYggScKDblrOg8ZRZ5zfoaL824HMFh6P0jdRhBEEVCsaZPFSK7y5N8AcBljrIwxdhiAEQDm5+haBEEQhAXZplBeyBjbBmAygBmMsfcAgHO+HMCLAFYAeBfAzZRZQxAEkX+yCrxyzl8F8KrFZ78B8Jtszk8QBEFkB01rQBAEUcSQyBMEQRQxJPIEQRBFDIk8QRBEEUMiTxAEUcSQyBMEQRQxJPIEQRBFDC0aQhAEYcFjV9Zh84GWQhcjK0jkCYJwxe8uPgq1vSsLXYyccsaYvoUuQtaQyBME4Ypv1g3OvBNRcMgnTxAEUcSQyBMEQRQxJPIEQRBFDIk8QRBEEUMiTxAEUcSQyBMEQRQxJPIEQRBFDIk8QRBEEcM454UugwZjrBHA6kKXIw3VAPYVuhAW+LlsgL/LR2Vzj5/L5+eyAd6WbyjnvMbsA7+NeF3NOa8rdCGsYIwt9Gv5/Fw2wN/lo7K5x8/l83PZgPyVj9w1BEEQRQyJPEEQRBHjN5F/tNAFyICfy+fnsgH+Lh+VzT1+Lp+fywbkqXy+CrwSBEEQ3uI3S54gCILwEBJ5giCIIibvIs8YG8wY+4gxtoIxtpwxdovJPqcwxuoZY4vVv7vyXMZNjLGl6rUXmnzOGGN/ZoytY4wtYYwdk6dyjZLuyWLGWANj7FbDPnm9d4yxJxhjexhjy6RtvRhjHzDG1qr/e1oce5W6z1rG2FV5KtsDjLFV6u/2KmOsh8WxaetAjsp2N2Nsu/TbTbc4dhpjbLVa/+7wumxpyveCVLZNjLHFFsfm+t6Zaogf6l2ashWu3nHO8/oHoD+AY9TXVQDWABhj2OcUAG/lu2zS9TcBqE7z+XQA7wBgACYBmFeAMgYB7IIyCKJg9w7ASQCOAbBM2vY7AHeor+8A8FuT43oB2KD+76m+7pmHsp0JIKS+/q1Z2ezUgRyV7W4A/23jd18PYBiACICvjM9Prspn+PwPAO4q0L0z1RA/1Ls0ZStYvcu7Jc8538k5/0J93QhgJYCB+S5HlpwP4F9c4XMAPRhj/fNchtMBrOecb87zdXVwzmcDOGDYfD6Ap9TXTwG4wOTQswB8wDk/wDk/COADANNyXTbO+fuc85j69nMAg7y8pl0s7psdJgJYxznfwDnvAPA8lPvtKenKxxhjAL4J4Dmvr2uHNBpS8HpnVbZC1ruC+uQZY7UAjgYwz+TjyYyxrxhj7zDGxua3ZOAA3meMLWKMXWfy+UAAW6X325D/huoyWD9khbx3ANCXc75Tfb0LgNlqyH64h9dA6ZGZkakO5IofqF36JyzcDX64bycC2M05X2vxed7unUFDfFXv0uhbXutdwaY1YIx1BfAygFs55w2Gj7+A4oZoUv2SrwEYkcfincA5384Y6wPgA8bYKtWy8QWMsQiA8wD81OTjQt87HZxzzhjzXZ4uY+xnAGIAnrHYpRB14G8A7oHyoN8DxSVyTY6v6YbLkd6Kz8u9M2qI0sFQKHS9s9K3QtS7gljyjLEwlBvwDOf8FePnnPMGznmT+vptAGHGWHW+ysc5367+3wPgVShdZJntAOSl6gep2/LF2QC+4JzvNn5Q6Hunslu4r9T/e0z2Kdg9ZIxdDeBcAN/mqiPUiI064Dmc892c8zjnPAHgHxbXLGjdY4yFAFwE4AWrffJx7yw0xBf1zkrfClXvCpFdwwA8DmAl5/yPFvv0U/cDY2wilHLuz1P5KhljVeI1lIDJMsNubwC4kilMAlAvdRPzgaUlVch7J/EGAJG1cBWA1032eQ/AmYyxnqpb4kx1W05hjE0DcDuA8zjnLRb72KkDuSibHNe50OKaCwCMYIwdpvboLoNyv/PFGQBWcc63mX2Yj3uXRkMKXu+sylbQeudlFNfOH4AToHRHlwBYrP5NB3ADgBvUfX4AYDmUzIHPAUzJY/mGqdf9Si3Dz9TtcvkYgL9CyXJYCqAuj+WrhCLa3aVtBbt3UBqbnQCiUPyb1wLoDWAWgLUAZgLope5bB+Ax6dhrAKxT/76bp7Ktg+KTFXXv7+q+AwC8na4O5KFs/1br0xIogtXfWDb1/XQoWRvrc1E2q/Kp2/8p6pq0b77vnZWGFLzepSlbweodTWtAEARRxNCIV4IgiCKGRJ4gCKKIIZEnCIIoYkjkCYIgihgSeYIgiCKGRJ4gCKKIIZEnCIIoYv4/oHaEsb7LLV8AAAAASUVORK5CYII=\n",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "plt.plot(t_vec, eye_ratio_v)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "d0c9ca0e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "blink_closing = (eye_ratio_v > th1)\n",
+    "blinking = blink_closing.copy()\n",
+    "n_smp = blinking.shape[0]\n",
+    "blink = np.zeros((n_smp,))\n",
+    "\n",
+    "befor_closing = round(n_smp_blink / 3) - 1\n",
+    "after_closing = round(2 * n_smp_blink / 3) - 1\n",
+    "\n",
+    "for i, bc in enumerate(blink_closing):\n",
+    "    if bc and (i > befor_closing) and (i < n_smp-after_closing):\n",
+    "        for j in range(1, befor_closing+1):\n",
+    "            blinking[i-j] = True\n",
+    "        for j in range(1, after_closing+1):\n",
+    "            blinking[i+j] = True\n",
+    "\n",
+    "blink[blinking] = th1"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 61,
+   "id": "fd3bb860",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "2 3\n"
+     ]
+    },
+    {
+     "ename": "NameError",
+     "evalue": "name 'a2' is not defined",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[1;31mNameError\u001b[0m                                 Traceback (most recent call last)",
+      "\u001b[1;32m~\\AppData\\Local\\Temp/ipykernel_12744/4152700513.py\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m      4\u001b[0m \u001b[1;32mdel\u001b[0m \u001b[0ma1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0ma2\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      5\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 6\u001b[1;33m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0ma2\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
+      "\u001b[1;31mNameError\u001b[0m: name 'a2' is not defined"
+     ]
+    }
+   ],
+   "source": [
+    "a1 = 2\n",
+    "a2 = 3\n",
+    "print(a1, a2)\n",
+    "del a1, a2\n",
+    "\n",
+    "print(a2)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "a348192e",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[<matplotlib.lines.Line2D at 0x1d8004f6a30>]"
+      ]
+     },
+     "execution_count": 11,
+     "metadata": {},
+     "output_type": "execute_result"
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD4CAYAAAAJmJb0AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAABNzElEQVR4nO2dd5zcxPn/PyPt7t35zv3c29nGBhewwYdxoYPB9JIQSkIJJARIIU7yJUDyJSSQQEISQso3gQAJofOjg+nFGBww2ODeOz73dne+urua3x/SSCOtdlfa1Zbbe96vl1++1aqMtKNnnvnMM88wzjkIgiCI0kQpdAEIgiCI3EFGniAIooQhI08QBFHCkJEnCIIoYcjIEwRBlDChQhdAprq6mtfU1BS6GARBEB2KhQsX7uGc93H7rqiMfE1NDRYsWFDoYhAEQXQoGGObk31Hcg1BEEQJQ0aeIAiihMnayDPGyhljnzLGFjPGljPGfmlsH84Ym88YW8cYe5oxFsm+uARBEIQfgvDk2wCczDmfAGAigJmMsSkAfgvgXs75IQD2A7gmgGsRBEEQPsjayHOdg8bHsPGPAzgZwLPG9kcAnJ/ttQiCIAh/BKLJM8ZUxtgiALsAvA1gPYADnPOYsctWAIOSHHstY2wBY2zB7t27gygOQRAEYRCIkeecxznnEwEMBjAZwGE+jn2Ac17LOa/t08c1zJMgCILIkECjazjnBwC8D2AqgB6MMRGHPxhAXZDXIgjCnS17mzF3DfWKCZ0gomv6MMZ6GH9XAJgBYCV0Y/9VY7crAbyU7bUIgkjP8fe8jyse/rTQxSCKhCBmvA4A8AhjTIXeaDzDOX+VMbYCwFOMsTsBfAHgoQCuRRAEQfggayPPOV8C4EiX7Rug6/MEQRBEgaAZrwRBECUMGXmCIIgShow8QRBECUNGniAIooQhI08QBFHCkJEnCIIoYcjIEwRBlDBk5AmCIEoYMvIdiPaYhrZYvNDFIAiiA0FGvgNx0u/n4NCfv1HoYhAE0YEgI9+BqDvQUugiEATRwSAjTxAEUcKQkScIgsgxK7Y1oObm2Vi3qzHv1yYjTxAEkWNeWbINAPDm8p15vzYZeYIgiBKGjDxBEEQJQ0aeIAiihCEjTxAEkWM4L9y1ycgTBEEESFzjaI26z0xnLM+FARl5giCIQLn2Pwtw2P8Wz8x0MvIEQRAB8u6qXYUugg0y8gRBECUMGXmCIIgShow8QZQovJAhHUTRQEaeIEoUsvHFA0fhfgwy8gRRopCNLz4Y8h9DSUaeIEoUkmsIgIw8QRBESUNGniBKFPLjCYCMPEGULKTWEAAZeYIoWQoZ0UE4oARlBEEEDXnyxQclKCMIgigRiiW6iYw8QRBEDigSG09GniBKlXRGpj2m4ap/fYrl2+p9n/vTjfuweW9ThiXrHBSJjScjTxClSrqB12Xb6jFn9W7c+sIy3+f+2v0f44R75mRYMu9s3tuEZXX+G6FioFjkmlChC0AQRG4oEhuTFaIh2XT3WYUtSAZoRfL8yZMniE5KAQI9OhVyT6qQ9j5rI88YG8IYe58xtoIxtpwxdqOxvRdj7G3G2Frj/57ZF5cgCK8UiSPZaXHrSRWiYQ3Ck48B+DHnfCyAKQC+yxgbC+BmAO9yzkcBeNf4TBBEnigWTbizUiyPP2sjzznfzjn/3Pi7EcBKAIMAnAfgEWO3RwCcn+21OhovflGH217yP6hFEEFQJDam06IViZUPVJNnjNUAOBLAfAD9OOfbja92AOiX5JhrGWMLGGMLdu/eHWRxCs4Pn16E/3y8udDFIDopRWJjOi3F8vgDM/KMsSoAzwH4Iee8Qf6O6/1G13vmnD/AOa/lnNf26dMnqOIQBEEUlGKRywIx8oyxMHQD/zjn/Hlj807G2ADj+wEAdgVxLYIgPJLGxhSHCSpd5BDKQhr8IKJrGICHAKzknP9R+uplAFcaf18J4KVsr0UQhHe8ZqGkUMoc4RZdU4CHHcRkqOkALgewlDG2yNh2K4C7ATzDGLsGwGYAXwvgWgRBeKRI1IJOS7EMvGZt5DnnHyG5M3BKtucnCCIzisPEdF6K5fnTjFeCKFGKZeCvs1IsnjwZeYIgiBxQJDaejDxBlCpebYxfW0Q9BG8Uy/KLZOQJokRJZ4sztdXFkl2x2OHc/e98Q0beBwea27Gvqb3QxSAIT6TzJDP1yItFay523BOU5T+GkvLJ+2Dir94G0DFzWxOdkDS2WHjkfs0OGXlvFMtzIk+eIDopmRqhIrFdRU+xPCYy8gRRoqQzMpka+TiJ8p7QiuQ5kZEniBIldwOvxWG8CG+Qkc8DFHJGFIJ0A6+ZGusicVCLHlt0TeGKQUY+H5CNJwpBunqXqbEudaelqS2Gnz67BPUt0azO49aIFiJBGRn5PEDdW6IYIU/enSc/3YKnF3yJv7y7NqvzFMtjIiOfB4J+KUrdkyKCIV0toTh5dxTD3Y5l+eIWy3MiI58Hgp7eXCR1hyhy0hlxTdP/9yshFEvUSK4IqcLIa1mdp1jeUzLyeSDoH7tI6g5R5KTX5Lmn/RKPy7BAAPYcbMPeg22ZnyAPqIpu5LMNFS2WHjfNeM0DQXfb9MpD6/kQ2ZGpDcumPtfe+Q6A4p41HlZ03zcaz9LIB1GYACBPPg8ErskHezqi00KavBvBefLuf+cbMvJ5IHhPPtDTESVK7kIoMzuuo2Bp8jTw2mnxO/AUvCZfHJWHyC/1LVGs3tHoef9cTYYq9bQGIUOuicVp4LXT4reFD3oAplgqD5FfLr7/Y5z+p7me98+VJ18sHmquEHJNLjz5Qjw6MvIZ4NeTKXHHh8gTq3x48YD3OHnfIZQlXp8NG5+1J+9GIXrhZOQzIOozfpY0eaIYyTzVcGlXQHF35Ml3YuI+Q6tIkycKgdfJUH4pdU9ePLdAo2uMd7YQj46MfAb49eRJkycKQa7yyZe6Ji9seywHcfLkyXcQCq3Jl/YrRgRFrvLJl3p0jWjEsk1r4CrXkCbfMfDbwgfh+ci9gVLXRInUeP/9cxNCWerVz/TkA5RrUm3LNWTkM8Dvjx+MkZf+zvpsREfGa/WjEMrMEI1o1nJNkTwnMvIZEPetyWd/Tc3myWd/PqLjEpSR7UiafD4NphbUwKvbtgI8OzLyGeA3cVEQvytP+oHobHitT2nj5DO8fiEk+XzaRuHD5SLVMMk1HQT/A69ByzVk5TszXutT+oHXDuTJ5/FaZgrmgM4DWL8FhVB2EKI+Z8IFYuSRWGGIzol3Tz5dnHyGRr4Arnw+GxbzUllekjz5DkwhQihp4JUQeO3J5W7gNbPjsiGvck1AnrxbT4lCKDsI/hcTCFiuIVe+UxOUke1IaQ3yaRzjxv1l23ugyVAdmIJ48rJck/3piA5McJp8ptfP7LhsyK8nH8w13SdD5R8y8hngd9Q98IFXsvKdGu6x+uUqn3zQk/uKDW7KNcFNhjLvl0IoOwa+Z7wGkLHUNlJPvnynptCafDwAQ+W3N5zfEEoeyDW5y9/kyXcQ/HoyQRhlipMnBEFr8n6XhA/CCy/ErHHv19L/D1KuMQdzO6omzxh7mDG2izG2TNrWizH2NmNsrfF/zyCuVQz4NvIUXUMESFAGL+M4+YB7pl4oSJx8gK68FSffceWafwOY6dh2M4B3OeejALxrfC4J/HpSgbyUpMkTBkENvBYyd02hl9BMfS3j/yzPY5dY7efOJ4EYec75XAD7HJvPA/CI8fcjAM4P4lqFpB/2YSSrQ8WBdUB7U9r9B2IPRrI6hOo3Z/3rapyb11fqN2d1LqJj0hf7MZLVQd2zBoi1J92vGvUYyeoQrt+Yst5xLYqRrA6DYluApr2ey6FpGkawbRjJ6oDGnb7uQbwTfG/qsgn6GPeMA1t8XScblFgzRrI6DNO2Avs3+T6+Fxp0O1G/HtDiAAA13oaRrA69WjYC0ZaAS5waFlQLyRirAfAq53y88fkA57yH8TcDsF98TkZtbS1fsGBBIOUJnL3rgb8cZX2uOQ646tXk+29bBDxwgvX5q/8Cxl+Y8eUP1K1Bj38ebW24ajZQc2zG5yM6GAe+hHbv4VCY8b5OuAy44O+J+7U2oO2u4ShjMf3zBfcDEy5xPeXCB67HpG1P6B/KugM/3QgoatqirHj+Loxdcrf+ganAT9YClb3T38O2L4AHTrQ+X/oUcOgZyfdv3IH47w+DKu75ipeBESck3z8gtv7pVAw+8Jm14evPAaNO9XZwtBXNdw5FF9amfz7xFuDEm7HiD2dibOM8fdthZwOXPB5omRljCznntW7f5WXglestiWtrwhi7ljG2gDG2YPfu3fkoTmY0657O32LnYl+Pw83PyfffAwC4P3aW7fhMYcb5XoxP0zc07cnqfEQHo2UfFMbxQOwsxLoNTV6f2hpQxmJ4Lm44ACnqXZf2fdjNu2Fu2QlAWz2gxTwVJdy6D3HO8EhsBsDjQOsBb/dg1Nl/xU5PWzb9+31QGcer8WOMz/mp8+XR/VikjcAflG8a1/Xx7kab0YW14dn48YipXcxjK2MHsFIbgt0VI7K2BX7JpZHfyRgbAADG/7vcduKcP8A5r+Wc1/bp0yeHxckS4wX4rzYOzRUDzW5Y8v310amPtbH6Z6/BzUngxvWWaCNt5SE6CcbvP187DFpFz+T1ydhvsVlPUtRTruEgr8AGdXj6fW3HxRFDCAu10eZ5vB2n7/eJNsbb9Yz9l2uifAGM+HqAcQ3beDXmMsMx5h6fC2CWeYk2HJpaZt6jAg07eS8cDFd7f84BkUsj/zKAK42/rwTwUg6vlXuMHyYOFRpT0v/wxvdRhGzHZ4ow8u3ifFk2GkQHw/i941DAoSDpsKCz3qWop4xr0KAgLsyAR2PGeBxxKNDEcV7rtua9bPL37TAkpDw5NozHoUFBjPu8P2lfDQo4U817EM8sztS8O2hBhVA+CeBjAIcyxrYyxq4BcDeAGYyxtQBONT53XIwfJs6Nyp3uhzJeyhj3WKHTXl8/n2nkyZPvXBj1STesLGkjz+N6vYh6cQacxtqr46AZBstn4yD2iwmjne56pmMT9nedLGGcZ3Z/0r5xKOBMMe+RQW9QOTw4iAETCuIknPNLk3x1ShDnLwqkH09javquo+m1eKzQ6S4vvBpuVHgy8p0KrsXAIDxEljQyhWtxMADtwrlIUU8ZNN2YcWM6lGe5hkMD8904iLJZvds0xxnnjebZsWGIQwNDTDwXP++u3ONiinmPjGuSJ186ck1poUlGHqrnrmZQco2oLIGdj+hQxONW/UvpyfuQRIQsEc/AWGtQoMFf42BKjl57t87981TnhYwVE48jA7mGgxmevKXJa2C6hENGvkjRrK6m3kJ7lGtMTz5bTV6/Hsk1nZN4zNB6ueEhJtHkhVzjxXCb3qVPj9XUl33KGaJsnuWaoB0ljzCuIc4VtGfkyRuNMVfAoVpyjTn+0UE1+U6B0OSh6pXbY3RNu9euaRpMucbUJ2ngtTMRN+qf7j0nl2tk3Vt3Rnx48j6iazQoegAC4Lluc7NsHo12QrBBPj15hqjmU8YCzGcRF7KaiK4R4x9egjYChoy8V2RN3odcE5Qnn1DhyZPvVMRjloeua/Kp5RpndIcbwphZnrxXI6/3ADTu05N3Oj5p36FCavKKZeQzGHhNiK4xxj+4l6CNgCEj7xG5G6xJAypJ95c0fJ5CQ/VMgj5JRr4zoWnO6JpkA6+SXCNFd7hhyjV+o2u40Jf9R+UAHUGu0aNrolkMvGrO6BqugYN5C9oIGDLyHuG2gdf0rbHYn/NgBlssL0hE19DAa2ciHo8C0OUaPQwvtSfvR67RfEbXMC2ujw34lHk056Bw2gg1o87nOaJMfy4sqzh53bmz3nsFmvfw64AhI+8VmyafXq6xefJB6HCaI/6ZjHynQgy8mnJNsslQcXmAVk3tyRuGJ+bbk5f0ZeOzp8P8RteYnnwwYcheUYyYditENBO5htlkNUUOoSRNvjjREiZDpTPy1gAMggibIk2+U5Mo1yTx5LlwRph3uYb7M/LiOM2cierPyHuVhwoVUZYgY/mRV+Qev9TIMrnhIE++SImLbrDx46eTa7iIlxWefLYLEDgmWeTZGyAKiybLNanqk/DkzUE+D3KNz3h3JKRD8Fi3nb3bNNfTnIPI+dLkTYOciSavP4u44/mLgVcNpMkXLdwMYRPdOJ66cpsVmgVilOUBNc5C5Ml3MuTJUBxI4clLg5tp6h0Dd8zs9J67Rh8b8HecFfnD0kb+6DtKvZc8yhwiph3iHjOQa7jDGVOMwWoKoSxmpJcnztJ3U7nm8LwDGnjVuAIo+e/yEYWFSwYvVYIyuzOQRpM3tHX/mrxmRY8AvuUaK4eLjwi1PNZ53es2GjDFZw8iYSzO0ORBk6GKHhFCaRuQSfFj2eWa7L0Qxp2ePE2G6kxo8mSoFHHy9iyI6eQaw/D4zV1jJCjjzN8ckIRghLTjWpbnjzzWecX05OG/ByEnKJPlGjMZHKU1KFqs0DRv07ltHpUHryUtpifHDO+CPPnOhGaTa1InKANkuSa9J+93MpSlL/uPygESY8iT7m7z5PNU5zk3vW4AaZ9h4vFWj1t+7xVwaY4NefLFSYImj9QtcsByTbKZdETnwPRqeWojD6funcaTzySKxJzYk2GCMrMXkHbRECkaJ191XgQ4GBFH3MPERxvS89eY7MmLZ6YC4HntiZOR94jsIcU9pCowJ0OZA6/BrAxlvSDkyXcmEjT5NHJNuvQHgJS7xu/MTu6Ua7xnrzTL5mEWuK1hy1ed51KPGchOrrFp8iKpW0BpTnxARt4rUlqDuAcPxj7wGoAOJy8aoVB0TWfDnq4A8D7wmrzeKYbsEvOZgyaxB+Bdy9dL7rE3KsfV+x0AzRRpTAOAIRP5GXi13lM5x5UqJC6WQdKzLCEj7xGuxQyPh3mr3AmafDCToaxGo2MMvM5esh0/f3FpoYvR4fEcmSLqCVfTOxdGWF/Mp+zCNGcPwKNcwzXzGE8Dr7Y8MOmN7S9fWY4fPvWFp7Ikv6hjwpbfkEfu6Elpccvwc3kCWf6cNDLyXjEiCgB4k2u4ZuQECUausVWeDjTw+t0nPsdjn2wpdDE6PJrZyIvcNUk8eZuOnbreKVyDxv3PeAX0um32aH0uG6ifwfvAq9dxqH/N24QXF23zVpbkFwVgGXnfQROS3GPmk5e2kVxTxHAtZv5A5kuRytBqcVPXSzfz0BOyJkuToToddk0eyQ2PFOqbzkAlLP/nV67xmcBLrCgFwJvjIzVseYsok8bSAG89DrfjzXz7XLM5aL4XPw8AMvJe4XEzRaoXuYbLXksQkQEJAzoUXdOpkKK7Uq0MZU3aSx/VpRgDr+aMV6/RNUKu8TnjVfbkPTk+fqNxgsDpyaeZUJZ4vGNwWYvbDH/c5wSyICAj75V4zPTMPSVYkiZU+A7Dcj2dI5yMslB2KjR5IB9IWvcSorBSyTUJA6/+omuiflPxctnx8SnX5C1O3m7kkaEnb0tQZjP8pMkXLZxrlifvoXJzSa7RAlg0RIvHLI2/A2nyRDDY8r6k0uTl6BCWOu+KlU3S72QoPeeNtTKUtwRlNrnGSzCCjwaLZ5sA0HFNm1yTQYIybv5Ocbsm73OWcBCQkfeKFjMrqCXXpDC03DnIlN2PqsldXRbKiyf/wZrdWL6tPufXITzgCO1LZni0mJ6t0ovEIWZ2xvwuGsKzl2s8pevmcoOV+l6a263vNC0Lg++IrvEtjTr1d4dc43eBliAgI+8VzdLkrYROqUIoub1RyDZBWVwetMqPJ3/lw5/irD9/lPPrEOnhkoxgZkF1wUp/oKb1QnUjz3zLNaIH4HflJM651bv1NPDqDKFMXucbWqPm362xLN41N00+U7lG9OClc1rJDUmuKT60mN0LAVLLNTwmyTVBePIxu3dRQLmGc46/vrcWG/c0+TqGyBy7XJM8rUE87j1nkqXJ+081bF8b1qsnH3PINWk0ee6Ua5Jfp7HVeh9a2rN415xyjd93V/Q+xIpbXLMPvOZ5lSuAjLx3tLipxXsacJLj6nkAaQ3iklyjhPJaSZwcaI7i92+twdf/+YnnY7LpQROQjE/qdAVa3DHj1YNc43vmKud2mcdHimI/cg2TG7Y0M08PNMuefBbvhmhYuBwZ56Pyyj0ubtwjefIdBMmT9uTBcFmuYVlH12hy3H2Bc9eIKt/kw2OKk5XPCs41Sx5JmYXSmpntRa6xZ5P09hsx6HXR94pS0sCr18lQ4l7SRbnsOdhm/i08+b0H2/DMZ196K5t5UWtlJwBWGKRXpIYpbso1hdXkQ3m7UkdHi0uToTz8UPJALc9eruE2uUYFtLY0R+SOmDlN27vh9rMvkYg9Wiu5gYzLPT6mALw96TlDyCy6RnFq8n5CL4WH7GXFJe4MNkh+L3slI98a1c/7w6cX4cO1ezB5eC/UVFd6K6PIGqkogAZPM21tRdb04WhNPB8etxt+8uSLGO6QX4xtSXeXuqaBDLzaomty78nH4slf3GjcCBMz7HZbLJ7WU8+XJ3/rC0tx1b8+zcu18gnT5PBDhmQDr/YB+hT1TsqnkkquufSBTzD1rnftZeHxtMe5X9MRJ+9hMpQm75/kfatvjmLtroPmZ2Hk9xzUG4Wmdh/viriGohtjvzNe5XkKcdELkCQgzYsKEDBk5D3C45YnHfUyocHMXQNduwwg1TDPYwhlewojLxoA4Z0f+vM3cN1jC1OeL54nT/6J+VswZ/XujI/nnOPQn7+Of8/bGGCpskeTnIZUaXq1eFTaL0VsuVi5jClSvHvivh9v2Ivt9a3gnOO+d/TBdmYsgGGtgerdk7fkmvQzSe2OTfJUHhN+9Rb+8/Fm83OLYeQjIf3YNj8avVEmZnjcXmQlt+M1KIhrQq6xJCBz4JVCKPNPXOP4y7trUd8Sdf3+0w27pUGjRLnmjWU77DHlkhcS5z4XA3bBPvCa+7QGbdH0nrwswby9YmfK8+V7nLix1f13TEdM42iLabj9lRUAgAPN7bjrtZWIpmj08oIt70vqlaFMQ5JiMpQItWSK6kmu2by3Gfe+swY3PvWFuSi1cQLvdVE6ToQXzlu3B49+sjnF/vLMU2+/QatRdyOqfq2DrT48eeOdVlTJk/cp1wAwsnvao2s4KHdNQXl/1S784e01uPPVFa7fh2C9PFGXqILrHltojym3Za0MQK7h+ZVrUnk/wuBx7n3iSb48ecGG3d7DO2Wcxvy3b6zC/XM34LWl24MolmfiGsd/1+2xNnBp4N3oGc7801zc9fpK23EJOZOSGMZoXG8EmaImlV1kyW5bfQsAIKQwY+DVOCZJ1MtrS7fjQLNDQ9cSo2u+/uB8/O+Ly9wfgtQocA9rKPTsEgaQ6Mk3+jHywqDLnryfoAkp7DPGHXKNLZKJNPm8I+SJZPqdiEQAgBj3KNeY+2cv10Cz5J/8GPnkjVJM0uTlxkDTON5Ytt3V8Oc7ukaOtvBDNGYvp4jUEL2XXFHfHMX8DXvNz//8cAMue3A+PlhjSE+SgeRM1+RX7WjE/R9ssJ1HlhVTRdfEonr90T15d7lmZ6P1DLfsbQYA9OlaZg686idIHJjcdqAFNzz+Ob7/pCO3uyTXxL3IIM65IWk86l6VEQBAa3scew62mZ2dBh+9uhe/2ApAfy7prlt75zv46bNLzM+cc3y4Wu/RWgOvlief0dyCAChZI7/3YBuuePhTrNzegJcW1SWVYQSiQjDRDZWIaxwh5hhIBaxumJuXKnleMZ7ck39t6XZ8tHaP63e20yUMvAZfSZ7+bAum3/0eOOcpPfl2SZMXXhMAfP/JL3DdY5/j+S/q9HJKz2Xmn+amHMwNGl/em4RzLIKxxPqQC6555DNc/MAnZuO63hhI3HZA96BlA8nBzBmwTuQoLKSYyBOLuck1+jmXbD2AjXuasKuh1dx/k2Hke1REYFvoWknsLYgUA1/uaza3/XPuBtQ3tTrGFdINvGpJNfl1uw7iW48sMAdZAaB3VRkAfcZr7Z3v4L/r9UbTj3T3yEfr9dtSpLED6V1bVlePe99eA0B3JJ5eYIVort99EAs26u+ynooZ+j2aIZRSPnny5DPn3/M24q3lO7BqRyPmrtmNM+77EDc+tShBhuGcY8W2ButzstSt0L17BZoZVeOcIeg6SKlpkiYPV6+Fc44bHv8c33hofvobs0UapB949WJQ1+xsxJa9zZi9ZDua2mL46XNLUXegBW0xLaUmLw+8Nks9n9mGpCG86Jjkve9tase+pnas2tGAk38/B/ubkofDBcFf31+HKx5OHmWzbtdB0zuVybf2/tHaPWiLxbGkTh/PaWrTf1dVMbRr0VDaBi1TafKSJJIiMiSWQq4596/zcNLv59jywdQZjU1rLA4mG3mXiVmiTiiK1UD++rWV2LKnMW2qj70H23CwLZZ4z457+cXLy/DOyp34bNM+c1t1le7JO2e8/ua1VZ5kxbimq+YAoKh6dLkzueDZf/kI97271ta4WMcDRo5Qy5OXJkPJM16bWttRc/NsvL9qV9pyZUvJGPm4phvt219ZgWsfXWiroIDlXazb1Yj65ij+/d9NOPPPH5qVxHxnmG58v/vE5zjsf1/HW8t3oKkthhCk3DWOOHlnpXpj2Q7saWxBHAoiISMlq4vX4i8tgCP+OYknsPjLA1i4eT8O+dnr+O/61D2E0+6di+PveR/ffeJzvLzYWlGnpT3uKtfE4hqW1dVLA69wrewhheHWF5Zi9M9ft21vi2n4w1trsGFPEz6WpIkgEcZx3a6DmLsmeZTNqX/8AMff837C9mRG3ovxX7uzETU3z7Zr6dB7a+N/8WbCs1q6tR7feGg+fvfGajMiUgwSih5ES3scze0xrN/ZKHnoqT15W46jZPH0MSHXhGxyjdxoy+9Qs2F4m9vjrnJNNK6ZdUbcg2rcgwhIEJJnSGEJs8CF4zDpzndw5n0fGsWR88/b70U1PO2YJKN1LdM1ebORkNjVmF6+29XYahrpdHLNjvrWhG3RuAaVWRknTZlWShMtnvWuet3B+MPbq9OWK1tKxsg/9slmnPnnD83PzQ5tvW83vSt36h/n4oK/z8NSw3PasFvvFouXmEGvyLOXbEdrVMO1jy5EU1sMquS9RB1hUM4G5brHFmLznkZoYKgIq9YAjIP1fgYHneFkjop35n0f4vR75+K8v83Drc/ra6rOXZNo5JvaYq5evuwYNkfjrnLNbS8vx9l/+QhbpG54S3vifmFVwRPztyQ4m63ROPYZHjyDHtOerCvd0Bp1bUB21Ldi3jr3xotzniCdyWMBbyzbjpqbZydotJxzPPnpFtS3RJMa8yaH4Xjwww2Y8pt3zeu1tMdxwf/9FwDw4qI6276/fGU5DrbFcPZfPsIzUvdeDGY+9NFGszd4sC2GpVvrTanjztkrMfa2N/VU02Y+lRRjPFrMHDOSJZHz/zYP//hgvblbVBh51R67vX6XVSfld0jUh5b2uF2uMaTDmX+ai8N/8RYAoMEw8oph5EVAggoNleURjOxTlTAZqi2mmc9S1K/1O+utBsuRXjtkNOayQVcUhoiquEqzQlbc2dCKi+//GDslKSoa11B3oAVT73rPNNKqKdco0ntuXUv0bJzX0Hv8+ixd3RnkiMXazXOJdzis6Odsaovj8NvfxN/eX5dwvqAoGSO/3dGy3vjUItvnuMaxea9egTfsbkJEtcfQirArxljC4OvBtjhUaGb2Sc0xGcpp5AGYjUJ5WEFU072BuMZNmeKv763FQx9tSDgOAN5ZsRPX/PsztLTH8dKiOr3y27quiQOvK7Y3YPXORgDAJuM+//HBevzxrdX4zqMLsHFPEx757yaM+8Wb+P6TXyR4IrLn3tIes33+vznr8NKiOrxqePu3vmAtzN3iYoiF8XJysC1mzkx84tMteGL+FpvhkTni9rdwuYuMde5fP8LXH5zvOg4SjfOEHDmNrVGzq/6X9/QXyRl5s7SuHrc8vxQ/e2Ep2qWB1/1N7eZgaFNbHAea23GwLYaam2fjztkrsaOh1aw/D8/baBqclqiGpVvrzd/a6kUexE3PLsGnG/cZZUv0OA+2xXDOXz/CR46GTGGaWe/c5Jrt9S04874PcbCl3TF2o5dv0ZcHcPfrq3D9YwsRjWuIGyGUikOu2bDHmlQk91DFO9EspEuh4yu6J79+dxPa4xpWbGswG25FYTZjqDAOVVWhKsyQa6yGqi2m2fLPvL96F/Y0tlrBBlDAtThqbp6Nv89Zb/bYZGOtMKAsrNhi5p338uzCrZi/cR8emLsBuxpbsXJ7A2Y9vQjT735PP4ch18QdcfJPf7bF7GEAQN3+xDre3K7bCXM+jVH2tna9zsuefNx4vxpbY2hsjeGeN3Pn0ec8rQFjbCaA+wCoAB7knN+di+sIPS4Z9S1RnHDPHKlc+v/vrNyFY4b3thmr5ja74broH//Fy6pLCKWLXCOMo5gwUhZSEYsC4By/eW0lHvpoIyKqkqDj/3r2CizYvB8v3DAd3/rPAgDATc8twSuLt6Fft3J015xyjT2qRaY8rJrG58+GYduyrwUrt+tjEK8v24HXl+2wHSN77s3tcZsm/7s3kldANyPvjPiQz7vXMHxlIf1ZLpfGRQTCWH62aT8AXQb5+5z1uOXMMWa3u7Ethm7levf81SXbMLJPFQb1rEg413WPLcTiL+vxk9MPNWdFnv+3ebZ9xBjCq0u246uTBpvbL3twPrYZjeG8dXtw7ztrcNbhA2zHrtjegCfmb0F52PKXvtzXjHP++hG6loew8OczEoz51+7/GJvuPstmoAQJYYcGijkBKdGTn79hL95fvRsrtjegLnwQVSx5dM3ry3bg8837UdmqX7uiLGKTa2QvWC63kGCa2+PGoiFWpA+TrnHmnz/EFVOH6WVmMI2nfg8amKIipLKE6Jq2mNXLA4Bv/uszPFSlWTNPpRDK376xCmeM7w/ALpsojCU867DKEI1bAQJ9uuo9+i/3NWPqXe8lRH0JuaaiTLcn+qSmOH763FLbflulxuuzTftw+KDuaGmPQQE3Jy0KWbe1tQ2VELHzhuQb1Z9zfUtux6aAHBt5pk8b+xuAGQC2AviMMfYy59w9GD0L3Lr2Mpscg2z7m/SHPHfNbsxdsxtXTx9ufnfjU/bQr2icQ1U1HDmsN45X+iB20OiOazG8vWKnraLsNoyQauTqNiu0FserS3RP2G2g9p8fbky4jznGoMxnG/fhVNtygpYnv+dgGy5zZIMMq4kRIat2JBpTGfm61/5nIYb26pJyf0Gzi/6ZjKa2mPkS7jeM2ZzVu7GroRV9u5Wb5Xh7hdUAcc7xtfs/xv7mKKYdUm1u33uwHfPW7sH1j39ubptc0yvhmp9s0L3mO5LMf9i4p8nmlck9QNEoAsCnxtjNbEe8/IWGRDOyj5UbZYVxXGNrDMf85h3X61758KdmXLfMY/O3uO4v6hOgGx65J3PxA9bvH4I8M1bvQTrluf3NUUSbdeNYWR6xRdfIRlL2ws1eSrtdugRTsbvB7tXO37APbqiGkVcY0z10Wa6Jath+wN7otUej5gAoZ4o50QiwpKDtUkPpFgjVo0sEuxvbTEeszajnX+5vcQ3r7dc1BLQBNdVVwBagsV3Dhv2J745cZy76x8cAgBlj++FoOdRaMzz5Nt0maLDyBImU0LkOzQVy78lPBrCOc74BABhjTwE4D0CwRn7XShy5/p+YFdqfdJfwLoaTQ9YD7b+9HGNCUqWaD8wKAYO3VWDr/hac7Hgy1awe3bp2QaiFod1ooaMrXsGK1XMA6McCgPLBJ5gV2ophyk7U8WpEVGNSRGs9vh1/Gk2h1Ebx2d+/jFkho3WPAwgB2vtAt9Bm7EA3AIZcE2sFf+/X2Lh5P87auxdnSeUNtTPEQv4qT5+FZZgVMganmvV/0z3UjjVPP2veezq2vfgSZoUMT7EOON44bunjb6N2WE90rwhj6aZ92LJ+r3nODx94B1e1HwBCwM6XnsMs476+eOQ1fLm/2X7trd7KLPPyn57FpKE9rboTRWZvxX7HceLv9iTnMzo7Cc9uAzDJZf/D2BbLsAJQtBhmhZ5N2G8U24pW6F4oZwrQ1gjtvd9gVsiSxbY89wK6cl3aq6oot+SaDXMwMbbDfBYjNlSiZ0iXtiKtCtpDGipbRcSYZeT3rJmPWSHLAIcPMERDHF0OqDhN2j6Q7UWL0lsfeIUCNO0272Hf7DnoEtUwK2Q1EIfwLVKUiwoWbTb3r9nVBaNDzei/tRyjjfd4wvYe6B06YHsevVgE+0LtGPDFPGBrFQ7btA+zQnuh7mGY6fKOnNerBdgOnDNxCG77fB8a2jQMZvUJz7rnugiGhhxe+BrgaGV1glxTseIZ85kJT77nptcwK9TNfvz6CmDkyQllypZcG/lBAORcn1sBHCPvwBi7FsC1ADB06NDMrrJ7FabXPZT+BZe/b4b73Tcm2Q4A/cchskXB7ngl0HUgwuvfxo3OfRfB3PZu/EiEVIa1fAgQfQ/fwjPpn3hb8ut/oI0FADR1OwQAB5v7OxwN4Gi3/f2OtrQkv25gJDOgO41/QOL9bLcaAwDWfaX6nfyyDTi2A+RjnR2fDADYXzkCHMCNoedd93s+fiwAoKHrKCDagsi8e+z11LBtrTwM9BoBgCHa+zCEN8/DdMyz3qMDSGy4NH1ltPV8IAAg3mcMxjS8hTGKowdi7Ov8jRZWjUQECjapwzDt4BzrHoxUQdMc+y+tPBloBJp6jAJ43Nq/3ji3/B7vAE50/o6ikTXcyqTvi2A7gLLu6NJnCIB92BIajulsbuKzTtZ4A1igjTZuaRCghNFz4yto4BXYxnujp9IDqOyLPnXvJNiOvUv7o3cHNPJp4Zw/AOABAKitrc2s7zLuAty0cgTeW7ULh/StMrvoAHD3hYdj9tLt+NDDhKNUPPntKZg6sjfCT36BJi0M/GQlltXV4+y/2JfHO3/iQLy4yApHnKgo+I92Bm654z5Mu+tdU+PNhm01F2Dcmdeh5ubZnvY/pG8V1klZ+jLhX1cdjT+9uxaLvzyQ1XlSsenus3DTs4sxd80efHzLyXh7xU5c+2jqxGcA8Nas43HavXPNzz+eMRp/MCasVIRV13EDmZ5dwhg/qDt2N7Zh1Y7G7G7ChcW3nYYJv3rL9btU5auMqDh2VDVuO2ecTdt+K3QSXhk/B88s2JryuhtGXIoJX/kf7GxoxTG/eRe/vmA8bn95uU0i+H3fCQAWY8fX38dxv7OHlA7qUeEaRVIZUdEkIk6+8oR5bxdNGoxXlmwzgxjcuGX0YSjfuA9PlF2Ey37y57R1eNb40cCONdg+4iIoR12BGdLvLJgxth++Ob0GU0f0xvBbXrN9d+bh/fHa0h343VeOwPrdB3H/XPfxIsGr3z8W4wd1R0jjABbjpW6X4pY9p6c8BgAmDO6OxVut3FXVVRHMxSTgtj14c/kOfOfRhQgpDCOVCuB/1uLlxdvwA8eM4KuUGtye9kr+yXV0TR2AIdLnwca2wGluj6NbeRjXHj/Ctv2SyUMxxKO+nAzGgJpq/RxhlZmDlM9/nngrn285YPs8vLrS1P7k2ZOXHZPYa7l8yjC8Pet487OqMPzzilo8de0U235+l9Kr6Z38/m87eyz6GoNRqejXrRw/OPmQtPv1SXOuH80Ybfv818uORJmRY6S5PYb6lii6V4TBGMNJh/U19xvhyAf+y3PH4bhR1fj3N4/G6H5d8cNTR2HSsJ4AgAE9KjD/1lMAAGeM74+bZh5qHvfMd6bi0slDbOfa3xzF4J5dUFkWjM9TGVFx1bQaALoRryxTk+5bW2OUuXu5uU0EEYwb2B33X16LQT3sA8qPfrI5rYGXEYPoZSEV3SusAIUjBnc353e75ft3GnhRRnmxmP3SQHEkpKB7ReI4g0xFREV5WEnZEMhMGNIdgN75SDYLe0SfSkwbWe06O7mPMQv2s0370hp4AGb5xTwuVUk8p9u2qnJ73RlRXWU2pmKco3tF2Hx32xwN+49mjMZtZ49NW75MyLWR/wzAKMbYcMZYBMAlAF7OxYWa2+OoiKg2L2XCkB4AgK5ZvryXTR6KAd31F60spCAa1yeNPCylo/3WsfrA7ZZ9zagyrjeoRwUG9ahwTc5VHkp88avKQxjVrytumnkoXr/xOKz79RmYMbYfpozojdvOHotvTNEbBo0nzgNIheJS+WeO06MTIiElIQZcIB9WXRVxrdz6d/qL9OAVtea9J+OGE0faPp99xEDcef54AMAXWw7gzeU70d0YkAyriu6VXzQBr3z/WAw2omc23X0WrpxWg0evOQYnHqo3BD88dTSe+c5U3HfJRJw/cSD6dSvHS9+djru+cjhuOPEQzL/1FNx94eE4uqYnQkpitQ+rzAyrBYCBktGtHdbTjObwwrJfno7bzh6Ly6cMw/M3TENIOu/vL5pg2/dMI1rnpMP6ms9b5GBxybDhC1Ht2o1wybKQgh7Gs73uhJF49rppEI/Cy2SvMQMsDVnEqe+QBj4jIQU9KlJHuakKQ3lIdQ2UcDZmN54yymo0ePJ8Sr0rk19TDOiLsGLBQ1fW4vBB3RP272oYa8YYFOae5sTtZykPqZg6orf5eXh1pbm4ToMRsSQ3gM7gi9H9utpmCAdJTo085zwG4HsA3gSwEsAznPPlQV9n/oa9eG/VLlRGQjhmeC8M6VWBV79/LF767nQA1g8nEN5mN2P7HeePxws3TEs47+nj+gGArTKEjfBHeRLGRZMG49Yzx5gvZ5eIbsBra3pCURjiWuIknYqI/uhD0g8rDOQNJx6CMQO62TyTq48djsun1ADQK/vY294EAPzqvHF4a9bxGNmn0hb+J+MMK3v5e9PRs9KqcM5l/FSFYdUdM03jCwA9K5MbeeHhRUKK6RGOqK7EuRMGon83y1jOu/lkm8G78MhBAIBq4/f4+oN6XLx8le+fMgpfmTQYlWUhvPOjE7Dsl8m7zqrCcN7EQeY1JgzpYYZq9utWjksmDwVjDGGpDMOMXs60kdVobLPCB2eO143v8zdMw7PXT8PfvzEp6XV7V0ZwgXEvgGEgFIY7zh9vGsaeXcK4fMqwhN/owqMG4Yqpw3D9CSPNxrhvV/2ZXVxr9Tg+vOkkPHvd1KRlcENUOTP1bkgxeytdy0OIhBTzmvKktl5JjOaYAV3Nv0NqYpx6JKSYDXRF2HJihBMgylQecTfy824+2Xw+RwzujhtOGmm+Axzc7JHcfeHhuLh2CCYaTlyvyuS9x+qqCBhDwgSpU8b0ww9OGWV+/slpo1E7rKcZlgsAIUVx7eHI9UdQHlbxyNWTMevU0RjZpxJdy0OIGj0PMfmua0XYWmjH0ZMRvdlckPPJUJzz1zjnoznnIznnv87FNWIax5FDe+CcCQPQo0sEH950MsZLhlkYz0P6VmHT3WeZ4W6i8oUUhiOH9sSr3z8Wj1w9GU98+xh8rXYwBhqexWGSBxNWFURjmi2W/lfnjYeiMLOlvnTyUBzarytmnTraNOIat3vGwpOXY7vTecHCyMpGuyykYHS/rnj3xyfi9xdNsBkC0bjJxgvQX2Lxcjsbn3MmDMTrNx6H8rBq82zDqpLUyIsY8bCqmM/0uhNH4s+XHok/Xmx5rk5P7Y8XTwRgdakFzolt1nXUtM/IC/IzP31cf6y6YyZmju9vMwRXH1uDxbedhqOG9jS3fXjTSXhLktNOH9cPT3z7GCz83xm417iXZHxx22m4w2g0F//iNHN7WUjFr84bjyG9uphpAC6dPBTPXT8NX5EahCG9uuDwwYmep+ArRw3G7756hG2b+GWF11gWUlBlSEeyxwrY5zu4hXYCQO/KMpwzQR9wFdVmt5QuoEy15JqRfS15TQ7p5YDhybv3HMScj8unDENZSDVlEznj6ZgB3fDbrx5h1s9UnnzX8jAqwqr523YtC+HUMXrvTzhj4wd1w/dOHoVnr59m86YVxZ42QeCUZgB9ElYkpODGU0fh3R+fiJCqIGrcS0NLDJURFSGFmTmynJ58JIdGvuADr0Ew/ZBqTJdiqJ1Uldt1tqG9uuCTDftQXVWGDXuazAorNwzTRlajuT2G40f3MT0GQP8xonFumxUrT4QBgIlDemCWoT0Lw9jcHsNWKba23PB05BY8nQETDYbsBZWH7bJPbU0vvPfjE8AYQ3VVBIff/hauOXY4Zj292Nynd2WZaeSdscLXHjcCo/vpHpvwWEx9MklGRvH8VIVhwuAeWLh5vznByE0qcjKyTxV6VUbMyTDp5jxky1XTavDSojos2VqP3pUR8xnWGzMuH76qFoN7Jo5jiLEdUdZ/fGNSRlkqu1eE8fi3jknw3sSpupaHzPEFmbKQak7ucfKLc8eiW3kYNzlS3wKW1xgJKaiM6HVM/C9+W/mZ96qMuKbcKA+rOGN8f7yyeJstzYFAURh6GEZ+VN+uWFbXgDPG97cP1nNuaPJx16RhwnMW742QSzQORIXsFBZx6HoZ5LGU+y6ZaJvrUFUWQkVYxX7jt339h8eZv6343ZPVa5UxRF1yyVeVhbC7sQ1HDO6OJcZgqxiDEURUZkpgja1RdKsI64s2JvHk3XoHQVEyaQ1SIYynMDgj+lRBVRj6GPlskmWg7BIJ4aRD+9q2CblGZAu84cSRCS+6PPgoKuvPHQsjiIZB1vxG9atKeR9Wg2G9WG7dvBF9qjC8uhJdy8PYdPdZuOBIyyN85XvHoiKiSlkO7ceWSQ2W8HjFPiGXSVaA5TEyBnz3pJE46dA+OH/iIFuZU1ERUTFT0rwfvLI27THZoCoMz3xnKn40YzSuml5jbp82UncUjj2kT8rjX/rudPzf14/KKg3x9EOqUeuYvCXqZ6qBWtmgCe/vw5tOMmWGz/93Bp673i7r/PiZRQD0RkK8C2KcSHH15C3P+D9XTzb/Lg8rNofHeZzGgZF9q8x7eO76qbj34om259S/ewXKwypiGsd7LhkYexjXFvfJTE+e4w1jlraQ4Eb17WqU1+p5nDfRks0AS5Zqj1kNnYW9QXGiKsx1rEI8w1vOGINnvjMVm+4+C0cM7mHbJ6Qq4Fx3ohpao+hWHoaiMDMza3s8DlVhpuNGnnyWiK6pqNCXTxmGo2t6Yc5qvZL5kQCEURXdv1PH9jO/mzm+P/4+Zz36S4N2wktwJtUqM7wI2U7IA1tuCCNrM/Lh5AZBRmH6S2g2LiIzg0OukRsN8UJHTI/eKuxfLzsS33viCz3TrHEOBj2n97++aRkGL548APQzdOijhvZIeGFyQXlYtWmyAHDvxRPx04bWtC/ckF5dXCO2upWHXDMgekXYmopw8vpYGQmZOV5uPGUULps8FD0luaJXZQRNbfqzFL+sCNstkzR5Mdju5snLRr4iYtWvirCKgT0q8IeLJmDd7oP4+5z19rxNnOPQ/rrh3dnQhknD7I3YHeeNw6lj+mKTkX1VpO+QuWnmoRjWuwtmjOln297YGjNDk0Udvf3ccTh7wgCM6tc14TyCruVhmxEvU637EQ3KkZIkJ6MqzFWu+eqkwVhaV4+hvbskSJAC4ZnvbGjFm8t3onZYT5QbslE0ruFv7+uT0/R3i9uk0aDpFEZeaG8ikqCyTO8Ojx/UDf27l+OcIwZ6PpfQF0XomOj2AsBPTjsUV06tsQ00iQpWWRYyV4+XzzOwRwV+NGM0tuxrTttlE+dqabdr8l6IhPSwNdFF/eqkwfjXvE04ZUw/vLtyl5n6t0yK+gmrCp789hQzpE9cf2D3cswwGrdvThuOL77UZ0i62XOz253G1ld31a+RagHxXFMRUTHcEarph09/dmpW1xcNYpdI8oZbdkjCKrMZeIH5rDlMDxbQ68oNJ43Ehj0HzZ6W8LJtRl46p1wnRUjrVyYNxuPzNyccp3Hg2EOq8Y0pQ3HVNCtNiCjPCaP7gjGWIG/KdImE8M3picd+ud9KSyLqfEVExXGjUve6upaHbPcgN+Aj+1Th5e9NT+pcqYqS4MkrDLhi6jBccNQg2yCtE/F+3zlbn4W1emcjpozojbZoHJvlFCsssVxB0ymMvJAbnNpbWUjF148Z5utcosII/VZ+IVWF2bx4sQ2wNwbdykNm7osR1ZU4bZy38DwR+id7T05NPhkRVTfyQo4ZN7A7Nt19FgDgsW8dg5G36pNInJVt6kgrLEzcS0jVE68tvf00VEZCWLG9AXe8ugLjBiYODIpnLnv08289JaFBE4OvXuOnixGvv0UyxCNKaeSlQT/VJRRUP48VkSKnco6EFAzoXoHHv2XNuxC/iqyt95Iir8T2iUN62O5PyIx2I88RVhXcef7hrvclZFG33uchfd2lSnGdL/dZ41lee6+AbuRlT95Zv1P1GlUlMbdM13J9DkcqAw9YdkI4TY2tMTNxoDwOJorWoaNrioGxA7phzIBuuO2c7CcbiEpywMgel24CjeXJWxVz9g+OwwVHDcbYAd1w9bHDkx2aeC7jbWmWB15d4u3dmDxc7zrL3VVnGYHUlc0y8vr/XQ2dcfyg7nj6O1NdjZywQ3ID269beUKYngijdC7A0pk42tDoK1IYebm+JRkiMQ035/ZILLeeomh8WyWPX5ZrRverQnlYwS1nHGa/hnER2eFINkXvljPGoCykoJ8RTlvhqCfVVRH8v++4h4eK+uPmyXuhIqyaureqME9jRAKVMVtytx/PGI3/5zGMVVxHhGnfdeHhKA/pA86iYRzVt8p8/uTJZ0lFRMXrNx4XyLnEi7LfxZN3Q1Qw8WMeMbi7qee+5rNMqvFWt9o0eW+V475LjsSqHY1mHHMyvLxAfvRDr3KN8ORTLSBe6vz50iOxdtdBdE3hJVZJzoKa5HewPGe7kXcaV8Ayos7oGkHvqjKsuuOMhONEo93i8OTdOPPwAeakL8Dq8YhxoqeuneoqOwGWJ79Nmn0b8mGoGbMMu1/dW1WZrWf5fccYTipEGUUjeMnRQ7BiWwNao3Hzmf3yvHH49iP6uARF1xQREUmuURWW1igq0o/NGPDoNcek3D8VzooDeJcIxDhE2mukqGxiEMpPhVRc5Bo3xDhGOo21lKksCyVEryTsE5E9+fTGTsg193z1iCT6faKx7pYmNYF+nP6/zZP3mG1DaPIaB44bVZ1UqgEsOaOxNYZelRFsvOtMz1FNn/5MT22RaQSLytyjazwdK97VaByRkALGdFvRGtXMBrU8rJrvhZffMlPIyPtElmu6RNS0FU5UsKa2GA4f1D1tbo9UyBVHEJSW54w0cUMMiiYLpXTDrMRpvK+KiIoP/udE3HXh4Sn36+zIck0yj1Z4v5xbS/GNHeg+uGjKNZKx9hJtpkr1emD3cpwzYSCuO2FkmqN0ZMekLI3cKN6X+pYoKsLp3zcZMXNYzdDIK0mia7wQknrdZYZTpGvyllxTEVbxn2sm46JJg9GtIneiSqeQa4LElGuaojavKhnmy9Aes0XdZILZRbZNxMpusE/woxmjE5KHOellaLVC3/eCV7kGAIb1zjyypbMgG+BkuU7kgU7hyScbKBSnkD15P/W6JRpHddcy/OXSI9MeI6iwGfnUhlfufaSTRgWH9utqLoUJWAELfh2ikMIyjvZSpSAJ0biUhxVoUsNbHlYxZkC3pCGcQUFG3icRKU6+S4pJKwLL44l71s+ToSh60qR0k6FyRU11Jd784fG2VZDS4RZdQ2SON09eR/bknfmbBKYnL2nPXjxe4VE3t8fNOQ5ekUMo09XfsKonVTvQHPVs5P/f9VOxVwpXztiTlwZep0lRZl4IOeQawHLIxBKPbmMkuYDkGp/IcfKePB5mefJBeN0hRbFFoORywMaNQ/t3TanbOxEDezlKsNfpkAdek2YtlAZedxqTu5LJhMzhyV93wkj065a+x2lGerXFfDsvskTjxfCKHrDX96dbedg230FIJ74HXhVmhl/feuYYX8cqUq9b3GOZaeT13lWq+QJBQkbeJ6KiNLfHPXkWwovgPBhpRVFg5s2R86QXK+L+/YSuEcnpEvGuyQN6TvhBPSqSatnOePdZM0Z50r2FvWxqj6fV1Z3IIaJeeqK9Hdld/SLqXiZyjcBvTzQkyVkR1S4X7TeNPHnyRYnseXhZZEI2bhUBtNwhxVpwwe9ErkIgXo5s8rwQFrKhS2t4OEfd/pakU+/1c+j/7znYjsqI6tlgy7+nX49UNm5ePPnexozrVPMHUpFpdI0zI6UfRLhzS3uiXFPf0g7G8ie1kibvE1ke8ePJA8G03F4nLhUbuQwR60yUS3UuqSdvbI5rHFv2NeOUw/q67gdYhmxXQ6stsd5Fkwa7ZuIUqCzzel0ekjX59McKqSlVTp9UZKrJy/eYqSdvG3gVkXnNUZSH/EUKZQMZeZ/IRt5PFAIQjJGXX+xcJjUKCs3MdljggpQI8mBdMglMbH11yXbsa2q3LaPoRJxib1M7RkgD6vc4VrByYq/XPmUQVTES23kzvGJyWKZyjRVd4+941SbX+LumnDE2rNo9+f3N0bzp8QDJNb6RvWc/0TWA3YPJFHG+sMpytlxYkAgPqNrDOrJEejwZeeOZL9i8H9VVZeZSj6n2BeArxFex1Wv/xlfUCy+9UZEaINNxHXGcX7lHvp5fr1teYlJOqAbovaYuHhzEoCBP3idyRfE0aUSqHH4SKyXD1Bc7gBcP6Hlqfn3BeJzqSB1LZEZFxLsnDwBDe1WkdAZkGSLdIuz246y/M+mhKgyIw5snL8a+nAvceEW8M5VZGHm/co3bsaInsrepHSen6F0FDRl5n8ihaF5aY3l2aCCavAgH60B6fEcYIO4oyKGBXgyPWIA+GbKx7p1irVQnNuclg7qoR/VwTxKKkDtiLqs0eUEYXL/eczZyjSyrikXE5eu7LSKeKzqOpSgSbJq8B7lGySIKwQ3xcnUkI08ER1hVzARiydJLyLZ/QPfUE5XkcMvuPqbWK1mONVlL96U/VhhMt2UPvSCek2+5JouBV7mBEIudyz2JET4mFGYLWYoscFsdyImszQWxCHWmkQJE6cAdS/c5kQ23SO+bDPkU6TKUymTrvAjlxbmIuxuizmcq14gOQDZyjd9AGPlYEfLcRXr//fSasoUsRRaMTrHsmECOr002tdwPotHoKJo8ETzmmrtpZrwC6edyyMY63UIYMnL1y2asycs4gHCm/KTTkBH5ZyqykmsyC6EEgPMm6ivPyYPmPSszT1ToF9Lks2Bgmq4w4PTks/9hLU8+P7PliOJDhKUmG3gNSzJO+lTY1t9+MqQqWcTJy3gx8lNG9MYz35nqKVW2G2IJRL8hmEENvP7p4okJ25yL5uQSMvIZcNW0GqzZ2ehr+jcQjFwT6oADr0SwiLztyYy8PJiZLq+MzZPP1MhnURe9Nix+Mp86EQvR+DXyPST5yv/Aq/VM3OyEvPpWriEjnwG3nzvO877yOpxByDVmHg6SazotPI0nb5+bkdqwyafw48nbZl5n4cnnY9Znm+nJ+3v/+lRZPXW/5VTTrLmQr7w1ABn5nCOP0AfiydPAa6dHS+PJy6Tz5GXjlbFck0Fd/OB/TsS+pvb0OwaAWN3J78CrLCVlE0JZaMjI5xi5RfeS0Cwd+Vj4lyhuTE3eg3eZLg49U21d1vIz8UqH9a7M2yIxQpP32+OwG/nMNflCQ0Y+x8gvYhCGWWjyHSk5GREsIpDQkyefpp5kaoqySVCWb4Rc4/edsRl5n0Y7mSf/q/PGZRwKmilk5HNMtUiTGtCLIDR+8uQ7L+k0eZl0BjjTFbtko1fsDkd7hkZelq8yTVDm5IqpNf5OFABk5HNMSFWw/JenB6Y/drTcNUTwpI2Tl0jryRun6OljIhRg9+TDRW7krUVD/Dla2SwaQnJNJ6OyLBSIHg9YMdDkyXdezPTNAQy8inNNGeFvDVPZ6IWLyKC58Y9vTMLzn2/FkF6p8/g4kVOYkJEn8obIYU9GvvPCfXjy6UIoh/WuxAOXT8Jxo/r4KoM88Opnzd9CUFNdiR+d5n+pTDk3kF9VK+R3KakcQka+gyF6BGTkiSA8eQA4LUW++WTInmqyRGkdnbCSuSdfRI485a7paIiFSmgyFOHFk8/V2I1drinNuig3Xn6NdjGtaVyav04JYw4eFVElIgqDF+8yV1KKfO1S9eTtRr7j3iMZ+Q6GGOSKxTNbQIEoHQo5q9Im1xSTNhEgYVv+mQIWJEvIyHcwRLhaLM8TKojiI1UEx2H906fBzgb50sUkTQSJPOaR6T36DU3NBTTw2sEQYV1iggfReUlleJ67fhoOtsVydu2OsIh8oXn+hmkY3MNf2GYuyMqTZ4xdxBhbzhjTGGO1ju9uYYytY4ytZoydnl0xCUFEFUuhkZHvrPzglFFp96ksC6VdFSobvOTN6ewcNbQn+ubwN/BKtp78MgAXArhf3sgYGwvgEgDjAAwE8A5jbDTnPJ7l9To9wpMnI995+dGM0fjRjNEFLUMxTfYhUpOVJ885X8k5X+3y1XkAnuKct3HONwJYB2ByNtcidCYM6QEAOPHQvoUtCNGpIUe+45ArTX4QgE+kz1uNbQkwxq4FcC0ADB06NEfFKR3GDOiGlb+a6XvleYIIEpJrOg5pjTxj7B0AblPifsY5fynbAnDOHwDwAADU1tZSyIgHyMAThYbkmo5DWiPPOT81g/PWARgifR5sbCMIogQo1bDJUiRXcfIvA7iEMVbGGBsOYBSAT3N0LYIgCCIJ2YZQXsAY2wpgKoDZjLE3AYBzvhzAMwBWAHgDwHcpsoYgCCL/ZDXwyjl/AcALSb77NYBfZ3N+giAIIjsorQFBEEQJQ0aeIAiihCEjTxAEUcKQkScIgihhyMgTBEGUMGTkCYIgShgy8gRBECUMLRpCEASRhAevqMXmfc2FLkZWkJEnCCIjfvfVI1DTu7LQxcgpp47tV+giZA0ZeYIgMuJrtUPS70QUHNLkCYIgShgy8gRBECUMGXmCIIgShow8QRBECUNGniAIooQhI08QBFHCkJEnCIIoYcjIEwRBlDCMc17oMpgwxhoBrC50OVJQDWBPoQuRhGIuG1Dc5aOyZU4xl6+YywYEW75hnPM+bl8U24zX1Zzz2kIXIhmMsQXFWr5iLhtQ3OWjsmVOMZevmMsG5K98JNcQBEGUMGTkCYIgSphiM/IPFLoAaSjm8hVz2YDiLh+VLXOKuXzFXDYgT+UrqoFXgiAIIliKzZMnCIIgAoSMPEEQRAmTdyPPGBvCGHufMbaCMbacMXajyz4nMsbqGWOLjH+35bmMmxhjS41rL3D5njHG/swYW8cYW8IYOypP5TpUeiaLGGMNjLEfOvbJ67NjjD3MGNvFGFsmbevFGHubMbbW+L9nkmOvNPZZyxi7Mk9lu4cxtsr43V5gjPVIcmzKOpCjst3OGKuTfrszkxw7kzG22qh/NwddthTle1oq2ybG2KIkx+b62bnakGKodynKVrh6xznP6z8AAwAcZfzdFcAaAGMd+5wI4NV8l026/iYA1Sm+PxPA6wAYgCkA5hegjCqAHdAnQRTs2QE4HsBRAJZJ234H4Gbj75sB/NbluF4ANhj/9zT+7pmHsp0GIGT8/Vu3snmpAzkq2+0AfuLhd18PYASACIDFzvcnV+VzfP8HALcV6Nm52pBiqHcpylawepd3T55zvp1z/rnxdyOAlQAG5bscWXIegP9wnU8A9GCMDchzGU4BsJ5zvjnP17XBOZ8LYJ9j83kAHjH+fgTA+S6Hng7gbc75Ps75fgBvA5iZ67Jxzt/inMeMj58AGBzkNb2S5Ll5YTKAdZzzDZzzdgBPQX/egZKqfIwxBuBrAJ4M+rpeSGFDCl7vkpWtkPWuoJo8Y6wGwJEA5rt8PZUxtpgx9jpjbFx+SwYO4C3G2ELG2LUu3w8C8KX0eSvy31BdguQvWSGfHQD045xvN/7eAcBtNeRieIZXQ++RuZGuDuSK7xld+oeTyA3F8NyOA7CTc742yfd5e3YOG1JU9S6FfctrvStYWgPGWBWA5wD8kHPe4Pj6c+gyxEFDl3wRwKg8Fu9YznkdY6wvgLcZY6sMz6YoYIxFAJwL4BaXrwv97GxwzjljrOjidBljPwMQA/B4kl0KUQf+DuAO6C/6HdAlkatzfM1MuBSpvfi8PDunDdE7GDqFrnfJ7Fsh6l1BPHnGWBj6A3icc/6883vOeQPn/KDx92sAwoyx6nyVj3NeZ/y/C8AL0LvIMnUA5KXqBxvb8sUZAD7nnO90flHoZ2ewU8hXxv+7XPYp2DNkjF0F4GwAX+eGEOrEQx0IHM75Ts55nHOuAfhnkmsWtO4xxkIALgTwdLJ98vHsktiQoqh3yexboepdIaJrGICHAKzknP8xyT79jf3AGJsMvZx781S+SsZYV/E39AGTZY7dXgZwBdOZAqBe6ibmg6SeVCGfncTLAETUwpUAXnLZ500ApzHGehqyxGnGtpzCGJsJ4CYA53LOm5Ps46UO5KJs8rjOBUmu+RmAUYyx4UaP7hLozztfnApgFed8q9uX+Xh2KWxIwetdsrIVtN4FOYrr5R+AY6F3R5cAWGT8OxPAdQCuM/b5HoDl0CMHPgEwLY/lG2Fcd7FRhp8Z2+XyMQB/gx7lsBRAbR7LVwndaHeXthXs2UFvbLYDiELXN68B0BvAuwDWAngHQC9j31oAD0rHXg1gnfHvm3kq2zromqyoe/8w9h0I4LVUdSAPZXvUqE9LoBusAc6yGZ/PhB61sT4XZUtWPmP7v0Vdk/bN97NLZkMKXu9SlK1g9Y7SGhAEQZQwNOOVIAiihCEjTxAEUcKQkScIgihhyMgTBEGUMGTkCYIgShgy8gRBECUMGXmCIIgS5v8DDR3eHuy/3BMAAAAASUVORK5CYII=\n",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "plt.plot(t_vec, eye_ratio_v)\n",
+    "plt.plot(t_vec, blink)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "venv",
+   "language": "python",
+   "name": "venv"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

codes/jupyter_notebook/rtn_2mdl_et.ipynb

@@ -0,0 +1,791 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "s4mSPpqO8oEz"
+   },
+   "source": [
+    "## Retraining 'eye_tracking' model for subject and predicting eye track (pixel coordinate)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "id": "Ks_AbdKIb-xZ"
+   },
+   "outputs": [],
+   "source": [
+    "from tensorflow.keras.callbacks import EarlyStopping\n",
+    "from tensorflow.keras.models import load_model\n",
+    "import numpy as np\n",
+    "import pickle\n",
+    "from sklearn.utils import shuffle\n",
+    "from joblib import load as j_load\n",
+    "from joblib import dump as j_dump\n",
+    "import time\n",
+    "import os"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "id": "T5XUNQBvcGf6"
+   },
+   "outputs": [],
+   "source": [
+    "# Parameters\n",
+    "path2root = \"../../\"\n",
+    "models_fol = \"models/\"\n",
+    "models_et_fol = \"et/\"\n",
+    "trained_fol = \"trained/\"\n",
+    "subjects_dir = \"subjects/\"\n",
+    "data_et_fol = \"data-et-clb/\"\n",
+    "sbj_scalers_boi_fol = \"scalers-boi.bin\"\n",
+    "sbj_model_boi_fol = \"model-boi\"\n",
+    "sbj_num = 24\n",
+    "sbj_num_test = 25\n",
+    "model_num = 5\n",
+    "r_train = 0.85\n",
+    "n_epochs = 40\n",
+    "patience = 5\n",
+    "trainable_layers = 2\n",
+    "chosen_inputs = [0, 1, 2, 6, 7, 8, 9]\n",
+    "y_scale = 1000.0"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "sbj_dir = path2root + subjects_dir + f\"{sbj_num}/\"\n",
+    "trained_dir = path2root + models_fol + models_et_fol + trained_fol"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "wGy7E9iA8oFA"
+   },
+   "source": [
+    "### Retraining 'eye_tracking' model with subject calibration data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "id": "bvzEN-j4cGFQ",
+    "outputId": "8ae84f7e-2cca-4f92-d03d-43e5e5f431a4"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "Loading subject data in ../../subjects/24/data-et-clb/\n",
+      "Samples number: 4200\n"
+     ]
+    }
+   ],
+   "source": [
+    "data_et_dir = sbj_dir + data_et_fol\n",
+    "print(f\"\\nLoading subject data in {data_et_dir}\")\n",
+    "with open(data_et_dir + \"x1.pickle\", \"rb\") as f:\n",
+    "    x1_load = pickle.load(f)\n",
+    "with open(data_et_dir + \"x2.pickle\", \"rb\") as f:\n",
+    "    x2_load = pickle.load(f)\n",
+    "with open(data_et_dir + \"y.pickle\", \"rb\") as f:\n",
+    "    y_load = pickle.load(f)\n",
+    "n_smp, frame_h, frame_w = x1_load.shape[:-1]\n",
+    "print(f\"Samples number: {n_smp}\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {
+    "id": "ygdBBcjud--4"
+   },
+   "outputs": [],
+   "source": [
+    "import matplotlib.pyplot as plt"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/",
+     "height": 336
+    },
+    "id": "x4GOLYKnAWR-",
+    "outputId": "2352588f-265d-49a7-d627-2ea272a4c1f2"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[ 2.92193524  0.12354425 -0.86240532 -1.57755936  5.01365275 70.57027415\n",
+      "  0.51105064  0.5061323   0.46361178  0.46345159]\n",
+      "[0.00530988 0.016     ]\n"
+     ]
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD7CAYAAACscuKmAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAABP0UlEQVR4nO29baht23ke9oz9dc69V7IkS0WokqhviUkxhtTh4jq4FGMl1HVD1B/GOAlBSVXunzZR0pRYSn+khhRsCHH0o6SIuMEUUzlWTGXUkNRV7R/9o1qKTSNLUazain2FZKkgO7q+55z9Nftj73fdZz37ed8x5lp777WOz3xgs/acc8wx3vHxfo6P2aZpwoIFC/7w42DXBCxYsOB+sDD7ggXPCBZmX7DgGcHC7AsWPCNYmH3BgmcEC7MvWPCMYCtmb639QGvtC621L7bWPnhbRC1YsOD20TadZ2+tHQL4VwD+FIBXAPwKgD87TdPnbo+8BQsW3BaOtnj3uwF8cZqm3wSA1tpHAbwXQMrsb3zjG6e3ve1tq+sQNCxwpmkaun95ebl2DwAuLi7Wyru8vFy7zvLWNO7/6zqmaUeh71TX7lmvTPdc6e7d72HkPU0zSsMoTVm6qH/veYWRsTHn3bntvGmbAMDp6SnOzs7sC9sw+zsB/A5dvwLgP9BErbWXAbwMAG9961vxYz/2Y6tn5+fnuLi4WA3ii4sLnJ2drf6/vLxcPT8/P8fl5eXqncePH+PRo0dr77766qsrBp+mCWdnZzg/P18r7/T0FADW0ul1CBMVFj1mDwHUS6PlxDW3RZTPNET7cN5aj95ga62t/g4ODm7U6eDgpmeXDT795Ty0jEhzdHRk0wYODw8tDVn+DBaGGW3n5+elQAVyJcH33T33btDbqw+ncdeuLnHNaT/72c/m5ZRU3AKmafrINE0vTdP00hvf+Ma7Lm7BggUJttHsXwbwbrp+1/W9rdBa28g8jnfd/9W9beEkeS+N0+pz6pxZD067R3tWWlDTVvc4P/6NZ5x/aw2Xl5drmknzUnovLy/LfuJ3tLxo04ODg43H0Eh/3gW4PtEGmSWp97mNK2zD7L8C4Ntbay/iisl/BMCf673kTGMm/uLiAgcHB6sKhB+e+elVXg6Rtw4Uvj/id4XZzXECNUmnaVq5DZnZzmndc26vcGv4HXUdKjfCmfNax9vwx52Zz2Z9dg+4asPDw8O1vNSM5fLimY6ruD/CBD1EeUFXtDvTEeWroHGmvsIxccbwzoWJsdvDxsw+TdN5a+2/AvDPABwC+J+mafr1zjtl0IsrwczHTO78YpdfSMoqMMSNrAyvtGneAG4wI4C1gRq0RjqmnYWY/nIcQpndDaRRZs/aoboewaiAcMx+eHhY+uwuDZD7sUoTC+AofxMB4KxGHissdFTgb2ItxDhygo2vgz+4vAzbaHZM0/RPAPyTbfJYsGDB/WBZQbdgwTOCrTT7XLAZlwV1Dg8PV357PO+ZapmPnfmkmX/em8bSaTr1oS8uLtam+qZpwpMnTwBgdd+Z6+5+5D9ilo8ENUfnjkcDm+oC9d5xwbzMhWCzVP15fs/5rnqfp/puw38P+oIWXduh/rMrs2fWZ3TGe1G/uGbXscK9MjuwXhH14YPZDw4OcHZ2tmJMF01m6ACIvLS8i4uLNB9ldL6ntMb98/PzNeZ2c+Mxr5sxtvt15bvnI1F3TZfVL/JwDOiYOQv8Vffcc+0T9dk54BUMdnx8vMqD/fn4P/zyg4MDHB8fr+Wv8+wOPT87aNJ+c0G1EUHo0BunQYcKnDvz2W8bm0QY9w0uWLIJo28ydTSi4av8qym028acfHWqz4Gj7y6YxumqceWUhMIJSQ7w7isWn33BgmcE9+6zV352pOFfNuPZrB8paxTOfHamr1samU17qOnO9zhtNW2W3eulqfx61ZBqjt/GVFxGg+apbaTTSEoX085z3my681g5PDxcc7Miva6NUJrmaOc51tQcqBXjrC6OHYxgZz67C4a11lYBlaOjo1WamG/mTo9O1DlUzdcFimINOqfj5/HLgbi4p8EznQd3a995UOu11kFpqXzj8Ne0DTW9zgW7erv8s/apzOlKeCmTAzc3L6nA4aBuXB8dHa3usxLg8RF/R0dHZbD36OjIBvFG/Wodz/qeixNpnaNNqns6LjnQPSqcduaz8+o4hkZaDw8PV/5Q/PWWVHIZVZDLNbAyuNPcLpiXlRHX2Z/CMQTn4xhW61Rp50ywuLTZAOyl6b2bCdqMftffytS6Yi4L7jLzA1cBOxYUvBIu8qmggtSt5NMxpPEAtxAngxuXHI2v3l189gULnhHsVTResY0vdBcRZGD+Rom78OcU2/iN6iZU+cwp4zZ92bvqyz8MmLN24N6ZvVrHy/6H+rRhfrHPHiZ+FqhwG2fU13br7t11NofuNqYw/ZXZ3vPRFSPz2dW0kwsY9vLL3ImMFmbyylWo8nB5ZnBBR36WbShhuP0N1ZqMCm68MVyeMY4raGyI3RaOOVQ03yuz9wazC4oBr1eI32fmrhbeuKCaK4eDgfHLzB2r5fRdt/CGfVIWGnEP8Ezl0IuOOx/c+X/ut8fsvb7Kyh6hpRJKozMCFaPHcy0zEwqMKlrfs1hcHEoxIuTc86gPK70IMI7s8lt89gULnhEszL5gwTOCnQXoMpNXr8MsYp857o2Ywtn0FCPbAMP3o1w14TVfV4dtoAG0kQBhz+/PzN547v7fJG1mWqsprHXswR2AwXPt/MzdczRl9cs2tWRtuMlmm5HDLdgdDJ9d71V1AXYcoFNfVoNjzGBx4OTZ2dnqHleYfZqA+uecb+a7Rzr+P3azPXnyZC2dEzZuU0IvwDT6rBJmPbjAmQ7MucGxkTKZTm7veLbJWvJg4Nj4oszMf3oIhDsIY2QXmmP8YPie0HPz6txvbqGZUzj8bgQQNZC4V8xeQZk97vGzOXmN3JuTZjQqXSHSb6vxN8ljbjS/ujenzCw4tk0baHDQBQuztPcNV+cRq7S365HvD1lE3RQLFiz4Q4G90Ow9Ce/87pG8ttEc2bTUiGl9X9h24cq2mq5XNtNX+cXqs/fWDvAfP6/M+bvCXeQdbcJLbXuuwsg42Cmzu40jGuCKNLGAJf7cAhg97dXNg6v/nplPjh4dONnuuE1QzSVzmhF/U+noMc8IHVUZGUOP+K+aRn1qx8jhQ+tCKnfyqm5y2ZT5dRzw75x97BxHCITfrQd1MCL/w8NDu/tyBDvX7Fk0np8zY7p07n7VAMz0PWQroG5Lo/eYvGJG56u5aHeVz1xmzyybXowkK08Z1n0RxjGX/q9BR5d2NDCpdeEAsIIFT6TtwQk4DjS7NCrUVCGOYPHZFyx4RrAw+4KnBruMj/xhwM7NeCA34dU843nN+It7sdc9/KGDg4P0cEE1u6opjRHa+f/R6ZIMbopKn4/mM/JOL+A4YrZnZrCmcQtORsrTfKKPdXOUfgDCfRDCzbOP0qTz7FmQcNMgLtPaM803Cc7ulNkzHxxY/yKMO4UkwPd0E7/61xzhZL9n06h2b2BWqAbcnGcj87UZlClHBqmuf8j6hftV+2EuzW4h0+Hh4eqP+z8Ccs5nj/tzmMqhilO4gzbm+PEuEs/PXYR+dNXeU2vG3+V0yoIFI3jaxuBTy+wLFiyYh51uhHH3RuaN457z5/h/Z/KGGcfmFvtk/P/Ikt27moJzU2gj02QjCy5cbGSkPODmV0ozM17zyNKM0MvuWZxtEPGZ8Ms1/2xOXX327LRivuemX7N+38XZ8Xs5z67MzJtS3K6dat6d/3jBATNwdArnwYE+twBHd7NF/tkuN0Vv8FaDXX3ekXereeuof7ZGIQuKVte9uIQLWvH/+rxX3uHh4eqrLtG/x8fHaweQss/OAVpgfVGNy98hW7Ph7rk+Y+EUnzLL4hWbYFOBshfR+E3hNPucxuTjeOcEqar7rvN7AzrTiC4qP6LNXYRY6dH/R4OCvbQ9GipBUOWlmlsFhtPk2ZnqI1970XycJejorALNGQ33ZQksPvuCBc8I9l6zj/rE2RRehU3WGPfy7/l/zi9WbZm5CpkGrPKf2yYj6Jn+mrayHnquidPmvfPWKnp601ShiV0MYyT/24Qz17OpuRHcO7M7MzkLymW+ZbzDn0HunSTjfH1OF/47+7iuPDbznD9bffgi0mSMrfe03jGPzObk2dlZ6sqMBL+ydA4jcQRXTtTZ+etcH+DmRxzCZ4//1WTnhTM6785+PvvT1ZdxnHDkuJL65b1FWSMMmW16iWd83fs8cyXMdr6oZtM0yhAcROP7mgen1w7sdY7m6VZozdGiVUAuC04yw3C9Oc9N/ONRwZDFNKr3M+3NjBtwzO6YHFj3yTlNlJFF6XXxVXX6UNxjTX95eWk3r7j3XJv1wNZFXHOeVVyitGpmUbFgwYKnFl1mb629u7X2S621z7XWfr219oHr+9/aWvvF1tpvXP++5e7JXbBgwaYYMePPAfz1aZr+eWvtjQA+01r7RQB/EcAnp2n68dbaBwF8EMCPVhlN07T2CV13uAR/iIH96/C1Yl17vM/+Ovvs6pur6Z7Rp8iCRfosQ2XCVvk4F4TnkeO94+NjW5/KhHT+9Mg8cPjYnM6Z1+5/9a+BsY0q6oe7uqhLoB9RYGiMIMrQPF27xXw5/7m4Df8C/QNGXIzgLtBl9mmavgLgK9f/f7O19nkA7wTwXgDfd53spwH8MgaY/fT0NH3OQTA+LpoDLSwg+GSaOAGWA2QcdOMy4l7lHzN0YFdwEVw38IHaF3PgRSRcr8yPdkFJV+csUq70A1fMwswXPnAWOMsWvVQCNws4Rl5aX47OBzMHDbEgR9MyMh+eg3Gu7S4vL1dfKnLtrW0XOD8/v1GPrD2csuC6jIybVfqhVK8X9G0AvgvApwC8/VoQAMBXAbw9eefl1tqnW2uffvXVV7tlVBKvSvs0w1kP7k/Tunc4zZyBMJfeKu/qrHZm3LlnrM+pyzADbHDOe1bWXbT1bWK4pq21NwD4xwD+6jRN/4afTVdcZzlvmqaPTNP00jRNL73hDW/YitgFCxZsjqGpt9baMa4Y/Wemafr569u/21p7xzRNX2mtvQPA10byUh+dtTP73WyeX9MAADg7O8Pp6enqYxFsUrGZzfPt8VtZAs58y9IFnEk5B+wvOx+78uX5/8qMV2h5+kyvnX8bZm+Y7zFdFn9hSscvm9m9s9o0jqNw58fx3vbw0fUwE36/0uZR32pKjfPKFuBU7awY3fy1LbrM3q6o/SkAn5+m6e/So18A8D4AP379+/FeXpeXl3j8+PHqOvxzZtjw6YORIw378hcXFzg9PV3l5UzDaZrw2muv4ezsrBvA4nwZuqDBzWlr/Zy/XnV4NlD4GadVGnuM7gbcqH8O3JzHffjwIR4+fLjG2CcnJ6v/Dw8P8fDhQxwdHeHhw4c4Pj5e0eI2EylOT09vLJY6Oztbq5POqbOwiQU4GhjM6qMBW113wXRnG4p0PT1fV4KL+515IOjkejD02W1+svl7AfwFAP+itfZr1/f+Jq6Y/B+11t4P4F8D+OGBvG40rGrnaFD+7JM7RtqtZnMN04tsasfNwajkDvTSOk3fSz8HI3n2tDsH5FibBqMdHh7i5OTkBrPrSsWsPir89Nhw4PWAGlsOMehVk7strXytGjxboqp+Obd9dqrMCJSWuZgT+xiJxv9fADLq3zODrgULFuwQ975cViVjZmKyxOZtglXUeY65ymk2jVpvIsHnmNCbovLZbyv/0L4cL2ENnu05qDR7pNNprk0xspR0zpi5rYMpsnychVCZ8HNxr8zeWsODBw9W1zwvHh3Mn2gOn11NeDbv1d/ngFyUxSYhuw0XFxdriyx4zlT9sOiIKsDofGr9iqj6i1mn9YJzkcb57Gr2OtNTy3BCz/mD5+fneO211wBgzW/ngFgcLnFycrL6gkn0ZfjfIcR5PLh6ZoOdFQEH6CI/7tMK0zStpYlFW9x+ul6DF3bx+ABw40TjiCEwOC4Qc/UaUFZkPrxu7NnWZ781uMGjB0jE87iOwRE+OmuMeOYCbK21lb8YDRsdEQ0aA1H9JtVUwbAabFJmdx3FPqMy+iaoGJa1Ig+ozHJxwkYHjmpH185cRxeNjzY9OzvDkydP1tI9//zz9hNNAd7pl2k+FjK8qEaDc1kbZjMcvHiLrUumg8dv0MvtE/XMyuYFO6poHNx46u2EW9EylGrBggVPPRZmX7DgGcG9m/Hsw4T5pXON/H+Y2mwe6TRd+O9PnjxZ80+Pjo5u+NDxxUz+cqYLGob/HSZb0MHmW6xxrpB9gaQy4zMf200R6nw6m69ZENTdd1OQ6o8CN9cSRFvG/601nJ6e3pgSivaMdte2DDx48OCGnxv5XlxcrPpB97IfHR3h+Ph4zWdnMz7rJzbXo/664YX7ntcAcIxG5+s5fxcEVHeB+5z7LhsDWnfOO8O9MrsGZNTvdZHYaNwIzqm/Ev7648eP8ejRozUfShs/GFQ3VHC+IVQ4UMg+mvpVXIaLsroAkevILGqs/qTmryu4+GQW54tG/u5ZtmhIffbRiLSbNVEfOoJ98fxNb3qTFULx/PLyEicnJ2txl1jY8/DhQ7zwwgvd8l0MQscg15GDwRz30fUeEWNSIcfQvucxnzE458+7RiM4d3JycqPODnthxmfTHyqZNWLMnaedyHn1ptZ0EFZTNI7WXv4B1TKuPpvk7wa1i6xnAqVCJjBG33XWiHvumMyhtyLNtan+Zu3Dvy5Qye/z/9XYGS2vwsiimZExuBfMvmDBgrvHzhbV8P/q1zhNoNIx8zErLZnRMYf2KC98Up2GclKerQ6ewsmQbRbR/HmKUE1zR3M2xaTPGJWbtUkbjlgHPB40vds+G+Z9nGnAdLk98NV0nJv+GrVoRrRr9KFzSbkv3Xvcz5GPLhHeG58duNnQma+kTKVpomMjSBJBtPBfpmlazelyZ4VPHkGQrOGz55oma1yOASijK036TqC3iIfXHXAeyqC6KKRidKXBbRAaxYip6to1+kfTxQCPfo41+FHnx48fr5g90p6cnJTui9sYw4u1dFxywDYWZem45fx01WBrr+8pmBO7UWXH+wF4YVhl8u/0dNkRzaAVdoM/GitrQEW1XLE3sJ1PlpURz6Nj4i/zTZ3W1nuangWSo0e1fk9L6cAfEUpz4gm9vAK8Sk3BcRpd1BIBrOz0V1dedixVFkeI9ozxUlmUmZbOrrNYQ9Q7/qKcOdH4xWdfsOAZwcLsd4RsznXU/xvdtrgp5kbW3bvZ9JGbRhqJyFdwGqt6JyvXXTstPqevRq2mbbDNBpjAvS+qcWampgHWg1TaGdlgcsEknkNl/ynyqYJgvNPOmffqDrC/F+C12lwvZ7qp6armJS/qiPd4vpnjF67NNF4wwmCZ0KqgAbwwO3nTRtDG7eXiHPxOtEeYznEwSebPHhwc3Jj3dnEWdV10LQG3J8dJpun1vRtcJ24fN8aqaeL4VTeF25Dru7eLaoB1jZX5x45gZVLH8ArulLiO9JXkdpHzaHgNhjFDnp2drT7HFL5yBHM4T+7ArL7c4dwGOgiz+EOmbXUgap114Ct6AsMJ7/BvXTCMhbFGmjMG5o1QHDhz6bUN3eef3EIlrSuPpSg/hG+msOJdbWc+TUffYSbWdojgJLdjBOhc2YrFjF+w4BnBwuwLFjwj2ItPNquZk00XsRmVzU0ydLorfp1Jm02XsCkf5pXSw2n5zDU3VQLcPNBCnzOyaZ+svupXZu2r0DbXZ9o23Ka99M6kjnd1XTcvEon2dnu4eVot3CV2i9j9Yncx3CqmVdvYjbuqPXtTj730zoXS9lKznn+BsYDuzpnd+X7O59FTbTSdgzInb2LgQFek5YEUAyL+dwOAf4Gb/hPDra2O+xWjO2ZXhmO/PfxJ3mHm1uS7dlfmdXENpdX9r3lqgI7peO6559bef/jw4Y0AnQal2G+OnXCx5oD7CvBxIW1rrWO1lkPHKZ9u5PLP2s8xdkDXEbAPH6cCcT68qGavAnSMTaYqekG16l6lFUfBefSETaBidM0nC7htg0wDb4pMM+k1l+vSuIHPaXvlbIrRMdALPgZ6mn0uKmuhaq8eFp99wYJnBDtfLqv+9ByJBtzUHpmGn2P6a57Z1NDImvFsem1kkcSmWiF+VUOqxt3UysliLHNo0mcuLb9zm5bJiOXEZnvVTiPjao6Pz1Drx7XNnMU2O9/1pj6WNjKfvBl+Gf+xwIhTTeP94+Pj1bx33Ds5OVnz7bQzw8+N8oD1deqcl+s4nYcf9c+5PA4K6smnjKCV/fY4eDH+zwJujkFHzEM3V6+CI+qogTldY6AbVdj/5EGsH4PgNuzNmffgmDkbj/rMvaP9G/nz4idd5JPtk+CTeqPucc31H2X6nWp2t6NM4TpZG5AHVMWMnEbfVwnNZSnTx3ON7LKg0Hpynno/axuFMijnq7MC/I775boydKBqmjnWQKaZGCwAXL00+ryNz+ro7zE7X7u6syLK4LbLKsM6hnf15/aKtlhOl12wYMEaFmZfMIxtZzIYt+mD3yZdTws2ab97N+N7CxXUX4q109UBAJeXVx8gePz48ZrJrWY9gLV7BwcHa6eFMpzboHWIdAHnI48EZDTw4tornqu5GIcoqBvD72cm/AiTbJKGP/yoc8X6SeWYNw6a+DrSZQE+5670fPaR6bSRe9l6ejWpNe7i6qj9Hr56+OhsxusXbNUN2iufnRtJP8fkwM/Df46VahFsi5NNomGAm0JDA2fBGBw3YCHAm0xcng6OsbUjXB7xjh47rYtq3EA6Pz9fBTHdgRdcdzer0EOvzplwC0Z2C2QiXTA/18tt7HC+ugakRoOOvej6SB3nBAG1PzKGZYHGY1mDkxzg4/sjWMx4wW2alwsW7BMWZl+w4BnBvR9e8fDhw9cLJ/PTgc13PWMtTEL2zdl/ifT6ldgKm5h4Lg8G181Nz/QsCX5HD7dQVyPaIytf/feRumocoDdNyj6pruPWde66yYX9+wCnd22iuIs+nmO2a/7qmrl5cTXHeQ+B2/BSTb3tjc9+dHSEt73tbatr3TUG3AzUKfHxDn/COfxW/mxQpOOTQhkj88fRyKOdzSfh8Dx/dJh+cqo3gPWe80cjrxCIIUCjzhzgjLbgfKrB4fxBpdvNpcfCD/XZnQ//4MGDtXq6TxBzmdomPcYdWeW4iYDPxicfxhHPeWefLi6K9lIGV7/ebYSJ/EZdz3vX7L2AVcbs8cuWAAekRqLet73BJIMu7NH/FZWmGkG0I2vhqi0qWtx9x9x67TQUv1tt6x3JP97ZB1TCIXs22q7V4iFNW+XtMNx6rbXD1tqvttY+cX39YmvtU621L7bWfra1dtLLY8GCBbvDHM3+AQCfB/At19c/AeAnp2n6aGvtfwTwfgB/v8ogTJaATokxKjNezX/nU95FVN1ZIlW6TaTvXHrc1KGmYa1RLU+eSyPnC9w8903vq/k6imjLu7DM5vrwm8QEnCs053rUcu1hiNlba+8C8J8C+O8B/NftqtTvB/DnrpP8NID/Dh1mPzk5wYsvvri61krw4hk9SDB+Hz16hEePHuEb3/gGvvrVr64xv24aGWkcFg4xqJygcULELaKJwZyZyr2Oq4JhTYIxHJTkNQLRDurHh6Dl4GXFQG7g6jx4zJWzqR3XvKgm/FAW9urWuTo7hXCbzKlw7cEnJAXYhw5BFIu7tA+fe+65G+6L+t3RPioYe+Nljkk/qtn/HoC/AeCN19dvBfB70zTFcS+vAHhnQszLAF4GgHe/+91405vetCJKI60cUOIVSZz2m9/8Jg4PD/EHf/AHa9F64OZqolEf7+DgYG0HmZOsGVNw2pHy5kjokUUivIpQr7ksjnNwm3F+Ln6iAS4OHkV5buGHY3YNoroytX3cbM1dxl6ccOGgMKeLOsY78RurMlt7PSibxTniWn11vb6NuEX3zdbanwbwtWmaPrNJAdM0fWSappemaXqJI/ELFiy4X4xo9u8F8Gdaaz8I4CGufPYPA3hza+3oWru/C8CX5xY+umRzwYJ9w9M4brvMPk3ThwB8CABaa98H4L+ZpunPt9Z+DsAPAfgogPcB+Hgvr7OzM7zyyiura52mcWZ8zNnGXORrr72Gx48fr8z4MPMuLi5wenq6+kJHtVgnygoTshck5PjBNv4jv6OmauTPGDHZeGFR5O38S/bZwxx1m4B668B1rlwDcUo391FmImudlZE2/YrspsiCdhpc1PEQPjzXXw+bAF6fhlT/XM12Z+5vg23m2X8UwEdba38bwK8C+KneC0+ePMGXvvSl9HkwbQyM1l4/bfTBgwc4OjrC6ekpLi4u8OjRo7Wdc+fn53jy5AlOT08BvC441Keey6xZAA4YO7m0ynfbzouBoIFJZXhddcf/V8yXxRd40M+JTkcAKysv8u61yybtlglOJ+h1rYcLlPHpw0wTb8ZqbX3XWtzT/Dgw5/z1TeusmMXs0zT9MoBfvv7/NwF899wCewsSHDNyAKSXx0g5zzJY4MXAUgZ37zzt0KXGu8Iuzf/d137BggX3gntdLhtr1TOobxdSMA7iPz8/x9nZGS4uLvD48WM8evRoZa6fnp4Of0hiU60/arZuki/gTcpqaiyeszuhG394+kfzqRY1MbIDEau6Zu/cBnptPEd7Kk28QEn7W+fKXVnO3Ncv2mSbYXT+/bZxr8x+eXmJV199NX3Og48HMJuasQHm9PR0xezhsz969GhNmPBCGw7kxbXLnzu4mgN2gyGrc5aXIvOtGVk+cd/NBbv8q0UaI6h281UCxDHH3PKrNs/WSmRlZGsqeDy4L6oqY0Y9NCB3cnKytraAFz3pJhf98o2LmTg/vgqwMu5ds/ci5KqZ2V8P7R6f69XPOTm/Pjp/JDjnBsg2Czh6dc0GZG/Lb/bMpYu6a76bLFu9C+y6fMAL2WwhFTN6XFfPVBg460CfZ4FRpWMuFp99wYJnBDs7cLL3TDU0L43NNFyl+TZFJeE3gVocnFc2E8FwU1V8X6e6sm2vGp2ufNQ5qKyZKLOKQ+yDph9xz9w7zj1gTc/3szzuEvfK7BcXF/j93//99LnzlTlgp4toYtNBmPcxBx95sJmv+XJnsJmvi1M4bTXnru/oPV6bruvTs7leN4+vp5L0gl5uWo2DR70BlgWMVHAo7fw7klbpqp5n6D137exO9wnwLkGeG9f/+ff4+Hjlk/M1t71euy/VMg29Oo8Kk3sP0D169Gj2OzHwWasHw7PPrtH40WizMrzOP2eS3gWmHLNrmqBdg2nK/HrqCW8q0XtBJ/9miIE5EmR089O9d3ttkD3j+EKF29SOWeyEx0RcuyCdMr87QVf7S5naBeSymEpP8FZYfPYFC54RLMy+YMEzgr0y450/pz57di98eM5Dp/ECGkjJfGcOfmV+e2Vmse+pi1CyjTqcxh1euMm8tKLyoUemHzmuMVpOlvc+ws29q8kd99xe/mwRjcuP89Rpucwty4J/Pdw7s3/zm99cu6cDzzG1+3NBrgjIBWLuPVAFe9wGkGBQ9d8z+vXkmsvLy7VNKgBuHMvVY4DK/+WNQNU71fuM0YEz8l4WaKoi8Q4uwj0nUj5nwU9WdrY7jT9nFQzuAnSVsOD8tSxHvxMSe3mUNHCT4XrRa73vouu3Pd12W8im7QJOgzDYogD8iji3YKhiqCogd3m5/gnqu0YvOLhv2NQyuQuLZpM8F599wYJnBPc+z85r41U6ZeZ5/KpJrte6Lp6n4jgPzl+n9NRNiLxceTp1FvllyEzyTEo7rafLOedMMfbyb61Z+ntxAt1ow++FqTmXvsql6LXbnHxHpim5XI2pOH+c6Yt3qliO2y/vltwqzW5evhp/98rsjx49wmc/+1nbWdUCkx5GN5Do4hb+ZcZ2fr5zH9w8+Zz81A3JFqO4Dt9kbXs1gOJ5zy/uzXG7HWHZOvIerb301aIg/SiF3s/or+6Hsjg4OFj1fRzG4fqKmb3KN4tLVG2UCcMqAH7vAbo4TirQ89tGBnM1SF20XbUyv+e0tXtfGdtdu3c034r53WILrXOvfTiNChFlwmp2YKQfeDUa513tkMvy1XzcO5tq+NH0PdpGEWOqEqRZnCWLyPfKcrj3AB2wvgIrrrdB1fjZYHFBLW7kypVgl0LTZQzMyAaptkfVLjo7UGkP1Rw6pTja/lnAkVfkZXS7ZbcundYjcxGyPCNfrVvlFmTLTVXoOvoqqKAbtZaUP24LO2F2YD/mXNX8dzQxU1fr29Xnj3vxO6KBR++rUHL/8/vOTHTmYs9E5zrzOyNR/LmCfYSpqjXt/L8yndLCVoSzDOfQxNjEFdo2TfXezphdobu3nCSeM58MbB7AqRbyxNdQK2Z3NM4Z7I4h+ZfL03Kq/OJ/1nqZMHB5MMPr+/qFWhYAmQaOZ0D+JZYenLXB+wXYdw9mZlqYtkp4OEGwS2wytveG2cN0yTZ1RGBkkwDeplBfnE+5UVOe02fIzLNMQiuDKV29cp3pqUw64jow3RmzO9fA1XWuz+7A9dFZE/4NIQTcPHI7nruNPkpf1Knna/dordq2Kn+knBHsDbPrMVSBLKiUrWhT9ISDPs980lgNN03TmrbggeD8fkej09QO7Av3gnk9QcPgAVSZ+E4YOD+WNSb3XXxjLs5S5+OddMlttGm0/9HR0WpMaJ9n1hP3nTIxMz3XJ9P22l6Zeb8tE/ZM/Sr/yipy2BtmZ0RH9TQgd5jzrappOH3ea3T2SVnDMzNmebln2a++xwOtF82v8hvx57m+Pe3P91nTR98dHBzg/Pz8hiaNj0xEe2qkOnB+fr627JTrkR1dFowb1+qLt9Zwfn6eCimuQyBzL2+D2as4QiV8+J2whkcYfi+ZPfPxeozBGGX0zB9W0xnom1wZbSPvVXlVjFwJqU1oZS2n5joLHx2UMfccz5npgdf3LpycnKC1dkOTa5/3FiepKZ8hW5OvJryjw5n5WtYcpq/6Q9uW27iKJajCqsq4d2avpm+AeT5Qhh6jO83oIu1ZXpkfNuJ7zWF6Z75ndLhyemVy3lk9MuuJnwVTKMPEH3/xhDeOcD5s4msZc7Sns8SyugPrU3sxNpUGXbyjq+S4zo4WLq9Hk1pySksWrxgpYy81e4asMqPBnCyPzBfM8ttESweqQM+I5TLHL+9pkvjd1Apwz9mtcaY+a9VIz/fdYqHRoB4Ln02YKzOrLy4ubsRpHHOrAHCCVNOO0Nh7NprXvTJ7dGK1KiqDVmhOcMKZ+NlyWZ1S4zzCPHWYK8XVVHNMN+LXa9mKkcHiBrl7n+lT/53TZn6t8/lZ46uG57yzpb6ZRTg6ntz3553WVmEUbebckcqS6iETECP92KvzU6XZ50679bTwnACegzJsvOs0i+u8bTSqalItJzP/ld5R9MxF5286s5SDSlwHXnOu02QaNMvWvd8GetNzSpe2p5vzz+IL1ZTfKOZYmTtbLjsKx3iqETPGqoI8LtASmjt2zykTq9ZnDcXvO7r02pntLj3XtWfqMyorRAeo+1/LcVOdmaDpXTsNH/TyJhOn3cOsZg3rmEa/ssp1yNpEhRKDd6fxcz6cxFkZ1XLfEcbfxmVU3Duzb+O/cFpucG20TRuI83ONn2k0/t+V74RRlsYxvPq1labumf49Zs3KycrjMlzfqvCoEAwfdLCprNearjLnq37jgLFbvenGGNOk1klA14qooLsL9Np4rz4SMTcPl9eoqa+DgzVLSOsRUzyzMno+Vs8s5nSV6Z0xXubCKL09jPrCzKgh8HRRUEYHsL43QX/ZQmNr4OTkxC5lHXGj+H4wfM89YOsmaHJrCdQScVaCpnXm/yie6kU1czDXj4+OdRouGpjTVP5poKdNe88q9OIOWdpNYgGZeV+Z/S7PytpQy0FXILr8tb8ijWr8gDKKmt7qFjHDK/S+0/4qaLIgnjL2bRwBNmf87z2zZ5W5rTXy2vGsldwgqsrNhAM/c4JiE2tHTefMUhjVcBXUXB2hK7uu0mfuQ+Ye6By5y7Myr53W1TQZqoUuLFgODw9vLPvVpcKOBidUtsXeMHvld951eXyP/VaX3jF89o7Lf64Z7dAz+7eFcw16mn2bcqsYhKZRH1kDiAxnTldLsZ2vHu+7pdVujDCTRrCRPxse6XS836UvH9gJs88JWPXuVfk7AeLm0bkj+ZNSjk5mbJbUMSDcDIAOzAxuAPJf1KPyo7lucwfQNgNOBRkjmyN3Qi8Twlm66sw8NaeBm6fpME1V/Znhua6qteM+pw2/nseOjoXM4sg0vFo0t7Y2vrX2ZgD/AMB3ApgA/OcAvgDgZwF8G4AvAfjhaZq+MZKfIhsk2UYVlsK96RRu3Grpq34jjp9pw7JfBmC1uePi4sL6lz06GcoMPOXjVvptwqC96LAy12gZKpziT5fPbitU4rfXpxylZ2bp+dSaj2OsypJTayPGBpv2bGW4sqpgoSqdzO1UjGr2DwP4p9M0/VBr7QTA8wD+JoBPTtP04621DwL4IIAfHcwPQD/YodBnmTDghnOMzgzt7ocWVl+vmrceMaHnuCfqGoz8jtIR6Z0vvqmLkTFwj+EzrV3FHbQe+j/nHQwVDBpRd6ZJt7/26sj5VT41a39diKMMqppdFaCbDp7bR11mb629CcB/BOAvXhdwCuC0tfZeAN93neynAfwyZjJ7IDogM62vy73xjj6r/DdOwya1vpMJh9Damld0ZryTMejcwBWwbs4xIzP9ut0zno1o0BFfPMAaSBlT3wsm4l/+U+Z3glT7S9vAWWBxT013pvPg4GrrLW9u4TlztdoyzaqB3FEtH+XxPXdQZ7ybMbwrrxfMG9HsLwL4OoB/2Fr7YwA+A+ADAN4+TdNXrtN8FcDb3cuttZcBvAwAJycnJUMGKkbvLXGtkDHcppF97YjbQubXujrOtRaq67iX5VWZlI6GzKznZ/xdcs5DV+w59yjT/lk7MQ2xLNcJ5yyIB3jffZThe1YMWx5Bg3Ol5mr0wAizHwH44wD+8jRNn2qtfRhXJjsXPrXWLAXTNH0EwEcA4IUXXpicRq6YpsfoygSVVnfPR5h0hEmyMh0DcBrHAJx2tGMzRuNy+H5lwmu5ztfuab7sL7R7MJpuH3WMpxaNHg/m3q2i82oSq8DJjrAKOPNbhVHmbgE3t9WqRcH7A5jGKGdTjEzivQLglWmaPnV9/TFcMf/vttbeAQDXv1+bU/Bc6TR3ZRyj8icz0+c25zejLMdwPTM605qMEY3KeTlT2tHD71T5VcKvR4srzwkFdgE4eJWVnZn7PTeA01XuXtATyKLlKgDUTXTCTN1GfXdTdDX7NE1fba39Tmvtj07T9AUA7wHwueu/9wH48evfjw/ktfaJphURR+MzgHMqy8s1Q/Lq4IjgCfvAzBT8G6gW+uhUkOblBp6mrcxBZxK6esXRT66M4+PjVbu4AVSZ3Y5ebuOoqzJoXMdW1jiXTqfJtL7Rrlz/ON337Ozsxuk4zCxMN9MZeXHwLqCalq0iF0WvPsvNbaL/c73iXabXCTtOp/9rmzmMctlfBvAz7SoS/5sA/hKurIJ/1Fp7P4B/DeCHRzJypvu20LzcvK5bJstpOViSmbdZec5UG6lfDJxMC/D/jiErF4GZlQcVD6ZMw3FejuFdPdw7PVOe+0YHMbcL9xH707wuXRm3aht+rn46MzL/H+3d61dnqbnPhHHZYbrzu1nQTttI71UYYvZpmn4NwEvm0XtG3leMMIJ2Fg+AOX6sk6KOFud79TS6+obceZnACJoijWoDzqe65zRt7x0u35mojt6MSRzc1FC27iDyrg6r0AEdZfMpvxcXFzd8+Ii2V2avCjrnw6uWZ7pU8GXrPsLaUzouLy9xenp64+OQyvDPPffcmiXk3Agn1B12vsV1xBRhxqgGjzNxgdrfrwZ3pmkDOhBVg7r/uT5sEjrGy8qu8u69G3RnWiDT4qMCdq7FljE6WwFMAwvKSBvMHwKM3UKeFmVk/roT9Mzo2WKXqt4avQ+cnZ2tTrt18/BHR0c4Pj5e0/I8pvk8vxGrY6dr47UjnbaZo8kd5kYvXYP1GLAXOGHt4OZ/49e5OJUQ2MQVUkGY0bKtm8VaXRlE8+drnf/m/ACsCcfIm6P0cf/k5GStrfXT3BnNTHvQxoymC3DmCMFMQGRWFNPixo0Kvr1ldpZkQL0abnTgzdHoc8EDwZndOi/M7zB9au7NMcfnxgR6qHx0rccm5XH+vVkP1eaq1QNsnke76Vdf9D6Ph2qcOT+eaeRNLdUpSApniY1E7xVqOToBWOHeD5x0c5fcca6hddDcJhP3oD6u0jcq2VW4cTCoF0/gcpjxsrJ7z92zyNcNtkoYqcDILLPoexd1ZwGTRfK5PJc3cJOZ9bjqaZpwdnbWHT9aJ54m049MjLhUyuDR7xx74LpGHTl/J3A5WFm5ZoGdnBsfUG3Z6wQ27yrtrQzF5UZjVmXFezx4WFMo/VVemTZmsz4E2uggrJi4N+Di/UqDZAzvaHE0ars8ePDAWgycrzI4M0AW12EtH30T2jem5eJd/hjn2dlZKkgzq1K1KE+9KTNzndx6++jvOGknNlJlrh7TxgIhmJyFXTU29mY/O3BTazsrgK/naNU5Zmg02pyFPNmKKh4I2aBVE441l/6pBnauQsZQWq6714s/KJwfHu+7CDa/p8ys2p0FgLaPCsmIzHN8gI8XC5p0bzk/z1bjOZNZ+1JXvI2Y1RHwGxmf1fhioVZhp1+EcRXMpiGq/ID8pE7XCGyaBZxGcs80n4D7aoj+ZhYJz7GqmcpaztV7rq+v9/mZCwQp1Lzm68ysPT4+vpFO/6KOfK2Mr+Mipt3UQuIxEW0eJvPR0RHOz89X1gD/6n3VpvE8+iTuKc3cVpVr5Ji85x66uXcOWO69Zmci5zA6v9uTbCoUMnMpm6Lp0ZCVp9qKn4/46tlAmWOpBHrm/0i+zr92f9m7/OvuqTXkmFxNf52OZdpYkAbzA1jNxYc1UB35HFCFwJbFwcHB2px5KJSe1p7bj5Xv3sO9B+gAr42zSmeVqObbAdwQACoUwufTr4jylIqW0QsOKoPzPefPcZlO2+qg1vpxvlkb8G+WhmnN1p07upW2TOipFlIGVssg0+QKx9Ss4XQRFtMZzMnvAK9bN5VLo2Y/9yMrk2D4rB6Z25qNbe5PZfCRgPFOFtVo4zsGcn5alScjC0BxXsz8lVvh6BqhN6Nt5JmaY8pgzs9XcB17NLCm1h1gGZNzGVX+OvgrAebyqtqI+5M1fNClWj/S8Q6yEPgslLSPR4KRnG7OeHRtkI0xHRdzsRdfhNGDK7J0+g6w3hEV0yrz8EBQIQBgZd6phnd0Oo2uGxiymMLh4eFqcxD7nlwPHogafc6YLRMO/I760IeHh6tNMmqRRGArotqs/appSQ18cXnOR3cMoPVSQVT5rC5CzfVWAaFLb4P+bC5b66z9zdtlR61XbQNt28xt7QmCnS+XBeqpIMdklVnE6LkJKvnnmOxapg5UNyC108PMYxodTXGfp1v4nZ5FkmnpCIrF79HR0er76cyUAFZrs09PT1cMr+vSgf7KQmZwpknr4gZzT6hxGVF+Zi0wXeyzHx0drbWvKgQnUNjkV2HNTKq06Hhz77g69pRZhp18EWaOqRtwna8+S4/51edhJuIGzqLSbtVUNVXoOo6f6zWXrUyjEeI5+fJBEVGPaZrw4MEDPHjwAMfHx3jw4AEODw9XTB1bUSM/1ijK7NpmStPJyclavlFOMFf8AlibeaiYQ6+5rXmMRV/znHzEa+IdNu2nacLx8fGNesUf+/lqVXI5KqAZGptwml1dTMXIzIliZ0dJj5o0jKqzR8waZnK1GHigaiNyo1e0s8DhuU/XmS5/LoOlujK5MnvPZA9Nenx8vBqwwNVil+eeew7PP/88Hj58uGLGg4ODlYaPPE9PT1eLUc7Pz9fMeZ6y0nqxBXF0dHSDydXK6EWvHdRyUWbX9fPBlGxFqWuiwjfq6dyC6Aett17rp6JC8GV1GtHWGoTduwCdQxbgyoIeFVzQBLjpS/I9vc/X7Lu5evRozN7TSHWvPs4s1jrqIFETk9NfXFysGPfs7GyllTSYFmldWcDNXVfO3w/tyRqe82GNXkX3q/YBXhd+2k5Mm4vXBA3VltN4l5WG064qiCMNuwhOKcypJ1+PBu12vhEGmD9vPHcQVAytaatrDcIwLRwgGnVL5mowR5uWp4EqpluZPRj99PR0NRUVJ9wwncDrpj/fv7y8vOHj8jusrSNfjhMwLRygc4yW1Z+v2fTN3mWzmf1r9tnjWTAqa3kO0rm+drRFGarZ2bpwDO+UU9UWvfG0s7XxlS82F1nn6yBUDefy4YZ1Wl07mvPcpB5zBNccza7WgLoJwOsbO87OzlZMeXR0tLYyzcUelAYeqMzIYarHdezN1ukvZ004hmcLamTwO6hAUbqZORWxCo8X4GQBu/hlgeH+j7KrMRlw43kOdrrrLeCmp/idCs5kzzR5JSlZ+6lA0OCYSvhMEKjJxqakC37dBtS046CS1vX09HTF8E+ePFn51ADWAmah8XRQaiBNz5fjc+aC6WNqLxjr9PT0Rh0qMz7rN+6XuJ+5czwG+QxCFWoamIt81Md39HHfa3yILYKIJ+hS20jnxnQlHCp+2ZvDK+J6boRRoYw+mnY0D2bQDD0BxYPSDZbM53ao6qi+ow46DagF40fEHaiZHVgXXJH2wYMHa6Y8+9GZZuX8ON+srvE/06Ru1EiQy9VJlVLkGdo/gm3xvrMW1ZXKXLy4p7ECxghPjLjC98rsMWidCc+dV5n4rF3VrFPNXVVe07ppI2YyNkfnSNVgbGXu0La8GYOtCsfg1bVqdG6XKEstjbOzs5UJn/nJfJ/zcEIwfPLnnnsOR0dHeOGFF3B8fAwAq+BcTMPF9k6n0dgKCPqjLaNsbR+2mjQgqlpb39XZEA00qgk+TdOaZRK/7MrxuGZXxblATsCNxn3mYKeaXf3BuOd+N0GP4Stz3mmEEU0xh97KDBzBCD2cr2q9GJy6P8DRwoM3EyzB7OHbPnny5AadITDjV4U7m7RubNyWu5NBrYQIXLJ7wG2nm2hUUalAVg3P5bEP31NUVUAvw06m3tyiiVHmZl+HzdDIt2eabzN41KLQ/ICbZ+CzL6mmIXc0B4fYmmAmc/Q6rei0i6NV65Axtz7L8omyY/CHS8D+PwuYqC9DmQvADUtHrTF2V5yFl9WX8+dfro+2u26Z5XpppD7q49wyth7cF2KifKZFaXbz+RV2suut+j+QEc6mUhXYq7Cpdsii+/G/Bpa04ys/Vf1adVEyn8+B/eIeo2fYJEbAtEXb8CYTDhZyfV1dmXnUbM8Y2Ll47vmcemVuDf/PgtyNEafh41f7PerN2jvre41N7JXPHnBaXBlFGUbT8WBgZglJyXk4Sa8M5spSGjWNDi7nfwYt/BUWpiloVn+YAz8jgUttt8iT6cvM9TmoBJbGJmJlna4SY+2rNPGg5bbOBK3+jc5NVxq/qmPQwpodWJ+W07auxiGXpz47m/XaNjrWRvp0L+bZXWBiVIMBecS89y4LhirvjNl62kQR+Wg65/dFx/YWiTCtfB2uQRbs24ThezSoWamBtQAzZVwrNADL6VRgjjD6qIWTMbxTTMz0sbgo6OA+dRZPvF9ZM1EuM73T4KOzWDvx2bUB3FQUd3AvIJFVlOdGdYDwfR5Y6isGra78iBZHXjxPzWkArKLP0WF6ZhnPb5+fn6993MDVw1kYjBgkzny9DdN+RKtk02dc72revhdz0CW8zuR3tEcZyjiZwmFmcv9fXl6uzqmP/uUTcMLCUVp4rGudnU+u49zRVGEvFtUEMu3pnqk0j3sugObSZRqd6dOB6jYtqAmWXfMuKwc+HTW0si5RrZh3hIEzDdNDZhU4Judn2t+VC+QW7LAb4pAF7uJZBo2D6Fy9G6Pat9wPvMQ4NHwIbK6DBvC4TOe+OiUX7/P4zdpYsfN59sDcIJsDm72Vv5a9O2rqZmZnb8okK4efabCH24vfzfJxGmvUdO2lCcbgd5QmNueZOeJ5rwzVbJW574TcNuNok0VdVT9oe+hKRjXLR91PLm9kzAV2tsWVMccfnQtnivWYrWLIyn9WSTzi/yk9OkXHA8aV7cpw5akfqO9XDK9Mpe9zGyv9cT9W4KnmjPplArSiUe/PdUscMsuzEp5KP8dM2AKIdOzLZ8ovo43HgPvuXJXXvCU4twD1q+aYk5oPUM8tzl1hxKj8UEeL1mGuhlAmcUyT0ZiZ05XUZwHQ+9M6Zr8jVk0PrsweXZtgWyUyWob64mq9lcw5MLbn1GMvjpLeFs78Un9MB0XPL4vrDNV8aq8DuKPU92KfjRdc6KDplVn5vCPmtP6vmt2VHdrMza64e0zPiJauzHhgXMBmbcZtP5oPn1PIQk8X33B7KR1KuzJyNVbnMP1TyexzOtitqnLvuWmNOSbWKLIVYwFlCP5lZndMx+kZjnnj/8y6yt5xdDpfPQa8CgCtv8YjKhdK6bkNsz2gU17b5MOzODENxzNHepDpNL2+1j4wMu7mWq47m3obRTVQez6di9Y6jQyszwnr9MocZNrCmdXKpFEWL50NfzemeOJP66DCgbWNCjiu6yjjqLAJ2ng9fCwc0nPmjo+PV/+zQHDxCabFmfNct2zVmoPzs921G1PMnI5OjalwnVTz8//6RRoW+k6hqfnvFFaFe/XZ5zI6YxuJOzoYnNbP3nVBpuy5ImuHTJvrPSc0XFqtH9dzDqNrGarJWXMzk8e1ni+ndai0vcNtzNxk5alZ3KOF4QKUkScfLc5tVVk9PXrnYkizt9b+GoD/AsAE4F8A+EsA3gHgowDeCuAzAP7CNE03TyK4JWhDcIerNov/+V0ndR0qgTTXfKx8rqw8/p/Pdjs/P1/T8NU00Rz6eq6K+tx80gz/xYKiuA4NH1H4qvzKQhsxrXWmJROkmd+s6ViDZ2U7K7JK46xLdb2qOEvPNx8RFN0UrbV3AvgrAF6apuk7ARwC+BEAPwHgJ6dp+iMAvgHg/d3StkAMMJaMGXqdmf31JOw2lkkF1QisAdi0d+m2RdUeamGo+a2mPGtuPlxSfXlH+6Z+eCZQ3WyGq4+2g+bl+ryKb7j6OJdKwX3q2ljp5zqOjodRM/4IwHOttSMAzwP4CoDvB/Cx6+c/DeA/G8wrJ0YqqINEzaAK7jTUQDWwNaiUmWZBL99zg6RHq9KgX0YJplFBx23D0IGVDa6MpoqxVZuzVtfjovWAyQwaJKyChgE3BkbcLa33qGDReIHSXaUNS4yZXmnhtuZrp4iqfuuha8ZP0/Tl1trfAfDbAB4B+N9xZbb/3jRN59fJXgHwzoSIlwG8DFydU57BSSfV0BExHaD5xvtsyvM9p/nd/xEwUxOvMqe1fiMdxaadDoJ4roPE1XnUD+/dyzQNW1osmFyaqi4V3ZUZ3UvD7aJ9NJLvHDdpxIR3pjnTyQqkWrjF9FVj16HL7K21twB4L4AXAfwegJ8D8AO99wLTNH0EwEcA4Fu+5VumStLKezeuNaikA0Sn2RRcdqZt2G9iRJ58aEak45VMKoHjt1pOq0IkfHU978zlEwKQl9ZWwTcnIBzU0uFvrIcGj9/Y5MMRdxUEzsrp0ZEJgoy54m/ERVCt7AK0ca+3oapHY9yr3EPOX99lOnvLhyuMBOj+JIDfmqbp69eE/TyA7wXw5tba0bV2fxeALw/kdYMJVKtlBMchCDxfqSZrthzWDTAeeKMaLjpdzzzP6jMaVAladA80n8l+eXm5tjFGmZ3bJODaITRdz7xXtyKOgY76s08ez2KHX/yyua27ATM4SyVjsCywV7WBS5vlo1NhlSLpCQGum47Bqhw31eraYAQjPfDbAL6ntfY8rsz49wD4NIBfAvBDuIrIvw/Ax0cLzZiMO3rE1Kryr0y7igYHlsxOoKi2zPLkueHQwFV5anqqL1+Z7C7iy+1aCTjWwi6GEW0R2tx9zYUtnJ7PXt13DOaYmX9H2jbT4o6mTRm9B7erTRHtNzJD0MOIz/6p1trHAPxzAOcAfhVXZvn/BuCjrbW/fX3vp3p5aRCCf6/LWqtMz8yMvGJJaVzzyiROD4xpW5a2msfh4eGaWc0DWpdMxmopzscti+yZY5F35HV+fr76bhvX9+BgfXkt77KqBKhqGva1OfDGc8VsxvPnnVhAjARStc49U330Nxs7TnBkzFwJHH0/e+7qr2PE3deFPJxf9ONcph+yraZp+lsA/pbc/k0A3z2rtGtUml21T0+SuTQc5NhUCgI3tQXnp1K58s1VE2naSjM7qIZlxq78Uq2bmuzOv2bTXRnZrYMfZW6mYzRNZZ5r+jkLqXqmfuUnz4VaWiNQ7c4YDRADe3JSjT5XLeQaxpk3nE6PehoFM3HG2Hpfy2Ca3If7WHiMdBTXNfJif17bS0/e1UHLwbzIP8pwAbjwueNXI+06XehMeI01xD3t10zzch7bMLpqf8fIerDJqBZ1ddRnfF25UgoOHHO0fpryz1UpdrYRpvJtGZUEZPNT1y3r/UCmTfk9F9XXtdjsGwZTx6CI+9FBmUTOymfooAwGik5mIRCmfryTuQlxL2P2CL4FnazlWSA4RneCb3QwjjJpVqfqPb3HbVr56vFbrUPnIHEVE+nBWbpubGfjvdcm935SjRKZmfRxT30UNtFZO7pAS+ThGBfI96azBuRffp/v8dc5OW8e6DolNyIAqimY2EkV5+DxgAOw+jijo5cHZrRR5BkanM/M4zQReWcTP55rHVXwxb2qjlUarl/PIspM7yzK7fKulAz388HBQfnNOn1XYySuHL6vbRf9rXXoCZh71+xqcgI1kTpwgJuSjSWgk476TsBpHe7oLGDjBkM2LRQ0sNbN6lsFgdyZY1wnFTbq1rA146yaYF4121W7sPmuFpK6Z2zW9pgzM5Urs7vKi38d4zut3tP0UZesD7UttG/c2KzcWX6XxzZblKMWIrDjLa497e40dGXK6IDXfcM9813fySLpPO2VDT7XwfrFUFdPhbMu+H2NazDNkUbPONf8WSM7850HlZqvWr/KQlOMmOYjmr+yBPT5SLCtYujKp86CthUDMg3VOMgCcXtrxjNGzfi5DB/3gZvmutP8GV1ublNdiGCuTAM4cMzAuRjZO5qmOumWGTTTmK5t2QdXk9zNMHB/VPWoZidGNL6+0xO0I+a+a5NsPPUQY8KdCefAZvnoeOTxp7SO5rXzDzvy7+g73FG9qYcRxmSwQHBCQq2FbTASZBoNRFXmYM9UDFTz4ny/ink4OI0/qqUZmVvlrucwOXBToM/BJu/06GGLbQSZ78+493PjdfDNqQxwU4I531+fq7/D0AZSBleNHhgx5UcwEoHOUJmU/NwNhMy3rvLWe8r0o4LJ5TEHvUDeprSMMEwm4EYF323QEOlUyOy1Zgf6c6ZVGhUeWYDHBacycy1jcMfc22iDgPrAjhZnIlaBmYpRexrVtSUPQCdcsw9uzMHoAK/SOktsLtM7JtL8M7oUI1ak5j8qbObEjwI7ZfYeM1eozNPIU31WDtxUHaR+ebyr1gE/63VU5derm6HSWlepBUYisZmAcmmcGTwyAKs69jDClM7dyJ7x802ZnoXrSNrq2ZwxzgE+p4T4f46TaKwmw841O7DeKD0zm9/ZJH93PfKuC9j18s0GWiYcNjXbs/d7wjCjaZNYxNy69N4PcFszrYAXRMwI21hco6jG7SbCby7mBBV3MvWmv9qZmX/u4KK0Wl7k4xqmisoqU2dznk7KujpzeSO+Omt1ly9Pm2m+2YIhV2ee3lOGV8E0IggqX3oOdNYiwLRpek5TWVMZRmIbLr3rVx2/Sp+a5D1k2n4k8Azc8+mygZ7pM5Kul+a2pGoW2HPgzuRlpVWU+zbg6soD6K61GzDP0nrWsUlb9d4ZyXPnml3/z9JnyKRtFbWfi8xsdzQ6X0+1YhWU03SjA8Plo/5/di8rX2nt0Z/ROleDxTs9K8mV0evn2xwXSk9lFThf3LXnNnyxlz77HMk2d6Cwr5T5epvAuRv6PKMn68BN/UoWHnPaZQS3oaE3jYSP5juK+7BoGFkkPwu69fK6bexMswN5cMNJumxdOe82U/+dBx1H4of8G9NpTAvDRbH5HY20u7iFIgtUOt88e9+t4Z8DNwugsQBeJqp9qwHNnm/J0WjehOPq1qvXSEykF9DsXWe0zZ1z30T5bZLPXkTjGdlOIWZeNY96K6tGn/Vo6UU9M02tQULNc7SzmdF12u82NUE2yJ2G4sBlZsFosNPB5TPXiuK8epgz1TeKihF7VsYcAVG1y16Z8U6z8iDY5MAJJ6FVOOhGBSfdR/xZoA5+ZYNoGyGjZesaaf519Kjl4SLbI9CNHtn7bLHNnfKc46dX93VabtMZgkzgO2yyzHYTbKrM9oLZXZq52g7wU3YuSOK06Rzfd2QDxih6pqpzcbiM2LjC7zka3VLS27IGeoEkt0ahyqNH1xwmqVZKAvPWNrD1kZVTIbNaFXc1P7+zwyvuKn9gfUBVEn0uc7q4gP5fzf2OYnS6b4757tYYVNHialD35rO3mWdX5nL1m7sRp1eO4raF4dxltiMW0Eg+ip1o9jnY1D/rTa/MpSMYnc3gjMEZ2RLIudpNmXXucs7MdK8W7WTQFWx3FX2PPDPanFuh6Gniu2boTcvYNHK/dz57dn+TCoYU7PnKalWMaOXqeTaH7abadFDODSC5KbbM/Iv33I4ovVdtrhmxLpjp52rvXhuMTFdto9lHaHY0jlpujgGr6P99TI3uRTS+2mEUWicL3DhGj/QxLdcLWmU0RFr91TPlnTDIBusm2q8Xxc7gNtawj6/BvU0xMsU3Zxqrx+i3EbnOZk4c5i5nzawyR4t71gugZouaeu2yF8yuqNZ1u/tZkA7IO2p0bbqCNWTWqbc5FRaMPmr5OKbK6B3Jb040epSmkbJucyrRIdq0Wk/h6Mrq37OEqsU2GW2j/TPizgA7YPaReXS+B/jpI9cBvGgjNLtO5WXBu5GorBuMc6eS5gxiZ1WolTMqAJwbkQXjqoGbDSq2eLJ35tJ923B9MU3TjbMGGSokq6BaD9nYd9OmLniq740GcgN7t1y2morQ+W2VllmwyEnJTcxpFhSj0veuglfVMuI5QanMZJ7LjD3BMHdg3jYygc/948aIi33c1dQYY0RTz7EAgB0dSzWCaNDqFJQRk3JEEo82WBVgydJsgxEa58zd6sCdOyXUwygtI+9tisz1Ysyx6ICbgnWOxdOLtWTT0b2xnR1uuVfR+BFk5vscQbHp1IXSsAu4wamaUS2GEQujWhSSlb0NdsHou7Icovzqvo7l+x5jO9sIowztvokGvH6MMm+EySSri2jqirOgofq6aqTZJcMzXMBohOnvYzpH83GCqaJjmwUyTquNCu4R7Z5tG9ayY9POXOuwChKOKKy5Su3eD69wvlsVOR+Z1skGipu+y/7cLqtZDTmgMUe0bxUJjuvMVO2ZdFnd5yLaqgoczQ3GVfv6szK0vF76bS24OVOLVToXXHMB1Ko892wvp956WpMb4vj4eHW/F9V1jVR955ppiV89rdOVVdHvLJdN4ebBez7gHO3Sy4vBae9y+m0ueoHYKv1cVL56L1/njup4cluARwTeqFWxEzPeTbMB60RHI8bniDhdJaGZUeOroy5y7zbFOMZ3ZWSLINz685HBVQ2inrbW/OeYds7k1ny3jX3MXV23Cea08ZwY0BzXoloHUpVXjekedKlzz1LbuwBdDEA3pdYza5w7oPdc/pp2Lu7Sv1fmnVvOyCDK0rhNL3eF3gKTEVT09fz2u8KIsN6G0TWvqk57w+y9teMjARWn5bL7czRvRqtbBOHmZbdFxfBzfONq2Sajtbb2ued4t0q/KUam60bo5f9d+9yGv91zb3r5Z9bTCMO7eNLcdt8Zs2dSaBtp6/ZwM/M5gXEX0v02TFd1OXp0Vh3PA2VT2rIlmc4t2xab5udiKrueURk10zcVJHOwE2bfZKrDYSTApJtWtslX71eLLUbP8h7diLGJZtXZCE4/Mh0GrPvubm03u1g9OkZod9pubmS/stwqv1lp2Bbb+ONBh7NyXL4jNO/k3Pj7xhxGv03cpY87By6AM5e2u2yzUVr2pT3nYJ9obvdp5rTWvg7gDwD8f/dW6HZ4G54eWoGni96niVbg6aH335mm6d9yD+6V2QGgtfbpaZpeutdCN8TTRCvwdNH7NNEKPH30OjwTZvyCBQsWZl+w4JnBLpj9Izsoc1M8TbQCTxe9TxOtwNNH7w3cu8++YMGC3WAx4xcseEawMPuCBc8I7o3ZW2s/0Fr7Qmvti621D95XuaNorb27tfZLrbXPtdZ+vbX2gev739pa+8XW2m9c/75l17QGWmuHrbVfba194vr6xdbap67b+Gdbaye7pjHQWntza+1jrbV/2Vr7fGvtT+xr27bW/tr1GPhsa+1/aa093Oe2HcW9MHtr7RDA/wDgPwHwHQD+bGvtO+6j7Bk4B/DXp2n6DgDfA+C/vKbxgwA+OU3TtwP45PX1vuADAD5P1z8B4CenafojAL4B4P07ocrjwwD+6TRN/x6AP4YruveubVtr7wTwVwC8NE3TdwI4BPAj2O+2HUNsSLnLPwB/AsA/o+sPAfjQfZS9Bc0fB/CnAHwBwDuu770DwBd2Tds1Le/CFYN8P4BPAGi4WuF15Np8x7S+CcBv4TogTPf3rm0BvBPA7wD4VlztHfkEgP94X9t2zt99mfHRgIFXru/tJVpr3wbguwB8CsDbp2n6yvWjrwJ4+67oEvw9AH8DQOxOeSuA35um6fz6ep/a+EUAXwfwD6/djn/QWnsBe9i20zR9GcDfAfDbAL4C4PcBfAb727bDWAJ0gtbaGwD8YwB/dZqmf8PPpiuxvvO5ytbanwbwtWmaPrNrWgZxBOCPA/j70zR9F672R6yZ7HvUtm8B8F5cCah/G8ALAH5gp0TdEu6L2b8M4N10/a7re3uF1toxrhj9Z6Zp+vnr27/bWnvH9fN3APjarugjfC+AP9Na+xKAj+LKlP8wgDe31mLb8j618SsAXpmm6VPX1x/DFfPvY9v+SQC/NU3T16dpOgPw87hq731t22HcF7P/CoBvv45onuAq4PEL91T2ENrVXsSfAvD5aZr+Lj36BQDvu/7/fbjy5XeKaZo+NE3Tu6Zp+jZcteX/OU3TnwfwSwB+6DrZXtAKANM0fRXA77TW/uj1rfcA+Bz2sG1xZb5/T2vt+esxEbTuZdvOwj0GPn4QwL8C8P8C+G93Haww9P2HuDIj/x8Av3b994O48oU/CeA3APwfAL5117QK3d8H4BPX//+7AP5vAF8E8HMAHuyaPqLz3wfw6ev2/V8BvGVf2xbAjwH4lwA+C+B/BvBgn9t29G9ZLrtgwTOCJUC3YMEzgoXZFyx4RrAw+4IFzwgWZl+w4BnBwuwLFjwjWJh9wYJnBAuzL1jwjOD/B0BzG+XtkItYAAAAAElFTkSuQmCC\n",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "# Displaying data\n",
+    "smp_num = 2\n",
+    "print(x2_load[smp_num])\n",
+    "print(y_load[smp_num])\n",
+    "plt.imshow(x1_load[smp_num].reshape((frame_h, frame_w)),\n",
+    "           cmap=\"gray\", vmin=0, vmax=255)\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "8AzfSjqn8oFD"
+   },
+   "source": [
+    "#### Getting those data that looking 'in' screen"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "id": "DWA57YKb8oFE",
+    "outputId": "bcf52386-ae85-4ff9-af25-1f9415bebd56"
+   },
+   "outputs": [],
+   "source": [
+    "# print(\"\\nNormalizing data...\")\n",
+    "# sbj_scalers_boi_dir = sbj_dir + sbj_scalers_boi_fol\n",
+    "# x2_chs_inp = x2_load[:, chosen_inputs]\n",
+    "# x1_scaler_boi, x2_scaler_boi = j_load(sbj_scalers_boi_dir)\n",
+    "# x1_boi = x1_load / x1_scaler_boi\n",
+    "# x2_boi = x2_scaler_boi.transform(x2_chs_inp)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "id": "SJ-ojxa18oFG",
+    "outputId": "92885191-edcc-4664-fa23-d57e7c2b770e"
+   },
+   "outputs": [],
+   "source": [
+    "# print(\"\\nLoading in_blink_out model...\")\n",
+    "# sbj_model_boi_dir = sbj_dir + sbj_model_boi_fol\n",
+    "# model_boi = load_model(sbj_model_boi_dir)\n",
+    "# print(model_boi.summary())"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "id": "KGqOn_458oFH",
+    "outputId": "9080e5e9-11ac-4e42-b39f-7c5aa5181ffb"
+   },
+   "outputs": [],
+   "source": [
+    "# print(\"\\nPredicting those data that looking 'in' screen.\")\n",
+    "# yhat_boi = model_boi.predict([x1_boi, x2_boi]).argmax(1)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "id": "ySFqmNyd8oFI",
+    "outputId": "75cea127-1407-47f9-a7a9-ab1be5bcdf09"
+   },
+   "outputs": [],
+   "source": [
+    "# # Choosing those data\n",
+    "# x1_new = []\n",
+    "# x2_new = []\n",
+    "# y_new = []\n",
+    "# for (x10, x20, y0, yht0) in zip(x1_load, x2_load, y_load, yhat_boi):\n",
+    "#     if True: # yht0 != 1:\n",
+    "#         x1_new.append(x10)\n",
+    "#         x2_new.append(x20)\n",
+    "#         y_new.append(y0)\n",
+    "\n",
+    "# x1_new = np.array(x1_new)\n",
+    "# x2_new = np.array(x2_new)\n",
+    "# y_new = np.array(y_new)\n",
+    "# n_smp_new = x1_new.shape[0]\n",
+    "# print(f\"New samples: {n_smp_new}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "tVhzAjUV8oFJ"
+   },
+   "source": [
+    "### Preparing modified calibration data to feeding in eye_tracking model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "id": "o2yHN0I9cF6o",
+    "outputId": "adfd1167-179f-48b5-c016-91e9249c46eb"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "Normalizing modified calibration data to feeding in eye_tracking model...\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "F:\\Education_Study\\University\\MSC\\Thesis\\EyeTracker\\venv\\lib\\site-packages\\sklearn\\base.py:324: UserWarning: Trying to unpickle estimator StandardScaler from version 1.0.2 when using version 1.0.1. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:\n",
+      "https://scikit-learn.org/stable/modules/model_persistence.html#security-maintainability-limitations\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "['../../subjects/24/scalers.bin']"
+      ]
+     },
+     "execution_count": 11,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "print(\"\\nNormalizing modified calibration data to feeding in eye_tracking model...\")\n",
+    "public_scalers_dir = trained_dir + \"scalers.bin\"\n",
+    "scalers = j_load(public_scalers_dir)\n",
+    "x1_scaler, x2_scaler = scalers\n",
+    "\n",
+    "x2_chs_inp = x2_load[:, chosen_inputs]\n",
+    "\n",
+    "x1_nrm = x1_load / x1_scaler\n",
+    "x2_nrm = x2_scaler.transform(x2_chs_inp)\n",
+    "\n",
+    "j_dump(scalers, sbj_dir + \"scalers.bin\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "id": "eNwyQAwL2pGS",
+    "outputId": "9812cee4-8a01-4734-b479-81feed0099c2"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "(3570, 100, 100, 1) (630, 100, 100, 1) (3570,) (630,) (3570, 7) (630, 7) (3570,) (630,)\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Shuffling and splitting data to train and val\n",
+    "x1_shf, x2_shf, y_hrz_shf, y_vrt_shf = shuffle(x1_nrm, x2_nrm, y_load[:, 0], y_load[:, 1])\n",
+    "\n",
+    "n_train = int(r_train * n_smp)\n",
+    "n_val = n_smp - n_train\n",
+    "x1_train, x2_train = x1_shf[:n_train], x2_shf[:n_train]\n",
+    "x1_val, x2_val = x1_shf[n_train:], x2_shf[n_train:]\n",
+    "y_hrz_train, y_vrt_train = y_hrz_shf[:n_train], y_vrt_shf[:n_train]\n",
+    "y_hrz_val, y_vrt_val = y_hrz_shf[n_train:], y_vrt_shf[n_train:]\n",
+    "\n",
+    "x_train = [x1_train, x2_train]\n",
+    "x_val = [x1_val, x2_val]\n",
+    "\n",
+    "print(x1_train.shape, x1_val.shape, y_hrz_train.shape, y_hrz_val.shape,\n",
+    "      x2_train.shape, x2_val.shape, y_vrt_train.shape, y_vrt_val.shape)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {
+    "id": "PqdkxbYFcpuf"
+   },
+   "outputs": [],
+   "source": [
+    "# Callback for training\n",
+    "cb = EarlyStopping(patience=patience, verbose=1, restore_best_weights=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "id": "dpT4PjQUcprr",
+    "outputId": "a3aa1dfd-0002-41de-a4b3-9719987a58ea"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Loading public eye_tracking models...\n",
+      "Model: \"model_5\"\n",
+      "__________________________________________________________________________________________________\n",
+      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
+      "==================================================================================================\n",
+      "input_11 (InputLayer)           [(None, 100, 100, 1) 0                                            \n",
+      "__________________________________________________________________________________________________\n",
+      "conv2d_18 (Conv2D)              (None, 100, 100, 16) 800         input_11[0][0]                   \n",
+      "__________________________________________________________________________________________________\n",
+      "max_pooling2d_14 (MaxPooling2D) (None, 50, 50, 16)   0           conv2d_18[0][0]                  \n",
+      "__________________________________________________________________________________________________\n",
+      "conv2d_19 (Conv2D)              (None, 50, 50, 32)   12832       max_pooling2d_14[0][0]           \n",
+      "__________________________________________________________________________________________________\n",
+      "max_pooling2d_15 (MaxPooling2D) (None, 25, 25, 32)   0           conv2d_19[0][0]                  \n",
+      "__________________________________________________________________________________________________\n",
+      "conv2d_20 (Conv2D)              (None, 25, 25, 64)   18496       max_pooling2d_15[0][0]           \n",
+      "__________________________________________________________________________________________________\n",
+      "max_pooling2d_16 (MaxPooling2D) (None, 12, 12, 64)   0           conv2d_20[0][0]                  \n",
+      "__________________________________________________________________________________________________\n",
+      "flatten_5 (Flatten)             (None, 9216)         0           max_pooling2d_16[0][0]           \n",
+      "__________________________________________________________________________________________________\n",
+      "input_12 (InputLayer)           [(None, 7)]          0                                            \n",
+      "__________________________________________________________________________________________________\n",
+      "concatenate_5 (Concatenate)     (None, 9223)         0           flatten_5[0][0]                  \n",
+      "                                                                 input_12[0][0]                   \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_25 (Dense)                (None, 256)          2361344     concatenate_5[0][0]              \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_26 (Dense)                (None, 128)          32896       dense_25[0][0]                   \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_27 (Dense)                (None, 32)           4128        dense_26[0][0]                   \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_28 (Dense)                (None, 8)            264         dense_27[0][0]                   \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_29 (Dense)                (None, 1)            9           dense_28[0][0]                   \n",
+      "==================================================================================================\n",
+      "Total params: 2,430,769\n",
+      "Trainable params: 2,430,769\n",
+      "Non-trainable params: 0\n",
+      "__________________________________________________________________________________________________\n",
+      "None\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(\"Loading public eye_tracking models...\")\n",
+    "public_model_et_dir = trained_dir + f\"model{model_num}\"\n",
+    "model_hrz = load_model(public_model_et_dir + \"-hrz.h5\")\n",
+    "model_vrt = load_model(public_model_et_dir + \"-vrt.h5\")\n",
+    "print(model_hrz.summary())"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "id": "lgxSezv4jY70",
+    "outputId": "5c8f1c31-0c46-4388-c243-de3247676f9d"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Model: \"model_5\"\n",
+      "__________________________________________________________________________________________________\n",
+      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
+      "==================================================================================================\n",
+      "input_11 (InputLayer)           [(None, 100, 100, 1) 0                                            \n",
+      "__________________________________________________________________________________________________\n",
+      "conv2d_18 (Conv2D)              (None, 100, 100, 16) 800         input_11[0][0]                   \n",
+      "__________________________________________________________________________________________________\n",
+      "max_pooling2d_14 (MaxPooling2D) (None, 50, 50, 16)   0           conv2d_18[0][0]                  \n",
+      "__________________________________________________________________________________________________\n",
+      "conv2d_19 (Conv2D)              (None, 50, 50, 32)   12832       max_pooling2d_14[0][0]           \n",
+      "__________________________________________________________________________________________________\n",
+      "max_pooling2d_15 (MaxPooling2D) (None, 25, 25, 32)   0           conv2d_19[0][0]                  \n",
+      "__________________________________________________________________________________________________\n",
+      "conv2d_20 (Conv2D)              (None, 25, 25, 64)   18496       max_pooling2d_15[0][0]           \n",
+      "__________________________________________________________________________________________________\n",
+      "max_pooling2d_16 (MaxPooling2D) (None, 12, 12, 64)   0           conv2d_20[0][0]                  \n",
+      "__________________________________________________________________________________________________\n",
+      "flatten_5 (Flatten)             (None, 9216)         0           max_pooling2d_16[0][0]           \n",
+      "__________________________________________________________________________________________________\n",
+      "input_12 (InputLayer)           [(None, 7)]          0                                            \n",
+      "__________________________________________________________________________________________________\n",
+      "concatenate_5 (Concatenate)     (None, 9223)         0           flatten_5[0][0]                  \n",
+      "                                                                 input_12[0][0]                   \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_25 (Dense)                (None, 256)          2361344     concatenate_5[0][0]              \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_26 (Dense)                (None, 128)          32896       dense_25[0][0]                   \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_27 (Dense)                (None, 32)           4128        dense_26[0][0]                   \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_28 (Dense)                (None, 8)            264         dense_27[0][0]                   \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_29 (Dense)                (None, 1)            9           dense_28[0][0]                   \n",
+      "==================================================================================================\n",
+      "Total params: 2,430,769\n",
+      "Trainable params: 273\n",
+      "Non-trainable params: 2,430,496\n",
+      "__________________________________________________________________________________________________\n",
+      "None\n"
+     ]
+    }
+   ],
+   "source": [
+    "for (layer_hrz, layer_vrt) in zip(model_hrz.layers[:-trainable_layers], model_vrt.layers[:-trainable_layers]):\n",
+    "    layer_hrz.trainable = False\n",
+    "    layer_vrt.trainable = False\n",
+    "\n",
+    "print(model_hrz.summary())"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "id": "diMNdUZZjeLQ",
+    "outputId": "282ee5b4-fde4-49ee-c57c-3fc9ae03c483"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "Start of training for model-hrz (x-pixels)\n",
+      "Epoch 1/40\n",
+      "112/112 [==============================] - 8s 42ms/step - loss: 84873.1875 - val_loss: 83177.8281\n",
+      "Epoch 2/40\n",
+      "112/112 [==============================] - 4s 35ms/step - loss: 84873.2031 - val_loss: 83178.6172\n",
+      "Epoch 3/40\n",
+      " 19/112 [====>.........................] - ETA: 3s - loss: 81323.5703"
+     ]
+    },
+    {
+     "ename": "KeyboardInterrupt",
+     "evalue": "",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[1;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
+      "\u001b[1;32m~\\AppData\\Local\\Temp/ipykernel_13436/1078372996.py\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"\\nStart of training for model-hrz (x-pixels)\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 2\u001b[1;33m results_hrz = model_hrz.fit(x_train,\n\u001b[0m\u001b[0;32m      3\u001b[0m                             \u001b[0my_hrz_train\u001b[0m \u001b[1;33m*\u001b[0m \u001b[0my_scale\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      4\u001b[0m                             \u001b[0mvalidation_data\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mx_val\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my_hrz_val\u001b[0m \u001b[1;33m*\u001b[0m \u001b[0my_scale\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      5\u001b[0m                             \u001b[0mepochs\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mn_epochs\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
+      "\u001b[1;32mF:\\Education_Study\\University\\MSC\\Thesis\\EyeTracker\\venv\\lib\\site-packages\\tensorflow\\python\\keras\\engine\\training.py\u001b[0m in \u001b[0;36mfit\u001b[1;34m(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)\u001b[0m\n\u001b[0;32m   1181\u001b[0m                 _r=1):\n\u001b[0;32m   1182\u001b[0m               \u001b[0mcallbacks\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mon_train_batch_begin\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mstep\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1183\u001b[1;33m               \u001b[0mtmp_logs\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtrain_function\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0miterator\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1184\u001b[0m               \u001b[1;32mif\u001b[0m \u001b[0mdata_handler\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mshould_sync\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1185\u001b[0m                 \u001b[0mcontext\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0masync_wait\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
+      "\u001b[1;32mF:\\Education_Study\\University\\MSC\\Thesis\\EyeTracker\\venv\\lib\\site-packages\\tensorflow\\python\\eager\\def_function.py\u001b[0m in \u001b[0;36m__call__\u001b[1;34m(self, *args, **kwds)\u001b[0m\n\u001b[0;32m    887\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    888\u001b[0m       \u001b[1;32mwith\u001b[0m \u001b[0mOptionalXlaContext\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_jit_compile\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 889\u001b[1;33m         \u001b[0mresult\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_call\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    890\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    891\u001b[0m       \u001b[0mnew_tracing_count\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mexperimental_get_tracing_count\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
+      "\u001b[1;32mF:\\Education_Study\\University\\MSC\\Thesis\\EyeTracker\\venv\\lib\\site-packages\\tensorflow\\python\\eager\\def_function.py\u001b[0m in \u001b[0;36m_call\u001b[1;34m(self, *args, **kwds)\u001b[0m\n\u001b[0;32m    922\u001b[0m       \u001b[1;31m# In this case we have not created variables on the first call. So we can\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    923\u001b[0m       \u001b[1;31m# run the first trace but we should fail if variables are created.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 924\u001b[1;33m       \u001b[0mresults\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_stateful_fn\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    925\u001b[0m       \u001b[1;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_created_variables\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    926\u001b[0m         raise ValueError(\"Creating variables on a non-first call to a function\"\n",
+      "\u001b[1;32mF:\\Education_Study\\University\\MSC\\Thesis\\EyeTracker\\venv\\lib\\site-packages\\tensorflow\\python\\eager\\function.py\u001b[0m in \u001b[0;36m__call__\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m   3021\u001b[0m       (graph_function,\n\u001b[0;32m   3022\u001b[0m        filtered_flat_args) = self._maybe_define_function(args, kwargs)\n\u001b[1;32m-> 3023\u001b[1;33m     return graph_function._call_flat(\n\u001b[0m\u001b[0;32m   3024\u001b[0m         filtered_flat_args, captured_inputs=graph_function.captured_inputs)  # pylint: disable=protected-access\n\u001b[0;32m   3025\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
+      "\u001b[1;32mF:\\Education_Study\\University\\MSC\\Thesis\\EyeTracker\\venv\\lib\\site-packages\\tensorflow\\python\\eager\\function.py\u001b[0m in \u001b[0;36m_call_flat\u001b[1;34m(self, args, captured_inputs, cancellation_manager)\u001b[0m\n\u001b[0;32m   1958\u001b[0m         and executing_eagerly):\n\u001b[0;32m   1959\u001b[0m       \u001b[1;31m# No tape is watching; skip to running the function.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1960\u001b[1;33m       return self._build_call_outputs(self._inference_function.call(\n\u001b[0m\u001b[0;32m   1961\u001b[0m           ctx, args, cancellation_manager=cancellation_manager))\n\u001b[0;32m   1962\u001b[0m     forward_backward = self._select_forward_and_backward_functions(\n",
+      "\u001b[1;32mF:\\Education_Study\\University\\MSC\\Thesis\\EyeTracker\\venv\\lib\\site-packages\\tensorflow\\python\\eager\\function.py\u001b[0m in \u001b[0;36mcall\u001b[1;34m(self, ctx, args, cancellation_manager)\u001b[0m\n\u001b[0;32m    589\u001b[0m       \u001b[1;32mwith\u001b[0m \u001b[0m_InterpolateFunctionError\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    590\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mcancellation_manager\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 591\u001b[1;33m           outputs = execute.execute(\n\u001b[0m\u001b[0;32m    592\u001b[0m               \u001b[0mstr\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msignature\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mname\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    593\u001b[0m               \u001b[0mnum_outputs\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_num_outputs\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
+      "\u001b[1;32mF:\\Education_Study\\University\\MSC\\Thesis\\EyeTracker\\venv\\lib\\site-packages\\tensorflow\\python\\eager\\execute.py\u001b[0m in \u001b[0;36mquick_execute\u001b[1;34m(op_name, num_outputs, inputs, attrs, ctx, name)\u001b[0m\n\u001b[0;32m     57\u001b[0m   \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     58\u001b[0m     \u001b[0mctx\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mensure_initialized\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 59\u001b[1;33m     tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,\n\u001b[0m\u001b[0;32m     60\u001b[0m                                         inputs, attrs, num_outputs)\n\u001b[0;32m     61\u001b[0m   \u001b[1;32mexcept\u001b[0m \u001b[0mcore\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_NotOkStatusException\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
+      "\u001b[1;31mKeyboardInterrupt\u001b[0m: "
+     ]
+    }
+   ],
+   "source": [
+    "print(\"\\nStart of training for model-hrz (x-pixels)\")\n",
+    "results_hrz = model_hrz.fit(x_train,\n",
+    "                            y_hrz_train * y_scale,\n",
+    "                            validation_data=(x_val, y_hrz_val * y_scale),\n",
+    "                            epochs=n_epochs,\n",
+    "                            callbacks=cb)\n",
+    "print(\"End of training\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "id": "vQz7jjQ5CTS8",
+    "outputId": "f9c25499-ae97-4d66-8bfd-d5e13d35585d"
+   },
+   "outputs": [],
+   "source": [
+    "print(\"\\nStart of training for model-vrt (y-pixels)\")\n",
+    "results_vrt = model_vrt.fit(x_train,\n",
+    "                            y_vrt_train * y_scale,\n",
+    "                            validation_data=(x_val, y_vrt_val * y_scale),\n",
+    "                            epochs=n_epochs,\n",
+    "                            callbacks=cb)\n",
+    "print(\"End of training\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "id": "DCu14ufb8oFN",
+    "outputId": "55beefd3-f9e9-4338-8b26-4486771d830e"
+   },
+   "outputs": [],
+   "source": [
+    "# print(\"\\nSaving models...\")\n",
+    "# model_hrz.save(sbj_dir + \"model-et-hrz.h5\")\n",
+    "# model_vrt.save(sbj_dir + \"model-et-vrt.h5\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "e9L074ihCXD6"
+   },
+   "outputs": [],
+   "source": [
+    "# Predicting outputs for train and val data\n",
+    "y_hrz_prd_train = model_hrz.predict(x_train).reshape((n_train,)) / y_scale\n",
+    "y_hrz_prd_val = model_hrz.predict(x_val).reshape((n_val,)) / y_scale\n",
+    "y_vrt_prd_train = model_vrt.predict(x_train).reshape((n_train,)) / y_scale\n",
+    "y_vrt_prd_val = model_vrt.predict(x_val).reshape((n_val,)) / y_scale"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "hrz_train_eval = model_hrz.evaluate(x_train, y_hrz_train * y_scale)\n",
+    "hrz_val_eval = model_hrz.evaluate(x_val, y_hrz_val * y_scale)\n",
+    "vrt_train_eval = model_vrt.evaluate(x_train, y_vrt_train * y_scale)\n",
+    "vrt_val_eval = model_vrt.evaluate(x_val, y_vrt_val * y_scale)\n",
+    "\n",
+    "print(hrz_train_eval, hrz_val_eval, vrt_train_eval, vrt_val_eval)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "min_out_ratio = 0.005\n",
+    "max_out_ratio = 0.995\n",
+    "\n",
+    "y_hrz_prd_train[y_hrz_prd_train < min_out_ratio] = min_out_ratio\n",
+    "y_hrz_prd_val[y_hrz_prd_val < min_out_ratio] = min_out_ratio\n",
+    "y_vrt_prd_train[y_vrt_prd_train < min_out_ratio] = min_out_ratio\n",
+    "y_vrt_prd_val[y_vrt_prd_val < min_out_ratio] = min_out_ratio\n",
+    "\n",
+    "y_hrz_prd_train[y_hrz_prd_train > max_out_ratio] = max_out_ratio\n",
+    "y_hrz_prd_val[y_hrz_prd_val > max_out_ratio] = max_out_ratio\n",
+    "y_vrt_prd_train[y_vrt_prd_train > max_out_ratio] = max_out_ratio\n",
+    "y_vrt_prd_val[y_vrt_prd_val > max_out_ratio] = max_out_ratio"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/",
+     "height": 338
+    },
+    "id": "FDC8rCcVjjDa",
+    "outputId": "48c3b4f3-1946-4554-b0e9-6ae7c0ca6120"
+   },
+   "outputs": [],
+   "source": [
+    "# Displaying data\n",
+    "smp_num = 0\n",
+    "print(\"Train\")\n",
+    "sample_train = (int(y_hrz_train[smp_num] * y_scale),\n",
+    "                int(y_vrt_train[smp_num] * y_scale))\n",
+    "sample_prd_train = (int(y_hrz_prd_train[smp_num] * y_scale),\n",
+    "                    int(y_vrt_prd_train[smp_num] * y_scale))\n",
+    "print(sample_train)\n",
+    "print(sample_prd_train)\n",
+    "\n",
+    "print(\"Validation\")\n",
+    "sample_val = (int(y_hrz_val[smp_num] * y_scale),\n",
+    "               int(y_vrt_val[smp_num] * y_scale))\n",
+    "sample_prd_val = (int(y_hrz_prd_val[smp_num] * y_scale),\n",
+    "                   int(y_vrt_prd_val[smp_num] * y_scale))\n",
+    "print(sample_val)\n",
+    "print(sample_prd_val)\n",
+    "\n",
+    "_, ax = plt.subplots(1, 2)\n",
+    "ax[0].imshow((x1_train[smp_num] * x1_scaler).astype(np.uint8).\n",
+    "             reshape((frame_h, frame_w)), cmap=\"gray\", vmin=0, vmax=255)\n",
+    "ax[1].imshow((x1_val[smp_num] * x1_scaler).astype(np.uint8).\n",
+    "             reshape((frame_h, frame_w)), cmap=\"gray\", vmin=0, vmax=255)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "sbj_test_dir = path2root + subjects_dir + f\"{sbj_num_test}/\"\n",
+    "data_et_test_dir = sbj_test_dir + data_et_fol\n",
+    "print(f\"\\nLoading subject data in {data_et_test_dir}\")\n",
+    "with open(data_et_test_dir + \"x1.pickle\", \"rb\") as f:\n",
+    "    x1_load_test = pickle.load(f)\n",
+    "with open(data_et_test_dir + \"x2.pickle\", \"rb\") as f:\n",
+    "    x2_load_test = pickle.load(f)\n",
+    "with open(data_et_test_dir + \"y.pickle\", \"rb\") as f:\n",
+    "    y_load_test = pickle.load(f)\n",
+    "n_smp_test= x1_load_test.shape[0]\n",
+    "print(f\"Test samples number: {n_smp_test}\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Displaying data\n",
+    "smp_num = 0\n",
+    "print(x2_load_test[smp_num])\n",
+    "print(y_load_test[smp_num])\n",
+    "plt.imshow(x1_load_test[smp_num].reshape((frame_h, frame_w)),\n",
+    "           cmap=\"gray\", vmin=0, vmax=255)\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "x2_chs_inp_test = x2_load_test[:, chosen_inputs]\n",
+    "\n",
+    "x1_test = x1_load_test / x1_scaler\n",
+    "x2_test = x2_scaler.transform(x2_chs_inp_test)\n",
+    "\n",
+    "x_test = [x1_test, x2_test]\n",
+    "\n",
+    "# Predicting outputs for train and test data\n",
+    "y_hrz_test_prd = model_hrz.predict(x_test).reshape((n_smp_test,)) / y_scale\n",
+    "y_vrt_test_prd = model_vrt.predict(x_test).reshape((n_smp_test,)) / y_scale\n",
+    "\n",
+    "hrz_test_eval = model_hrz.evaluate(x_test, y_load_test[:, 0] * y_scale)\n",
+    "vrt_test_eval = model_vrt.evaluate(x_test, y_load_test[:, 1] * y_scale)\n",
+    "\n",
+    "print(hrz_test_eval, vrt_test_eval)\n",
+    "\n",
+    "min_out_ratio = 0.005\n",
+    "max_out_ratio = 0.995\n",
+    "\n",
+    "y_hrz_test_prd[y_hrz_test_prd < min_out_ratio] = min_out_ratio\n",
+    "y_vrt_test_prd[y_vrt_test_prd < min_out_ratio] = min_out_ratio"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "smp_num = 8000\n",
+    "print(\"Test\")\n",
+    "sample_test = (int(y_load_test[smp_num, 0] * y_scale),\n",
+    "                int(y_load_test[smp_num, 1] * y_scale))\n",
+    "sample_test_prd = (int(y_hrz_test_prd[smp_num] * y_scale),\n",
+    "                    int(y_vrt_test_prd[smp_num] * y_scale))\n",
+    "print(sample_test)\n",
+    "print(sample_test_prd)\n",
+    "\n",
+    "plt.imshow((x1_test[smp_num] * x1_scaler).astype(np.uint8).\n",
+    "             reshape((frame_h, frame_w)), cmap=\"gray\", vmin=0, vmax=255)"
+   ]
+  }
+ ],
+ "metadata": {
+  "colab": {
+   "collapsed_sections": [],
+   "name": "rtn_2mdl_et_35p_predict_tst.ipynb",
+   "provenance": []
+  },
+  "kernelspec": {
+   "display_name": "venv",
+   "language": "python",
+   "name": "venv"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 1
+}

codes/jupyter_notebook/rtn_mdl_boi.ipynb

@@ -0,0 +1,505 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "370e2b14",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from tensorflow.keras.callbacks import EarlyStopping\n",
+    "from tensorflow.keras.models import load_model\n",
+    "from tensorflow.keras.utils import to_categorical\n",
+    "from sklearn.utils import shuffle\n",
+    "from joblib import load as j_load\n",
+    "from joblib import dump as j_dump\n",
+    "import pickle\n",
+    "import tuning_parameters as tp\n",
+    "import numpy as np\n",
+    "import os"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "f1344641",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "path2root = \"../\"\n",
+    "subjects_fol = \"subjects/\"\n",
+    "models_fol = \"models/\"\n",
+    "models_boi_fol = \"boi/\"\n",
+    "trained_fol = \"trained/\"\n",
+    "data_boi_fol = \"data-boi/\"\n",
+    "r_train = 0.8\n",
+    "n_epochs = 2\n",
+    "patience = 1\n",
+    "trainable_layers = 1\n",
+    "chosen_inputs = [0, 1, 2, 6, 7, 8, 9]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "5c062b07",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "trained_dir = path2root + models_fol + models_boi_fol + trained_fol\n",
+    "public_model_dir = trained_dir + f\"model{tp.MODEL_BLINK_OUT_IN_NUM}.h5\"\n",
+    "public_scalers_dir = trained_dir + f\"scalers{tp.MODEL_BLINK_OUT_IN_NUM}.bin\"\n",
+    "sbj_dir = path2root + subjects_fol + f\"{tp.NUMBER}/\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "43563004",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "Loading subject data in in_blink_out folder...\n",
+      "Sapmles number: 60\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(\"\\nLoading subject data in in_blink_out folder...\")\n",
+    "data_boi_dir = sbj_dir + data_boi_fol\n",
+    "with open(data_boi_dir + \"x1.pickle\", \"rb\") as f:\n",
+    "    x1_load = pickle.load(f)\n",
+    "with open(data_boi_dir + \"x2.pickle\", \"rb\") as f:\n",
+    "    x2_load = pickle.load(f)\n",
+    "with open(data_boi_dir + \"y.pickle\", \"rb\") as f:\n",
+    "    y_load = pickle.load(f)\n",
+    "n_smp, frame_h, frame_w = x1_load.shape[:-1]\n",
+    "print(f\"Sapmles number: {n_smp}\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "b7c5f075",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import matplotlib.pyplot as plt"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "227878ef",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[ 2.90088087  0.06944803 -0.15008799  3.02945633 -3.65793277 56.22252096\n",
+      "  0.45436421  0.47427553  0.46212459  0.46236947]\n",
+      "2.0\n"
+     ]
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOkAAAD6CAYAAABEdWDWAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAad0lEQVR4nO2dfahmV3XGn5WZyXyP8xUnYyYmKUpFsFEYUsX+IbGBNBX1Dyl+UKYQyD8tRLTopIVSoYX4jx/QYhmIOAUxahUSgkXSOFKEEh1NtPkgZhTEhEmmk8xkZjLJmMTVP+478b7rPPee55773vfdNz4/GOaefffZZ599zrrnPOusvXZkJowx7XLJrDtgjFkcG6kxjWMjNaZxbKTGNI6N1JjGsZEa0zjLMtKIuDEiHouIYxFxcFKdMsb8jhj6nTQi1gD4OYAbADwB4EcAPpKZjyy0z7p163L9+vW1nUW3GazPQ/cbAmtHabvWmVSfWTuXXLJyL0m1T8p4KNeMnUctY+eljKPC0PP47W9/29uf2u+1a9eObZ8/fx4XLlygJ7KWFYpcB+BYZv5y1LE7AXwAwIJGun79elx77bVjZbXz69at6z3wpIxUaadeAAD4zW9+0yl7+eWXe/erZWvWrFm4syNeeuml3j7WP3wAcOmll/YeSzn/V155pVOnnmvdBrr9Zu3UG7X2GejeD0odoHtfsfujnj8b61p24cKFTp1axv6QbNmyZWx7165dY9tHjhzp7PNqewv+pp8rAPx63vYTozJjzARZzpNUIiJuAXALwP8KGmMWZzlP0icBXDlve9+obIzMPJSZ+zNzv/Iqa4wZZzlP0h8BeHNEXIM54/wwgI/27VR1WX1/Z1pOcTAo7dQyVqfC9NaLL744kf1YndonpuWqlmIaqL61sD+QVScBwMaNG3vbrhq4aktWxvRePTem9ZVrxKj9Vhxpyj2j3HvsWNUnUMdnMZ/KYCPNzJcj4m8AfBfAGgBfzsyHh7ZnjOEsS5Nm5ncAfGdCfTHGEBxxZEzjrLh3d6mwb3fKO7+iN2sZO1bVic8//3ynDtOktS2mN+v3NKbThgQKMD3zwgsvjG2z76S1DgBs2rRpbHvz5s2dOlXLsu+0VQMz3Vo1qKLR2XdKRj3flQyCUAIu+sZjsb74SWpM49hIjWkcG6kxjWMjNaZxpu446vuoPFTMV2eKEgTA6lTHBHMSsTIloFypM2SmjBI8zmDBA/XclAD/rVu3duookwcU6hgxh6DigFOcOQrKuCrHqsEmdhwZs4qxkRrTODZSYxpnqpo0Ijrv632Bxxf3mw/TckqgghJwoExoViY5M+2kTAyvKB/hh2ZvYMevmpxdDyXAvgbqM+p+QyehM5TAeKVORemjMjHdwQzGvIawkRrTODZSYxrHRmpM48w8mEGZ4TIkFacyw4V9BK91lMyAQNcBxhwBdb+VTGGpzAJhY1QdHHVWDKvDxrHCZsoozhNlVpAC2686oYYGYNT9mOOIOddU/CQ1pnFspMY0jo3UmMaZejBD1TPKB+WqJ5SsC0pmBDbL//z582PbTG+xwOxapmjZoUsv1P2YlqoalGnCbdu2dcpq1gUWPF/3Y+d65syZsW2W4aLqNCXDg6qtFW2vZGassHbqebD7o14jJfjm1WP29soYM1NspMY0jo3UmMaxkRrTOFMPZuhbZmLo2p9KEIKSdaE6iphThs3wqM4D5riqMKdQPZ6yZCH7UF4/qLM+7969u1N21VVX9bZdHSPsPE6dOjW2ffLkyU6dmlKUXQ8le4IyjkpQiDK7ilHHaFJZKS7iJ6kxjWMjNaZxbKTGNM7Ml5lQlk1XPjoPyaigLGvAPrCzoPN6HmwJByWjQp9mZ8dn2rJqUGX5DqCr25kmrAEObDyqTlOCO9iY1WvNrsfQbImKP6ReD6Y36xixMVOWXlwIP0mNaRwbqTGNYyM1pnFspMY0ztQdR30feoem4qwCn81eqc4LJaMAm2XPHCWKg0FBcXhUx8SOHTs6depslrNnz3bqsLLq4GBOkLpmKVvDtI41q1NnHDHnEiursGtUYY6beo2Gro2rBDPYcWTMaxgbqTGN02ukEfHliDgREQ/NK9sZEfdGxOOj/7vvW8aYiaBo0q8A+BcA/z6v7CCA+zLz9og4ONr+dF9DbJmJIVkXlI/OQ5ce3LBhw9g201JMA1WdzHTJkEyASpY7Fphe22GZEWr2hIX6VKnnr+g9RdszH0FtW9WtVScO1YR9y6KwMuW6KkE8r9bt62Rm/jeAZ0vxBwAcHv18GMAH+9oxxgxjqCbdk5nHRz8/BWDPhPpjjCks23GUc+9jC343iIhbIuJoRBxV3OnGmHGGGunTEbEXAEb/n1ioYmYeysz9mblf+Z5ljBlnaDDD3QAOALh99P9dQzugLJmgpLlUqGKd/dGojiMlCwPQdfAoaT8ZSjDDuXPnxraPHTvWqVPHiDk86rkCXWcOS+lZnXQsNWp1pLExq8diDpc6M+a5557rrcPaUtc1rdRxZOehBK4oTsOFUD7BfA3A/wD4w4h4IiJuxpxx3hARjwP409G2MWYF6H2SZuZHFvjVeyfcF2MMwRFHxjTOVAPsM7OjDZR3c0VPDFkij2lE5UM908R1vxrgDnQDypmWq+e6kkHf27dv75Tt3bt3bJstRVH7xDRhPTcWpKL4GpSsi+zc6jgO8X0A3XFkfox6HykB9hPVpMaY2WIjNaZxbKTGNI6N1JjGmfr6pH3rNCpOIuZMUBwFVeArTgDFKbNQWUWZhVOdS4rDRclmwRxZl19+eads586dY9sshWbtt7KkB5vhUq8jq1PPn10zVlb7qNwzrJ16XZVrr9RZSoCOn6TGNI6N1JjGsZEa0zg2UmMaZ+oRR32RQUpqFOZMqfuxqA9F4E9qbUnmcKkzSpijpJ4Hq1OdDEraTZb2k0Uc1fNnaVeUNVzqfqxOTfvCHFBK2lUlVc2kIo6UVDFKihXF0fjqMeWaxpiZYCM1pnFspMY0zszXJ60ompSh6DRFO9R22EdwRUuytmuWBzbDpMJ0mpJhompSFszAxrWm+WR1lCCEqjdZ2tHaztDsCUPWIgW0bAmKlhyS9rPenw5mMGYVYyM1pnFspMY0jo3UmMaZuuNIWfulMiRdCHMmKB+0ldk0daYKOz5zXNV+s5SaFTY+iuNISenB1idl59YHc/jUMuU8hq7XoqSYGTqOShoaxbnUNwNrsXP3k9SYxrGRGtM4NlJjGmfqmrTvA7KiJdk7f9UcLAig1lECqlk7SupJFgShaLBaNvTjedU8LOBfCQJg1P3YePR9vGf7DQ1kUTJTsOuhrKFa+83OQ/FHLEWDVvwkNaZxbKTGNI6N1JjGsZEa0zjNBTMMDUJQ1uMckplBSQUJDPswz9pRZmYoH+H7UqeyYzEUZ86kAkcYQ4+lOJwUx5GSmUGpsxz8JDWmcWykxjSOjdSYxpl5MMNKtausEakECigBD6zeUrLBLXb8oWtdKuOsBJ0rKNkblSB4hjIhQwnKUHwUSmaGoffvcu57P0mNaRwbqTGNYyM1pnF6jTQiroyIIxHxSEQ8HBG3jsp3RsS9EfH46P9uinRjzLJRvBsvA/hkZv4kIrYC+HFE3AvgrwDcl5m3R8RBAAcBfHrluvo7lA/8jOoEGJrmkTkBhgQhKM4ExXHEmNQMFyV4gPVHCUIY0kd1nyEOOOW6sjqTWiplIXqvdmYez8yfjH4+C+BRAFcA+ACAw6NqhwF8UD6qMUZmSZo0Iq4G8A4A9wPYk5nHR796CsCeBfa5JSKORsRRZfEdY8w4spFGxBYA3wLw8cwcS3Oec893+h6SmYcyc39m7meTYY0xiyN9cY+IdZgz0K9m5rdHxU9HxN7MPB4RewGcmESH2Du/osGGBLgrmnBosLSyhKKidycVzKAsqcFYinZaKpPScor+H1qnMrSPfVkoF9Painc3ANwB4NHM/Ny8X90N4MDo5wMA7upryxizdJQn6bsB/CWA/42IB0dlfwfgdgDfiIibAfwKwF+sSA+N+T2n10gz8wcAFnoveu9ku2OMqTjiyJjGmeosmIjo/RA+dGaG4qgZMhOBtatkFGCOgEkFUyhLH6yk42hSziRlKYqhQRnKjKdaNjRTxZCZOks5Lz9JjWkcG6kxjWMjNaZxpp6ZYQhDljVgTCowXdF3SlCG0s7Q7BGTWlZQCZ6fVGaIocElQwM+hoyJEoTPGJIl8yJ+khrTODZSYxrHRmpM49hIjWmcqTqONm7ciLe97W1jZTW1/6ZNmzr71SluzFFQyxQHEKtTl5U4f/58p87Zs2c7Zc8//3xvndoWW/u0ws51w4YNY9vKMgvsWGfOnOmUXbhwYWxb+XjPgjtq2dAMD0pQwubNmztl9Z5RlpAYuhbskCU1lhJY4yepMY1jIzWmcWykxjSOjdSYxpmq4ygzOwK6CnGWB0kR+EMiXNjao9XB8sILL3TqKM4kVqc6T7Zs2dKpUx0c7Lyqc+fcuXO9/WGOI9bHWo85c2oflVkxzHGjOAQV2LkNSdc5qaikoeu1LoSfpMY0jo3UmMaxkRrTODNfn1RJbajMMBnyzs+SdVcNyoISWBDAiy++OLbNNNjGjRvHtpkGq3167rnnOnVqGdOkNbii6lhAC0JgKBkmlOASJVChbx+Aa+Lqb2D+ByUzw5CZU0P170L4SWpM49hIjWkcG6kxjWMjNaZxpp7Ss++DtbKOhpKugzkq6kfv6uwBuk4Y5jhi+1UnBJvNU+swZ86pU6fGtp955plOndqn6iRibbMP/pNKocmuRw1UYMdSglSqA47dP4rjSElNqqzzosw4GpqGdCH8JDWmcWykxjSOjdSYxlkVKT2VwPiqFZgGqVpSCVRgQehMO7Fg+b62n3322U6dGqjAtKSyPEMNimDtKFkXFA3GUPwINXBEmVyhpO9kZSupSZey1ugQ/CQ1pnFspMY0jo3UmMaxkRrTODPPzFBhAn/IDH42w6U6jtjsEVZW2bZtW6esptlkTqHjx4+PbZ8+fbpTp54bC4qoZcxxonzMVxwczEmnzHCp15kFgNRABebcqo4j5lxiZco9MinHUR1/pR2vT2rMawgbqTGN02ukEbEhIn4YET+NiIcj4jOj8msi4v6IOBYRX4+I7ixnY8yyUTTpBQDXZ+a5iFgH4AcR8Z8APgHg85l5Z0T8G4CbAXxpsYbWrFmDrVu3jpVVPcECBZRlDeo7P9NANZiAacIamL59+/ZOHaYTa2B81Z9AV+++7nWv69TZuXPn2DbTv3XMWPaGEydOjG2fPHmyU4ftV7U8ux6KThuypizLZlH1Hrv2bL++doDueTA/hrJe7BD9q2TAuEjvkzTnuHh3rRv9SwDXA/iPUflhAB+Uj2qMkZE0aUSsiYgHAZwAcC+AXwA4nZkXXX9PALhiRXpozO85kpFm5iuZ+XYA+wBcB+At6gEi4paIOBoRR1kcrDFmcZbk3c3M0wCOAHgXgO0RcfEFfR+AJxfY51Bm7s/M/UzLGWMWp9dxFBGXAXgpM09HxEYANwD4LOaM9UMA7gRwAMBdfW1dcsklnbSWygyCWsY+sNcytjyEkuZSWfaCpfSsjhp2/DpTZu/evZ06r3/968e263gB3XMduoYoc67V/RRnCqOOI3Ou1LFl64xWpxC79ixQQ0kXOmSZCeU8WB0lfehCKN7dvQAOR8QazD15v5GZ90TEIwDujIh/AvAAgDvkoxpjZHqNNDN/BuAdpPyXmNOnxpgVxBFHxjTOzDMzKO/mVU8xfaUsD1G9y0zLVT3BMvGxtmugAtOye/bsGdt+4xvf2KlTgyfYudbACaYt635Mk7FgihpswrR1PX+mEytKMMP69es7dapOZeeqZkJU+lSpmlQJ8FeCK5aCn6TGNI6N1JjGsZEa0zg2UmMaZ+qOo+qs6dsGuk4Q5iioDgXm8FBmeFQnAAtlZMEM1VHBnDKXXXbZ2Pbu3bs7dWpUFnNc1fNn2STqDBc2HiwCrAZPMCdZvR5sjOp4sECBOv7McVQzXrD7gwVcKGlPlXShytq4SvaIPifVYuuV+klqTOPYSI1pHBupMY0zdU3atxwE+3hfy5gGqTqNBc9XPcO0Q9UG7EM900VVO7GMDlWnMg1Wj69kp2PnWrNOsDFjulnRibVMCXpXljWsY8iOxe4PpudqPdbH2icWhKAsX1LrqEthqPhJakzj2EiNaRwbqTGNYyM1pnFmPgtGmeFSnUIsmKGWsXaUWQ/1oz+bdcGcADX15q5duzp1lDVMq4ODHavODGHZG/raBYCnn366U/bUU0/1tqVkPVCcdMp5KFkPWNsVdv7VmabMXlEcm4y+zAyL3Zt+khrTODZSYxrHRmpM48xckyrBA0owQy1j2qWWsWNVbcsCF1hWux07doxt1wwHQFfzKMHa7DzqsVjWwWeeeWZsm2lrpuWUTIQVJTseG4+6pAYbV0X/KpqUUcef3Q9VJ7OAC9anSj0PJWjl1bq9rRtjZoqN1JjGsZEa0zg2UmMaZ+aOoyrWlVknQz4eA9qSFlXQsw/c7KN7zXLAZo8oSw0ojprqvLj88ss7darjiGVPYOuzVgec4txider5VycR0J2FwxxAylqoykyhoctDKBkm6j2izK6qdew4MmYVYyM1pnFspMY0jo3UmMaZquMoMztCvDpKWHqKWoeJ9yr6mRDvOzZrmzmJWFl1lDCHk9LHiuK4YbNr3vCGN4xtK+lkgG4KT8UpxBwl1SlUZwmx/ZhDsDqyFKch0O036+OQlJ6KA0o5lhJ9dhE/SY1pHBupMY1jIzWmcWYezKDoMkWTDtE3TJMpKS2Z3qzHUzSgsmanEvDAMlXU4Iq6xMVC+ylZHpRlFerxWZ16PVggixLswqjjyIJbah2mN+u1Vmb8sHPtC3iwJjVmFWMjNaZxZCONiDUR8UBE3DPaviYi7o+IYxHx9YiY7BrkxhgAS3uS3grg0XnbnwXw+cx8E4BTAG6eZMeMMXNIjqOI2AfgzwH8M4BPxJzKvR7AR0dVDgP4RwBf6mmnN40EcwpVUa2sdcmEuJIus84wUdNl1LbZGqaKU0hZM6QeiwWA1DLm7GJpR1makz7Y8YekxVECUBjMUTPEccTqVMchcyTWdthY9wW7TGJ90i8A+BSAiyO2C8DpzLw48k8AuEJsyxizBHqNNCLeB+BEZv54yAEi4paIOBoRR9mq1caYxVFed98N4P0RcROADQC2AfgigO0RsXb0NN0H4Em2c2YeAnAIAPbt29c/o9kYM0avkWbmbQBuA4CIeA+Av83Mj0XENwF8CMCdAA4AuEs54BBNWrWCkr1ACZ5X1sNkGoRpoFrGjq9kmKgZFFgqzhqEoARFKOMBaB/vlVSYQyZOKJMJlPuDlTGdWMuUOsp6rez+qHUUH8pFlvOd9NOYcyIdw5xGvWMZbRljFmBJYYGZ+X0A3x/9/EsA102+S8aY+TjiyJjGsZEa0zhTnwXTN+tFSauowJwZVdCzD/d1Fog6C0aZ1VD7xGbKDFmPU5lJpDh3WJkSXMFQUnEqwQxDA0CUQIVapqwzoziX2EwiZe3ThfCT1JjGsZEa0zg2UmMaZ6qaNCI67/1Vuwz96K1QdQoLZlA0ISuruojpvdpv5eM962M9lqLZmW5jZUpGxaollYCHodR21PVJlTGqOp2dRz1/VqdqUKY/+7IOOjODMasYG6kxjWMjNaZxbKTGNM7UHUd9aRSV2Rps9siQj9dM4CuzLpQP/MpMHSU9pBJwoAYqVJQAA+ZwUdZ5VRwuffuwY6nBFYqTrm8fQHPk1YCXIctV2HFkzCrGRmpM49hIjWmcqQfYV00xRKswvTlkWUPWjtK/ocHiStv1PNSl/vpQsjCyekyTKn1SAlCUa9bX7kJtK2OtLClSl8uo24AWPD9UWwN+khrTPDZSYxrHRmpM49hIjWmcmQczKA6XoccaUkf5CM5Ef3VoDJ0Fo/RRcToMbbsyqYCLoU4hJTWoMlOJzZSpgQlbtmzp1KnZO1gwQz3WkFlJDmYwZhVjIzWmcWykxjTOzLMFDtFXStC7orcUnaSiTAJQ9FWF6a2qi5TgjqGwMVLaHnKuDCVwQgmMZ2NUAxO2bdvWqbN58+axbaZth/hRHMxgzGsIG6kxjWMjNaZxbKTGNE5M0nnSe7CI/wPwKwC7AZyc2oEnw2rsM7A6+/372OerMvMy9oupGumrB404mpn7p37gZbAa+wyszn67z+P4ddeYxrGRGtM4szLSQzM67nJYjX0GVme/3ed5zESTGmN0/LprTONM3Ugj4saIeCwijkXEwWkfXyEivhwRJyLioXllOyPi3oh4fPT/jln2sRIRV0bEkYh4JCIejohbR+XN9jsiNkTEDyPip6M+f2ZUfk1E3D+6R74eEcOXyV4hImJNRDwQEfeMtlesz1M10ohYA+BfAfwZgLcC+EhEvHWafRD5CoAbS9lBAPdl5psB3DfabomXAXwyM98K4J0A/no0ti33+wKA6zPzWgBvB3BjRLwTwGcBfD4z3wTgFICbZ9fFBbkVwKPztleuz5k5tX8A3gXgu/O2bwNw2zT7sIS+Xg3goXnbjwHYO/p5L4DHZt3Hnv7fBeCG1dJvAJsA/ATAH2MuKGAtu2da+AdgH+b+4F0P4B4AsZJ9nvbr7hUAfj1v+4lR2WpgT2YeH/38FIA9s+zMYkTE1QDeAeB+NN7v0WvjgwBOALgXwC8AnM7MiwvctHiPfAHApwBcnH+3CyvYZzuOBpBzfy6bdItHxBYA3wLw8cw8M/93LfY7M1/JzLdj7ul0HYC3zLZHixMR7wNwIjN/PK1jTnvS95MArpy3vW9Uthp4OiL2ZubxiNiLub/8TRER6zBnoF/NzG+PipvvNwBk5umIOIK5V8XtEbF29GRq7R55N4D3R8RNADYA2Abgi1jBPk/7SfojAG8eecIuBfBhAHdPuQ9DuRvAgdHPBzCn+Zoh5tID3AHg0cz83LxfNdvviLgsIraPft6IOQ39KIAjAD40qtZUnzPztszcl5lXY+7+/V5mfgwr2ecZiO6bAPwcc9rj72ftBFigj18DcBzAS5jTFzdjTnfcB+BxAP8FYOes+1n6/CeYe5X9GYAHR/9uarnfAP4IwAOjPj8E4B9G5X8A4IcAjgH4JoD1s+7rAv1/D4B7VrrPjjgypnHsODKmcWykxjSOjdSYxrGRGtM4NlJjGsdGakzj2EiNaRwbqTGN8/8fWj1QE1QMJAAAAABJRU5ErkJggg==\n",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "smp_num = 0\n",
+    "print(x2_load[smp_num])\n",
+    "print(y_load[smp_num])\n",
+    "plt.imshow(x1_load[smp_num].reshape((frame_h, frame_w)), cmap=\"gray\", vmin=0, vmax=255)\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "b1785e54",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "Normalizing data...\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "['../subjects/7/scalers-boi.bin']"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "print(\"\\nNormalizing data...\")\n",
+    "x2_chs_inp = x2_load[:, chosen_inputs]\n",
+    "scalers = j_load(public_scalers_dir)\n",
+    "x1_scaler, x2_scaler = scalers\n",
+    "x1 = x1_load / x1_scaler\n",
+    "x2 = x2_scaler.transform(x2_chs_inp)\n",
+    "scalers_dir = sbj_dir + \"scalers-boi.bin\"\n",
+    "j_dump(scalers, scalers_dir)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "4b5e4faf",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "Shuffling data...\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(\"\\nShuffling data...\")\n",
+    "x1_shf, x2_shf, y_shf = shuffle(x1, x2, y_load)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "91de08eb",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "Splitting data to train and test...\n",
+      "Data shapes:\n",
+      "(51, 48, 44, 1) (9, 48, 44, 1) (51, 7) (9, 7) (51,) (9,)\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(\"\\nSplitting data to train and test...\")\n",
+    "n_train = int(r_train * n_smp)\n",
+    "n_test = n_smp - n_train\n",
+    "x1_train, x2_train = x1_shf[:n_train], x2_shf[:n_train]\n",
+    "x1_test, x2_test = x1_shf[n_train:], x2_shf[n_train:]\n",
+    "y_train = y_shf[:n_train]\n",
+    "y_test = y_shf[n_train:]\n",
+    "print(\"Data shapes:\")\n",
+    "print(x1_train.shape, x1_test.shape, x2_train.shape, x2_test.shape,\n",
+    "      y_train.shape, y_test.shape)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "f9e629e8",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "y_train_ctg = to_categorical(y_train)\n",
+    "y_test_ctg = to_categorical(y_test)\n",
+    "\n",
+    "x_train = [x1_train, x2_train]\n",
+    "x_test = [x1_test, x2_test]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "bbb4dd8d",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "Loading 'blink_out_in' model...\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(\"\\nLoading 'blink_out_in' model...\")\n",
+    "cb = EarlyStopping(patience=patience, verbose=1, restore_best_weights=True)\n",
+    "model = load_model(public_model_dir)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "47a5cae7",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "Model summary:\n",
+      "Model: \"model\"\n",
+      "__________________________________________________________________________________________________\n",
+      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
+      "==================================================================================================\n",
+      "input_1 (InputLayer)            [(None, 48, 44, 1)]  0                                            \n",
+      "__________________________________________________________________________________________________\n",
+      "conv2d (Conv2D)                 (None, 48, 44, 16)   416         input_1[0][0]                    \n",
+      "__________________________________________________________________________________________________\n",
+      "max_pooling2d (MaxPooling2D)    (None, 24, 22, 16)   0           conv2d[0][0]                     \n",
+      "__________________________________________________________________________________________________\n",
+      "conv2d_1 (Conv2D)               (None, 24, 22, 32)   12832       max_pooling2d[0][0]              \n",
+      "__________________________________________________________________________________________________\n",
+      "max_pooling2d_1 (MaxPooling2D)  (None, 12, 11, 32)   0           conv2d_1[0][0]                   \n",
+      "__________________________________________________________________________________________________\n",
+      "conv2d_2 (Conv2D)               (None, 10, 9, 64)    18496       max_pooling2d_1[0][0]            \n",
+      "__________________________________________________________________________________________________\n",
+      "max_pooling2d_2 (MaxPooling2D)  (None, 5, 4, 64)     0           conv2d_2[0][0]                   \n",
+      "__________________________________________________________________________________________________\n",
+      "flatten (Flatten)               (None, 1280)         0           max_pooling2d_2[0][0]            \n",
+      "__________________________________________________________________________________________________\n",
+      "dense (Dense)                   (None, 256)          327936      flatten[0][0]                    \n",
+      "__________________________________________________________________________________________________\n",
+      "input_2 (InputLayer)            [(None, 7)]          0                                            \n",
+      "__________________________________________________________________________________________________\n",
+      "concatenate (Concatenate)       (None, 263)          0           dense[0][0]                      \n",
+      "                                                                 input_2[0][0]                    \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_1 (Dense)                 (None, 128)          33792       concatenate[0][0]                \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_2 (Dense)                 (None, 32)           4128        dense_1[0][0]                    \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_3 (Dense)                 (None, 16)           528         dense_2[0][0]                    \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_4 (Dense)                 (None, 3)            51          dense_3[0][0]                    \n",
+      "__________________________________________________________________________________________________\n",
+      "dense_5 (Dense)                 (None, 3)            12          dense_4[0][0]                    \n",
+      "==================================================================================================\n",
+      "Total params: 398,191\n",
+      "Trainable params: 12\n",
+      "Non-trainable params: 398,179\n",
+      "__________________________________________________________________________________________________\n",
+      "None\n"
+     ]
+    }
+   ],
+   "source": [
+    "for layer in model.layers[:-trainable_layers]:\n",
+    "    layer.trainable = False\n",
+    "print(\"\\nModel summary:\")\n",
+    "print(model.summary())"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "id": "33258d12",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "Retraining the model...\n",
+      "Epoch 1/2\n",
+      "2/2 [==============================] - 1s 236ms/step - loss: 3.1723 - acc: 0.2941 - val_loss: 2.4191 - val_acc: 0.2222\n",
+      "Epoch 2/2\n",
+      "2/2 [==============================] - 0s 56ms/step - loss: 1.8350 - acc: 0.3137 - val_loss: 1.5906 - val_acc: 0.4444\n",
+      "End of retraining...\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(\"\\nRetraining the model...\")\n",
+    "results = model.fit(x_train,\n",
+    "                    y_train_ctg,\n",
+    "                    validation_data=(x_test, y_test_ctg),\n",
+    "                    epochs=n_epochs,\n",
+    "                    callbacks=cb)\n",
+    "print(\"End of retraining...\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "id": "0336e76a",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "INFO:tensorflow:Assets written to: ../subjects/7/model-boi\\assets\n"
+     ]
+    }
+   ],
+   "source": [
+    "model.save(sbj_dir + \"model-boi.h5\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "id": "b34c8d64",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "yhat_train = model.predict(x_train).argmax(1)\n",
+    "yhat_test = model.predict(x_test).argmax(1)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "id": "52d1aab5",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Test\n",
+      "0.0\n",
+      "1\n"
+     ]
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOkAAAD6CAYAAABEdWDWAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAX4klEQVR4nO2db6il1XXGnyfz72pmxtE0yOBItUQUP7QJDDbBfgimgrUhyYdQYkKZgjBfWjAkJY4tlAZa0C8xgZaUAUOmEGKSJqBISrDWUAJFM1GT+gczEyFEGZ2WRqyjzjhm9cM5o/eus+59n7Pvuefsq88PLve+++x37/W+56z7nrX22msxImCM6Zd3LVoAY8zaWEmN6RwrqTGdYyU1pnOspMZ0jpXUmM5Zl5KSvIHk0ySPkzw0K6GMMW/B1nVSklsA/BzA9QCeBfBjADdFxJOrnbN169bYvn17y1xTnzPv9V9Fxtyn5bpaz2u9H8p5rX1azlOvQ+mX7+NGfmbyXFu2bFlxfObMGZw9e7Z8Y7euY95rAByPiGfGQtwN4OMAVlXS7du348orr5x6one9a/oH/m9+85uJtpY3QVWIrVtX3spKZqVPblPGqWQ8e/bsiuPqfrzxxhsTbRlFufJc1XxVH0XG3EcZpxqrukf53lb3Q7lHQ+MCwLZt21Yc7969e8XxsWPHVh9vagne4hIAv1p2/Oy4zRgzQ9bzJJUgeRDAQWDyv4kxZpj1PEmfA3DpsuN947YVRMThiNgfEfvz1zRjzDDr0ZofA7iC5OUYKeenAHx62kHy9/fKLskofSoUR4Fig1T/bLJMiv1b2S7ZoaDY1pW9pTipqvnzfNX9yDbg66+/PiijYm9W5PNUX0Nuq65VuY/5/ajmapFxGj9Ls5JGxFmSfwHgBwC2APhaRDzROp4xpmZd3z8j4vsAvj8jWYwxBY44MqZz5urJITnxHT/bAcp3dcWWal0nzfZmdU7Lum0lkyJjZf9mOzHfU0CzSZXzFHuv6qOsgSr3Q7FbFRad3GAomGGt9Xg/SY3pHCupMZ1jJTWmc6ykxnTO3EOAhhxFswpUmBXVuK0y5mvdKJnXw5CDQzmnogqKUN77We3eqcZWnGuKY1NxtmUHoLJJ4s05V33FGNMFVlJjOsdKakznLHxbihKY3rLLXgl4UIKlK1uqsh8UGfNWPcXea0UJHlfOq2jJQlFl5MjvkRIUMcugBMW3kN8jJbik+swM2aRr4SepMZ1jJTWmc6ykxnSOldSYzpm742goQ1tLdjZA21WgLJ4r8rTu1MlOj1lda4Wym6fVSaRkC1Sy/Cnz52ut7pmSYUMJQqhoyYyoZHj0Lhhj3kZYSY3pHCupMZ0zd5u0JateRrH3WjMBZFoDDpSxlQVtJeCiQsloUF1by/vReh2nT59e87hCCZQHNJtPyTDRYjdX92NpaWlq+c7hJ6kxnWMlNaZzrKTGdI6V1JjOmbvjaGhBvzUIYZ4oQQiKc6tC2c3TkuFhlg6wfP1KMENViiL3UXYuKdkTAM0plM+rHD5KucosY1WYbCiYYS38JDWmc6ykxnSOldSYzpmrTRoRE/aMYqcpNuhG2a1qyUAl89ysaClrqAZF5LbK3jxz5szgOLmPYrfO6rMAzM5uz+MomyuqLBRDtq2DGYzZxFhJjekcK6kxnWMlNaZzFp7SMxvMSqBAZeC3OIqUBW61FmnL4nlr7c9Z9VGcQvm4Oq96z5QMF8p7raQmVWrIVoEKynutBB3kPlUwQ26bxrHoJ6kxnWMlNaZzBpWU5NdIniT5+LK2i0jeT/LY+PeFGyumMe9cFJv06wD+AcA/L2s7BOCBiLid5KHx8a1DA5FsCjpQFr03KnhAtXWVII1ZyTirLBSVvfnKK6+sOG4NQmjJgqGgBtgrmQ+UjBItZQ0rm1T1bVQMnhkR/wHgf1PzxwEcGf99BMAnmiUwxqxJq3pfHBEnxn8/D+DiGcljjEms23EUo+f/qkGXJA+SPEryqJIcyxizklYlfYHkXgAY/z65WseIOBwR+yNi/zTl3owxI1q15l4ABwDcPv59j3piy+4EpTxDS1pJZe7q6a9kS6gcJblPNXbOYKA4Rar7oWRGOHXq1OB5ilNI2b3SGjiiOICUgAOlzmyLIwmYdBQpgRPToCzBfBPAfwK4kuSzJG/GSDmvJ3kMwB+Oj40xG8Dgv46IuGmVlz4yY1mMMQWOODKmcxbuyZmVXZJpDXBXbLnWhfmWgPKq9EJL8LpqE+V+1bUqtr3ie1DKMyhZ9tS2TEuWkErGHTt2DM69nswdfpIa0zlWUmM6x0pqTOdYSY3pnIU7jhRjXXFCKM6U7BRqTWlZOZOU+fPYys4IxXFT0brrIo+tjFP1yQv8lTOlxSlUpctUZFI+M8o42UkEaKUoMnYcGfM2wkpqTOdYSY3pHCupMZ3TneOoNZqnJYWlEqmjpLRU51f6ZFrvR8tuo6qfEgHWGimk7EJR5qpQ7m1LihWl9qiyS8q1YIx5G2ElNaZzrKTGdM7CbdJZ7fJvsQkVm7Q1mKCiJcCgOqfF3lLsJECzE3NAgRKE0BoUodjIlT2n5NNSrnVpaWmwj1LDtNVHAPhJakz3WEmN6RwrqTGdYyU1pnPm6jgiOeFQUGqGtNQ+WW3+tY6rNiUNB9BWn7Q1zaNynpJmUgkeqBbvs+NIrc+SUdKgtqRhVfvl61fuR+sOl5Zx3uwr9zTGLAQrqTGdYyU1pnPmHswwtICs7JZXqOwbZfE6Z12oAuwr8uK5kuayNVvB0LjVOEoqSmDSBm2ttan4GloyVSibGyqUgAulPEQ1jpKFYmgcB9gbs4mxkhrTOVZSYzrHSmpM58zVcRQRg0EHrXVFFAdUnrvqoyzmV86LWQVTtKSHrPpkp1B1HYrjqDXgIt9b5Z5VDqA8TuXcqcbO7+OsUooqO1yUwBElIOTN8Vd9xRjTBVZSYzrHSmpM58w9wL4lO4EShJCp7AulhESL3QjMLutCSwa7VltKCVRQgueVwHjFllOyKVRUY2eZlOto3SignNNSY/ccfpIa0zlWUmM6x0pqTOcMKinJS0k+SPJJkk+QvGXcfhHJ+0keG/++cOPFNeadh+I4Ogvg8xHxCMldAH5C8n4AfwbggYi4neQhAIcA3LrWQCQnnB5KEICSMlFxVLTsupiVk6iipRxBJZPigKqcS0p6zMrB0ZKFQg1CGKL6vCj3sTWYoaXshuLIm+kumIg4ERGPjP/+PwBPAbgEwMcBHBl3OwLgE0NjGWOmZ6olGJKXAfgAgIcAXBwRJ8YvPQ/g4lXOOQjgIFCHoRlj1kb+3kZyJ4DvAvhsRLy0/LUYfd8pd95GxOGI2B8R+6t1OWPM2khPUpLbMFLQb0TE98bNL5DcGxEnSO4FcLJFgFml6FcyCirZ6fLYlb3VWvpBsW8UW6qlj1qeYaNoLR+SUTZgVG1qmY2hPoptqwSJzDRbIEd35S4AT0XEl5a9dC+AA+O/DwC4R57VGCOjPEmvBfCnAP6L5GPjtr8CcDuAb5O8GcAvAfzJhkhozDucQSWNiB8BWO170UdmK44xJuOII2M6Z+H1SdeTfn85SjmCvMtfcaYoC/UVqoNjqI8STDGr3RsVyg6X1lqwLc4kpRQE0FbSQwlUaH0/hhxHzsxgzCbGSmpM51hJjemcuWdmGIo6UuyLCsUGGjoH0Hb0t9pAswrcUEof5OtX7L+qn3KPFLtVmau61vx5aQ1KaEUJQMm0bBpZs6/c0xizEKykxnSOldSYzrGSGtM5c3UcLS0t4aqrrlrRpjgGMsrO9wqlzERuU8YFtOtQSi+89tprax4DWnkGpc5q1ZbPq5wg+TpaAx4UlPvfGnCiOHhOnTq15jGgZXg477zzVhyff/75six+khrTOVZSYzrHSmpM51hJjemche+CURwM0+wYOIfi8FBqmFbyKBE+2QEDTDqKFGeK4oBS5lKuFdB2ASn3aFapWjPKjpuKWTmSFJlmvSvJT1JjOsdKakznWEmN6Zy574LJC78tO1wqlEAFZfeGkr2hOi/bgK12Seui/xBqndV8HUqdVyWYYFbX1brjRrG/W+1GxSbd0JSexpjFYiU1pnOspMZ0jpXUmM6ZezDDUGCC4gSonEJ5XGWBXUmxUu1UUVKqKGk+qrFbggCUnSKVPNX8ShqavONHcea0Bgq00pK+RXGAtaZzHaoX45SexmxirKTGdI6V1JjOmbtNOvSdXqk92ppmUrGLlIX6yr5TbIwWm7g1XWWeX7UJlcwULUEZSkkPJbhEDTjI/VrvdUvwfnXPtm/fvuJ4aWlpcO43XxuUwBizUKykxnSOldSYzrGSGtM5c98Fk41qxVGTqfooWReUBfbcVtWuqdqUYAZlh4ly/fk8pRZrdT+qYIahuYDJe9SadrMlwEGtzZPnU7JptNZwyYEK2SlUtWVHkoMZjNnEWEmN6ZxBJSW5RPJhkj8l+QTJL47bLyf5EMnjJL9FcvvQWMaY6VFs0tMArouIl0luA/Ajkv8K4HMA7oyIu0n+E4CbAXx1rYEiYtAGVexEZWFaWRivyLZLlb1AKXOh2IBVH2XDQc4OWNmWylwVSj3OfE8U21bJ8qdcq5JhoULxUSiB8dXnIduXuYQEMGmTzjSYIUa8PD7cNv4JANcB+Jdx+xEAnxgayxgzPZJNSnILyccAnARwP4BfAHgxIs79C30WwCUbIqEx73AkJY2INyLi/QD2AbgGwFVrn/EWJA+SPEry6KuvvtompTHvYKby7kbEiwAeBPAhAHtInvuCvg/Ac6ucczgi9kfE/lz+zRgzzKDjiOR7AbweES+SPA/A9QDuwEhZPwngbgAHANwzC4Faywi01MNUHEBVn8p5oJR+UBbYlcwIGSWFpBIAUvVTFviV61DSoLamYVWcSdXYynudA1d27Ngx0Sc7irIjqTpvmswMind3L4AjJLdg9OT9dkTcR/JJAHeT/DsAjwK4SxjLGDMlg0oaET8D8IGi/RmM7FNjzAbiiCNjOmfuAfbZVtqoLHKtC9PZVqjslGrxXskgkOdrzWg4qywUrWUNW8oIVvcs2+1nzpyZ6KNsFKhQfBTT2IXnqOzNbJNWAfZ5rmnKrfhJakznWEmN6RwrqTGdYyU1pnO6S+nZmq0go6SQVNJVqrtplN0jmdadIco4GUXmqk25/mr+06dPr3kMAK+88sqK41bHUUvaTWAywKAaR3H25c9R5VxSnJar4SepMZ1jJTWmc6ykxnTOwm1SJaOB0kdZiFay2rVmjFMWxhX7Sslg11J2o0Kx5apxchBCtZkgb0vM9mfVVtmtOQiikqd6P1rsRCUoZFafDyXj45vjDc5ojFkoVlJjOsdKakznWEmN6ZxNUWaipWRAhZL1IBvw1aJztcsh96ucKS2lH5QdN60L/Eoq0CrAIDt4XnrppYk+L7/88orjapzcplyrsnOp6rdz586JPhdccMFgn5zyp3rvs1Oqkic7k+w4MuZthJXUmM6xkhrTOXO3SbOtoARrK6X2lOx4ip2mBEK3lpBoCVSoxsn27muvvTbRJ7dV4yhB71Wu5FOnTg2Oo5TCaMlEWAUlVHbihRdeuOYxAOzZs2fF8a5duyb65CD86vOQMzNUqWuHsjesq8yEMWaxWEmN6RwrqTGdYyU1pnPmvgtmaEd65dxR0oC2ZDBQMhNUKIvuSkYDxeGj9Kl2mGTnTjVOFXCRnUBVEEJLeQglxakSlFD1efe73z3Rlh01VXkIJX1rnr9yUuU2Za7sAHMwgzGbGCupMZ1jJTWmc6ykxnTO3COOsgHd4vCZVZ0TJTVI5SSqImyyg0VxuFTOnLx7pIr4aXEuqTtlFAeYkhpVSWma+1ROoexgqRw3VYSPEimkOAmHdq9UY8+6PqmfpMZ0jpXUmM6xkhrTOQvPzKDUaVRrUi5HCXio7EYlUKCyAfNYld2a2xSbVAlCUAIn1ACQbIMp5TqqPkrgiLLjKNuglU1aBTMoNqliNyvn5LGrYIZspzozgzFvI6ykxnSOrKQkt5B8lOR94+PLST5E8jjJb5Gc9DsbY9bNNE/SWwA8tez4DgB3RsT7APwawM2zFMwYM0JyHJHcB+CPAfw9gM9xZOVeB+DT4y5HAPwtgK8OjZUNb6VO46wCHrLDpQoUyI6ivJsE0GqWKKlJFAdUFUyRnQxKCslqx4tSV6VyaCjvodInO1OU1CjVtbam2czOG8VJVo2Tgykqx9E09Ugz6pP0ywC+AODcu/oeAC9GxLlP0LMALmmWwhizKoNKSvKjAE5GxE9aJiB5kORRkkerp5IxZm2UZ/C1AD5G8kYASwB2A/gKgD0kt46fpvsAPFedHBGHARwGgH379g1/bzXGrGBQSSPiNgC3AQDJDwP4y4j4DMnvAPgkgLsBHABwz9BYJKXg7IySmSEv3leBCrmPkplBqWsJTNp8VZ98rUqwdhU8nqmCPSp7e0geQAs6z3ZZzoJQ9WnNzDAUmF6NA0xem9KnkjHPpwT4K/anUl7lHOtZJ70VIyfScYxs1LvWMZYxZhWmcjlFxA8B/HD89zMArpm9SMaY5TjiyJjOsZIa0zlzT+mZjfNsQCs7EyoDPxvelYNBSTOpOLKUjAZVMEN25lTOneyAqubKfSonWXa4VEERFUp91jx25VzLbcrukereK8621iAExUm3e/fuFce5fgww6Tir7tnQ59y7YIzZxFhJjekcK6kxnbPw+qRKoELuUy3eKyUDsi2lBEUo9mfVVgXP5/krG0jJuqAE4Wd7VwnUBzSfgJIdT8nwoNitLUHwVT8lMD/bnwBwwQUXrDiuAjemKRmxGrZJjdnEWEmN6RwrqTGdYyU1pnMWXmYiO1yUcgiKYa7scGktRaHUJ1VKDVRBCEq5CiWYIc+vpkVV7pGyeyUzS6eQMraSZjOnAt21a9dgn+p9Ve7ZkHxr4SepMZ1jJTWmc6ykxnTO3APssx2iBNjn7/wti8XV3BVKMENlk2Y7sZor226KTVoF6ue5KjtJKTOhlp7IKLakEqSivB+tNqlSMjHboDt37pzok8+rZG4pzekAe2PeRlhJjekcK6kxnWMlNaZz5u44GkJJvag4KlpRHC6tzgxFxtynclRkGZXgCmXnDqA5jlpQaqFWtARXAJMOn6qGaXYcVX2U3VVKoIjiJFsNP0mN6RwrqTGdYyU1pnPmHmA/lNpfSfXfWh5BSe2fx6nsjdYA/zyfUjKwutZsgyrZ6dTABcXeVcZW5lLuo1IKQgmer7Iu5CwLSkkRxWZXAnKmsVH9JDWmc6ykxnSOldSYzrGSGtM53KjF63Iy8r8B/BLAbwH4n7lNPBs2o8zA5pT7nSjzb0fEe6sX5qqkb05KHo2I/XOfeB1sRpmBzSm3ZV6Jv+4a0zlWUmM6Z1FKenhB866HzSgzsDnltszLWIhNaozR8dddYzpn7kpK8gaST5M8TvLQvOdXIPk1kidJPr6s7SKS95M8Nv594SJlzJC8lOSDJJ8k+QTJW8bt3cpNconkwyR/Opb5i+P2y0k+NP6MfIvkZFDtgiG5heSjJO8bH2+YzHNVUpJbAPwjgD8CcDWAm0hePU8ZRL4O4IbUdgjAAxFxBYAHxsc9cRbA5yPiagAfBPDn43vbs9ynAVwXEb8H4P0AbiD5QQB3ALgzIt4H4NcAbl6ciKtyC4Cnlh1vnMwRMbcfAB8C8INlx7cBuG2eMkwh62UAHl92/DSAveO/9wJ4etEyDsh/D4DrN4vcAM4H8AiA38coKGBr9Znp4QfAPoz+4V0H4D4A3EiZ5/119xIAv1p2/Oy4bTNwcUScGP/9PICLFynMWpC8DMAHADyEzuUef218DMBJAPcD+AWAFyPi3B65Hj8jXwbwBQDn9ui9Bxsosx1HDcTo32WXbnGSOwF8F8BnI+Kl5a/1KHdEvBER78fo6XQNgKsWK9HakPwogJMR8ZN5zTnvRGTPAbh02fG+cdtm4AWSeyPiBMm9GP3n7wqS2zBS0G9ExPfGzd3LDQAR8SLJBzH6qriH5Nbxk6m3z8i1AD5G8kYASwB2A/gKNlDmeT9JfwzgirEnbDuATwG4d84ytHIvgAPjvw9gZPN1A0db/+8C8FREfGnZS93KTfK9JPeM/z4PIxv6KQAPAvjkuFtXMkfEbRGxLyIuw+jz++8R8RlspMwLMLpvBPBzjGyPv160E2AVGb8J4ASA1zGyL27GyO54AMAxAP8G4KJFy5lk/gOMvsr+DMBj458be5YbwO8CeHQs8+MA/mbc/jsAHgZwHMB3AOxYtKyryP9hAPdttMyOODKmc+w4MqZzrKTGdI6V1JjOsZIa0zlWUmM6x0pqTOdYSY3pHCupMZ3z//tAbTd9FHA/AAAAAElFTkSuQmCC\n",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "smp_num = 8\n",
+    "print(\"Test\")\n",
+    "print(y_test[smp_num])\n",
+    "print(yhat_test[smp_num])\n",
+    "plt.imshow(\n",
+    "    (x1_test[smp_num].reshape((frame_h, frame_w)) * 255)\n",
+    "    .astype(np.uint8),\n",
+    "    cmap=\"gray\",\n",
+    "    vmin=0,\n",
+    "    vmax=255\n",
+    "    )\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "id": "a711848e",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAzWElEQVR4nO3deVxVdf7H8deXHQR3RBAQUTbF1NyyXHBfWjSsbF8np2UcK9trZqppGy3by7Gycto0wzJzzQ13U1xAWVQUxBVEQUXW+/39cW6DP0cE5cK59/J5Ph48HnDu4dzPEX17+J7v+X6U1hohhBCOz8XsAoQQQtiGBLoQQjgJCXQhhHASEuhCCOEkJNCFEMJJuJn1xi1bttRhYWFmvb0QQjikLVu25Gmt/S/0mmmBHhYWxubNm816eyGEcEhKqayqXpMhFyGEcBIS6EII4SSqDXSllJdSapNSartSaqdS6uUL7POEUmqXUmqHUmqZUqpt3ZQrhBCiKjUZQy8BBmmtTyul3IE1SqmFWusN5+yzFeihtS5SSj0MTAbG1UG9QggHV1ZWRk5ODsXFxWaXYte8vLwIDg7G3d29xt9TbaBrY7GX09Yv3a0f+rx9Vpzz5QbgzhpXIIRoUHJycvDz8yMsLAyllNnl2CWtNcePHycnJ4d27drV+PtqNIaulHJVSm0DjgFLtdYbL7L7A8DCKo4zXim1WSm1OTc3t8ZFCiGcR3FxMS1atJAwvwilFC1atLjk32JqFOha6wqtdVcgGOillIqtoog7gR7AlCqOM11r3UNr3cPf/4LTKIUQDYCEefUu58/okma5aK1PAiuAERd48yHAC8ANWuuSS66khnJPlfDq/F3knymtq7cQQgiHVJNZLv5KqabWz72BoUDaeft0A/6NEebH6qDO/1q3N48Za/cxYPIKPl65h+Kyirp8OyGEE/L19TW7hDpRkyv0QGCFUmoH8DvGGPp8pdQrSqkbrPtMAXyBH5RS25RS8+qoXkZ3bcOSx/vTO7wFkxelM/CtlczZkkOFRRp1CCEatmoDXWu9Q2vdTWt9hdY6Vmv9inX737XW86yfD9FaB2itu1o/brj4UWunQys/PrunB9+Pv4pWfp48+cN2rvtgDYkZcqNVCFFzWmueeuopYmNj6dy5M7NmzQLg8OHD9O/fn65duxIbG8vq1aupqKjg3nvv/e++77zzjsnV/y/T1nKxhavCWzD3kWv4NfkwkxencfeMTfSLaMlzI2PoGNTY7PKEENV4+Zed7DpUaNNjdgxqzD+u71SjfRMSEti2bRvbt28nLy+Pnj170r9/f7799luGDx/OCy+8QEVFBUVFRWzbto2DBw+SkpICwMmTJ21aty04/KP/Li6K67sE8dsTA/jbdR1JPljAtR+sZtLs7Rw6edbs8oQQdmzNmjXcdtttuLq6EhAQwIABA/j999/p2bMnX3zxBS+99BLJycn4+fkRHh5OZmYmEyZMYNGiRTRubH8XjQ59hX4uTzdXHujbjpuuDObjlXv4Yt1+5u84xP192/FwXHsae9X8aSshRP2o6ZV0fevfvz+JiYn8+uuv3HvvvTzxxBPcfffdbN++ncWLFzNt2jRmz57NjBkzzC71/3H4K/TzNfFx57lRMSyfNIBrOwfyycq9DJi8gi/W7qO03GJ2eUIIO9KvXz9mzZpFRUUFubm5JCYm0qtXL7KysggICODBBx/kT3/6E0lJSeTl5WGxWBg7diyvvvoqSUlJZpf/P5zmCv18wc18mDquK/f3bccbC1N5+ZddfLluP08Pj2ZU59byYIMQghtvvJH169fTpUsXlFJMnjyZ1q1b89VXXzFlyhTc3d3x9fVl5syZHDx4kPvuuw+LxbgwfOONN0yu/n8pY6mW+tejRw9dXw0utNasysjljQVppB89RdeQprxwbQw9w5rXy/sLISqlpqYSExNjdhkO4UJ/VkqpLVrrHhfa3+mGXC5EKUVcVCsWTOzH5LFXcLjgLDdPW8/4mZvZm3u6+gMIIYQDaBCB/gdXF8UtPUNY+eRAnhwWybq9xxn2TiIv/pRM7qk6W61ACCHqRYMK9D94e7jyl0ERrHwqjjt6h/L9pgPETVnB+8t2U1RabnZ5QghxWRpkoP+hpa8nr4yOZcnj/ekX4c/UpRnETVnJ95uyZSkBIYTDadCB/odwf1+m3dWdOQ/1IbiZN88mJDPyvURWpB3DrJvGQghxqSTQz9EjrDk/Pnw1n9xxJaXlFu778ndu/3QjyTkFZpcmhBDVkkA/j1KKkZ0DWfrEAF6+oRPpR09x/YdrmPj9Vg7kF5ldnhBCVEkCvQruri7cc3UYK5+K49GB7VmUcoTBb6/i9QWpFBSVmV2eEKKeXGzt9P379xMbe8EGbqaQQK9GYy93nhoezcqn4hjdNYhPV2fSf8oKPk3MpKRcmmsIIeyH0z76b2uBTbyZcnMX7u/bjjcXpvHaglS+Wr+fp4ZHcf0VQbi4yFICQlyyhc/CkWTbHrN1Zxj5ZpUvP/vss4SEhPDoo48C8NJLL+Hm5saKFSs4ceIEZWVlvPrqq4wePfqS3ra4uJiHH36YzZs34+bmxtSpUxk4cCA7d+7kvvvuo7S0FIvFwo8//khQUBC33HILOTk5VFRU8Le//Y1x48bV6rShZi3ovJRSm5RS25VSO5VSL19gH0+l1Cyl1B6l1EalVFitK7NTMYGN+er+Xnz9QG8ae7kz8fttjPl4Lev3Hje7NCFEDYwbN47Zs2f/9+vZs2dzzz33MHfuXJKSklixYgWTJk265BluH330EUopkpOT+e6777jnnnsoLi5m2rRpTJw4kW3btrF582aCg4NZtGgRQUFBbN++nZSUFEaM+J82zZelJlfoJcAgrfVppZQ7sEYptVBrveGcfR4ATmitOyilbgX+BdT+vxs71jeiJfMn9OWnbQd5a3E6t326gcHRrXh2ZDQRAX5mlyeEY7jIlXRd6datG8eOHePQoUPk5ubSrFkzWrduzeOPP05iYiIuLi4cPHiQo0eP0rp16xofd82aNUyYMAGA6Oho2rZtS0ZGBn369OG1114jJyeH+Ph4IiIi6Ny5M5MmTeKZZ57huuuuo1+/fjY5t5q0oNNa6z8WPHG3fpz/X9do4Cvr53OAwaoBLGfo4qKIvzKY5U/G8ezIaDbtz2f4u4k8l7CDY4XFZpcnhKjCzTffzJw5c5g1axbjxo3jm2++ITc3ly1btrBt2zYCAgIoLrbNv+Hbb7+defPm4e3tzahRo1i+fDmRkZEkJSXRuXNnXnzxRV555RWbvFeNbooqpVyVUtuAYxhNojeet0sb4ACA1rocKABaXOA445VSm5VSm3Nznaf/p5e7Kw8NaE/iUwO59+p2zNmSw4ApK5m6NIPTJbKUgBD2Zty4cXz//ffMmTOHm2++mYKCAlq1aoW7uzsrVqwgKyvrko/Zr18/vvnmGwAyMjLIzs4mKiqKzMxMwsPD+etf/8ro0aPZsWMHhw4dwsfHhzvvvJOnnnrKZmur1yjQtdYVWuuuQDDQSyl1WfN0tNbTtdY9tNY9/P39L+cQdq1ZIw/+fn1HfntiAINjWvH+st3ETVnJ1xuyKK+Q5hpC2ItOnTpx6tQp2rRpQ2BgIHfccQebN2+mc+fOzJw5k+jo6Es+5iOPPILFYqFz586MGzeOL7/8Ek9PT2bPnk1sbCxdu3YlJSWFu+++m+TkZHr16kXXrl15+eWXefHFF21yXpe8HrpS6u9Akdb6rXO2LQZe0lqvV0q5AUcAf32Rg9fneuhm2Zp9gjcWpLFpfz7h/o14dkQ0QzsGSHMN0aDJeug1Z/P10JVS/kqpptbPvYGhQNp5u80D7rF+fhOw/GJh3lB0C23GrD9fxad390AB4/+zhXH/3sDW7BNmlyaEcEI1meUSCHyllHLF+A9gttZ6vlLqFWCz1noe8DnwH6XUHiAfuLXOKnYwSimGdgxgYJQ/szYf4J2lu7nx43Vce0UgTw+Pom2LRmaXKISoRnJyMnfdddf/2+bp6cnGjeffTjRXg2hBZ09Ol5TzaWIm0xMzKbdYuPOqtkwYFEHzRh5mlyZEvUhNTSU6OlqGHquhtSYtLU1a0NkzX083Hh8ayaqn4ripezBfrdvPgCkr+GTlXorLZCkB4fy8vLw4fvy4LE19EVprjh8/jpeX1yV9n1yhm2z30VO8uTCNZWnHCGrixaRhUdzYrY0sJSCcVllZGTk5OTab5+2svLy8CA4Oxt3d/f9tv9gVugS6nVi/9zivL0gl+WABHQMb89yoaPpFON/UTiFE7ciQiwPo074FPz96De/f1o3C4jLu+nwTd8/YROrhQrNLE0I4CAl0O+LiorihSxDLJg3gxWtj2H7gJKPeX82TP2zncMFZs8sTQtg5GXKxYwVFZXy0cg9frt2PUvBA33Y8FNeexl7u1X+zEMIpyRi6gzuQX8TbS9L5adshmjfyYOLgCG7rFYqHm/yCJURDI2PoDi6kuQ/v3tqNX/7Sl6gAP/4xbyfD3lnFwuTDMvVLCPFfEugOpHNwE759sDdf3NsTDzcXHv4mibGfrGPz/nyzSxNC2AEJdAejlGJgdCsWTuzPv8Z2JufEWW6atp4//2czmbmnqz+AEMJpyRi6gysqLefz1fuYtmovxeUWbu8VysQhEbT09TS7NCFEHZCbog1A7qkS3l+2m283ZePt7spDA8J5oG843h6uZpcmhLAhuSnaAPj7efLPMbEsebw/V7dvwVtLMoh7awWzfz9AhUVunArREEigO5n2/r5Mv7sHPzzUh6Cm3jz94w5GvbeaFenHZEaMEE5OAt1J9QxrTsLDV/PxHVdSXF7BfV/8zh2fbSTlYIHZpQkh6ogEuhNTSjGqcyBLHx/AS9d3JPVwIdd9sIbHZ20j50SR2eUJIWysJi3oQpRSK5RSu5RSO5VSEy+wTxOl1C9Kqe3Wfe6rm3LF5fBwc+Hea9qx6umBPBLXngXJhxn09ireWJBKQVGZ2eUJIWyk2lkuSqlAIFBrnaSU8gO2AGO01rvO2ed5oInW+hmllD+QDrTWWpdWdVyZ5WKeQyfP8vaSDBK25tDYy50JgzpwV5+2eLrJjBgh7F2tZrlorQ9rrZOsn58CUoE25+8G+Cmjp5QvRl/R8lpVLepMUFNv3r6lC79O6EeXkKa8+msqQ6auYt72Q1hkRowQDuuS5qErpcKARCBWa114znY/YB4QDfgB47TWv17g+8cD4wFCQ0O7Z2Vl1ap4YRurd+fy+oI0Ug8X0iW4Cc+NiuGq8BZmlyWEuACbPFiklPIFVgGvaa0TznvtJuAa4AmgPbAU6HJu6J9PhlzsS4VF89PWg7y1JJ3DBcUMiWnFMyOiiQjwM7s0IcQ5av1gkVLKHfgR+Ob8MLe6D0jQhj3APoyrdeEgXF0UY7sHs+LJOJ4eEcXGzHyGv5vIcwnJHCuU3o9COIKazHJRwOdAqtZ6ahW7ZQODrfsHAFFApq2KFPXHy92VR+I6sOrpgdzdJ4wfNh8g7q2VvLM0gzMlcltECHtWk1kufYHVQDJgsW5+HggF0FpPU0oFAV8CgYAC3tRaf32x48qQi2PYn3eGKYvT+TX5MC19PXl8aATjeoTg5iqPMAhhBlmcS9Ta1uwTvL4gld/3n6C9fyOeHRnDkJhWGL/ACSHqiyzOJWqtW2gzZv+5D9Pv6o4GHpy5mXHTN7D9wEmzSxNCWEmgixpTSjGsU2sWP9aff46JJTP3NKM/Wstfvk0i+7gsJSCE2WTIRVy20yXlTF+1l09X76PcYuGuq8KYMKgDzRp5mF2aEE5LxtBFnTpaWMw7SzOYvfkAjTzdeHRgB+69Ogwvd1lKQAhbkzF0UacCGnvx5tgrWPRYf3qGNefNhWkMfnsVCUk5spSAEPVIAl3YTGSAHzPu7cm3D/amWSN3npi9nes/XMPaPXlmlyZEgyCBLmzu6vYtmfdoX967tSsni8q447ON3DNjE2lHqlwJQghhAxLook64uChGd23DskkDeGFUDFuzTzDyvdU89cN2jhTIUgJC1AW5KSrqxcmiUj5asYev1mXh4gIP9G3HQwPa4+flbnZpQjgUmeUi7MaB/CLeWpLOz9sO0aKRBxOHRHBbr1DcZSkBIWpEZrkIuxHS3If3bu3GvL9cQ0SAL3//eSfD3klkUcphzLq4EMJZSKALU1wR3JTvHryKGff2wM1F8dDXSdw0bT1bsvLNLk0IhyWBLkyjlGJQdAALJ/bjzfjOHMgvYuwn63noP1vYl3fG7PKEcDgyhi7sRlFpOZ+t3se/V+2lpNzCHb1D+evgCFr4eppdmhB2Q26KCoeSe6qE95Zl8N2mA3i7u/JwXHvuv6Yd3h6ylIAQclNUOBR/P09eHdOZxY/1p0/7FkxZnM7At1Yye/MBKmQpASGqVJMWdCFKqRVKqV1KqZ1KqYlV7BenlNpm3WeV7UsVDU2HVr58encPZv+5D62bePH0nB1c+/5qVqYfkxkxQlxATVrQBQKBWuskpZQfsAUYo7Xedc4+TYF1wAitdbZSqpXW+tjFjitDLuJSaK35Nfkwkxelk51fxDUdWvDcyBhi2zQxuzQh6lWthly01oe11knWz08BqUCb83a7HUjQWmdb97tomAtxqZRSXHdFEL89MYC/X9eRnYcKuf7DNTwxaxsHT541uzwh7MIl3RRVSoUBiUCs1rrwnO3vAu5AJ8APeE9rPfMC3z8eGA8QGhraPSsrqza1iwas4GwZn6zcy4y1+wC475owHonrQBNvWUpAODebzHJRSvkCq4DXtNYJ5732IdADGAx4A+uBa7XWGVUdT4ZchC0cPHmWt5ekM3frQZp4uzNhUAR3XhWKp5vMiBHOqdazXJRS7sCPwDfnh7lVDrBYa31Ga52HcRXf5XILFqKm2jT1ZuotXZk/oS+d2zThn/N3MWTqKn7ZfkhunIoGpyazXBTwOZCqtZ5axW4/A32VUm5KKR+gN8ZYuxD1olNQE/7zQG++ur8XjTzcmPDdVsZ8tJaNmcfNLk2IeuNWg32uAe4CkpVS26zbngdCAbTW07TWqUqpRcAOwAJ8prVOqYN6hbioAZH+9O3QkoSkHN5eksG46RsYEhPAsyOj6NDKz+zyhKhT8qSocFpnSyuYsXYfn6zcy9myCsb1DOGxIRG08vMyuzQhLps8+i8atOOnS/hg+R6+3pCFh5sL4/uH82C/cBp51uQXVCHsiwS6EMC+vDNMWZzGguQj+Pt58viQSG7pEYybNNcQDkTWchECaNeyER/f0Z0fH76a0OY+PD83mRHvrea3XUdlRoxwChLoosHp3rYZcx7qw7Q7u1Nh0fxp5mZunb6BHTknzS5NiFqRQBcNklKKEbGtWfJ4f/45uhN7jp3mhg/XMuG7rRzILzK7PCEui4yhCwGcKi5jemImn67OxGKBu/q0ZcKgDjT18TC7NCH+H7kpKkQNHSko5p2lGfyw5QC+nm48OrAD91wdhpe7LCUg7IPcFBWihlo38eJfN13Bgon9uLJtM95YmMbgt1fx09aDWKS5hrBzEuhCXEB068Z8eV8vvvlTb5r6uPPYrG3c8NEa1u3JM7s0IaokgS7ERVzToSW//KUv747ryokzZdz+2Ubu/WIT6UdOmV2aEP9DAl2Iari4KMZ0a8OySQN4flQ0SVknGPleIk/P2c6RgmKzyxPiv+SmqBCX6MSZUj5csYeZ6/fj6qL4U99w/jwgHD8vaa4h6p7MchGiDhzIL2Ly4nR+2X6IFo08eGxIBLf2CsVdlhIQdUhmuQhRB0Ka+/DBbd34+dFr6NDKl7/9vJPh7ySyKOWILCUgTCGBLkQtdQlpyvfjr+Lze3rg4qJ46Ost3DxtPVuyTphdmmhgHC/QT+dC0kwoyje7EiH+SynF4JgAFk3sxxvxncnKL2LsJ+t4+Ost7Ms7Y3Z5ooGoSQu6EKXUCqXULqXUTqXUxIvs21MpVa6Uusm2ZZ5j92KYNwHeioRvboZt30FxQZ29nRCXws3Vhdt6hbLyyTgeGxLBqoxchk5dxUvzdnL8dInZ5QknV+1NUaVUIBCotU5SSvkBW4AxWutd5+3nCiwFioEZWus5FzvuZd8U1RoOb4OUBNg5FwoOgKsnRAyFTjdC5Ajw9L304wpRB46dKubd33Yz6/cD+Li78lBcex7o206WEhCXzaazXJRSPwMfaq2Xnrf9MaAM6AnMr7NAP5fWkLMZUn6EXT/BqcPg5g2RwyE2HiKGgbt37d5DCBvYc+wUby5M57fUowQ28eKJoZHEXxmMq4syuzThYGwW6EqpMCARiNVaF56zvQ3wLTAQmEEVga6UGg+MBwgNDe2elZV1CadRDYsFstfDzgTY9TOcyQUPX4gaCZ3iocNgcPO03fsJcRk2ZB7njQWpbM8pILq1H8+NimFApL/ZZQkHYpNAV0r5AquA17TWCee99gPwttZ6g1LqS+rrCr0qFeWQtcYYlkmdB2dPgGcTiL4WYsdC+ABwlYdAhDm01szfcZjJi9M4kH+WfhEteXZkNJ2CmphdmnAAtQ50pZQ7MB9YrLWeeoHX9wF//O7YEigCxmutf6rqmPX2YFFFGWSuMq7cU+dDSQF4N4eY641hmbZ9wVWaBYv6V1Jewdcbsvlg+W4KzpZxY7c2TBoWRZumMkwoqlarQFdKKeArIF9r/VgN3uxLzL5Cr0p5Cexdboy5py+E0tPQyB86jjaGZUL7gIvjzeQUjq3gbBkfr9zDF2v3A3D/Ne14ZGB7GstSAuICahvofYHVQDJgsW5+HggF0FpPO2//L7HXQD9X2VnYvcQYlslYDOVnwS8QOo4xrtyDe4KSG1ai/uScKGLqkgwSth6kmY87EwZFcOdVbfFwk4sMUUnWcqlOyWnIWGRMg9y9FCpKoEkIdBpjXLkHdZNwF/Um5WABbyxMZe2e44Q29+HpEVFc2zkQJX8HBRLol6a4ENIXGFfue5eBpRyatTOu2jvFQ0AnCXdR57TWrMrI5c2FaaQdOUWXkKa8MCqGXu2am12aMJkE+uUqyoe0+Ua470sEXQEtI41gj40H/yizKxROrsKi+TEph7eXpHO0sIShHQN4ZkQ0HVrJw3MNlQS6LZzJM+a375wL+9cAGlp1gtgbjYBv0d7sCoUTO1tawYy1+/hk5V7OllVwa88QHhsSib+fPFvR0Eig29qpI0a4pyTAgQ3GtsAuRrB3uhGatTW3PuG08k6X8P6y3Xy7MRtPNxfG92/Pg/3b4eMhU28bCgn0ulSQAzt/Mua5H9xibGvTw3iAqdMYaBxkZnXCSWXmnmbK4nQWphzB38+TJ4ZGcnP3YNykuYbTk0CvL/n7jCGZnQlwJBlQxtz22HhjrrtvK7MrFE5mS1Y+r/2aSlL2SSJa+fLsyGgGRbeSGTFOTALdDHl7jGBPSYDcVFAuENbXGJaJuQEatTC7QuEktNYs3nmEfy1KZ1/eGa4Kb87zo2K4Irip2aWJOiCBbrZjqdblfhPg+B5QrhAeZ1y5R18H3k3NrlA4gbIKC99tyubd33aTf6aUG7oE8dTwKEKa+5hdmrAhCXR7obUxFPPHlfvJLHD1gPaDjXCPGgmefmZXKRzcqeIypq3ay2er96E13HN1Wx4d2IGmPh5mlyZsQALdHmkNh5IqG3UUHgQ3L2ujjnhjTXePRmZXKRzY4YKzTF2SwZykHPw83fjLoA7c3SdMmms4OAl0e2exQM4mI9x3/QSnj4K7j9F9KTYeOgwFdy+zqxQOKu1IIW8sSGNVRi5tmnrz9Igorr8iCBdpruGQJNAdiaUCstZVNuooOg4efhA9yrqW+0Bwk1+dxaVbszuP1xeksutwIZ3bNOG5UdFc3b6l2WWJSySB7qgqymF/orVRxy9QfBK8mkLMdcawTLsBspa7uCQWi+anbQd5a3E6hwqKGRTdimdHRhMZIPduHIUEujMoL4XMlcZa7mm/Qukp8GlhTIGMjYe214CLjI2Kmikuq+DLdfv5aMUezpSUc0uPEB4fGklAYxnas3cS6M6mrBj2/GYMy6QvgrIz4BtQ2agjpLc06hA1cuJMKR8s38N/NuzHzcWFB/u1Y/yA9vh6ym9+9koC3ZmVFsHuxcawzO4lUF4MjdtUNupo012W+xXVyj5exOTFaczfcZiWvh5MHBLJrT1DcJelBOxObTsWhQAzgQBAA9O11u+dt88dwDMYfUVPAQ9rrbdf7LgS6HWg5JRxxb4zwbiCryiFpqGVy/22vkLCXVzUtgMnef3XVDbtzyfcvxHPjIhmWMcAWUrAjtQ20AOBQK11klLKD9gCjNFa7zpnn6uBVK31CaXUSOAlrXXvix1XAr2OnT1pjLXvTDDG3i3l0Lz9OY06OppdobBTWmt+Sz3GmwtT2Zt7hp5hzXhuVAxXhjYzuzSBjYdclFI/Ax9qrZdW8XozIEVr3eZix5FAr0dF+ZA6zxiW2b8atAX8oyuv3FtGmF2hsEPlFRZmbT7AO0t3k3e6hGs7B/LU8CjCWsoDb2ayWaArpcKARCBWa11YxT5PAtFa6z9d4LXxwHiA0NDQ7llZWTV+b2Ejp49VNurIWgdoCOhc2aijeTuzKxR25nRJOZ8mZjI9MZNyi4U7erflr4MjaN5Inocwg00CXSnlC6wCXtNaJ1Sxz0DgY6Cv1vr4xY4nV+h2oPBQZaOOnE3GtqBulY06moaYW5+wK8cKi3nnt93M+j2bRh5uPDywPfdf006WEqhntQ50pZQ7MB9YrLWeWsU+VwBzgZFa64zqjimBbmdOZhtX7SkJcHibsS2ktzXcx4BfazOrE3Zk99FT/GtRGr+lHiOwiReThkVxY7c2uMpSAvWitjdFFfAVkK+1fqyKfUKB5cDdWut1NSlKAt2OHd9rbdQxF46mAMp4cCn2RogZDb7+Zlco7MD6vcd5Y2EqO3IKiAlszPOjoukXIX836lptA70vsBpIBizWzc8DoQBa62lKqc+AscAfg+LlVb3hHyTQHURuRuVyv3npRqOOdv2tjTquB5/mZlcoTGSxaOYnH2byojRyTpylX0RLnhsZQ8egxmaX5rTkwSJRe1rDsV2VjTryM8HFzVgsLHassXiYVxOzqxQmKSmv4D/rs/hg+R4Ki8uI7xbMpGGRBDX1Nrs0pyOBLmxLazi83XrlPhcKso1GHR2GGtMgI0eAp6/ZVQoTFBSV8fHKPXyxbj8KuL9vOx6Oa09jL3ezS3MaEuii7mgNOZuNcN/5E5w6BG7eEDnMGJaJGAYe0gKtock5UcRbi9P5adshmvm489fBEdzRuy0ebrKUQG1JoIv6YbHAgQ2VjTrO5IJ7I6O1Xmw8dBgCbp5mVynqUcrBAl5fkMq6vcdp28KHp4dHM6pza1lKoBYk0EX9s1TA/jXWRh3z4Gw+eDaB6GuNcA+PA1f5Nbwh0FqzMiOXNxekkX70FF1DmvLCtTH0DJMb6pdDAl2Yq6IM9q2yNuqYDyUF4N3MmCXTKR7C+kmjjgagwqL5cUsOby9N52hhCcM6BvDMyGja+8v9lkshgS7sR3kJ7F1uhHv6Aig9DT4tjbXcY+MhtI806nByRaXlzFizj09W7qW43MJtvUKYODgSfz8ZjqsJCXRhn8rOwu6lxrBMxmIoKwLf1saTqZ3iIbinNOpwYnmnS3jvt918uykbLzcXHhrQngf6tcPHQ35buxgJdGH/Ss9AxiJro46lUFECjYMrFw0L6iZruTupvbmnmbwojcU7jxLQ2JMnhkZyU/cQWUqgChLowrEUF0L6QmujjmVgKYNmYZXL/QbESrg7od/35/P6glS2Zp8kMsCX50bGEBflLzNiziOBLhzX2RPGjdSdCZC5CnQFtIiobNTRKtrsCoUNaa1ZmHKEyYvS2H+8iD7hLXh+VAydg+Up5D9IoAvncCbvnEYdawANrTpWXrm3aG92hcJGSsstfLsxi/eX7yH/TCmjuwbx5LAoQprLQ2oS6ML5nDpqbdSRANnrjW2trzDWlel0IzRra259wiYKi8uYtnIvn6/Zh9Zw7zVhPBrXgSY+DfcZBgl04dwKcoxlB3YmwMEtxrY2PYyr9o5joMlFuyEKB3C44CxvL8ngx6QcGnu5M2FQB+7q0xZPt4Y3xVUCXTQcJ/ZXNuo4ssPYFtrHGJbpOBr8AkwtT9TOrkOFvLkojcSMXIKbefPU8CiuvyIIlwY0I0YCXTRMeXusjToSjKV/lYu1UUe80aijUQuzKxSXafXuXF5fkEbq4UKuCG7CcyNj6NO+Yfw8JdCFOJZW2ajj+G5QrhA+wNqo4zpjKQLhUCwWzdytB3l7STqHCooZHN2KZ0dGExHgZ3Zpdaq2HYtCgJlAAKCB6Vrr987bRwHvAaOAIuBerXXSxY4rgS5MobXRVi8lAVJ+hJNZ4OIOHQYb4R41Eryk244jKS6r4Iu1+/l4xR7OlJYzrmcIjw+JpFVjL7NLqxO1DfRAIFBrnaSU8gO2AGO01rvO2WcUMAEj0HsD72mte1/suBLownRaw6Ekaxemn6AwB1w9IeKcRh0ejcyuUtRQ/plSPli+m683ZOHm4sKD/cMZ3z8cX0/nWkrApkMuSqmfgQ+11kvP2fZvYKXW+jvr1+lAnNb6cFXHkUAXdsVigZzfKxt1nD4C7j4QOdzaqGMouEs7NUeQdfwMkxel82vyYVr6evLYkAhu7RmCm6tzrAtks0BXSoUBiUCs1rrwnO3zgTe11musXy8DntFabz7v+8cD4wFCQ0O7Z2VlIYTdsVQYc9tTEoy57kV54OFn9E3tFA/tB4Gbh9lVimpszT7B6wtS+X3/CcL9G/HsiGiGdgxw+KUEbBLoSilfYBXwmtY64bzXahTo55IrdOEQKsph/2pjvD31Fyg+aTTDjr7eWDis3QBp1GHHtNYs3XWUNxelkZl7hl5hzXluVDTdQh33JnitA10p5Q7MBxZrrade4HUZchHOr7wUMlcawzJpv0JJIXg3h443WBt19JW13O1UWYWFWb8f4N3fMsg7Xcq1VwTy9PAo2rZwvHsktb0pqoCvgHyt9WNV7HMt8Bcqb4q+r7XudbHjSqALh1ZWDHuXWRt1LISyM9CoVWWjjpCrZC13O3S6pJzpiZl8mphJucXCnVe1ZcKgCJo3cpwhtNoGel9gNZAMWKybnwdCAbTW06yh/yEwAmPa4n0XG24BCXThREqLYPeSykYd5cXgF2Q06ogdC226y3K/duZoYTHvLM1g9uYDNPJ045G4Dtx3TRhe7vb/G5Y8WCREfSk5XdmoY89SqCiFpqHGgmGd4iGwi4S7Hck4eop/LUxjWdoxgpp4MWlYFDd2a2PXSwlIoAthhuICY6w9JQEyV4ClHJqHVy7326qjhLudWLc3jzcWpJF8sICOgY15flQMfSNaml3WBUmgC2G2onxjlszOBNiXCNoCLaMqG3X4R5pdYYNnsWh+2XGIKYvTyTlxlv6R/jw3MpqYQPt6clgCXQh7cjoXUn+GlLmQtRbQRlu9P8K9eTuzK2zQSsormLkuiw+W7+ZUSTljrwxm0rBIApvYx4NlEuhC2KvCw8bDSyk/Qs4mY1tQNyPYO90ITUPMra8BO1lUykcr9vDVuiyUggf6tuOhuPY09jL3uQMJdCEcwckDlcv9HtpqbAvuVdmoo3GgqeU1VAfyi3hrSTo/bztE80YeTBwcwW29QvFwM2daqgS6EI4mP9PaqGMuHE0GFLS92rhq7zgGfP3NrrDBSc4p4PUFqazPPE5YCx+eGRHNiNjW9b6UgAS6EI4sb7d1RcgEyE0zGnWE9bM26rgBfJqbXWGDobVmRfox3liQxu5jp7kytCnPj4qhR1j9/Qwk0IVwFkd3VTbqyN8LLm4QPtAI9+hrjXVmRJ0rr7AwZ0sOU5dmcOxUCcM7BfDMiGjC/X3r/L0l0IVwNlrD4e3W5X7nwslscPWADkOsjTpGgKdzd+6xB0Wl5Xy2eh//XrWX4nILt/cKZeKQCFr6etbZe0qgC+HMtIaDW6zDMnPh1CFw84KIYcaVe8Rw8PAxu0qnlnuqhPeWZfDdpgN4u7vy0IBwHugbjreH7ZcSkEAXoqGwWODAxspGHWeOgXsj44q9U7xxBe/unK3Z7MHe3NP8a2EaS3YdJaCxJ5OGRjG2ezCuNlxKQAJdiIbIUmE8uPRHo46z+eDZ2Bhr7xQP4XHSqKOObNqXz+sLUtl24CRRAX48OyqauEh/m8yIkUAXoqGrKIN9q4xpkGm/GOvMeDWFmOuNYZmw/uDqXL03zaa1ZkHyESYvTiPreBFXt2/B86NiiG1TuxvXEuhCiErlpbB3ubVRxwIoPQU+LSsbdbS9Whp12FBpuYVvNmbx/rLdnCgq48ZubZg0LJLgZpd3X0MCXQhxYWXFxjK/KQnGsr9lReDb2tqoYywE95RGHTZScLaMaav2MmPNPm7vHco/ru90WceRQBdCVK/0jNGgY2cCZCyBihJoHGxt1BEPQVfKcr82cOjkWbzdXWl2mV2SatuxaAZwHXBMax17gdebAF9jdDByA97SWn9RXVES6ELYseJCo7XezgTYswwsZdAsrLJRR+vOEu4mqW2g9wdOAzOrCPTngSZa62eUUv5AOtBaa116seNKoAvhIM6eOKdRx0rQFdCiwzmNOmLMrrBBuVigV3tbW2udqJQKu9gugJ+1r6gvkA+UX06hQgg75N0Mut1pfJw5DqnzjCv31W9B4mTwjzHG22PjoUV7s6tt0Go0hm4N9PlVXKH7AfOAaMAPGKe1/rWK44wHxgOEhoZ2z8rKuvzKhRDmOnXUCPeUBMheZ2xrfYW1UceNxhCNsLla3xStJtBvAq4BngDaA0uBLlrrwosdU4ZchHAiBQdh109GuB+0/rtu093aqGMMNAk2szqncrFAt8V8pPuABG3YA+zDuFoXQjQUTdpAn0fhwWUwcQcMedloir3kBXinE3w+HDb+27iqF3XGFoGeDQwGUEoFAFFApg2OK4RwRM3aQt/H4M+JMCEJBr0IJadg4dPwdhR8eR38/jmcyTO7UqdTk1ku3wFxQEvgKPAPwB1Aaz1NKRUEfAkEAgp4U2v9dXVvLEMuQjQwuemVjTryMkC5QvgAY1gm5jrj5quoljxYJISwH1rD0ZTKcD+xH1zcof0g44Zq1Cjwamx2lXZLAl0IYZ+0Nhpi/7Hcb8EBcPWEiKHGTJmokeDRyOwq7Uqt5qELIUSdUQraXGl8DHnFmCGTkmDMmEmbD27eEDnc2qhjGLh7m12xXZMrdCGE/bFYIHt95ZV7UR54+BrDMbHxxvCMW921ebNnMuQihHBcFeWwf7UR7qm/GEsReDYxbqR2ijdurLq6m11lvZFAF0I4h4oyYz2ZlARjSKakELybn9Ooo5/Tr+UugS6EcD7lJcZKkDsTjJUhS09DI39jLfdO8RDaxynXcpebokII5+PmCdGjjI+ys7B7iXHlvvUb+P0z8Asylh3oFA/BPRrEcr9yhS6EcC4lp43uSykJRjemilJoElrZqCOwq0OHuwy5CCEapuICo2/qzgSjj6qlHJqHVzbqCOjkcOEugS6EEEX5xo3UlATYl2g06mgZWdmowz/K7AprRAJdCCHOdSYPdv0MO+fC/jWAhoBY48o9Nt64irdTEuhCCFGVwsPWcE+AAxuNbYFdKxt1NA01tbzzSaALIURNnDxQ2ajjUJKxLbhnZaOOxkFmVgdIoAshxKXL32cMyexMgCPJgDLmtsfGG3PdfVuZUpYEuhBC1EbeHiPYUxIgNxWUi/FUamw8xNwAPs3rrRQJdCGEsJWjuyrDPX8vuLhBeJwxLBN9LXg3rdO3r1WgK6VmANcBxy7UJNq6TxzwLkYnozyt9YDqipJAF0I4NK3hyI7KRh0ns8HVA9oPtjbqGAmefjZ/29oGen/gNDDzQoGulGoKrANGaK2zlVKttNbHqitKAl0I4TS0hoNJ1uV+50LhQXDzsjbqiIfIEeDhY5O3qtVaLlrrRKVU2EV2uR1I0FpnW/evNsyFEMKpKAXB3Y2Pof+EnE2VjTpSfwF3H+OKvVM8dBgC7l51UoYtFueKBNyVUisBP+A9rfXMC+2olBoPjAcIDbWvuZ1CCGETLi4QepXxMeINyFprhHvqPEj5ETwbw4Bn4Oq/2PytbRHobkB3YDDgDaxXSm3QWmecv6PWejowHYwhFxu8txBC2C8XV2jX3/gY9RbsW2UMy9TRfHZbBHoOcFxrfQY4o5RKBLoA/xPoQgjRYLm6QYfBxkcdscXq7z8DfZVSbkopH6A3kGqD4wohhLgE1V6hK6W+A+KAlkqpHOAfGNMT0VpP01qnKqUWATsAC/CZ1jql7koWQghxITWZ5XJbDfaZAkyxSUVCCCEui/M13BNCiAZKAl0IIZyEBLoQQjgJCXQhhHASEuhCCOEkTFs+VymVC2Rd5re3BPJsWI4jkHNuGOScG4banHNbrbX/hV4wLdBrQym1uarVxpyVnHPDIOfcMNTVOcuQixBCOAkJdCGEcBKOGujTzS7ABHLODYOcc8NQJ+fskGPoQggh/pejXqELIYQ4jwS6EEI4CbsOdKXUCKVUulJqj1Lq2Qu87qmUmmV9fWM1vU8dQg3O+Qml1C6l1A6l1DKlVFsz6rSl6s75nP3GKqW0Usrhp7jV5JyVUrdYf9Y7lVLf1neNtlaDv9uhSqkVSqmt1r/fo8yo01aUUjOUUseUUhdcTlwZ3rf+eexQSl1Z6zfVWtvlB+AK7AXCAQ9gO9DxvH0eAaZZP78VmGV23fVwzgMBH+vnDzeEc7bu5wckAhuAHmbXXQ8/5whgK9DM+nUrs+uuh3OeDjxs/bwjsN/sumt5zv2BK4GUKl4fBSwEFHAVsLG272nPV+i9gD1a60ytdSnwPTD6vH1GA19ZP58DDFZKqXqs0daqPWet9QqtdZH1yw1AcD3XaGs1+TkD/BP4F1Bcn8XVkZqc84PAR1rrEwBa62P1XKOt1eScNdDY+nkT4FA91mdzWutEIP8iu4wGZmrDBqCpUiqwNu9pz4HeBjhwztc51m0X3EdrXQ4UAC3qpbq6UZNzPtcDGP/DO7Jqz9n6q2iI1vrX+iysDtXk5xwJRCql1iqlNiilRtRbdXWjJuf8EnCntTPaAmBC/ZRmmkv9914tWzSJFiZQSt0J9AAGmF1LXVJKuQBTgXtNLqW+uWEMu8Rh/BaWqJTqrLU+aWZRdew24Eut9dtKqT7Af5RSsVpri9mFOQp7vkI/CISc83WwddsF91FKuWH8mna8XqqrGzU5Z5RSQ4AXgBu01iX1VFtdqe6c/YBYYKVSaj/GWOM8B78xWpOfcw4wT2tdprXeB2RgBLyjqsk5PwDMBtBarwe8MBaxclY1+vd+Kew50H8HIpRS7ZRSHhg3Peedt8884B7r5zcBy7X1boODqvaclVLdgH9jhLmjj6tCNeestS7QWrfUWodprcMw7hvcoLXebE65NlGTv9s/YVydo5RqiTEEk1mPNdpaTc45GxgMoJSKwQj03Hqtsn7NA+62zna5CijQWh+u1RHNvhNczV3iURhXJnuBF6zbXsH4Bw3GD/wHYA+wCQg3u+Z6OOffgKPANuvHPLNrrutzPm/flTj4LJca/pwVxlDTLiAZuNXsmuvhnDsCazFmwGwDhpldcy3P9zvgMFCG8RvXA8BDwEPn/Iw/sv55JNvi77U8+i+EEE7CnodchBBCXAIJdCGEcBIS6EII4SQk0IUQwklIoAshhJOQQBdCCCchgS6EEE7i/wBS80RvxTwM8QAAAABJRU5ErkJggg==\n",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "loss = results.history[\"loss\"]\n",
+    "val_loss = results.history[\"val_loss\"]\n",
+    "plt.plot(loss, label=\"loss\")\n",
+    "plt.plot(val_loss, label=\"val_loss\")\n",
+    "plt.legend()\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
+   "id": "1bcd977b",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA48AAAOjCAYAAAAbBzFCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAoLklEQVR4nO3debhkd13n8c+XNM2aREKCJJ0OSRDQToAAAWQVcAuJmaDDooAs4hNRERHRYUYfGHHwGWTXgZkBYZRFhJYtBEziODAsItmBLBAiCaTTcbLJErYmzW/+qOpwabv7e9NJdd3cer2e5z7cOufUOd97yVO575xTp2qMEQAAANiVW817AAAAAFY+8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8ArDqVdXtquoDVfXVqtp4E/bzlKo67eacbR6q6u+q6unzngOAWxbxCMCKUVVPrqozq+q6qrpiGjkPvxl2/fgkP5zkzmOMJ+zuTsYYbx9j/MzNMM8PqKpHVdWoqvdut/y+0+UfWeZ+/nNVva3bbozx2DHGX+3muAAsKPEIwIpQVc9P8pokf5JJ6B2S5PVJTrgZdn+3JBeNMa6/GfY1K1cleUhV3XnJsqcnuejmOkBN+Hc/ALvFv0AAmLuq2jfJS5L85hjjPWOMb4wxvjvG+MAY4/em29ymql5TVZunX6+pqttM1z2qqjZV1e9W1ZXTs5bPnK77oyQvSvKk6RnNZ21/hq6qDp2e4VszffyMqvpiVX29qi6pqqcsWf7xJc97aFWdMb0c9oyqeuiSdR+pqj+uqk9M93NaVe2/i1/DliTvS/KL0+fvleRJSd6+3e/qtVV1WVV9rarOqqpHTJcfk+Q/Lfk5P71kjpdW1SeSfDPJ4dNlvzpd/9+r6t1L9v+yqvqHqqrl/v8HwGIQjwCsBA9Jctsk793FNn+Q5MeTHJXkvkkelOQPl6y/a5J9k6xL8qwkr6uqO40xXpzJ2cx3jjHuOMZ4064Gqao7JPmzJI8dY+yd5KFJzt3Bdvsl+eB02zsneVWSD2535vDJSZ6Z5C5J1iZ5wa6OneQtSZ42/f5nk5yXZPN225yRye9gvyR/nWRjVd12jHHKdj/nfZc855eTnJhk7yRf2m5/v5vk3tMwfkQmv7unjzFGMysAC0Y8ArAS3DnJ1c1lpU9J8pIxxpVjjKuS/FEmUbTNd6frvzvG+FCS65Lcazfn+V6SI6vqdmOMK8YY5+9gm+OSfGGM8dYxxvVjjHck+VyS45ds87/GGBeNMb6V5F2ZRN9OjTH+Mcl+VXWvTCLyLTvY5m1jjGumx3xlktuk/zn/coxx/vQ5391uf9/M5Pf4qiRvS/JbY4xNzf4AWEDiEYCV4Jok+2+7bHQnDsoPnjX70nTZDfvYLj6/meSON3aQMcY3Mrlc9NlJrqiqD1bVjy5jnm0zrVvy+F92Y563JnlOkkdnB2diq+oFVXXh9FLZr2RytnVXl8MmyWW7WjnG+FSSLyapTCIXAP4N8QjASvDJJN9J8rhdbLM5kxvfbHNI/u0lncv1jSS3X/L4rktXjjFOHWP8dJIDMzmb+MZlzLNtpst3c6Zt3prkN5J8aHpW8AbTy0p/P8kTk9xpjPFDSb6aSfQlyc4uNd3lJahV9ZuZnMHcPN0/APwb4hGAuRtjfDWTm9q8rqoeV1W3r6pbV9Vjq+pPp5u9I8kfVtUB0xvPvCiTyyx3x7lJHllVh0xv1vMft62oqh+uqhOm7338TiaXv35vB/v4UJJ7Tj9eZE1VPSnJhiQn7+ZMSZIxxiVJfiKT93hub+8k12dyZ9Y1VfWiJPssWf//khx6Y+6oWlX3TPJfkjw1k8tXf7+qjtq96QFYzcQjACvC9P17z8/kJjhXZXKp5XMyuQNpMgmcM5N8Jslnk5w9XbY7x/r7JO+c7uus/GDw3Wo6x+Yk12YScr++g31ck+TnMrnhzDWZnLH7uTHG1bsz03b7/vgYY0dnVU9NckomH9/xpSTfzg9ekrpx+r/XVNXZ3XGmlwm/LcnLxhifHmN8IZM7tr51251sAWCbcjM1AAAAOs48AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQWjPvAZbaf//9x90OPWTeYwAL7OyLzpv3CMCCu/89j5z3CMAC+9KlX87VV19dO1q3ouLxbocekk986uPzHgNYYLc75p7zHgFYcJ84xd9CwPw87MEP3+k6l60CAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8ciqc9opp+U+G47KEfe6d17+slfMexxgUY2R/NOVyblXz3sSYAH5e4hZEI+sKlu3bs3znvv8vP/k9+acz56Vje/cmAsvuHDeYwGL6MvXJXdYM+8pgAXk7yFmRTyyqpxx+pm5+90Pz2GHH5a1a9fmCU98fE4+6eR5jwUsmm9vTa7+TrLuDvOeBFhA/h5iVsQjq8rmzZtz8PqDb3i87uB1uXzzFXOcCFhIF30lucc+854CWFD+HmJWZhqPVXVMVX2+qi6uqhfO8lgAsCJc9a1k7V7JPmvnPQkA3Kxm9maMqtoryeuS/HSSTUnOqKqTxhgXzOqYcNBBB2XTZZtueHz5psuz7qAD5zgRsHC+umUSkFd/O/neSK4fyXnXJkfuN+/JgAXh7yFmZZZnHh+U5OIxxhfHGFuS/E2SE2Z4PMjRD3xALr74n3PpJZdmy5Yt2fiuv81xxx8377GARfIj+yaPODB5+F0nwbjfWuEI7FH+HmJWZnkbuHVJLlvyeFOSB8/weJA1a9bk1a99ZY4/9oRs3bo1T3/G07LhiA3zHgsAYI/x9xCzMvd7iFfViUlOTJL1h6yf8zSsBscce0yOOfaYeY8BkOx3m8kXwB7m7yFmYZaXrV6eZGkNHjxd9gPGGG8YYxw9xjj6gAP2n+E4AAAA7K5ZxuMZSe5RVYdV1dokv5jkpBkeDwAAgBmZ2WWrY4zrq+o5SU5NsleSN48xzp/V8QAAAJidmb7ncYzxoSQfmuUxAAAAmL1ZXrYKAADAKiEeAQAAaIlHAAAAWuIRAACAlngEAACgJR4BAABoiUcAAABa4hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BAAAoCUeAQAAaIlHAAAAWuIRAACAlngEAACgJR4BAABoiUcAAABa4hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BAAAoCUeAQAAaIlHAAAAWuIRAACAlngEAACgJR4BAABoiUcAAABa4hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BAAAoCUeAQAAaIlHAAAAWuIRAACAlngEAACgJR4BAABoiUcAAABa4hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BAAAoCUeAQAAaIlHAAAAWuIRAACAlngEAACgJR4BAABoiUcAAABa4hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BAAAoCUeAQAAaIlHAAAAWuIRAACAlngEAACgJR4BAABoiUcAAABa4hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BAAAoCUeAQAAaIlHAAAAWuIRAACAlngEAACgJR4BAABoiUcAAABa4hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BAAAoCUeAQAAaIlHAAAAWuIRAACAlngEAACgJR4BAABoiUcAAABa4hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BAAAoCUeAQAAaIlHAAAAWuIRAACAlngEAACgJR4BAABoiUcAAABa4hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BAAAoCUeAQAAaIlHAAAAWuIRAACAlngEAACgJR4BAABoiUcAAABa4hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BAAAoCUeAQAAaIlHAAAAWuIRAACAlngEAACgJR4BAABoiUcAAABa4hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BAAAoCUeAQAAaIlHAAAAWuIRAACAlngEAACgJR4BAABoiUcAAABaa+Y9AMBK8q1TLpr3CMCCO+fq0+c9ArDAvvndb+x0nTOPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQOtGxWNV3amq7jOrYQAAAFiZ2nisqo9U1T5VtV+Ss5O8sapeNfvRAAAAWCmWc+Zx3zHG15L8QpK3jDEenOSnZjsWAAAAK8ly4nFNVR2Y5IlJTp7xPAAAAKxAy4nHlyQ5NcnFY4wzqurwJF+Y7VgAAACsJGu6DcYYG5NsXPL4i0n+/SyHAgAAYGXZaTxW1Z8nGTtbP8Z47kwmAgAAYMXZ1ZnHM/fYFAAAAKxoO43HMcZfLX1cVbcfY3xz9iMBAACw0izncx4fUlUXJPnc9PF9q+r1M58MAACAFWM5d1t9TZKfTXJNkowxPp3kkTOcCQAAgBVmOfGYMcZl2y3aOoNZAAAAWKHaj+pIcllVPTTJqKpbJ/ntJBfOdiwAAABWkuWceXx2kt9Msi7J5iRHTR8DAACwINozj2OMq5M8ZQ/MAgAAwAq1nLutHl5VH6iqq6rqyqp6f1UdvieGAwAAYGVYzmWrf53kXUkOTHJQko1J3jHLoQAAAFhZlhOPtx9jvHWMcf30621JbjvrwQAAAFg5dhqPVbVfVe2X5O+q6oVVdWhV3a2qfj/Jh/bciHDjnHbKabnPhqNyxL3unZe/7BXzHgdYQF6HgHl66fNelmOP+Pk85SeeOe9RWGV2debxrCRnJnlikl9L8uEkH0ny60meNPPJYDds3bo1z3vu8/P+k9+bcz57Vja+c2MuvMAnywB7jtchYN6OfdIxefU7XjbvMViFdnq31THGYXtyELg5nHH6mbn73Q/PYYdP/vF9whMfn5NPOjk/tuHH5jwZsCi8DgHzdr+H3DdXfPlf5j0Gq1D7UR1JUlVHJtmQJe91HGO8ZVZDwe7avHlzDl5/8A2P1x28LqeffuYcJwIWjdchAFar5XxUx4uT/Pn069FJ/jTJv1vG8948/WiP827ylAAAAMzVcu62+vgkP5nkX8YYz0xy3yT7LuN5f5nkmN0fDW68gw46KJsu23TD48s3XZ51Bx04x4mAReN1CIDVajnx+K0xxveSXF9V+yS5Msn67kljjI8mufYmzgc3ytEPfEAuvvifc+kll2bLli3Z+K6/zXHHHzfvsYAF4nUIgNVqOe95PLOqfijJGzO5A+t1ST45y6Fgd61Zsyavfu0rc/yxJ2Tr1q15+jOelg1HbJj3WMAC8ToEzNuLnv3HOecfz81Xrv1qTrjfE/Krv/eMHP9k/xGLm67GGMvfuOrQJPuMMT5zI7Y/eYxx5C62OTHJiUmy/pD1D7joi59b9jwAAKvNOVefPu8RgAX2Kz/za7nw05+vHa3b6ZnHqrr/rtaNMc6+OYYbY7whyRuS5AFH33/5JQsAAMAes6vLVl+5i3UjyWNu5lkAAABYoXYaj2OMR9+UHVfVO5I8Ksn+VbUpyYvHGG+6KfsEAABgPpZzw5zdMsb4pVntGwAAgD1rOR/VAQAAwIITjwAAALTaeKyJp1bVi6aPD6mqB81+NAAAAFaK5Zx5fH2ShyTZ9h7Gryd53cwmAgAAYMVZzg1zHjzGuH9VnZMkY4x/raq1M54LAACAFWQ5Zx6/W1V7ZfLZjqmqA5J8b6ZTAQAAsKIsJx7/LMl7k9ylql6a5ONJ/mSmUwEAALCitJetjjHeXlVnJfnJJJXkcWOMC2c+GQAAACtGG49VdUiSbyb5wNJlY4wvz3IwAAAAVo7l3DDng5m837GS3DbJYUk+n+SIGc4FAADACrKcy1bvvfRxVd0/yW/MbCIAAABWnOXcMOcHjDHOTvLgGcwCAADACrWc9zw+f8nDWyW5f5LNM5sIAACAFWc573nce8n312fyHsh3z2YcAAAAVqJdxmNV7ZVk7zHGC/bQPAAAAKxAO33PY1WtGWNsTfKwPTgPAAAAK9Cuzjyensn7G8+tqpOSbEzyjW0rxxjvmfFsAAAArBDLec/jbZNck+Qx+f7nPY4k4hEAAGBB7Coe7zK90+p5+X40bjNmOhUAAAAryq7ica8kd8wPRuM24hEAAGCB7CoerxhjvGSPTQIAAMCKtdO7rWbHZxwBAABYQLuKx5/cY1MAAACwou00HscY1+7JQQAAAFi5dnXmEQAAAJKIRwAAAJZBPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0Foz7wEAAPi+hz71qfMeAVhkl1y501XOPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPAAAAtMQjAAAALfEIAABASzwCAADQEo8AAAC0xCMAAAAt8QgAAEBLPAIAANASjwAAALTEIwAAAC3xCAAAQEs8AgAA0BKPrDqnnXJa7rPhqBxxr3vn5S97xbzHARaQ1yFgRRgj+acrk3OvnvckrBLikVVl69ated5zn5/3n/zenPPZs7LxnRtz4QUXznssYIF4HQJWjC9fl9xhzbynYBURj6wqZ5x+Zu5+98Nz2OGHZe3atXnCEx+fk086ed5jAQvE6xCwInx7a3L1d5J1d5j3JKwi4pFVZfPmzTl4/cE3PF538LpcvvmKOU4ELBqvQ8CKcNFXknvsM+8pWGVmFo9Vtb6qPlxVF1TV+VX127M6FgAAMHXVt5K1eyX7rJ33JKwys7wI+vokvzvGOLuq9k5yVlX9/RjjghkekwV30EEHZdNlm254fPmmy7PuoAPnOBGwaLwOAXP31S2TgLz628n3RnL9SM67Njlyv3lPxi3czM48jjGuGGOcPf3+60kuTLJuVseDJDn6gQ/IxRf/cy695NJs2bIlG9/1tznu+OPmPRawQLwOAXP3I/smjzgwefhdJ8G431rhyM1ij9x+qaoOTXK/JJ/aE8djca1Zsyavfu0rc/yxJ2Tr1q15+jOelg1HbJj3WMAC8ToEwGpVY4zZHqDqjkn+b5KXjjHes4P1JyY5MUnWH7L+ARd98XMznQcAYCW73TH3nPcIwCL71JUZX9tSO1o107utVtWtk7w7ydt3FI5JMsZ4wxjj6DHG0QccsP8sxwEAAGA3zfJuq5XkTUkuHGO8albHAQAAYPZmeebxYUl+Ocljqurc6dexMzweAAAAMzKzG+aMMT6eZIfXygIAAHDLMtP3PAIAALA6iEcAAABa4hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BAAAoCUeAQAAaIlHAAAAWuIRAACAlngEAACgJR4BAABoiUcAAABa4hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BAAAoCUeAQAAaIlHAAAAWuIRAACAlngEAACgJR4BAABoiUcAAABa4hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BAAAoCUeAQAAaIlHAAAAWuIRAACAlngEAACgJR4BAABoiUcAAABa4hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BAAAoCUeAQAAaIlHAAAAWuIRAACAlngEAACgJR4BAABoiUcAAABa4hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BAAAoCUeAQAAaIlHAAAAWuIRAACAlngEAACgJR4BAABoiUcAAABa4hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BAAAoCUeAQAAaIlHAAAAWuIRAACAlngEAACgJR4BAABoiUcAAABa4hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BAAAoCUeAQAAaIlHAAAAWuIRAACAlngEAACgJR4BAABoiUcAAABa4hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BAAAoCUeAQAAaIlHAAAAWuIRAACAlngEAACgJR4BAABoiUcAAABa4hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BAAAoCUeAQAAaIlHAAAAWuIRAACAlngEAACgJR4BAABoiUcAAABa4hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BAAAoCUeAQAAaIlHAAAAWuIRAACAlngEAACgJR4BAABoiUcAAABa4hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BAAAoCUeAQAAaIlHAAAAWuIRAACAlngEAACgJR4BAABoiUcAAABa4hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BAAAoFVjjHnPcIOquirJl+Y9B7do+ye5et5DAAvLaxAwb16HuKnuNsY4YEcrVlQ8wk1VVWeOMY6e9xzAYvIaBMyb1yFmyWWrAAAAtMQjAAAALfHIavOGeQ8ALDSvQcC8eR1iZrznEQAAgJYzjwAAALTEI6tCVR1TVZ+vqour6oXzngdYLFX15qq6sqrOm/cswGKqqvVV9eGquqCqzq+q3573TKw+LlvlFq+q9kpyUZKfTrIpyRlJfmmMccFcBwMWRlU9Msl1Sd4yxjhy3vMAi6eqDkxy4Bjj7KraO8lZSR7n7yFuTs48sho8KMnFY4wvjjG2JPmbJCfMeSZggYwxPprk2nnPASyuMcYVY4yzp99/PcmFSdbNdypWG/HIarAuyWVLHm+KF0sAYEFV1aFJ7pfkU3MehVVGPAIAwCpRVXdM8u4kzxtjfG3e87C6iEdWg8uTrF/y+ODpMgCAhVFVt84kHN8+xnjPvOdh9RGPrAZnJLlHVR1WVWuT/GKSk+Y8EwDAHlNVleRNSS4cY7xq3vOwOolHbvHGGNcneU6SUzN5c/i7xhjnz3cqYJFU1TuSfDLJvapqU1U9a94zAQvnYUl+Ocljqurc6dex8x6K1cVHdQAAANBy5hEAAICWeAQAAKAlHgEAAGiJRwAAAFriEQAAgJZ4BOAWp6q2Tm9Df15Vbayq29+Eff1lVT1++v1fVNWGXWz7qKp66G4c49Kq2n+5y7fb5robeaz/XFUvuLEzAkBHPAJwS/StMcZRY4wjk2xJ8uylK6tqze7sdIzxq2OMC3axyaOS3Oh4BIDVQDwCcEv3sSQ/Mj0r+LGqOinJBVW1V1W9vKrOqKrPVNWvJUlN/Leq+nxV/e8kd9m2o6r6SFUdPf3+mKo6u6o+XVX/UFWHZhKpvzM96/mIqjqgqt49PcYZVfWw6XPvXFWnVdX5VfUXSar7IarqfVV11vQ5J2637tXT5f9QVQdMl929qk6ZPudjVfWjO9jnc6vqgunP/ze7+fsFgCTJbv2XWQBYCaZnGB+b5JTpovsnOXKMcck0wL46xnhgVd0mySeq6rQk90tyryQbkvxwkguSvHm7/R6Q5I1JHjnd135jjGur6n8kuW6M8Yrpdn+d5NVjjI9X1SFJTk3yY0lenOTjY4yXVNVxSZ61jB/nV6bHuF2SM6rq3WOMa5LcIcmZY4zfqaoXTff9nCRvSPLsMcYXqurBSV6f5DHb7fOFSQ4bY3ynqn5oOb9TANgZ8QjALdHtqurc6fcfS/KmTC4nPX2Mccl0+c8kuc+29zMm2TfJPZI8Msk7xhhbk2yuqv+zg/3/eJKPbtvXGOPanczxU0k2VN1wYnGfqrrj9Bi/MH3uB6vqX5fxMz23qn5++v366azXJPlekndOl78tyXumx3hoko1Ljn2bHezzM0neXlXvS/K+ZcwAADslHgG4JfrWGOOopQumEfWNpYuS/NYY49Tttjv2ZpzjVkl+fIzx7R3MsmxV9ahMQvQhY4xvVtVHktx2J5uP6XG/sv3vYAeOyyRkj0/yB1V17zHG9TdqOACY8p5HAFarU5P8elXdOkmq6p5VdYckH03ypOl7Ig9M8ugdPPefkjyyqg6bPne/6fKvJ9l7yXanJfmtbQ+q6qjptx9N8uTpsscmuVMz675J/nUajj+ayZnPbW6VZNvZ0ydncjns15JcUlVPmB6jquq+S3dYVbdKsn6M8eEk/2F6jDs2cwDATolHAFarv8jk/YxnV9V5Sf5nJlfcvDfJF6br3pLkk9s/cYxxVZITM7lE9NP5/mWjH0jy89tumJPkuUmOnt6Q5oJ8/66vf5RJfJ6fyeWrX25mPSXJmqq6MMl/zSRet/lGkgdNf4bHJHnJdPlTkjxrOt/5SU7Ybp97JXlbVX02yTlJ/myM8ZVmDgDYqRpjzHsGAAAAVjhnHgEAAGiJRwAAAFriEQAAgJZ4BAAAoCUeAQAAaIlHAAAAWuIRAACAlngEAACg9f8B0stDya7eYS0AAAAASUVORK5CYII=\n",
+      "text/plain": [
+       "<Figure size 1152x1152 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "from sklearn.metrics import confusion_matrix\n",
+    "\n",
+    "n_class = y_train_ctg.shape[1]\n",
+    "cm_test = confusion_matrix(y_test, yhat_test)\n",
+    "plt.figure(figsize=(16, 16))\n",
+    "plt.imshow(cm_test, cmap=plt.cm.Greens)\n",
+    "class_arr = np.arange(n_class)\n",
+    "plt.xticks(class_arr)\n",
+    "plt.yticks(class_arr)\n",
+    "plt.title('Confusion Matrix')\n",
+    "plt.xlabel('Predicted labels')\n",
+    "plt.ylabel('True labels')\n",
+    "\n",
+    "for i in range(n_class):\n",
+    "    for j in range(n_class):\n",
+    "        plt.text(i, j, cm_test[j][i])"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "venv",
+   "language": "python",
+   "name": "venv"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

codes/sampling.py

@@ -0,0 +1,395 @@
+"""This module contains Smp class to collect inputs from the user. To understand this module you should know about Mediapipe library."""
+
+
+import numpy as np
+import cv2
+import time
+from codes.base import eyeing as ey
+from codes.calibration import Clb
+import math
+import random
+from datetime import datetime
+
+
+class Smp(object):
+    running = True
+
+    def sampling(self, num, camera_id=0, gui=True):
+        """
+        Collecting inputs (eyes images and face vectors) from the user as the main sampling data.
+
+        Parameters:
+            num: subject number
+            camera_id: Camera ID
+            gui: If it's False, for having the ability to stop the program, a little window will be appeared. So, you can press "q" to stop it. If it's True, you can stop program using GUI.
+        
+        Returns:
+            None
+        """
+
+        face_saving_time = 80
+        return_face1 = True
+        win_name = "Sampling"
+        little_win_name = "smp"
+        tx0 = [["Sampling", (0.25, 0.5), 2, ey.RED, 3]]
+        tx1 = [["SPACE --> start/pause", (0.05, 0.3), 1.5, ey.RED, 3],
+        ["ESC --> Stop", (0.05, 0.6), 1.6, ey.RED, 3]]
+
+        some_landmarks_ids = ey.get_some_landmarks_ids()
+
+        (
+            frame_size,
+            camera_matrix,
+            dst_cof,
+            pcf
+        ) = ey.get_camera_properties(camera_id)
+
+        face_mesh = ey.get_mesh()
+
+        cap = ey.get_camera(camera_id, frame_size)
+        ey.pass_frames(cap, 100)
+
+        print("Sampling started...")
+        t_mat = []
+        sys_time_mat = []
+        eyes_mat = []
+        inp_scalars_mat = []
+        eyes_ratio_mat = []
+        face_vec = []
+        fps_vec = []
+
+        ey.big_win(win_name, math.floor(len(ey.monitors) / 2)*ey.monitors[0].width)
+        ey.show_clb_win(win_name, win_color=ey.WHITE, texts=tx0)
+        cv2.waitKey(4000)
+        cv2.destroyWindow(win_name)
+
+        t0 = time.perf_counter()
+        while self.running:
+            j = 0
+            ey.big_win(win_name, math.floor(len(ey.monitors) / 2)*ey.monitors[0].width)
+            ey.show_clb_win(win_name, win_color=ey.WHITE, texts=tx1)
+            button = cv2.waitKey(0)
+            cv2.destroyWindow(win_name)
+            if (button == ord('q')) or (button == ord('Q')) or (button == 27):
+                break
+            elif button == ord(' '):
+                t_vec = []
+                sys_time_vec = []
+                eyes_vec = []
+                inp_scalars_vec = []
+                eyes_ratio_vec = []
+                t1 = time.perf_counter()
+                while self.running:
+                    frame_success, frame, frame_rgb = ey.get_frame(cap)
+                    if frame_success:
+                        return_face = False
+                        if ((time.perf_counter() - t0) < face_saving_time) and return_face1:
+                            return_face = True
+
+                        results = face_mesh.process(frame_rgb) # Get inputs of the models
+                        
+                        # Get inputs of the models
+                        (
+                            features_success,
+                            _,
+                            eyes_frame_gray,
+                            features_vector,
+                            eyes_ratio,
+                            face_img
+                        ) = ey.get_model_inputs(
+                            frame,
+                            frame_rgb,
+                            results,
+                            camera_matrix,
+                            pcf,
+                            frame_size,
+                            dst_cof,
+                            some_landmarks_ids,
+                            return_face=return_face
+                        )
+                        if features_success:
+                            t_vec.append(round(time.perf_counter() - t0, 3))
+                            sys_time_vec.append(str(datetime.now())[-15:-3])
+                            eyes_vec.append(eyes_frame_gray)
+                            inp_scalars_vec.append(features_vector)
+                            eyes_ratio_vec.append(eyes_ratio)
+                            if return_face:
+                                face_vec.append(face_img)
+                            j += 1
+                            if not gui:
+                                ey.show_clb_win(little_win_name, win_color=ey.RED, win_size=(50, 50))
+                                button = cv2.waitKey(1)
+                                if (button == ord('q')) or (button == ord('Q')) or (button == 27) or (button == ord(' ')):
+                                    break
+            fps_vec.append(ey.get_time(j, t1, True))
+            t_mat.append(np.array(t_vec))
+            sys_time_mat.append(sys_time_vec)
+            eyes_mat.append(np.array(eyes_vec))
+            inp_scalars_mat.append(np.array(inp_scalars_vec))
+            eyes_ratio_mat.append(np.array(eyes_ratio_vec))
+            
+            if not gui:
+                cv2.destroyWindow(little_win_name)
+            if (button == ord('q')) or (button == ord('Q')) or (button == 27):
+                break
+            return_face1 = False
+
+        print("Sampling finished")
+        ey.get_time(0, t0, True)
+        print(f"Mean FPS : {np.array(fps_vec).mean()}")
+
+        cv2.destroyAllWindows()
+        cap.release()
+
+        smp_dir = ey.create_dir([ey.subjects_dir, f"{num}", ey.SMP])
+        ey.save(
+            [t_mat, sys_time_mat, eyes_mat, inp_scalars_mat, eyes_ratio_mat, [np.array(face_vec)]],
+            smp_dir,
+            [ey.T, "sys_time", ey.X1, ey.X2, ey.ER, ey.FV])
+
+
+    def accuracy(self, num, camera_id=0, clb_grid=(2, 2, 10)):
+        """
+        Collecting inputs (eyes images and face vectors) from the user to assess the accuracy of the models. It shows a point in a grid
+        and the user must look at that. So, with the true outputs (locations of the point) and the predicted outputs, it's possible to
+        evaluate the models. The evaulation part is in the eye_track.py module.
+
+        Parameters:
+            num: subject number
+            camera_id: Camera ID
+            clb_grid: The grid that you want to use for comparison
+
+        Returns:
+            None
+        """
+
+        # Collecting data for testing
+        tx0 = [["Track WHITE point", (0.05, 0.25), 1.5, ey.RED, 3],
+        ["SPACE --> start", (0.05, 0.5), 1.5, ey.RED, 3],
+        ["ESC --> Stop", (0.05, 0.75), 1.5, ey.RED, 3]]
+        clb_points = Clb().create_grid(clb_grid)
+
+        some_landmarks_ids = ey.get_some_landmarks_ids()
+
+        (
+            frame_size,
+            camera_matrix,
+            dst_cof,
+            pcf
+        ) = ey.get_camera_properties(camera_id)
+
+        face_mesh = ey.get_mesh()
+
+        i = 0
+        fps_vec = []
+        t_mat = []
+        eyes_mat = []
+        inp_scalars_mat = []
+        points_loc_mat = []
+        eyes_ratio_mat = []
+        cap = ey.get_camera(camera_id, frame_size)
+        ey.pass_frames(cap, 100)
+        t0 = time.perf_counter()
+
+        win_name = "Information"
+        ey.big_win(win_name, math.floor(len(ey.monitors) / 2)*ey.monitors[0].width)
+        ey.show_clb_win(win_name, texts=tx0, win_color=ey.WHITE)
+        cv2.waitKey(10000)
+        cv2.destroyWindow(win_name)
+        for (i_m, m) in enumerate(ey.monitors):
+            if not self.running:
+                break
+            win_name = f"Calibration-{i_m}"
+            ey.big_win(win_name, i_m * m.width)
+            for item in clb_points:
+                if not self.running and (i_m != 0):
+                    break
+                pnt = item[0]
+                t_vec = []
+                eyes_vec = []
+                inp_scalars_vec = []
+                points_loc_vec = []
+                eyes_ratio_vec = []
+                ey.show_clb_win(win_name, pnt, win_color=ey.GRAY)
+
+                button = cv2.waitKey(0)
+                if (button == ord('q')) or (button == ord('Q')) or (button == 27):
+                    break
+                elif button == ord(' '):
+                    ey.pass_frames(cap)
+                    t1 = time.perf_counter()
+                    s = len(item)
+                    for pnt in item:
+                        ey.show_clb_win(win_name, pnt)
+                        button = cv2.waitKey(1)
+                        if (button == ord('q')) or (button == ord('Q')) or (button == 27):
+                            break
+                        while True:
+                            frame_success, frame, frame_rgb = ey.get_frame(cap)
+                            if frame_success:
+                                results = face_mesh.process(frame_rgb)
+                                (
+                                    features_success,
+                                    _,
+                                    eyes_frame_gray,
+                                    features_vector,
+                                    eyes_ratio,
+                                    _
+                                ) = ey.get_model_inputs(
+                                    frame,
+                                    frame_rgb,
+                                    results,
+                                    camera_matrix,
+                                    pcf,
+                                    frame_size,
+                                    dst_cof,
+                                    some_landmarks_ids
+                                )
+                                if features_success:
+                                    t_vec.append(round(time.perf_counter() - t1, 3))
+                                    eyes_vec.append(eyes_frame_gray)
+                                    inp_scalars_vec.append(features_vector)
+                                    points_loc_vec.append([(pnt[0] + i_m)/len(ey.monitors), pnt[1]])
+                                    eyes_ratio_vec.append(eyes_ratio)
+                                    i += 1
+                                    break
+                        if not self.running:
+                            break
+                    fps_vec.append(ey.get_time(s, t1))
+                    t_mat.append(np.array(t_vec))
+                    eyes_mat.append(np.array(eyes_vec))
+                    inp_scalars_mat.append(np.array(inp_scalars_vec))
+                    points_loc_mat.append(np.array(points_loc_vec))
+                    eyes_ratio_mat.append(np.array(eyes_ratio_vec))
+                    
+                if not self.running:
+                    break
+                if (button == ord('q')) or (button == ord('Q')) or (button == 27):
+                    break
+            cv2.destroyWindow(win_name)
+            if (button == ord('q')) or (button == ord('Q')) or (button == 27):
+                break
+        cap.release()
+
+        cv2.destroyAllWindows()
+        ey.get_time(0, t0, True)
+        print(f"Mean FPS : {np.array(fps_vec).mean()}")
+
+        acc_dir = ey.create_dir([ey.subjects_dir, f"{num}", ey.ACC])
+        ey.save(
+            [t_mat, eyes_mat, inp_scalars_mat, points_loc_mat, eyes_ratio_mat],
+            acc_dir,
+            [ey.T, ey.X1, ey.X2, ey.Y, ey.ER])
+        print("Accuracy data collected!")
+
+
+    def latency(self, num, camera_id=0):
+        """ Collecting data to assessing latency. The user should look at the left and right side of the screen when they see red or blue colors.
+
+        Parameters:
+            num: Subject number
+            camera_id: Camera ID
+        
+        Returns:
+            None
+        """
+        tx1 = [["SPACE --> start", (0.05, 0.2), 1.3, ey.BLACK, 2],
+            [f"ESC --> stop", (0.05, 0.4), 1.3, ey.BLACK, 2],
+            ["RED --> Left", (0.05, 0.6), 1.3, ey.RED, 2],
+            ["BLUE --> Right", (0.05, 0.8), 1.3, ey.BLUE, 2]]
+        some_landmarks_ids = ey.get_some_landmarks_ids()
+
+        (
+            frame_size,
+            camera_matrix,
+            dst_cof,
+            pcf
+        ) = ey.get_camera_properties(camera_id)
+
+        face_mesh = ey.get_mesh()
+
+        fps_vec = []
+        t_mat = []
+        eyes_mat = []
+        inp_scalars_mat = []
+        cap = ey.get_camera(camera_id, frame_size)
+        ey.pass_frames(cap, 100)
+        t0 = time.perf_counter()
+
+        win_name = "Information"
+        ey.big_win(win_name, math.floor(len(ey.monitors) / 2) * ey.monitors[0].width)
+        ey.show_clb_win(win_name, texts=tx1, win_color=ey.WHITE)
+        button = cv2.waitKey(0)
+        if button == ord(' '):
+            cv2.destroyWindow(win_name)
+            win_name = "Latency"
+            time.sleep(2)
+            for j in range(6):
+                if not self.running:
+                    break
+                t_vec = []
+                eyes_vec = []
+                inp_scalars_vec = []
+                i = 0
+                t1 = time.perf_counter()
+                dt = random.random()*3
+                ey.big_win(win_name, math.floor(len(ey.monitors) / 2) * ey.monitors[0].width)
+                if j % 2 == 0:
+                    win_color = ey.BLUE
+                else:
+                    win_color = ey.RED
+                ey.show_clb_win(win_name, win_color=win_color)
+                cv2.waitKey(ey.LATENCY_WAITING_TIME)
+                cv2.destroyWindow(win_name)
+                while (time.perf_counter()-t1) < (3 + dt):
+                    if (button == ord('q')) or (button == ord('Q')) or (button == 27):
+                        break
+                    while True:
+                        frame_success, frame, frame_rgb = ey.get_frame(cap)
+                        if frame_success:
+                            results = face_mesh.process(frame_rgb)
+                            (
+                                features_success,
+                                _,
+                                eyes_frame_gray,
+                                features_vector,
+                                eyes_ratio,
+                                _
+                            ) = ey.get_model_inputs(
+                                frame,
+                                frame_rgb,
+                                results,
+                                camera_matrix,
+                                pcf,
+                                frame_size,
+                                dst_cof,
+                                some_landmarks_ids
+                            )
+                            if features_success:
+                                t_vec.append(round(time.perf_counter() - t1, 3))
+                                eyes_vec.append(eyes_frame_gray)
+                                inp_scalars_vec.append(features_vector)
+                                i += 1
+                                break
+                    if not self.running:
+                        break
+                fps_vec.append(ey.get_time(i, t1))
+                t_mat.append(np.array(t_vec))
+                eyes_mat.append(np.array(eyes_vec))
+                inp_scalars_mat.append(np.array(inp_scalars_vec))
+                
+                if not self.running:
+                    break
+                if (button == ord('q')) or (button == ord('Q')) or (button == 27):
+                    break
+        cap.release()
+
+        ey.get_time(0, t0, True)
+        print(f"Mean FPS : {np.array(fps_vec).mean()}")
+
+        ltn_dir = ey.create_dir([ey.subjects_dir, f"{num}", ey.LTN])
+        ey.save(
+            [t_mat, eyes_mat, inp_scalars_mat],
+            ltn_dir,
+            [ey.T, ey.X1, ey.X2])
+        print("Latency data collected!")

codes/see_data.py

@@ -0,0 +1,383 @@
+"""This module is for seeing the inputs and the outputs."""
+
+import cv2
+from codes.base import eyeing as ey
+import time
+import matplotlib.pyplot as plt
+import numpy as np
+import os
+import math
+from openpyxl import load_workbook
+
+
+class See(object):
+    running = True
+
+    @staticmethod
+    def data_features(num, target_fol=ey.CLB):
+        """
+        Seeing the inputs in each one of the folders
+        
+        Parameters:
+            num: subject number
+            target_fol: targeted folder
+        
+        Returns:
+            None
+        """
+        sbj_dir = ey.create_dir([ey.subjects_dir, f"{num}"])
+        if target_fol == ey.CLB:
+            target_dir = ey.create_dir([sbj_dir, ey.CLB])
+            data = ey.load(target_dir, [ey.X1, ey.X2, ey.Y])
+        elif target_fol == ey.IO:
+            target_dir = ey.create_dir([sbj_dir, ey.IO])
+            data = ey.load(target_dir, [ey.X1, ey.X2, ey.Y])
+        elif target_fol == ey.SMP:
+            target_dir = ey.create_dir([sbj_dir, ey.SMP])
+            data = ey.load(target_dir, [ey.X1, ey.X2, ey.T])
+        elif target_fol == ey.ACC:
+            target_dir = ey.create_dir([sbj_dir, ey.ACC])
+            data = ey.load(target_dir, [ey.X1, ey.X2, ey.T, ey.Y])
+        elif target_fol == ey.LTN:
+            target_dir = ey.create_dir([sbj_dir, ey.LTN])
+            data = ey.load(target_dir, [ey.X1, ey.X2, ey.T])
+        else:
+            data = None
+            print("The folder isn't valid!!")
+            quit()
+
+        win_name = "Eyes"
+        cv2.namedWindow(win_name)
+        if len(ey.monitors) == 1:
+            cv2.moveWindow(win_name, int(ey.monitors[0].width / 2), int(ey.monitors[0].height / 2))
+        else:
+            cv2.moveWindow(win_name, ey.monitors[0].width + int(ey.monitors[0].width / 2), int(ey.monitors[0].height / 2))
+
+        x1 = data[0]
+        print(f"Number of vectors : {len(x1)}")
+        time.sleep(2)
+
+        i = 0
+        for (k, x1_vec) in enumerate(x1):
+            for (s, img) in enumerate(x1_vec):
+                d = []
+                for (j, _) in enumerate(data):
+                    if j == 0:
+                        continue
+                    d.append(data[j][k][s])
+                if True: #i % 10 == 0:
+                    print(f"{i}, {d}")
+                    cv2.imshow(win_name, img)
+                    q = cv2.waitKey(20)
+                    if q == ord('q') or q == ord('Q'):
+                        break
+                i += 1
+            if q == ord('q') or q == ord('Q'):
+                break
+        cv2.destroyAllWindows()
+
+
+    def pixels_smp(self, num, n_monitors_data=len(ey.monitors), show_in_all_monitors=False, win_size=(1280,720), show_fixations=False):
+        """
+        See the eye viewpoint of the user during sampling.
+
+        Parameters:
+            num: subject number
+            n_monitors_data: The number of monitors while the data was collecting.
+            show_in_all_monitors: Just for the moment that we have more than one monitor. So we tune the parameters to show the data in all of them
+            win_size: size of the appeared window
+            show_fixations: It shows the fixations
+        
+        Returns:
+            None
+        """
+        little_win = False
+        smp_dir = ey.create_dir([ey.subjects_dir, f"{num}", ey.SMP])
+        try:
+            sheet_et = load_workbook(smp_dir + "eye_track.xlsx")["Sheet"]
+            prd_et = []
+            for i in range(3,sheet_et.max_row+1):
+                et_splited = sheet_et[f"C{i}"].value[1:-1].split(',')
+                prd_et.append([float(sheet_et[f"A{i}"].value), float(et_splited[0]), float(et_splited[1])])
+            prd_et = np.array(prd_et)
+
+            if show_fixations:
+                sheet_fxn = load_workbook(smp_dir + "fixations.xlsx")["Sheet"]
+                fixations = []
+                for i in range(3, sheet_fxn.max_row+1):
+                    fxn_splited = sheet_fxn[f"D{i}"].value[1:-1].split(',')
+                    fixations.append([float(sheet_fxn[f"A{i}"].value), float(sheet_fxn[f"C{i}"].value),
+                        float(fxn_splited[0]), float(fxn_splited[1])])
+                fixations = np.array(fixations)
+
+            if show_in_all_monitors:
+                win_names = []
+                for (i, m) in enumerate(ey.monitors):
+                    win_name = f"Calibration-{i}"
+                    ey.big_win(win_name, i * m.width)
+                    win_names.append(win_name)
+            elif (n_monitors_data == 1):
+                win_name = "Calibration"
+                ey.big_win(win_name, math.floor(len(ey.monitors)/2)*ey.monitors[0].width)
+            else:
+                win_name = "Calibration"
+                little_win = True
+
+            for prd1 in prd_et:
+                t0 = prd1[0]
+                fxn_exist = False
+                if show_fixations:
+                    time_comparison = t0 - fixations[:, 0]
+                    time_comparison[time_comparison<0] = 1000
+                    matched_t_fxn_arg = time_comparison.argmin()
+                    if (t0 > fixations[matched_t_fxn_arg, 0]) and (t0 < (fixations[matched_t_fxn_arg, 0]+fixations[matched_t_fxn_arg, 1])):
+                        fxn_exist = True
+                prd0 = prd1[1:]
+                tx0 = [[f"time: {t0} sec", (0.05, 0.25), 1, ey.GREEN, 2]]
+                if show_in_all_monitors:
+                    y_prd_show = [None] * len(ey.monitors)
+                    texts = y_prd_show.copy()
+                    texts[math.floor(len(ey.monitors) / 2)] = tx0
+                    pw_prd = prd0[0] * n_monitors_data
+                    for (i, _) in enumerate(ey.monitors):
+                        if prd0[0] != -1:
+                            win_color = ey.WHITE
+                            if i != 1:
+                                t0 = None
+                            if (pw_prd > i) and (pw_prd < (i + 1)):
+                                y_prd_show[i] = prd0
+                                y_prd_show[i][0] = pw_prd - i
+                        else:
+                            prd0 = None
+                            win_color = ey.GRAY
+                        ey.show_clb_win(win_names[i], pnt_prd=y_prd_show[i], texts=texts[i], win_color=win_color)
+                else:
+                    if prd0[0] != -1:
+                        win_color = ey.WHITE
+                    else:
+                        prd0 = None
+                        win_color = ey.GRAY
+                    if little_win:
+                        if fxn_exist:
+                            ey.show_clb_win(win_name, pnt=fixations[matched_t_fxn_arg,2:],
+                                pnt_prd=prd0, texts=tx0, win_color=win_color, win_size=win_size, pnt_color=ey.RED)
+                        else:
+                            ey.show_clb_win(win_name, pnt_prd=prd0, texts=tx0, win_color=win_color, win_size=win_size)
+                        cv2.moveWindow(win_name, 0, 0)
+                    else:
+                        if fxn_exist:
+                            ey.show_clb_win(win_name, pnt=fixations[matched_t_fxn_arg,2:],
+                                pnt_prd=prd0, texts=tx0, win_color=win_color, pnt_color=ey.RED)
+                        else:
+                            ey.show_clb_win(win_name, pnt_prd=prd0, texts=tx0, win_color=win_color)
+
+                q = cv2.waitKey(50)
+                if q == ord('q') or q == ord('Q'):
+                    break
+                if not self.running:
+                    break
+            cv2.destroyAllWindows()
+        except FileNotFoundError:
+            print(f"Data does not exist in {smp_dir}")
+
+
+    def pixels_acc(self, num, n_monitors_data=len(ey.monitors), show_in_all_monitors=False):
+        """
+        See the eye viewpoint of the user during testing.
+
+        Parameters:
+            num: subject number
+            n_monitors_data: The number of monitors while the data was collecting.
+            show_in_all_monitors: Just for the moment that we have more than one monitor. So we tune the parameters to show the data in all of them
+        
+        Returns:
+            None
+        """
+        acc_dir = ey.create_dir([ey.subjects_dir, f"{num}", ey.ACC])
+        if ey.file_existing(acc_dir, 'y_mdf.pickle'):
+            [y, y_prd] = ey.load(acc_dir, ['y_mdf', 'y_prd_mdf'])
+            if show_in_all_monitors:
+                win_names = []
+                for (i, m) in enumerate(ey.monitors):
+                    win_name = f"Calibration-{i}"
+                    ey.big_win(win_name, i * m.width)
+                    win_names.append(win_name)
+            else:
+                win_name = "Calibration"
+                ey.big_win(win_name, math.floor(len(ey.monitors) / 2)*ey.monitors[0].width)
+
+            for (y0, y_prd0) in zip(y, y_prd):
+                if show_in_all_monitors:
+                    y_show = [None] * len(ey.monitors)
+                    y_prd_show = [None] * len(ey.monitors)
+                    pw = y0[0] * n_monitors_data
+                    pw_prd = y_prd0[0] * n_monitors_data
+                    for (i, _) in enumerate(ey.monitors):
+                        if (pw > i) and (pw < (i + 1)):
+                            y_show[i] = y0
+                            y_show[i][0] = pw - i
+                        if (pw_prd > i) and (pw_prd < (i + 1)):
+                            y_prd_show[i] = y_prd0
+                            y_prd_show[i][0] = pw_prd - i
+                        ey.show_clb_win(win_names[i], pnt=y_show[i], pnt_prd=y_prd_show[i], win_color=ey.WHITE, pnt_color=ey.RED)
+                else:
+                    ey.show_clb_win(win_name, pnt=y0, pnt_prd=y_prd0, win_color=ey.WHITE, pnt_color=ey.RED)
+
+                q = cv2.waitKey(50)
+                if q == ord('q') or q == ord('Q') or q == 27:
+                    break
+                if not self.running:
+                    break
+
+            cv2.destroyAllWindows()
+        else:
+            print(f"Data does not exist in {acc_dir}")
+
+    @staticmethod
+    def blinks_plot(num, threshold=ey.DEFAULT_BLINKING_THRESHOLD, target_fol="er"):
+        """
+        Plotting the eyes aspect ratio (EAR) vector to tune threshold
+        
+        Parameters:
+            num: subject number
+            threshold: the threshold of ear velocity
+            target_fol: targeted folder
+        
+        Returns:
+            None
+        """
+        sbj_dir = ey.create_dir([ey.subjects_dir, f"{num}"])
+        if target_fol == ey.ER:
+            target_dir = ey.create_dir([sbj_dir, ey.ER])
+        elif target_fol == ey.CLB:
+            target_dir = ey.create_dir([sbj_dir, ey.CLB])
+        elif target_fol == ey.SMP:
+            target_dir = ey.create_dir([sbj_dir, ey.SMP])
+        elif target_fol == ey.ACC:
+            target_dir = ey.create_dir([sbj_dir, ey.ACC])
+        else:
+            print("The folder isn't valid!!")
+            quit()
+        er_dir = ey.create_dir([sbj_dir, ey.ER])
+        
+        t_mat, eyes_ratio_mat = ey.load(target_dir, [ey.T, ey.ER])
+
+        threshold = ey.get_threshold(er_dir, threshold)
+
+        print(f"Blinking threshold is {threshold}")
+        eyes_ratio_v_mat, _, eyes_ratio_v_blink_mat = ey.get_blinking(t_mat, eyes_ratio_mat, threshold)
+
+        if len(eyes_ratio_v_mat) > 1:
+            eyes_ratio_v_vec = eyes_ratio_v_mat[0]
+            eyes_ratio_v_blink_vec = eyes_ratio_v_blink_mat[0]
+            for (i, erv) in enumerate(eyes_ratio_v_mat):
+                if i == 0:
+                    continue
+                eyes_ratio_v_vec = np.concatenate([eyes_ratio_v_vec, erv])
+                eyes_ratio_v_blink_vec = np.concatenate([eyes_ratio_v_blink_vec, eyes_ratio_v_blink_mat[i]])
+        else:
+            eyes_ratio_v_vec = eyes_ratio_v_mat[0]
+            eyes_ratio_v_blink_vec = eyes_ratio_v_blink_mat[0]
+
+        # print(eyes_ratio_v_vec)
+        plt.figure()
+        plt.plot(eyes_ratio_v_vec)
+        plt.plot(eyes_ratio_v_blink_vec)
+        plt.title(f"Velocity of Eyes Ratio ({target_fol})")
+        plt.xlabel("# of Sample")
+        plt.ylabel("ER/sec")
+        blink_img_dir = target_dir + 'blinking.png'
+        plt.savefig(blink_img_dir, dpi=300, bbox_inches='tight')
+        blink_img = cv2.imread(blink_img_dir)
+        cv2.imshow("Blinking", blink_img)
+        cv2.waitKey(0)
+        cv2.destroyAllWindows()
+        os.remove(blink_img_dir)
+
+
+    def user_face(self, num, threshold="d", save_threshold=False):
+        """
+        Show the user's face to tune blinking threshold.
+
+        Parameters:
+            num: subject number
+            threshold: the blinking threshold
+            save_threshold: save the tuned threshold
+        
+        Returns:
+            None
+        """
+        scaling_frame = 5
+        sbj_dir = ey.create_dir([ey.subjects_dir, f"{num}"])
+        smp_dir = ey.create_dir([sbj_dir, ey.SMP])
+        er_dir = ey.create_dir([sbj_dir, ey.ER])
+
+        threshold = ey.get_threshold(er_dir, threshold)
+        if save_threshold:
+            ey.save([threshold], er_dir, ["oth_usr"])
+        print(f"Blinking threshold is {threshold}")
+
+        if ey.file_existing(smp_dir, ey.T+".pickle"):
+            t_mat, face_mat, eyes_ratio_mat = ey.load(smp_dir, [ey.T, ey.FV, ey.ER])
+
+            eyes_ratio_v_mat = ey.get_blinking(t_mat, eyes_ratio_mat)[0]
+
+            face_vec = face_mat[0]
+            vec120_len, fh, fw = face_vec.shape[:-1]
+            little_vec_len = int(vec120_len / 10)
+            before_len = int(2 * little_vec_len / 3)
+            after_len = int(little_vec_len - before_len)
+            eyes_ratio_v_vec = eyes_ratio_v_mat[0][:vec120_len]
+            min_eyes_ratio_v, max_eyes_ratio_v = eyes_ratio_v_vec.min(), eyes_ratio_v_vec.max()
+            new_fw, new_fh = fw*scaling_frame, fh*scaling_frame
+            shift_edge = int(new_fh / 90.0)
+            red_area_h = int(0.85 * fh)
+            red_area_w = int(0.3 * fw)
+
+            thr_in_img_y = fh - int((fh / (max_eyes_ratio_v - min_eyes_ratio_v)) * (threshold - min_eyes_ratio_v))
+            zero_in_img_y = fh - int((fh / (max_eyes_ratio_v - min_eyes_ratio_v)) * (0.0 - min_eyes_ratio_v))
+
+            for i, fr in enumerate(face_vec):
+                fr = cv2.resize(fr, (new_fw, new_fh),interpolation=cv2.INTER_AREA)
+                frb = fr[-(fh+shift_edge):, :, :]
+                frb[:, :, 0:2] = 200
+                for j in range(i-before_len, i+after_len):
+                    if (j>0) and (j<vec120_len):
+                        if j != i:
+                            marker_color = (0, 0, 255)
+                            marker_size = 5
+                        else:
+                            marker_color = (0, 0, 0)
+                            marker_size = 8
+                        eye_ratio_in_img_x = int(j / vec120_len * new_fw)
+                        eye_ratio_in_img_y = fh - int((fh / (max_eyes_ratio_v - min_eyes_ratio_v)) * (eyes_ratio_v_vec[j] - min_eyes_ratio_v))
+                        frb = cv2.circle(frb, (eye_ratio_in_img_x, eye_ratio_in_img_y+shift_edge), marker_size, marker_color, cv2.FILLED)
+                        frb = cv2.line(frb, (0, thr_in_img_y+shift_edge), (new_fw, thr_in_img_y+shift_edge), (0, 0, 0), 2)
+                        frb = cv2.line(frb, (0, 0), (new_fw, 0), (0, 0, 0), 10)
+                        frb = cv2.line(frb, (0, fh+shift_edge), (new_fw, fh+shift_edge), (0, 0, 0), 10)
+                        frb = cv2.line(frb, (0, zero_in_img_y+shift_edge), (new_fw, zero_in_img_y+shift_edge), (53, 18, 80), 1)
+                        frb = cv2.putText(frb, "erv = 0", (10, zero_in_img_y), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 81, 140), 2)
+                        frb = cv2.putText(frb, f"erv = {threshold}", (10, thr_in_img_y), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 81, 140), 2)
+                fr[new_fh-(fh+shift_edge):, :, :] = frb
+
+                if eyes_ratio_v_vec[i] > threshold:
+                    fr[-(fh+red_area_h):-fh, :, 2] = 255
+                    fr[:red_area_h, :, 2] = 255
+                    fr[:-fh, :red_area_w, 2] = 255
+                    fr[:-fh, -red_area_w:, 2] = 255
+                    
+                win_name = "User"
+                
+                if len(ey.monitors) == 1:
+                    x_disp = 0
+                else:
+                    x_disp = ey.monitors[0].width
+                ey.big_win(win_name, x_disp)
+                cv2.imshow(win_name, fr)
+                q = cv2.waitKey(100)
+                if q == ord('q') or q == ord('Q'):
+                    break
+                i += 1
+            cv2.destroyAllWindows()
+        else:
+            print(f"Data does not exist in {smp_dir}")

codes/show.py

@@ -0,0 +1,109 @@
+"""This module is for seeing the camera stream. This module includes Camera class."""
+
+import cv2
+import time
+from codes.base import eyeing as ey
+from screeninfo import get_monitors
+
+
+monitors = get_monitors()
+
+class Camera(object):
+    running = True
+    def raw(self, camera_id):
+        """
+        See camera stream
+        
+        Parameters:
+            camera_id: Camera ID
+        
+        Returns:
+            None
+        """
+        frame_size, _, _, _ = ey.get_camera_properties(camera_id)
+        cap = ey.get_camera(camera_id, frame_size)
+        ey.pass_frames(cap, 100)
+        i = 0.0
+        win_name = "Webcam"
+        if len(monitors) == 1:
+            x_disp = 0
+        else:
+            x_disp = monitors[0].width
+        ey.big_win(win_name, x_disp)
+        t0 = time.perf_counter()
+        print("Showing camera..")
+        while self.running:
+            frame_success, frame, _ = ey.get_frame(cap)
+            if frame_success:
+                i += 1
+                cv2.imshow(win_name, frame)
+                q = cv2.waitKey(1)
+                if q == ord('q') or q == ord('Q'):
+                    break
+
+        cv2.destroyAllWindows()
+
+        fps = ey.get_time(i, t0, True)
+        print(f"FPS : {fps}")
+
+
+    def features(self, camera_id):
+        """
+        See camera stream with landmarks and features
+        
+        Parameters:
+            camera_id: Camera ID
+        
+        Returns:
+            None
+        """
+        some_landmarks_ids = ey.get_some_landmarks_ids()
+
+        (
+            frame_size,
+            camera_matrix,
+            dst_cof,
+            pcf
+        ) = ey.get_camera_properties(camera_id)
+
+        face_mesh = ey.get_mesh()
+
+        cap = ey.get_camera(camera_id, frame_size)
+        ey.pass_frames(cap, 100)
+        win_name = "Features"
+        if len(monitors) == 1:
+            x_disp = 0
+        else:
+            x_disp = monitors[0].width
+        ey.big_win(win_name, x_disp)
+        t0 = time.perf_counter()
+        i = 0
+        print("Showing features..")
+        while self.running:
+            frame_success, frame, frame_rgb = ey.get_frame(cap)
+            if frame_success:
+                results = face_mesh.process(frame_rgb)
+                features_success, frame = ey.get_model_inputs(
+                    frame,
+                    frame_rgb,
+                    results,
+                    camera_matrix,
+                    pcf,
+                    frame_size,
+                    dst_cof,
+                    some_landmarks_ids,
+                    True
+                )[:2]
+                
+                if features_success:
+                    i += 1
+                    cv2.imshow(win_name, frame)
+                    q = cv2.waitKey(1)
+                    if q == ord('q') or q == ord('Q'):
+                        break
+
+        cap.release()
+        cv2.destroyAllWindows()
+
+        fps = ey.get_time(i, t0, True)
+        print(f"FPS : {fps}")

codes/tune_models_params.py

@@ -0,0 +1,198 @@
+"""This module is for retraining the base (et) models. It contains the Tuning class."""
+
+from tensorflow.keras.callbacks import EarlyStopping
+from tensorflow.keras.models import load_model
+from tensorflow.keras.utils import to_categorical
+from sklearn.utils import shuffle
+from joblib import load as j_load
+from joblib import dump as j_dump
+import pickle
+import numpy as np
+import os
+from codes.base import eyeing as ey
+from openpyxl import Workbook
+
+
+PATH2ROOT_ABS = os.path.dirname(__file__) + "/../"
+
+
+class Tuning(object):
+    @staticmethod
+    def et_mdl(
+        subjects,
+        models_list=[1],
+        r_train_list=[0.99],
+        n_epochs_patience=[[3, 3]],
+        trainable_layers=[1],
+        shift_samples=None,
+        blinking_threshold='uo',
+        show_model=False,
+        delete_files=False
+        ):
+        """
+        You can retrain the base (et) models using this method. You have the possibility to retrain the models with various hyper parameters
+        to see which one has better performance. So you can enter lists as inputs.
+
+        Parameters:
+            subjects: list of subjects
+            models_list: a list of models' number
+            r_train_list: the ratio of the data that you want for training
+            n_epochs_patience: number of epochs and patience
+            trainable_layers: The number of trainable layer's (ending layers of the network)
+            shift_samples: shift the inputs to align with outputs. This is because of the delay.
+            blinking_threshold: Blinking threshold for removing the samples that are during blink
+            show_model: Show the structure of the model
+            delete_files: delete the dataset after retraining
+        
+        Returns:
+            None
+        """
+        print("\nStarting to retrain eye_tracking model...")
+        x1_scaler, x2_scaler, y_scaler = j_load(ey.scalers_dir + f"scalers_et_main.bin")    # Loading the scaler
+
+        # Going through each subject's folder
+        kk = 0
+        for num in subjects:
+            print(f"Subject number {num} in process...")
+            sbj_dir = ey.create_dir([ey.subjects_dir, f"{num}"])
+
+            # ### Retraining 'eye_tracking' model with subject calibration data
+            clb_dir = ey.create_dir([sbj_dir, ey.CLB])
+            if ey.file_existing(clb_dir, ey.X1+".pickle"):
+                print(f"Loading subject data in {clb_dir}")
+                (
+                    x1_load0,
+                    x2_load0,
+                    y_load0,
+                    t_mat,
+                    eyes_ratio
+                ) = ey.load(clb_dir, [ey.X1, ey.X2, ey.Y, ey.T, ey.ER])
+                if shift_samples:
+                    if shift_samples[kk]:
+                        ii = 0
+                        for (x11, x21, y1, t1, eyr1) in zip(x1_load0, x2_load0, y_load0, t_mat, eyes_ratio):
+                            t_mat[ii] = t1[:-shift_samples[kk]]
+                            x1_load0[ii] = x11[shift_samples[kk]:]
+                            x2_load0[ii] = x21[shift_samples[kk]:]
+                            y_load0[ii] = y1[:-shift_samples[kk]]
+                            eyes_ratio[ii] = eyr1[shift_samples[kk]:]
+                            ii += 1
+                kk += 1
+                er_dir = ey.create_dir([sbj_dir, ey.ER])
+                blinking_threshold = ey.get_threshold(er_dir, blinking_threshold)
+
+                blinking = ey.get_blinking(t_mat, eyes_ratio, blinking_threshold)[1]
+
+                # Removing the samples that are during blink
+                x1_load = []
+                x2_load = []
+                y_load = []
+                k1 = 0
+                k2 = 0
+                for (x11, x21, y1, b1) in zip(x1_load0, x2_load0, y_load0, blinking):
+                    for (x10, x20, y0, b0) in zip(x11, x21, y1, b1):
+                        k2 += 1
+                        if not b0:
+                            k1 += 1
+                            x1_load.append(x10)
+                            x2_load.append(x20)
+                            y_load.append(y0)
+
+                print(f"All samples of subjects: {k2}, Not blinking: {k1}")
+                x1_load = np.array(x1_load)
+                x2_load = np.array(x2_load)
+                y_load = np.array(y_load)
+                n_smp, frame_h, frame_w = x1_load.shape[:-1]
+                print(f"Samples number: {n_smp}")
+
+                # Displaying data
+
+                # ### Preparing modified calibration data to feeding in eye_tracking model. Going through each model to predict the output
+                print("Normalizing modified calibration data to feeding in eye_tracking model...")
+                for mdl_num in models_list:
+                    print("Loading public eye_tracking models...")
+                    mdl_name = ey.MDL + f"{mdl_num}"
+                    info = ey.load(ey.et_trained_dir, [mdl_name])[0]
+                    x2_chosen_features = info["x2_chosen_features"]
+                    x2_new = x2_load[:, x2_chosen_features]
+
+                    x1 = x1_load / x1_scaler
+                    x2 = x2_scaler.transform(x2_new)
+
+                    # Shuffling and splitting data to train and val
+                    x1_shf, x2_shf, y_hrz_shf, y_vrt_shf = shuffle(x1, x2, y_load[:, 0], y_load[:, 1])
+
+                    # Going through each training ratio in the ratio list
+                    for rt in r_train_list:
+                        n_train = int(rt * n_smp)
+                        x1_train, x2_train = x1_shf[:n_train], x2_shf[:n_train]
+                        x1_val, x2_val = x1_shf[n_train:], x2_shf[n_train:]
+                        y_hrz_train, y_vrt_train = y_hrz_shf[:n_train], y_vrt_shf[:n_train]
+                        y_hrz_val, y_vrt_val = y_hrz_shf[n_train:], y_vrt_shf[n_train:]
+
+                        x_train = [x1_train, x2_train]
+                        x_val = [x1_val, x2_val]
+
+                        print(x1_train.shape, x1_val.shape, y_hrz_train.shape, y_hrz_val.shape,
+                              x2_train.shape, x2_val.shape, y_vrt_train.shape, y_vrt_val.shape)
+
+                        # Callback for training. Going through each epoch and patience in epochs list
+                        for nep in n_epochs_patience:
+                            cb = EarlyStopping(patience=nep[1], verbose=1, restore_best_weights=True)
+
+                            # Going through each nubmer for trainable_layers list
+                            for tl in trainable_layers:
+                                # Retraining the models and saving them
+                                model_hrz = load_model(ey.et_trained_dir + mdl_name + "-hrz.h5")
+                                model_vrt = load_model(ey.et_trained_dir + mdl_name + "-vrt.h5")
+                                info["trained_mdl_num"] = mdl_num
+                                info["r_retrain"] = rt
+                                info["n_epochs_patience_retrain"] = nep
+                                info["trainable_layers"] = tl
+                                for (layer_hrz, layer_vrt) in zip(model_hrz.layers[:-tl], model_vrt.layers[:-tl]):
+                                    layer_hrz.trainable = False
+                                    layer_vrt.trainable = False
+
+                                if show_model:
+                                    print(model_hrz.summary())
+
+                                sbj_mdl_dir = ey.create_dir([sbj_dir, ey.MDL])
+                                retrained_mdl_num = ey.find_max_mdl(sbj_mdl_dir, b=-7) + 1
+
+                                print(f"\n<<<<<<< {retrained_mdl_num}-sbj:{num}-model-hrz:{mdl_num}-r_train:{rt}-epoch_patience:{nep}-trainable_layers:{tl} >>>>>>>>")
+                                model_hrz.fit(x_train,
+                                              y_hrz_train * y_scaler,
+                                              validation_data=(x_val, y_hrz_val * y_scaler),
+                                              epochs=nep[0],
+                                              callbacks=cb)
+                                hrz_train_loss = model_hrz.evaluate(x_train, y_hrz_train * y_scaler)
+                                hrz_val_loss = model_hrz.evaluate(x_val, y_hrz_val * y_scaler)
+                                info["hrz_retrain_train_loss"] = hrz_train_loss
+                                info["hrz_retrain_val_loss"] = hrz_val_loss
+                                retrained_mdl_name = ey.MDL + f"{retrained_mdl_num}"
+                                mdl_hrz_dir = sbj_mdl_dir + retrained_mdl_name + "-hrz.h5"
+                                model_hrz.save(mdl_hrz_dir)
+                                print("Saving model-et-hrz in " + mdl_hrz_dir)
+
+                                print(f"\n<<<<<<< {retrained_mdl_num}-sbj:{num}-model-vrt:{mdl_num}-r_train:{rt}-epoch_patience:{nep}-trainable_layers:{tl} >>>>>>>>")
+                                model_vrt.fit(x_train,
+                                              y_vrt_train * y_scaler,
+                                              validation_data=(x_val, y_vrt_val * y_scaler),
+                                              epochs=nep[0],
+                                              callbacks=cb)
+                                vrt_train_loss = model_vrt.evaluate(x_train, y_vrt_train * y_scaler)
+                                vrt_val_loss = model_vrt.evaluate(x_val, y_vrt_val * y_scaler)
+                                info["vrt_retrain_train_loss"] = vrt_train_loss
+                                info["vrt_retrain_val_loss"] = vrt_val_loss
+                                mdl_vrt_dir = sbj_mdl_dir + retrained_mdl_name + "-vrt.h5"
+                                model_vrt.save(mdl_vrt_dir)
+                                print("Saving model-et-vrt in " + mdl_vrt_dir)
+
+                                ey.save([info], sbj_mdl_dir, [retrained_mdl_name])
+
+                                if delete_files:
+                                    ey.remove(clb_dir)
+            else:
+                print(f"Data does not exist in {clb_dir}")
+
+

codes/work.py

@@ -0,0 +1,143 @@
+from PyQt5.QtCore import pyqtSignal, QObject
+from codes.show import Camera
+from codes.calibration import Clb
+from codes.sampling import Smp
+from codes.tune_models_params import Tuning
+from codes.eye_track import EyeTrack
+from codes.see_data import See
+
+
+# Change parameters use_io, clb_grd for accuracy and del_files, nep
+
+
+class Worker(QObject, Camera, Clb, Smp, Tuning, EyeTrack, See):
+    num = 0
+    camera_id = 0
+    thb = 0.0
+    ss = 0
+    mfr = 0.0
+    dft = 0.0
+    st = 0.0
+
+    cam = False
+    clb = False
+    smp = False
+    acc = False
+    tbt = False
+    mdl = False
+    uio = False
+    gps = False
+    gpa = False
+    sps = False
+    spa = False
+    gfx = False
+    
+    running = True
+        
+    cam_started = pyqtSignal()
+    clb_started = pyqtSignal()
+    smp_started = pyqtSignal()
+    acc_started = pyqtSignal()
+    tbt_started = pyqtSignal()
+    mdl_started = pyqtSignal()
+    gps_started = pyqtSignal()
+    gpa_started = pyqtSignal()
+    sps_started = pyqtSignal()
+    spa_started = pyqtSignal()
+    gfx_started = pyqtSignal()
+    finished = pyqtSignal()
+
+    def __init__(self):
+        super().__init__()
+        
+    def do_work(self):
+        if self.cam and self.running:
+            print("\nCamera")
+            self.cam_started.emit()
+            self.features(camera_id=self.camera_id)
+        if self.clb and self.running:
+            print("\nCalibration")
+            self.clb_started.emit()
+            self.et(
+                num=self.num,
+                camera_id=self.camera_id,
+                info=self.info,
+                clb_grid=self.clb_grid
+                )
+        if self.smp and self.running:
+            print("\nSampling")
+            self.smp_started.emit()
+            self.sampling(
+                num=self.num,
+                camera_id=self.camera_id,
+                gui=True
+                )
+        if self.acc and self.running:
+            print("\nTesting")
+            self.acc_started.emit()
+            self.accuracy(
+                num=self.num,
+                camera_id=self.camera_id,
+                clb_grid=(5, 7, 20)
+                )
+        if self.tbt and self.running:
+            print("\nSee user blinking")
+            self.user_face(
+                num=self.num,
+                threshold=self.thb,
+                save_threshold=True
+                )
+        if self.mdl and self.running:
+            print("\nTuning params")
+            self.mdl_started.emit()
+            self.et_mdl(
+                subjects=[self.num],
+                shift_samples=[self.ss],
+                delete_files=False
+                )
+        if self.gps and self.running:
+            print("\nGetting pixels")
+            self.gps_started.emit()
+            self.get_pixels(
+                subjects=[self.num],
+                shift_samples=[self.ss],
+                use_io=self.uio,
+                delete_files=False
+                )
+        if self.gpa and self.running:
+            print("\nGetting test pixels")
+            self.gpa_started.emit()
+            self.get_pixels(
+                subjects=[self.num],
+                target_fol="acc",
+                shift_samples=[self.ss],
+                use_io=True,
+                delete_files=False
+                )
+        if self.sps and self.running:
+            print("\nSeeing sampling data")
+            self.sps_started.emit()
+            self.pixels_smp(num=self.num, show_in_all_monitors=True)
+        if self.spa and self.running:
+            print("\nSeeing testing data")
+            self.spa_started.emit()
+            self.pixels_acc(
+                num=self.num,
+                show_in_all_monitors=True
+                )
+        if self.gfx and self.running:
+            print("\nGetting fixations")
+            self.gfx_started.emit()
+            self.get_fixations(
+                subjects=[self.num],
+                t_discard=self.dft,
+                x_merge=self.mfr[0],
+                y_merge=self.mfr[1],
+                vx_thr=self.st,
+                vy_thr=self.st
+                )
+        
+
+        print("\nEye Tracking finished!")
+        self.finished.emit()
+        self.running = True

crt_exe_gui.spec

@@ -0,0 +1,46 @@
+# -*- mode: python ; coding: utf-8 -*-
+# Activate environment, then type "pyinstaller crt_exe_gui.spec" in command line
+# to generate .exe file for gui
+
+
+block_cipher = None
+
+
+a = Analysis(['main_gui.py'],
+             pathex=[],
+             binaries=[],
+             datas=[('env/Lib/site-packages/mediapipe/modules', 'mediapipe/modules'),
+			 ('models', 'models'),
+			 ('docs', 'docs'),
+             ('other_files', 'other_files')],
+             hiddenimports=[],
+             hookspath=[],
+             hooksconfig={},
+             runtime_hooks=[],
+             excludes=[],
+             win_no_prefer_redirects=False,
+             win_private_assemblies=False,
+             cipher=block_cipher,
+             noarchive=False)
+pyz = PYZ(a.pure, a.zipped_data,
+             cipher=block_cipher)
+
+exe = EXE(pyz,
+          a.scripts,
+          a.binaries,
+          a.zipfiles,
+          a.datas,  
+          [],
+          name='Owleye',
+          debug=False,
+          bootloader_ignore_signals=False,
+          strip=False,
+          upx=True,
+          upx_exclude=[],
+          runtime_tmpdir=None,
+          console=True,
+		  icon='docs/images/logo.ico',
+          disable_windowed_traceback=False,
+          target_arch=None,
+          codesign_identity=None,
+          entitlements_file=None )

docs/USE_APP.md

@@ -0,0 +1,86 @@
+## This document shows you how to use Owleye application
+
+When you run the program, a window like the one in below will be appeared:
+
+![Screenshot 2024-03-25 142116](https://github.com/owleye/Owleye/assets/53625380/1d311246-4273-4092-a1ab-d7dd2b84173f)
+
+Also, after running, a folder called "subjects" will be created. In this folder, a folder will be created for each subject based on the subject number in the UI. Then, you can adjust the items that you need, and finally "start" the program. In following, the items will be explained.
+
+**1. Subject number:** A specific number that you enter as the ID of the subject.
+
+**2. Camera ID:** Usually it is 0, but you can try other numbers if you have several webcams on your system.
+
+**3. Camera:** by activating this checkbox, after clicking on the "start" button, the webcam stream will be shown. So you can see yourself and the landmarks that already are detected on your face.
+
+**4. Calibration:** By activating this checkbox, after clicking on the "start" button, the program creates a folder with the name of subject number. Then, it will be ready to collect data (inputs and outputs of the models of [the sixth block in owleye's structure](https://github.com/owleye/Owleye/blob/main/docs/images/Owleye%20structure.png)) from the user. So, a white point will be appeared on the screen. As soon as you press the "SPACE" key on keyboard, the program starts collecting data for a particlular time. While this, the background bacomes black and the user should look at the white point during this time. If the data collection ends, the screen will become gray and the point will go to another location. The user can look everywhere and actually rest in this situation. Again, the "SPACE" key should be pressed for data collection and all of the explained process get repeated. Also, a folder called "clb" (calibration) will be created in the user's folder. In this folder, 5 ".pickle" files will be created which were collected:
+
+- t.pickle: Time
+- x1.pickle: Eyes images
+- x2.pickle: Face vecotrs
+- y.pickle: White point locations
+- er.pickle: Eye aspect ratio vector (Go to [11th section](https://github.com/owleye/Owleye/blob/main/docs/USE_APP.md#:~:text=from%20the%20default.-,11.%20Threshold,-%3A%20To%20detect) to know about this)
+
+**5. Subject name:** This item is arbitrary. you can enter the user's name. This is activated when you want to do calibration.
+
+**6. Describtion:** This section is arbitrary too. you can enter any information that your user has. This is activated when you want to do calibration.
+
+**7. Calibration grid:** This item can have three types of integer numbers.
+
+- 2 numbers (n, c): The white point starts to move just horizontally in the screen in n rows. Each row contains c locations that the point goes through.
+- 3 numbers (n, m , c): The white point does not move. just in a grid by size of n x m (like a matrix) remains fixed in each location.
+- 4 numbers (n, c, m, d): The white point moves both horizontally and vertically, in n rows with c locations and m columns with d locations.
+
+**8. Sampling:** If you activate this checkbox, the program will start collecting data from you while you are looking in the screen. So, this item is for using the program for your goal. Also a folder called "smp" (sampling) will be created in the user's folder. In this folder these files will be made:
+- t.pickle
+- sys_time.pickle
+- x1.pickle
+- x2.pickle
+- er.pickle
+- fv.pickle: face vector, this is a series of face images for seeing the face during the sampling, to choose the blinking threshold.
+
+**9. Testing:** This checkbox is for seeing how well Owleye works. If you activate this checkbox, after starting the program, it creates a folder named "acc" (accuracy).  The program will start showing you a white point that you must look at that. Actually, the user is looking in the white point. So, it is clear that what should be the best possible prediction of Owleye (The position of white points). Also, you can see mean squared error (MSE). In this folder these files will be made:
+- t.pickle
+- x1.pickle
+- x2.pickle
+- y.pickle
+- er.pickle
+
+**10. Tune blinking threshold:** By activating this checkbox, you can change the blinking threshold (11) from the default.
+
+**11. Threshold:** To detect the blinks, The eyes aspect ratio (EAR) method is used. In the samples that are collected during sampling, EAR will be calculated. It is a vector during time. Then using that vector, the velocity of EAR will be calculated as a vector. So, the values above the determined threshold are considered as a blink. The default value for threshold is gained using try and error with my face. every face can have a different threshold.
+
+**12. Tune eye tracking model:** By activating this checkbox, after pressing "start" button, the program starts retraining the two base models of [the sixth block in the Owleye's structure](https://github.com/owleye/Owleye/blob/main/docs/images/Owleye%20structure.png) which already are in the models folder in root. After this, a folder called "mdl" (modeling) will be created in the subject's folder. In this folder, there are 3 files:
+- mdl1.pickle: scaler of the face vector
+- mdl1-hrz.h5: model for predicting the horizontal direction
+- mdl1-vrt.h5: model for predicting the vertical direction
+
+**13. SS:** Abbreviation of shift samples. While the white point is moving during calibration, the inputs and outputs (point locations) are not exactly aligned. Because of processing problems, the images (inputs) are a little later than the outputs. For example, if this parameter is equal to 20, it means you want to shift inputs in 20 samples. So, the 21th input will be aligned with 1st output.
+
+**14. Sampling data:** It means the later calculations are for the sampling data, not testing data.
+
+**15. Test data:** It means the later calculations are for the testing data, not sampling data.
+
+**16. Use IO model:** If the user activates this checkbox, after the program predicted the sampling data, it will remove the samples that are out of the screens range.
+
+**17. Get pixels:** If the user activates this checkbox, the program will predict the sampling or testing data. If the sampling data is chosen, these data will be created in the "smp" folder:
+- y_prd.pickle
+- eye_track.xlsx: An excel file containing the eye movements and times during sampling. This is the file that you should use for your future analysis.
+- fixations.xlsx: An excel file containing the fixations during sampling. This is the file that you should use for your future analysis.
+
+Otherwise, if the test data is chosen, these data will be created:
+- y_mdf.pickle: This is same as the actual y, but the points that are detected during a blink, are removed.
+- y_prd_mdf.pickle: This is the predictions of Owleye, while the blink moments are removed.
+
+**18. See pixels:** If the user activates this checkbox, the program will show the locations that the user has looked.
+
+**19. Get fixations:** The program will calculate the fixations of the user by three parameters of 20, 21, and 23.
+
+**20. ST:** Abbreviation for saccade threshold. To separate fixations, it is needed to firstly calculate the velocity of eye movement. So, by putting a threshold on the velocity, it's possible to compute the moments that the user changed their viewpoint.
+
+**21. DFT:** Abbreviation for discard fixation time. Fixations that last less than this time, will be removed.
+
+**22. MFR:** Abbreviation for merged fixations ratio. Fixations that are close together, will be added. Two numbers are for two directions.
+
+**23. Start:** This button starts the program. If you activate one of the checkboxes, the program just do that specific one. If you select some of the checkboxes, the program goes for all of the selected ones, one after another.
+
+**24. Stop:** This button stops the program in every step that it's running.

main.py

@@ -0,0 +1,159 @@
+"""The project "Owleye" turns your webcam to an eye tracker. You can use it to know which point in the screen you are looking.
+The project has several parts that you can get familiar with, using the documentations that I've provided in README.md and docs/USE_APP.md files.
+Before using this project, make sure that you have read these documentations. main.py is like main_gui.py with some advantages.
+Using main.py you can retrain the models for several subjects all together.
+Also, you can do predictions for all subjects together. You can train the models using this file, but with main_gui.py you can just retrain
+the models. In addition, you can test the latency of Owleye by main.py. To sum up, use main_gui.py to collect calibration data and sampling data
+or for seeing data, but use main.py for training, retraining, and predictions for a group of subjects that you already have their data.
+In the following, you can uncomment each section (between star signs **) to do your work.
+Also, for a faster understanding of the code, it is recommended to print the varibales shape. There are a lot of lists and lists of lists
+that may confuse you.
+"""
+
+
+from codes.show import Camera
+from codes.calibration import Clb
+from codes.sampling import Smp
+from codes.see_data import See
+from codes.crt_train_models import Modeling
+from codes.tune_models_params import Tuning
+from codes.eye_track import EyeTrack
+
+# *********************** PARAMETERS ***********************
+NUMBER = 6   # The subject number that we want to do subsequent works on them
+
+TRAINING_SUBJECTS = [71, 72, 73, 74, 81, 82, 83, 84, 85, 86, 122, 123,
+124, 125, 126, 144, 145, 146, 147, 201, 203, 204, 206, 207, 211,
+212, 213, 214, 215, 216, 217, 221, 222, 224]        # You can retrain the models and predict the sampling data for several users at a same time
+CAMERA_ID = 0       # Check camera id by uncommenting Camera().raw()
+SHIFT_SAMPLES = 0       # Because of the delay that the sampling has, you can shift inputs to reach to the appropriate output
+
+# # *********************** SEE CAMERA ***********************
+# Camera().raw(camera_id=CAMERA_ID)     # You can see the webcam stream
+# Camera().features(camera_id=CAMERA_ID)
+# """You can see the webcam stream with the detected landmarks. You can check whether Mediapipe and Opencv work properly or not."""
+
+# # *********************** CALIBRATION **********************
+NAME = "Mostafa Lotfi"
+Descriptions = "Test for shifting"
+INFO = [NAME, Descriptions]
+CALIBRATION_GRID = 4, 200, 6, 100
+
+# Clb().et(num=NUMBER, camera_id=CAMERA_ID, info=INFO, clb_grid=CALIBRATION_GRID)
+# """This method collects data (input and output) for eye tracking"""
+# Clb().out(num=NUMBER, camera_id=CAMERA_ID, n_smp_in_cls=100)
+# """This method collects data of the subjects while looking at out of the screen
+# This is used for in-out model to see whether the user is looking inside of the screen or outside of that. This is not in main_gui.pyp"""
+
+# Clb().calculate_threshold(num=NUMBER, camera_id=CAMERA_ID)
+# """This method collects data for calculation of blink threshold. This is not in main_gui.py"""
+
+# # *********************** SAMPLING *************************
+# Smp().sampling(num=NUMBER, camera_id=CAMERA_ID, gui=False)    # The method collects inputs during sampling time
+
+# # *********************** ACCURACY **************************
+# Smp().accuracy(num=NUMBER, camera_id=CAMERA_ID, clb_grid=(5, 7, 30))
+# """The method collects data (input and output), This is for testing Owleye's performance.
+
+# # *********************** LATENCY **************************
+# Smp().latency(num=NUMBER, camera_id=CAMERA_ID)
+# """This method collects data to calculates the delay of Owleye. When it's run, you should look at the left and write side of the screen
+# based on the color. This method is not in main_gui.py"""
+
+# # ********************* SEE FEATURES ***********************
+# """This method is to see the data collected in previous sections. It isn't in main_gui.py"""
+# See().data_features(num=NUMBER, target_fol="clb")
+# See().data_features(num=NUMBER, target_fol="io")
+# See().data_features(num=NUMBER, target_fol="smp")
+# See().data_features(num=NUMBER, target_fol="acc")
+# See().data_features(num=NUMBER, target_fol="ltn")
+
+# See().user_face(num=NUMBER, threshold=5, save_threshold=True)  # See user's face during sampling, to tune eye ratio threshold
+
+# """This method is for ploting the eye aspect ration in varous data."""
+# See().blinks_plot(num=NUMBER, target_fol="er")
+# See().blinks_plot(num=NUMBER, threshold="ao", target_fol="clb")
+# See().blinks_plot(num=NUMBER, threshold=9, target_fol="smp")
+# See().blinks_plot(num=NUMBER, threshold=4.5, target_fol="acc")
+
+# # ***************** CREATE BASE MODELS *******************
+# """This section is just for creation of eye tracking and in-out models (not training). You can change the structure in the method.
+# This method is not in main_gui.py"""
+# Modeling().create_io()
+# Modeling().get_models_information(show_model=True)
+# Modeling().create_et()
+# Modeling().get_models_information(io=False, show_model=True)
+
+# # ****************** TRAIN BASE MODELS *******************
+# """You can train the base models in this section. You should enter a list of subjects that you want to create the model using them.
+# This method is not in main_gui.py"""
+# Modeling().train_io(
+#     subjects=[1, 2],
+#     models_list=[1, 2],
+#     min_max_brightness_ratio=[[0.65, 1.45], [0.6, 1.5]],
+#     r_train_list=[0.8, 0.9],
+#     n_epochs_patience=[[2, 1], [3, 2]],
+#     save_scaler=False,
+#     show_model=False)
+# Modeling().get_models_information(io=True, raw=False, show_model=False)
+# Modeling().train_et(subjects=TRAINING_SUBJECTS,
+#     models_list=[1],
+#     min_max_brightness_ratio=[[0.65, 1.45]],
+#     r_train_list=[0.8],
+#     n_epochs_patience=[[2, 1]],
+#     shift_samples=[SHIFT_SAMPLES] * len(TRAINING_SUBJECTS),
+#     blinking_threshold="d",
+#     save_scaler=False,
+#     show_model=False)
+# Modeling().get_models_information(io=False, raw=False, show_model=False)
+
+# # *********************** Tuning *************************
+# # To retrain the base models
+# Tuning().et_mdl(subjects=TRAINING_SUBJECTS,
+# 	models_list=[1],
+#     r_train_list=[0.99],
+#     n_epochs_patience=[[3, 3]],
+#     trainable_layers=[1],
+#     shift_samples=[SHIFT_SAMPLES] * len(TRAINING_SUBJECTS),
+#     blinking_threshold='uo',
+#     show_model=False,
+#     delete_files=False)
+
+# # *********************** GET PIXELS-Sampling ***********************
+# # To predict the sampling pixels
+# EyeTrack().get_pixels(
+#     subjects=TRAINING_SUBJECTS,
+#     models_list=[1],
+#     target_fol="smp",
+#     shift_samples=[SHIFT_SAMPLES] * len(TRAINING_SUBJECTS),
+#     blinking_threshold="uo"
+#     )
+
+# # ******************* GET PIXELS-GET PIXELS-Accuracy *******************
+# # To predict testing pixels and calculate the loss
+# EyeTrack().get_pixels(
+#     subjects=[NUMBER],
+#     models_list=[1],
+#     target_fol="acc",
+#     shift_samples=[SHIFT_SAMPLES],
+#     blinking_threshold="uo"
+#     )
+# EyeTrack().get_models_information(show_model=False)
+
+# # ******************** GET PIXELS-Latency *****************************
+# To calculate the delay of Owleye
+# EyeTrack().get_pixels(subjects=[NUMBER], models_list=[1], target_fol="ltn", shift_samples=[1])
+
+# # ******************** GET FIXATIONS ***********************
+# # To calculate fixations
+# EyeTrack().get_fixations(subjects=TRAINING_SUBJECTS, n_monitors_data=3, x_merge=0.15/2, y_merge=0.18/2, vx_thr=0.8, vy_thr=0.8, t_discard=0.1)
+
+# # ***************** SEE SAMPLING PIXELS ********************
+# To see the predictions of sampling data
+# See().pixels_smp(num=NUMBER, n_monitors_data=3, show_in_all_monitors=False, win_size=(3 * 1280, 720), show_fixations=True)
+
+# # ***************** SEE ACCURACY PIXELS *********************
+# # To see the predictions of testing data
+# See().pixels_acc(num=NUMBER, n_monitors_data=1)
+
+

main_gui.py

@@ -0,0 +1,514 @@
+"""The project "Owleye" turns your webcam to an eye tracker. You can use it to know which point in the screen you are looking.
+The project has several parts that you can get familiar with, using the documentations that I've provided in README.md and docs/USE_APP.md files.
+Before using this project, make sure that you have read these documentations.
+This file contains the code for a GUI. There are some points that you should know about a GUI of PyQt5 to understand the following code.
+Also, unfortunately I didn't add proper comments in this file and now it's a little hard to understand it (Now I am really embarrassed for this :)). But, totally, the GUI is connected
+To the modules in the codes folder, using a worker. the worker gives the ability for multithreading. For understanding the code of eye tracker,
+I suggest you to just visit the modules in the codes folder and see how I used them in main.py.
+Also, for a faster understanding of the code, it is recommended to print the varibales shape. There are a lot of lists and lists of lists
+that may confuse you.
+"""
+
+
+from PyQt5 import QtCore, QtGui, QtWidgets
+from PyQt5.QtCore import QThread
+from codes.work import Worker
+import os
+
+PATH2ROOT_ABS = os.path.dirname(__file__) + "/"
+
+
+class Ui_MainWindow(object):
+    def setupUi(self, MainWindow):
+        MainWindow.setObjectName("MainWindow")
+        MainWindow.resize(362, 462)
+        MainWindow.setAcceptDrops(True)
+        MainWindow.setWindowIcon(QtGui.QIcon(PATH2ROOT_ABS + "docs/images/logo.ico"))
+        self.centralwidget = QtWidgets.QWidget(MainWindow)
+        self.centralwidget.setObjectName("centralwidget")
+        self.gridLayout = QtWidgets.QGridLayout(self.centralwidget)
+        self.gridLayout.setObjectName("gridLayout")
+        self.l_num = QtWidgets.QLabel(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(10)
+        self.l_num.setFont(font)
+        self.l_num.setObjectName("l_num")
+        self.gridLayout.addWidget(self.l_num, 0, 0, 1, 2)
+        self.le_num = QtWidgets.QLineEdit(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        self.le_num.setFont(font)
+        self.le_num.setObjectName("le_num")
+        self.gridLayout.addWidget(self.le_num, 0, 3, 1, 2)
+        self.l_cam = QtWidgets.QLabel(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(10)
+        self.l_cam.setFont(font)
+        self.l_cam.setObjectName("l_cam")
+        self.gridLayout.addWidget(self.l_cam, 0, 5, 1, 4)
+        self.le_cam = QtWidgets.QLineEdit(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        self.le_cam.setFont(font)
+        self.le_cam.setObjectName("le_cam")
+        self.gridLayout.addWidget(self.le_cam, 0, 9, 1, 1)
+        self.chb_cam = QtWidgets.QCheckBox(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(10)
+        self.chb_cam.setFont(font)
+        self.chb_cam.setLayoutDirection(QtCore.Qt.LeftToRight)
+        self.chb_cam.setObjectName("chb_cam")
+        self.gridLayout.addWidget(self.chb_cam, 1, 0, 1, 2)
+        self.chb_clb = QtWidgets.QCheckBox(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(10)
+        self.chb_clb.setFont(font)
+        self.chb_clb.setLayoutDirection(QtCore.Qt.LeftToRight)
+        self.chb_clb.setObjectName("chb_clb")
+        self.gridLayout.addWidget(self.chb_clb, 2, 0, 1, 2)
+        self.l_name = QtWidgets.QLabel(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(10)
+        self.l_name.setFont(font)
+        self.l_name.setLayoutDirection(QtCore.Qt.LeftToRight)
+        self.l_name.setObjectName("l_name")
+        self.gridLayout.addWidget(self.l_name, 3, 0, 1, 2)
+        self.le_name = QtWidgets.QLineEdit(self.centralwidget)
+        self.le_name.setEnabled(False)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(8)
+        self.le_name.setFont(font)
+        self.le_name.setObjectName("le_name")
+        self.gridLayout.addWidget(self.le_name, 3, 3, 1, 4)
+        self.l_dcp = QtWidgets.QLabel(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(10)
+        self.l_dcp.setFont(font)
+        self.l_dcp.setObjectName("l_dcp")
+        self.gridLayout.addWidget(self.l_dcp, 4, 0, 1, 2)
+        self.te_dcp = QtWidgets.QTextEdit(self.centralwidget)
+        self.te_dcp.setEnabled(False)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        self.te_dcp.setFont(font)
+        self.te_dcp.setObjectName("te_dcp")
+        self.gridLayout.addWidget(self.te_dcp, 4, 3, 1, 7)
+        self.l_clg_grd = QtWidgets.QLabel(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(10)
+        self.l_clg_grd.setFont(font)
+        self.l_clg_grd.setObjectName("l_clg_grd")
+        self.gridLayout.addWidget(self.l_clg_grd, 5, 0, 1, 3)
+        self.le_clb_grd = QtWidgets.QLineEdit(self.centralwidget)
+        self.le_clb_grd.setEnabled(False)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        self.le_clb_grd.setFont(font)
+        self.le_clb_grd.setObjectName("le_clb_grd")
+        self.gridLayout.addWidget(self.le_clb_grd, 5, 3, 1, 3)
+        self.chb_smp = QtWidgets.QCheckBox(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(10)
+        self.chb_smp.setFont(font)
+        self.chb_smp.setLayoutDirection(QtCore.Qt.LeftToRight)
+        self.chb_smp.setObjectName("chb_smp")
+        self.gridLayout.addWidget(self.chb_smp, 6, 0, 1, 2)
+        self.chb_tst = QtWidgets.QCheckBox(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(10)
+        self.chb_tst.setFont(font)
+        self.chb_tst.setLayoutDirection(QtCore.Qt.LeftToRight)
+        self.chb_tst.setObjectName("chb_tst")
+        self.gridLayout.addWidget(self.chb_tst, 6, 6, 1, 3)
+        self.chb_blink = QtWidgets.QCheckBox(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(10)
+        self.chb_blink.setFont(font)
+        self.chb_blink.setLayoutDirection(QtCore.Qt.LeftToRight)
+        self.chb_blink.setObjectName("chb_blink")
+        self.gridLayout.addWidget(self.chb_blink, 7, 0, 1, 5)
+        self.l_blink = QtWidgets.QLabel(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(10)
+        self.l_blink.setFont(font)
+        self.l_blink.setObjectName("l_blink")
+        self.gridLayout.addWidget(self.l_blink, 7, 6, 1, 3)
+        self.le_blink = QtWidgets.QLineEdit(self.centralwidget)
+        self.le_blink.setEnabled(False)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        self.le_blink.setFont(font)
+        self.le_blink.setObjectName("le_blink")
+        self.gridLayout.addWidget(self.le_blink, 7, 9, 1, 1)
+        self.chb_tune_mdl = QtWidgets.QCheckBox(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(10)
+        self.chb_tune_mdl.setFont(font)
+        self.chb_tune_mdl.setLayoutDirection(QtCore.Qt.LeftToRight)
+        self.chb_tune_mdl.setObjectName("chb_tune_mdl")
+        self.gridLayout.addWidget(self.chb_tune_mdl, 8, 0, 1, 5)
+        self.l_shift = QtWidgets.QLabel(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(10)
+        self.l_shift.setFont(font)
+        self.l_shift.setObjectName("l_shift")
+        self.gridLayout.addWidget(self.l_shift, 8, 6, 1, 1)
+        self.le_shift = QtWidgets.QLineEdit(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        self.le_shift.setFont(font)
+        self.le_shift.setObjectName("le_shift")
+        self.gridLayout.addWidget(self.le_shift, 8, 7, 1, 3)
+        self.rb_smp = QtWidgets.QRadioButton(self.centralwidget)
+        self.rb_smp.setChecked(True)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(10)
+        self.rb_smp.setFont(font)
+        self.rb_smp.setObjectName("rb_smp")
+        self.gridLayout.addWidget(self.rb_smp, 9, 0, 1, 3)
+        self.rb_tst = QtWidgets.QRadioButton(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(10)
+        self.rb_tst.setFont(font)
+        self.rb_tst.setObjectName("rb_tst")
+        self.gridLayout.addWidget(self.rb_tst, 9, 3, 1, 2)
+        self.chb_io = QtWidgets.QCheckBox(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(10)
+        self.chb_io.setFont(font)
+        self.chb_io.setLayoutDirection(QtCore.Qt.LeftToRight)
+        self.chb_io.setObjectName("chb_io")
+        self.gridLayout.addWidget(self.chb_io, 9, 6, 1, 4)
+        self.chb_pxl = QtWidgets.QCheckBox(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(10)
+        self.chb_pxl.setFont(font)
+        self.chb_pxl.setLayoutDirection(QtCore.Qt.LeftToRight)
+        self.chb_pxl.setObjectName("chb_pxl")
+        self.gridLayout.addWidget(self.chb_pxl, 10, 0, 1, 2)
+        self.chb_see_pxl = QtWidgets.QCheckBox(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(10)
+        self.chb_see_pxl.setFont(font)
+        self.chb_see_pxl.setLayoutDirection(QtCore.Qt.LeftToRight)
+        self.chb_see_pxl.setObjectName("chb_see_pxl")
+        self.gridLayout.addWidget(self.chb_see_pxl, 10, 3, 1, 2)
+        self.chb_fix = QtWidgets.QCheckBox(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(10)
+        self.chb_fix.setFont(font)
+        self.chb_fix.setLayoutDirection(QtCore.Qt.LeftToRight)
+        self.chb_fix.setObjectName("chb_fix")
+        self.gridLayout.addWidget(self.chb_fix, 10, 6, 1, 4)
+        self.l_st = QtWidgets.QLabel(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(10)
+        self.l_st.setFont(font)
+        self.l_st.setObjectName("l_st")
+        self.gridLayout.addWidget(self.l_st, 11, 0, 1, 1)
+        self.le_st = QtWidgets.QLineEdit(self.centralwidget)
+        self.le_st.setEnabled(False)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(8)
+        self.le_st.setFont(font)
+        self.le_st.setObjectName("le_st")
+        self.gridLayout.addWidget(self.le_st, 11, 1, 1, 2)
+        self.l_dft = QtWidgets.QLabel(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(10)
+        self.l_dft.setFont(font)
+        self.l_dft.setObjectName("l_dft")
+        self.gridLayout.addWidget(self.l_dft, 11, 3, 1, 1)
+        self.le_dft = QtWidgets.QLineEdit(self.centralwidget)
+        self.le_dft.setEnabled(False)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(8)
+        self.le_dft.setFont(font)
+        self.le_dft.setObjectName("le_dft")
+        self.gridLayout.addWidget(self.le_dft, 11, 4, 1, 1)
+        self.l_mfr = QtWidgets.QLabel(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(10)
+        self.l_mfr.setFont(font)
+        self.l_mfr.setObjectName("l_mfr")
+        self.gridLayout.addWidget(self.l_mfr, 11, 6, 1, 2)
+        self.le_mfr = QtWidgets.QLineEdit(self.centralwidget)
+        self.le_mfr.setEnabled(False)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        self.le_mfr.setFont(font)
+        self.le_mfr.setObjectName("le_mfr")
+        self.gridLayout.addWidget(self.le_mfr, 11, 8, 1, 2)
+        self.pb_start = QtWidgets.QPushButton(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(12)
+        self.pb_start.setFont(font)
+        self.pb_start.setObjectName("pb_start")
+        self.gridLayout.addWidget(self.pb_start, 12, 0, 1, 2)
+        self.l_monitor = QtWidgets.QLabel(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(9)
+        self.l_monitor.setFont(font)
+        self.l_monitor.setObjectName("l_monitor")
+        self.gridLayout.addWidget(self.l_monitor, 12, 2, 2, 3)
+        self.pb_stop = QtWidgets.QPushButton(self.centralwidget)
+        font = QtGui.QFont()
+        font.setFamily("Times New Roman")
+        font.setPointSize(12)
+        self.pb_stop.setFont(font)
+        self.pb_stop.setObjectName("pb_stop")
+        self.gridLayout.addWidget(self.pb_stop, 13, 0, 1, 2)
+        MainWindow.setCentralWidget(self.centralwidget)
+        self.menubar = QtWidgets.QMenuBar(MainWindow)
+        self.menubar.setGeometry(QtCore.QRect(0, 0, 362, 22))
+        self.menubar.setObjectName("menubar")
+        MainWindow.setMenuBar(self.menubar)
+        self.statusbar = QtWidgets.QStatusBar(MainWindow)
+        self.statusbar.setObjectName("statusbar")
+        MainWindow.setStatusBar(self.statusbar)
+
+        self.retranslateUi(MainWindow)
+        QtCore.QMetaObject.connectSlotsByName(MainWindow)
+
+    def retranslateUi(self, MainWindow):
+        _translate = QtCore.QCoreApplication.translate
+        MainWindow.setWindowTitle(_translate("MainWindow", "Owleye"))
+        self.l_num.setText(_translate("MainWindow", "Subject Number :"))
+        self.le_num.setText(_translate("MainWindow", "1"))
+        self.l_cam.setText(_translate("MainWindow", "Camera ID :"))
+        self.le_cam.setText(_translate("MainWindow", "0"))
+        self.chb_cam.setText(_translate("MainWindow", "Camera"))
+        self.chb_clb.setText(_translate("MainWindow", "Calibration"))
+        self.l_name.setText(_translate("MainWindow", "Subject Name :"))
+        self.le_name.setText(_translate("MainWindow", ""))
+        self.l_dcp.setText(_translate("MainWindow", "Descriptions :"))
+        self.te_dcp.setHtml(_translate("MainWindow", "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.0//EN\" \"http://www.w3.org/TR/REC-html40/strict.dtd\">\n"
+"<html><head><meta name=\"qrichtext\" content=\"1\" /><style type=\"text/css\">\n"
+"p, li { white-space: pre-wrap; }\n"
+"</style></head><body style=\" font-family:\'Times New Roman\'; font-size:8.1pt; font-weight:400; font-style:normal;\">\n"
+"<p style=\" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;\"><span style=\" font-family:\'MS Shell Dlg 2\';\"></span></p></body></html>"))
+        self.l_clg_grd.setText(_translate("MainWindow", "Calibration Grid :"))
+        self.le_clb_grd.setText(_translate("MainWindow", "4, 200, 6, 100"))
+        self.chb_smp.setText(_translate("MainWindow", "Sampling"))
+        self.chb_tst.setText(_translate("MainWindow", "Testing"))
+        self.chb_blink.setText(_translate("MainWindow", "Tune Blinking Threshold"))
+        self.l_blink.setText(_translate("MainWindow", "Threshold :"))
+        self.le_blink.setText(_translate("MainWindow", "4.5"))
+        self.chb_tune_mdl.setText(_translate("MainWindow", "Tune Eye Tracking Model"))
+        self.l_shift.setText(_translate("MainWindow", "SS :"))
+        self.le_shift.setText(_translate("MainWindow", "0"))
+        self.rb_smp.setText(_translate("MainWindow", "Sampling data"))
+        self.rb_tst.setText(_translate("MainWindow", "Test data"))
+        self.chb_io.setText(_translate("MainWindow", "Use IO Model"))
+        self.chb_pxl.setText(_translate("MainWindow", "Get Pixels"))
+        self.chb_see_pxl.setText(_translate("MainWindow", "See Pixels"))
+        self.chb_fix.setText(_translate("MainWindow", "Get Fixations"))
+        self.l_st.setText(_translate("MainWindow", "ST :"))
+        self.le_st.setText(_translate("MainWindow", "2.5"))
+        self.l_dft.setText(_translate("MainWindow", "DFT :"))
+        self.le_dft.setText(_translate("MainWindow", "0.3"))
+        self.l_mfr.setText(_translate("MainWindow", "MFR :"))
+        self.le_mfr.setText(_translate("MainWindow", "0.125, 0.165"))
+        self.pb_start.setText(_translate("MainWindow", "Start"))
+        self.l_monitor.setText(_translate("MainWindow", "Not Running..."))
+        self.pb_stop.setText(_translate("MainWindow", "Stop"))
+
+
+    def do(self):
+        self.pb_start.clicked.connect(self.b_start_action)
+        self.pb_start.clicked.connect(lambda: self.pb_start.setEnabled(False))
+        self.pb_stop.clicked.connect(self.b_stop_action)
+        self.chb_clb.clicked.connect(self.clb_uncheck)
+        self.chb_blink.clicked.connect(self.blink_uncheck)
+        self.rb_smp.clicked.connect(self.smp_uncheck)
+        self.rb_tst.clicked.connect(self.tst_uncheck)
+        self.chb_fix.clicked.connect(self.fix_uncheck)
+        
+    def b_start_action(self):
+        # # After it's activated, the algorithm receives user's data and start to do all the needed actions
+        self.num = int(self.le_num.text())
+        self.cam_id = int(self.le_cam.text())
+        self.name = self.le_name.text()
+        self.dcp = self.te_dcp.toPlainText()
+        
+        clb_grid_txt = self.le_clb_grd.text()
+        clb_grid_txt = " " + clb_grid_txt + " "
+        sep = []
+        sep.append(0)
+        for pos, s in enumerate(clb_grid_txt):
+            if s == ',':
+                sep.append(pos)
+
+        sep.append(len(clb_grid_txt)-1)
+        grid_len = len(sep)
+        self.clb_grid = []
+        for i in range(grid_len-1):
+            self.clb_grid.append(int(clb_grid_txt[sep[i]+1:sep[i+1]]))
+
+        self.thb = float(self.le_blink.text())
+        self.ss = int(self.le_shift.text())
+        self.st = float(self.le_st.text())
+        self.dft = float(self.le_dft.text())
+        mfr = self.le_mfr.text()
+        for (i, char) in enumerate(mfr):
+            if char == ",":
+                break
+        self.mfr = float(mfr[:i]), float(mfr[i+1:])
+
+        self.worker = Worker()
+        """ Worker is created for gaining the ability of multithreading. Unless, you couldn't stop the program" while it is running """
+
+        # # Giving the data that the user entered, to the program.
+        self.worker.num = self.num
+        self.worker.camera_id = self.cam_id
+        self.worker.info = (self.name, self.dcp)
+        self.worker.clb_grid = self.clb_grid
+        self.worker.thb = self.thb
+        self.worker.ss = self.ss
+        self.worker.st = self.st
+        self.worker.dft = self.dft
+        self.worker.mfr = self.mfr
+        
+
+        if self.chb_cam.checkState() == 2:
+            self.worker.cam = True
+        if self.chb_clb.checkState() == 2:
+            self.worker.clb = True
+        if self.chb_smp.checkState() == 2:
+            self.worker.smp = True
+        if self.chb_tst.checkState() == 2:
+            self.worker.acc = True
+        if self.chb_blink.checkState() == 2:
+            self.worker.tbt = True
+        if self.chb_tune_mdl.checkState() == 2:
+            self.worker.mdl = True
+        if self.chb_io.checkState() == 2:
+            self.worker.uio = True
+        if (self.chb_pxl.checkState() == 2) and self.rb_smp.isChecked():
+            self.worker.gps = True
+        if (self.chb_pxl.checkState() == 2) and self.rb_tst.isChecked():
+            self.worker.gpa = True
+        if (self.chb_see_pxl.checkState() == 2) and self.rb_smp.isChecked():
+            self.worker.sps = True
+        if (self.chb_see_pxl.checkState() == 2) and self.rb_tst.isChecked():
+            self.worker.spa = True
+        if self.chb_fix.checkState() == 2:
+            self.worker.gfx = True
+
+        self.thread = QThread()
+        self.worker.moveToThread(self.thread)
+
+        self.thread.started.connect(self.worker.do_work)
+
+        self.thread.start()
+
+        self.worker.finished.connect(self.thread.quit)
+        self.worker.cam_started.connect(lambda: self.monitor("Camera"))
+        self.worker.clb_started.connect(lambda: self.monitor("Calibration"))
+        self.worker.smp_started.connect(lambda: self.monitor("Sampling"))
+        self.worker.acc_started.connect(lambda: self.monitor("Testing"))
+        self.worker.tbt_started.connect(lambda: self.monitor("Seeing Blinking"))
+        self.worker.mdl_started.connect(lambda: self.monitor("Tuning params"))
+        self.worker.gps_started.connect(lambda: self.monitor("Getting sampling pixels"))
+        self.worker.gpa_started.connect(lambda: self.monitor("Getting test pixels"))
+        self.worker.sps_started.connect(lambda: self.monitor("Seeing sampling pixels"))
+        self.worker.spa_started.connect(lambda: self.monitor("Seeing test pixels"))
+        self.worker.gfx_started.connect(lambda: self.monitor("Getting fixations"))
+        
+        self.worker.finished.connect(lambda: self.monitor("Eye Tracking finished!"))
+        self.worker.finished.connect(lambda: self.pb_start.setEnabled(True))
+
+
+    def b_stop_action(self):
+        self.worker.running = False
+
+    def monitor(self, txt):
+        self.l_monitor.setText(txt)
+
+    def clb_uncheck(self):
+        if self.chb_clb.checkState() == 2:
+            self.le_name.setEnabled(True)
+            self.te_dcp.setEnabled(True)
+            self.le_clb_grd.setEnabled(True)
+        else:
+            self.le_name.setEnabled(False)
+            self.te_dcp.setEnabled(False)
+            self.le_clb_grd.setEnabled(False)
+
+    def blink_uncheck(self):
+        if self.chb_blink.checkState() == 2:
+            self.le_blink.setEnabled(True)
+        else:
+            self.le_blink.setEnabled(False)
+
+    def smp_uncheck(self):
+        if self.rb_smp.isChecked():
+            self.chb_fix.setEnabled(True)
+            self.chb_io.setEnabled(True)
+        else:
+            self.chb_fix.setEnabled(False)
+            self.chb_io.setEnabled(False)
+        
+        if self.rb_smp.isChecked() and (self.chb_fix.checkState() == 2):
+            self.le_st.setEnabled(True)
+            self.le_dft.setEnabled(True)
+            self.le_mfr.setEnabled(True)
+
+    def tst_uncheck(self):
+        if self.rb_tst.isChecked():
+            self.chb_fix.setEnabled(False)
+            self.chb_io.setEnabled(False)
+            self.le_st.setEnabled(False)
+            self.le_dft.setEnabled(False)
+            self.le_mfr.setEnabled(False)
+        else:
+            self.chb_fix.setEnabled(True)
+            self.chb_io.setEnabled(True)
+
+    def fix_uncheck(self):
+        if self.rb_smp.isChecked() and (self.chb_fix.checkState() == 2):
+            self.le_st.setEnabled(True)
+            self.le_dft.setEnabled(True)
+            self.le_mfr.setEnabled(True)
+        else:
+            self.le_st.setEnabled(False)
+            self.le_dft.setEnabled(False)
+            self.le_mfr.setEnabled(False)
+
+
+if __name__ == "__main__":
+    import sys
+    app = QtWidgets.QApplication(sys.argv)
+    MainWindow = QtWidgets.QMainWindow()
+    ui = Ui_MainWindow()
+    ui.setupUi(MainWindow)
+    ui.do()
+    MainWindow.show()
+    sys.exit(app.exec_())

requirements.txt

@@ -0,0 +1,74 @@
+absl-py==2.0.0
+altgraph==0.17.4
+astunparse==1.6.3
+attrs==23.1.0
+cachetools==5.3.1
+certifi==2023.7.22
+cffi==1.16.0
+charset-normalizer==3.3.0
+contourpy==1.1.1
+cycler==0.12.1
+dill==0.3.8
+et-xmlfile==1.1.0
+flatbuffers==23.5.26
+fonttools==4.43.1
+gast==0.5.4
+google-auth==2.23.2
+google-auth-oauthlib==1.0.0
+google-pasta==0.2.0
+grpcio==1.59.0
+h5py==3.9.0
+idna==3.4
+importlib-metadata==6.8.0
+importlib-resources==6.1.0
+joblib==1.3.2
+keras==2.14.0
+kiwisolver==1.4.5
+libclang==16.0.6
+Markdown==3.5
+MarkupSafe==2.1.3
+matplotlib==3.8.0
+mediapipe==0.10.5
+ml-dtypes==0.2.0
+numpy==1.26.0
+oauthlib==3.2.2
+opencv-contrib-python==4.8.1.78
+opencv-python==4.8.1.78
+openpyxl==3.1.2
+opt-einsum==3.3.0
+packaging==23.2
+pefile==2023.2.7
+Pillow==10.0.1
+protobuf==3.20.3
+pyasn1==0.5.0
+pyasn1-modules==0.3.0
+pycparser==2.21
+pyinstaller==6.5.0
+pyinstaller-hooks-contrib==2024.3
+pyparsing==3.1.1
+PyQt5==5.15.9
+PyQt5-Qt5==5.15.2
+PyQt5-sip==12.12.2
+python-dateutil==2.8.2
+pywin32-ctypes==0.2.2
+requests==2.31.0
+requests-oauthlib==1.3.1
+rsa==4.9
+scikit-learn==1.3.1
+scipy==1.11.3
+screeninfo==0.8.1
+six==1.16.0
+sounddevice==0.4.6
+tensorboard==2.14.1
+tensorboard-data-server==0.7.1
+tensorflow==2.14.0
+tensorflow-estimator==2.14.0
+tensorflow-intel==2.14.0
+tensorflow-io-gcs-filesystem==0.31.0
+termcolor==2.3.0
+threadpoolctl==3.2.0
+typing_extensions==4.8.0
+urllib3==2.0.6
+Werkzeug==3.0.0
+wrapt==1.14.1
+zipp==3.17.0