Upload 80 files

I guess GH isn't ready for all this winning.

.gitattributes

@@ -33,3 +33,26 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf filter=lfs diff=lfs merge=lfs -text
+BoostedTree.pdf filter=lfs diff=lfs merge=lfs -text
+code/creditcard.csv filter=lfs diff=lfs merge=lfs -text
+code/data/cifar-10-batches-py/data_batch_1 filter=lfs diff=lfs merge=lfs -text
+code/data/cifar-10-batches-py/data_batch_2 filter=lfs diff=lfs merge=lfs -text
+code/data/cifar-10-batches-py/data_batch_3 filter=lfs diff=lfs merge=lfs -text
+code/data/cifar-10-batches-py/data_batch_4 filter=lfs diff=lfs merge=lfs -text
+code/data/cifar-10-batches-py/data_batch_5 filter=lfs diff=lfs merge=lfs -text
+code/data/cifar-10-batches-py/test_batch filter=lfs diff=lfs merge=lfs -text
+cs229_linear_algebra_review.pdf filter=lfs diff=lfs merge=lfs -text
+cs229_probability_review.pdf filter=lfs diff=lfs merge=lfs -text
+cs229/cs229-notes-dt.pdf filter=lfs diff=lfs merge=lfs -text
+cs229/cs229-notes-ensemble.pdf filter=lfs diff=lfs merge=lfs -text
+cs229/cs229-notes1.pdf filter=lfs diff=lfs merge=lfs -text
+cs229/cs229-notes2.pdf filter=lfs diff=lfs merge=lfs -text
+cs229/cs229-notes3.pdf filter=lfs diff=lfs merge=lfs -text
+cs229/cs229-notes4.pdf filter=lfs diff=lfs merge=lfs -text
+cs229/cs229-notes5.pdf filter=lfs diff=lfs merge=lfs -text
+gcp_intro_ml_slides.pdf filter=lfs diff=lfs merge=lfs -text
+machine-learning-yearning.pdf filter=lfs diff=lfs merge=lfs -text
+matrixcookbook.pdf filter=lfs diff=lfs merge=lfs -text
+mml-book.pdf filter=lfs diff=lfs merge=lfs -text
+probability_cheatsheet.pdf filter=lfs diff=lfs merge=lfs -text

Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4ee767e0a6b04fa05ba7e599e9dbb4637a94a4407ccedf0b4d316b1fd7c8ec64
+size 18090775

BoostedTree.pdf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6819dcf6b1bf49fa47c6c4f018418dcbaaa020bf90340d89850f1b6c0bfcefd3
+size 1463610

README.md

@@ -1,3 +1,155 @@
----
-license: mit
----
+# Courses 
+
+<img src="https://upload.wikimedia.org/wikipedia/en/d/d6/Stanford_University.png" width="200px">
+
+
+## Stanford - Machine Learning - CS229 
+Course Description   This course provides a broad introduction to machine learning and statistical pattern recognition. Topics include: supervised learning (generative/discriminative learning, parametric/non-parametric learning, neural networks, support vector machines); unsupervised learning (clustering, dimensionality reduction, kernel methods); learning theory (bias/variance tradeoffs, practical advice); reinforcement learning and adaptive control. The course will also discuss recent applications of machine learning, such as to robotic control, data mining, autonomous navigation, bioinformatics, speech recognition, and text and web data processing.
+
+[course website](https://cs229.stanford.edu/)
+
+## Stanford - Natural Language Processing with Deep Learning - CS224n 
+
+Natural language processing (NLP) or computational linguistics is one of the most important technologies of the information age. Applications of NLP are everywhere because people communicate almost everything in language: web search, advertising, emails, customer service, language translation, virtual agents, medical reports, politics, etc. In the last decade, deep learning (or neural network) approaches have obtained very high performance across many different NLP tasks, using single end-to-end neural models that do not require traditional, task-specific feature engineering. In this course, students will gain a thorough introduction to cutting-edge research in Deep Learning for NLP. Through lectures, assignments and a final project, students will learn the necessary skills to design, implement, and understand their own neural network models, using the Pytorch framework.
+
+Note: 2019 lectures by Anand Avati are better and have review of concepts for first lectures.
+
+[course website](https://web.stanford.edu/class/cs224n/)
+
+## Stanford - Machine Learning with Graphs - CS224W
+
+Complex data can be represented as a graph of relationships between objects. Such networks are a fundamental tool for modeling social, technological, and biological systems. This course focuses on the computational, algorithmic, and modeling challenges specific to the analysis of massive graphs. By means of studying the underlying graph structure and its features, students are introduced to machine learning techniques and data mining tools apt to reveal insights on a variety of networks.
+Topics include: representation learning and Graph Neural Networks; algorithms for the World Wide Web; reasoning over Knowledge Graphs; influence maximization; disease outbreak detection, social network analysis.
+
+[course website](https://snap.stanford.edu/class/cs224w-2023/index.html#content)
+
+
+## Stanford - Deep Learning for Computer Vision - CS231n
+
+Computer Vision has become ubiquitous in our society, with applications in search, image understanding, apps, mapping, medicine, drones, and self-driving cars. Core to many of these applications are visual recognition tasks such as image classification, localization and detection. Recent developments in neural network (aka “deep learning”) approaches have greatly advanced the performance of these state-of-the-art visual recognition systems. This course is a deep dive into the details of deep learning architectures with a focus on learning end-to-end models for these tasks, particularly image classification. During the 10-week course, students will learn to implement and train their own neural networks and gain a detailed understanding of cutting-edge research in computer vision. Additionally, the final assignment will give them the opportunity to train and apply multi-million parameter networks on real-world vision problems of their choice. Through multiple hands-on assignments and the final course project, students will acquire the toolset for setting up deep learning tasks and practical engineering tricks for training and fine-tuning deep neural networks.
+
+[course website](http://cs231n.stanford.edu/)
+
+## Stanford - Introduction to Statistical Learning - STAT216
+
+Overview of supervised learning, with a focus on regression and classification methods. Syllabus includes: linear and polynomial regression, logistic regression and linear discriminant analysis; cross-validation and the bootstrap, model selection and regularization methods (ridge and lasso); nonlinear models, splines and generalized additive models; tree-based methods, random forests and boosting; support-vector machines; Some unsupervised learning: principal components and clustering (k-means and hierarchical). Computing is done in R, through tutorial sessions and homework assignments. This math-light course is offered via video segments (MOOC style), and in-class problem solving sessions. Prerequisites: first courses in statistics, linear algebra, and computing.
+
+[Tibshirani Course website - 2018](https://tibshirani.su.domains/stats216.html)
+[awesome ISLR course videos](https://www.dataschool.io/15-hours-of-expert-machine-learning-videos/)
+
+## Stanford - Machine Learning Systems Design - CS 329S
+
+This course aims to provide an iterative framework for developing real-world machine learning systems that are deployable, reliable, and scalable.
+
+It starts by considering all stakeholders of each machine learning project and their objectives. Different objectives require different design choices, and this course will discuss the tradeoffs of those choices.
+
+Students will learn about data management, data engineering, feature engineering, approaches to model selection, training, scaling, how to continually monitor and deploy changes to ML systems, as well as the human side of ML projects such as team structure and business metrics. In the process, students will learn about important issues including privacy, fairness, and security.
+
+
+[course website](https://stanford-cs329s.github.io/syllabus.html)
+
+
+<img src="https://upload.wikimedia.org/wikipedia/commons/thumb/d/d0/Princeton_seal.svg/320px-Princeton_seal.svg.png?1663207635322" width="200px">
+
+[Overview of AI/ML courses](https://aiml.cs.princeton.edu/course.html)
+
+## Princeton - Computer Vision - COS 429
+
+An introduction to the concepts of 2D and 3D computer vision. Topics include: low-level image processing methods such as filtering and edge detection; segmentation and clustering; optical flow and tracking; recognition; shape reconstruction from stereo, motion, texture, and shading; and recent developments in deep learning. Throughout the course, we also look at aspects of human vision and perception that guide and inspire computer vision techniques.
+
+https://www.cs.princeton.edu/courses/archive/fall19/cos429/
+
+<img src="https://upload.wikimedia.org/wikipedia/commons/thumb/9/93/Seal_of_the_University_of_Michigan.svg/320px-Seal_of_the_University_of_Michigan.svg.png?1663207811796" width="200px">
+
+
+## University of Michigan - Deep Learning for Computer Vision - EECS 498.008 / 598.008
+
+
+Computer Vision has become ubiquitous in our society, with applications in search, image understanding, apps, mapping, medicine, drones, and self-driving cars. Core to many of these applications are visual recognition tasks such as image classification and object detection. Recent developments in neural network approaches have greatly advanced the performance of these state-of-the-art visual recognition systems. This course is a deep dive into details of neural-network based deep learning methods for computer vision. During this course, students will learn to implement, train and debug their own neural networks and gain a detailed understanding of cutting-edge research in computer vision. We will cover learning algorithms, neural network architectures, and practical engineering tricks for training and fine-tuning networks for visual recognition tasks. 
+
+Excellent course and essentially an updated version of Stanford CS231n. 
+
+[YouTube Playlist](https://www.youtube.com/playlist?list=PL5-TkQAfAZFbzxjBHtzdVCWE0Zbhomg7r)
+
+[Course Website](https://web.eecs.umich.edu/~justincj/teaching/eecs498/WI2022/)
+
+<img src="https://upload.wikimedia.org/wikipedia/commons/thumb/a/aa/University_of_Wisconsin_seal.svg/640px-University_of_Wisconsin_seal.svg.png?1663207698837" width="200px">
+
+## UW - Madison - Introduction to Machine Learning and Statistical Pattern Classification - STAT 451
+
+Introduction to machine learning for pattern classification, regression analysis, clustering, and dimensionality reduction. For each category, fundamental algorithms, as well as selections of contemporary, current state-of-the-art algorithms, are being discussed. The evaluation of machine learning models using statistical methods is a particular focus of this course. Statistical pattern classification approaches, including maximum likelihood estimation and Bayesian decision theory, are compared and contrasted to algorithmic and nonparametric approaches. While fundamental mathematical concepts underlying machine learning and pattern classification algorithms are being taught, the practical use of machine learning algorithms using open source libraries from the Python programming ecosystem will be of equal focus in this course.
+
+[course website](https://sebastianraschka.com/teaching/stat451-fs2021/)
+
+[youtube playlist](https://www.youtube.com/playlist?list=PLTKMiZHVd_2KyGirGEvKlniaWeLOHhUF3)
+
+Sebastian also has an excellent textbook [info](https://sebastianraschka.com/blog/2022/ml-pytorch-book.html) 
+
+
+<img src="https://upload.wikimedia.org/wikipedia/en/thumb/4/44/MIT_Seal.svg/1200px-MIT_Seal.svg.png" width="200px">
+
+## MIT - Matrix Methods in Data Analysis, Signal Processing, and Machine Learning - MATH 18.065
+
+Linear algebra concepts are key for understanding and creating machine learning algorithms, especially as applied to deep learning and neural networks. This course reviews linear algebra with applications to probability and statistics and optimization–and above all a full explanation of deep learning.
+
+Taught by Gil Strang! - Also 💙 the intro linear algebra course with Strang 
+
+[OCW link](https://ocw.mit.edu/courses/18-065-matrix-methods-in-data-analysis-signal-processing-and-machine-learning-spring-2018/)
+
+## MIT - Distributed Systems - CS 6.824
+
+6.824 is a core 12-unit graduate subject with lectures, readings, programming labs, an optional project, a mid-term exam, and a final exam. It will present abstractions and implementation techniques for engineering distributed systems. Major topics include fault tolerance, replication, and consistency. Much of the class consists of studying and discussing case studies of distributed systems.
+
+[course website](https://pdos.csail.mit.edu/6.824/)
+
+## MIT - Introduction to Deep Learning - CS 6.S191 
+
+Pretty high level view of DL. 
+
+[youtube playlist](https://www.youtube.com/playlist?list=PLtBw6njQRU-rwp5__7C0oIVt26ZgjG9NI)
+
+<img src="https://cdn5.euraxess.org/sites/default/files/news/eth_1.png" width="200px">
+
+## ETH Zurich - Digital Design and Computer Architecture
+
+The class provides a first introduction to the design of digital circuits and computer architecture. It covers technical foundations of how a computing platform is designed from the bottom up. It introduces various execution paradigms, hardware description languages, and principles in digital design and computer architecture. The focus is on fundamental techniques employed in the design of modern microprocessors and their hardware/software interface.
+
+[course website](https://safari.ethz.ch/digitaltechnik/spring2021/doku.php?id=start)
+[YouTube Playlist](https://www.youtube.com/watch?v=LbC0EZY8yw4&list=PL4YhK0pT0ZhXVMJMffEq_XqAIQM_uWSdi)
+
+<img src="https://bair.berkeley.edu/images/BAIR_Logo_BlueType_Tag.png" width="200px"> 
+
+## UC Berkely - Advanced Robotics - CS 287
+
+Over the past ten years advances in optimization, in probabilistic reasoning, and in machine learning have had a large impact in robotics, with many of the current state-of-the-art algorithms heavily relying on these tools. At the same time these three tools have wide applicability in many other fields. The current curriculum of CS287 is centered around these three tools---making it both a treatment of these tools (in the context of a specific application domain, namely robotics), as well as a treatment of the state of the art in (algorithmic) robotics. Problem sets are a mix of mathematical/algorithmic questions and programming problems. There is a substantial final project. NOTE: This course is about algorithms for robotics, and does *not* cover hardware aspects. PREREQS: Familiarity with mathematical proofs, probability, algorithms, linear algebra; ability to implement algorithmic ideas in code. 
+
+[course website](https://people.eecs.berkeley.edu/~pabbeel/cs287-fa19/)
+
+<img src="https://cmu-multicomp-lab.github.io/mmml-course/assets/img/cmu-logo.png" width="200px"> 
+
+## Carnegie Mellon University - MultiModal Machine Learning - 11-777 • Fall 2022 
+
+Multimodal machine learning (MMML) is a vibrant multi-disciplinary research field which addresses some of the original goals of artificial intelligence by integrating and modeling multiple communicative modalities, including linguistic, acoustic, and visual messages. With the initial research on audio-visual speech recognition and more recently with language & vision projects such as image and video captioning, this research field brings some unique challenges for multimodal researchers given the heterogeneity of the data and the contingency often found between modalities. This course will teach fundamental mathematical concepts related to MMML including multimodal alignment and fusion, heterogeneous representation learning and multi-stream temporal modeling. We will also review recent papers describing state-of-the-art probabilistic models and computational algorithms for MMML and discuss the current and upcoming challenges.
+
+[course website](https://cmu-multicomp-lab.github.io/mmml-course/fall2022/)
+
+<img src="https://logos-world.net/wp-content/uploads/2022/01/University-of-Washington-Symbol.png" width="200px">
+
+## University of Washington 
+
+Welcome to the Computer Vision course (CSE/ECE 576, Spring 2020)
+
+This class is a general introduction to computer vision. It covers standard techniques in image processing like filtering, edge detection, stereo, flow, etc. (old-school vision), as well as newer, machine-learning based computer vision.
+
+[course website](https://courses.cs.washington.edu/courses/cse576/20sp/)
+
+<img src="https://ai.google/static/images/share.png" width="200px">
+
+## Google Machine Learning Crash Course 
+
+Nice quick and easy overview ML topics.
+
+
+[course website](https://developers.google.com/machine-learning/crash-course/ml-intro)
+
+

code/README.md

@@ -0,0 +1,74 @@
+---
+license: apache-2.0
+tags:
+- image-classification
+- generated_from_trainer
+datasets:
+- beans
+metrics:
+- accuracy
+model-index:
+- name: ''
+  results:
+  - task:
+      name: Image Classification
+      type: image-classification
+    dataset:
+      name: beans
+      type: beans
+      args: default
+    metrics:
+    - name: Accuracy
+      type: accuracy
+      value: 0.9774436090225563
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# 
+
+This model is a fine-tuned version of [google/vit-base-patch16-224-in21k](https://huggingface.co/google/vit-base-patch16-224-in21k) on the beans dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.0666
+- Accuracy: 0.9774
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 0.0002
+- train_batch_size: 16
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 4
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Accuracy |
+|:-------------:|:-----:|:----:|:---------------:|:--------:|
+| 0.0742        | 1.54  | 100  | 0.0806          | 0.9774   |
+| 0.0185        | 3.08  | 200  | 0.0666          | 0.9774   |
+
+
+### Framework versions
+
+- Transformers 4.14.1
+- Pytorch 1.10.0
+- Datasets 2.0.0
+- Tokenizers 0.10.3

code/activation_functions.md

@@ -0,0 +1,45 @@
+https://stats.stackexchange.com/questions/11859/what-is-the-difference-between-multiclass-and-multilabel-problem
+
+https://machinelearningmastery.com/choose-an-activation-function-for-deep-learning/
+
+https://web.stanford.edu/~nanbhas/blog/sigmoid-softmax/
+
+
+Binary Classification: One node, sigmoid activation.
+Multiclass Classification: One node per class, softmax activation.
+Multilabel Classification: One node per class, sigmoid activation.
+
+Multi-class vs Binary-class is the question of the number of classes your classifier is modeling. In theory, a binary classifier is much simpler than multi-class, so it's important to make this distinction. For example, the Support vector machine (SVM) can trivially learn a hyperplane to separate two classes, but 3 or more classes makes it complex. In the neural networks, we commonly use Sigmoid for binary, but Softmax for multi-class as the last layer of the model.
+
+Multi-label vs Single-Label is the question of how many classes any object or example can belong to. In the neural networks, if single label is needed we use a single Softmax layer as the last layer, thus learning a single probability distribution that spans across all classes. If the multi-label classification is needed, we use multiple Sigmoids on the last layer, thus learning separate distribution for each class.
+
+# Cross-Entropy or Log Likelihood in Output Layer (StackExchange)
+
+*negative log likelihood* is also known as multi class cross-entropy 
+
+
+### all of the normal loss functions and their applications in PyTorch 
+
+https://neptune.ai/blog/pytorch-loss-functions
+
+
+https://medium.com/deeplearningmadeeasy/negative-log-likelihood-6bd79b55d8b6 
+
+It’s a cost function that is used as loss for machine learning models, telling us how bad it’s performing, the lower the better.
+
+I’m going to explain it word by word, hopefully that will make it. easier to understand.
+
+Negative: obviously means multiplying by -1. What? The loss of our model. Most machine learning frameworks only have minimization optimizations, but we want to maximize the probability of choosing the correct category.
+
+We can **maximize by minimizing the negative log likelihood,** there you have it, we want somehow to maximize by minimizing.
+
+Also it’s much easier to reason about the loss this way, to be consistent with the rule of loss functions approaching 0 as the model gets better.
+
+
+cross entropy loss is same as negative log likelihood 
+
+
+NLL uses a negative connotation since the probabilities (or likelihoods) vary between zero and one, and the logarithms of values in this range are negative. In the end, the loss value becomes positive.
+
+
+

code/all_results.json

@@ -0,0 +1,12 @@
+{
+    "epoch": 1.0,
+    "eval_accuracy": 0.9774436090225563,
+    "eval_loss": 0.10330713540315628,
+    "eval_runtime": 10.9904,
+    "eval_samples_per_second": 12.101,
+    "eval_steps_per_second": 1.547,
+    "train_loss": 0.3343511177943303,
+    "train_runtime": 232.7119,
+    "train_samples_per_second": 4.443,
+    "train_steps_per_second": 0.279
+}

code/classification.md

@@ -0,0 +1,88 @@
+# Logistic Regression  
+
+References: 
+https://www.kdnuggets.com/2020/01/guide-precision-recall-confusion-matrix.html
+https://developers.google.com/machine-learning/crash-course/classification/thresholding
+https://machinelearningmastery.com/roc-curves-and-precision-recall-curves-for-classification-in-python/ 
+
+https://developers.google.com/machine-learning/crash-course/logistic-regression/model-training
+
+### Thresholding 
+
+A logistic regression model that returns 0.9995 for a particular email message is predicting that it is very likely to be spam. Conversely, another email message with a prediction score of 0.0003 on that same logistic regression model is very likely not spam. However, what about an email message with a prediction score of 0.6? In order to map a logistic regression value to a binary category, you must define a classification threshold (also called the decision threshold). A value above that threshold indicates "spam"; a value below indicates "not spam." It is tempting to assume that the classification threshold should always be 0.5, but thresholds are problem-dependent, and are therefore values that you must tune.
+
+### Accuracy 
+
+$ 
+Accuracy = Total correct / total predictions
+$ 
+
+Using the Confusion Matrix values
+
+$ 
+Accuracy = TP + TN / TP + FP + TN + FN
+$
+
+Accuracy alone doesn't tell the full story when you're working with a **class-imbalanced data** set, like this one, where there is a significant disparity between the number of positive and negative labels.
+
+### Precision 
+
+Precision — Also called Positive predictive value
+The ratio of correct positive predictions to the *total predicted positives.*
+
+$
+Precision = \frac{TP}{TP + FP}
+$
+
+
+### Recall 
+
+Recall — Also called Sensitivity, Probability of Detection, True Positive Rate
+
+The ratio of correct positive predictions to the *total positives examples.*
+
+$
+Recall = \frac{TP}{TP + FN}
+$
+
+### ROC & AUC 
+
+* ROC Curves summarize the trade-off between the true positive rate and false positive rate for a predictive model using different probability thresholds.
+    
+* Precision-Recall curves summarize the trade-off between the true positive rate and the positive predictive value for a predictive model using different probability thresholds.
+
+* ROC curves are appropriate when the observations are balanced between each class, whereas precision-recall curves are appropriate for imbalanced datasets.
+
+
+### sklearn functions
+
+https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html
+https://scikit-learn.org/stable/modules/generated/sklearn.metrics.precision_recall_curve.html
+
+
+### What about using XGBoost for classification? 
+
+We can still use XGBoost but logistic regression is linear and XGBoost is *not* linear. 
+
+For example we can see here that we are drawing linear boundaries between classifications in the iris dataset. 
+
+https://scikit-learn.org/stable/auto_examples/linear_model/plot_iris_logistic.html#sphx-glr-auto-examples-linear-model-plot-iris-logistic-py 
+
+
+### What about difference between SVM and logistic regression? 
+
+
+### Logistic Regression Loss function 
+
+**This always trips me up because some people call it *log loss* or cross entropy or logits or something else!**
+
+The loss function for linear regression is squared loss. The loss function for logistic regression is Log Loss, which is defined as follows:
+$
+put formula in here later
+$
+
+is the data set containing many labeled examples, which are
+pairs.
+is the label in a labeled example. Since this is logistic regression, every value of
+must either be 0 or 1.
+is the predicted value (somewhere between 0 and 1), given the set of features in . 

code/config.json

@@ -0,0 +1,32 @@
+{
+  "_name_or_path": "google/vit-base-patch16-224-in21k",
+  "architectures": [
+    "ViTForImageClassification"
+  ],
+  "attention_probs_dropout_prob": 0.0,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.0,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "angular_leaf_spot",
+    "1": "bean_rust",
+    "2": "healthy"
+  },
+  "image_size": 224,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "angular_leaf_spot": "0",
+    "bean_rust": "1",
+    "healthy": "2"
+  },
+  "layer_norm_eps": 1e-12,
+  "model_type": "vit",
+  "num_attention_heads": 12,
+  "num_channels": 3,
+  "num_hidden_layers": 12,
+  "patch_size": 16,
+  "qkv_bias": true,
+  "torch_dtype": "float32",
+  "transformers_version": "4.14.1"
+}

code/cracks.py

@@ -0,0 +1,32 @@
+from autodistill_grounded_sam import GroundedSAM
+from autodistill.detection import CaptionOntology
+from autodistill.utils import plot
+import cv2
+
+# define an ontology to map class names to our OWLv2 prompt
+# the ontology dictionary has the format {caption: class}
+# where caption is the prompt sent to the base model, and class is the label that will
+# be saved for that caption in the generated annotations
+# then, load the model
+classes = ["crack"]
+
+base_model = GroundedSAM(ontology=CaptionOntology({"crack": "crack"}))
+
+results = base_model.predict("crack.png")
+
+image = cv2.imread("crack.png")
+
+# Print image properties
+print("Image shape:", image.shape)  # Shows (height, width, channels)
+print("Image dtype:", image.dtype)  # Shows data type of the image array
+print("Image size:", image.size)  # Shows total number of pixels * channels
+
+
+plot(
+    image=image,
+    detections=results,
+    classes=[classes[i] for i in results.class_id],
+)
+
+
+print(results)

code/creditcard.csv

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:76274b691b16a6c49d3f159c883398e03ccd6d1ee12d9d8ee38f4b4b98551a89
+size 150828752

code/cs224N_pytorch_tutorial_nlp.ipynb

@@ -0,0 +1,981 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import torch \n",
+    "import torch.nn as nn\n",
+    "\n",
+    "import pprint \n",
+    "pp = pprint.PrettyPrinter()\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Tensors "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[[1, 2, 3], [4, 5, 6]]\n"
+     ]
+    }
+   ],
+   "source": [
+    "list_of_lists = [[1 ,2, 3], [4,5,6]]\n",
+    "print(list_of_lists)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "tensor([[0, 1],\n",
+       "        [2, 3],\n",
+       "        [4, 5]])"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "#init a tensor\n",
+    "data = torch.tensor([\n",
+    "    [0,1],\n",
+    "    [2,3],\n",
+    "    [4,5]\n",
+    "])\n",
+    "data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "tensor([[0., 1.],\n",
+       "        [2., 3.],\n",
+       "        [4., 5.]])"
+      ]
+     },
+     "execution_count": 10,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# we can cast them to different types -- usualloy float32 or int\n",
+    "data = torch.tensor([\n",
+    "    [0,1],\n",
+    "    [2,3],\n",
+    "    [4,5],\n",
+    "] ,   dtype=torch.float32)\n",
+    "data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "torch.Size([3, 2])"
+      ]
+     },
+     "execution_count": 11,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "data.shape"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 24,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "data2 = torch.tensor([\n",
+    "    [1, 2, 3],\n",
+    "    [4, 5, 6]\n",
+    "], dtype=torch.float32)\n",
+    "\n",
+    "# must be same type \n",
+    "data3 = data @ data2"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "tensor([1, 2, 3, 4, 5, 6, 7, 8, 9])"
+      ]
+     },
+     "execution_count": 16,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "rr = torch.arange(1, 10)\n",
+    "rr"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 22,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "tensor([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15])\n",
+      "tensor([[ 1,  2,  3],\n",
+      "        [ 4,  5,  6],\n",
+      "        [ 7,  8,  9],\n",
+      "        [10, 11, 12],\n",
+      "        [13, 14, 15]])\n",
+      "torch.Size([5, 3])\n"
+     ]
+    }
+   ],
+   "source": [
+    "# can reshape a vector into a matrix!  \n",
+    "# evidently this can make batch operations easier \n",
+    "# e.x. take a 15 element row vector and turn it into a 5 x 3 matrix \n",
+    "vec = torch.arange(1,16)\n",
+    "print(vec)\n",
+    "matrix = vec.view(5,3)\n",
+    "print(matrix)\n",
+    "print(matrix.shape)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 23,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# one of the reason we use tensors is vectorized operations: \n",
+    "# operations that be conducted in parallel over a paarticular dimension of a tensor \n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 28,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "tensor([42., 57., 72.])"
+      ]
+     },
+     "execution_count": 28,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# things we can do like sum, mean, and std dev \n",
+    "\n",
+    "# https://stackoverflow.com/questions/55691819/why-does-dim-1-return-row-indices-in-torch-argmax\n",
+    "\n",
+    "# dim=1 is ROWS \n",
+    "data3.mean(dim=1)\n",
+    "data3.sum(dim=1)\n",
+    "\n",
+    "# dim=0 is COLUMNS\n",
+    "data3.mean(dim=0)\n",
+    "data3.sum(dim=0)\n",
+    "\n",
+    "# look at the official pytorch tutorial for a good example of all the way you can slice a tensor\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Autograd \n",
+    "\n",
+    "Pytorch is known for it's auto differentation feature.. We can call the `backward()` method to ask Pytorch to calculate the gradients which are then stored in the `grad` attribute"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 33,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "None\n"
+     ]
+    }
+   ],
+   "source": [
+    "x = torch.tensor([2], dtype=torch.float32, requires_grad=True)\n",
+    "\n",
+    "pp.pprint(x.grad)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 53,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "<class 'torch.Tensor'>\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "tensor([84.])"
+      ]
+     },
+     "execution_count": 53,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "y = 3 * x * x \n",
+    "# z = 3 * x * x\n",
+    "print(type(y))\n",
+    "y.backward()\n",
+    "# if we just want to manually reset the grad instead of using an optimizer \n",
+    "# we can just do it manually like this... \n",
+    "# x.grad = torch.tensor([0.])\n",
+    "# x.zero_grad()\n",
+    "x.grad"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Neural Network Module\n",
+    "\n",
+    "### Linear Layer \n",
+    "\n",
+    "we can use `nn.Linear(H_in, H_out)` to create a linear layer. This will take a matrix of (N, *, H_in) dimensions and ouptput a matrix of (N, *, H_out). the `*` denotes that there could be arbitrary number of dimensions in between. The linear layer performs the operation `Ax+b`., whewr e`A` and `b` are intialized randomly. If we don't wnat the linear layer to learn the bias parameters, we can intialize our layer with `bias=False`\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 88,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "tensor([[[ 0.1580, -0.3979],\n",
+       "         [ 0.1580, -0.3979],\n",
+       "         [ 0.1580, -0.3979]],\n",
+       "\n",
+       "        [[ 0.1580, -0.3979],\n",
+       "         [ 0.1580, -0.3979],\n",
+       "         [ 0.1580, -0.3979]]], grad_fn=<AddBackward0>)"
+      ]
+     },
+     "execution_count": 88,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "import torch.nn as nn\n",
+    "\n",
+    "# create the inputs \n",
+    "# two, 3x4 matrices of ones. \n",
+    "# 3 rows by 4 columns \n",
+    "inputs = torch.ones(2,3,4)\n",
+    "# print(inputs)\n",
+    "\n",
+    "# make a linear layers transforming N,*,H_in dimensional inputs to N,*,H_out dimensional outputs \n",
+    "linear = nn.Linear(4,2)\n",
+    "linear_output = linear(inputs)\n",
+    "linear_output.shape\n",
+    "linear_output"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We get different values in the tensor each time we run it because `Linear()` randomly initializes the function `Ax + b` to different values each time!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 90,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[Parameter containing:\n",
+       " tensor([[-0.2625, -0.1494,  0.0883,  0.1954],\n",
+       "         [ 0.0474,  0.1861,  0.1699, -0.3701]], requires_grad=True),\n",
+       " Parameter containing:\n",
+       " tensor([ 0.2862, -0.4312], requires_grad=True)]"
+      ]
+     },
+     "execution_count": 90,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "list(linear.parameters())"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Activation Function Layer\n",
+    "\n",
+    "we can also use `nn` module to apply activations function to our tensors. Activation functions are used to add non-linearity to our network. Some examples of activations functions are `nn.ReLU()`, `nn.Sigmoid()` and `nn.LeakyReLU()`. Activation functions operate on each elementate seperatel, so sthe sahpe of the tensorwe get as an output ar ethe same as the ones we pass in."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 92,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "tensor([[[0.5394, 0.4018],\n",
+       "         [0.5394, 0.4018],\n",
+       "         [0.5394, 0.4018]],\n",
+       "\n",
+       "        [[0.5394, 0.4018],\n",
+       "         [0.5394, 0.4018],\n",
+       "         [0.5394, 0.4018]]], grad_fn=<SigmoidBackward0>)"
+      ]
+     },
+     "execution_count": 92,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "sigmoid = nn.Sigmoid()\n",
+    "\n",
+    "output = sigmoid(linear_output)\n",
+    "output\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Putting the Layers Together\n",
+    "\n",
+    "So far we have just created lyaers and pass the output of one as the input of the next. Instead of creating intermediate tensors and passing them around, we can use `nn.Sequeuntial` which puts layers together sequentially"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 101,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "block = nn.Sequential(\n",
+    "    nn.Linear(4,2),\n",
+    "    nn.Linear(2,1),\n",
+    "    nn.Sigmoid(),\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 102,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "tensor([[[0.4056],\n",
+       "         [0.4056],\n",
+       "         [0.4056]],\n",
+       "\n",
+       "        [[0.4056],\n",
+       "         [0.4056],\n",
+       "         [0.4056]]], grad_fn=<SigmoidBackward0>)"
+      ]
+     },
+     "execution_count": 102,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "input = torch.ones(2,3,4)\n",
+    "output = block(input)\n",
+    "output"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Custom Modules \n",
+    "\n",
+    "Instead of using the predefined modules, we can also build our own by extending the `nn.Module` class. For example , we can build the `nn.Linear` on our own using te tensor introduced earlier! We can also build new, more complex modules, such a s a custom neural network. \n",
+    "\n",
+    "To create a custom module, the first thing we have to do is to extend the `nn.Module` . We can then initialize our parameters in the `__init__` function, staring with a call to the `__init__` function of the super class. Al the class attributes we define which are `nn` module objects are treated as parameters , which can then learned during the training. Tensors are not parameters , but they can be turned into parameters if they are wrapp in `nn.Parameters` class \n",
+    "\n",
+    "All classes extending the `nn.Module` are also expected to implement a `forward(x)` function, where `x` is the tensor. this is the function that is called when a parameters is passed to our modul, for example `modelFooBarBaz(x)`"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 107,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class MultiLayerPerceptron(nn.Module):\n",
+    "    def __init__(self, input_size, hidden_size):\n",
+    "        # Call to the super() class \n",
+    "        super(MultiLayerPerceptron, self).__init__()\n",
+    "\n",
+    "        # saving initialization params\n",
+    "        self.input_size = input_size \n",
+    "        self.hidden_size = hidden_size\n",
+    "\n",
+    "        #define our model \n",
+    "        # doesn't have to be named model \n",
+    "        self.model = nn.Sequential(\n",
+    "            nn.Linear(self.input_size, self.hidden_size),\n",
+    "            nn.ReLU(),\n",
+    "            nn.Linear(self.hidden_size, self.input_size),\n",
+    "            nn.Sigmoid()\n",
+    "        )\n",
+    "\n",
+    "    def forward(self, x):\n",
+    "        output = self.model(x)\n",
+    "        return output\n",
+    "\n",
+    "        \n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 108,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "tensor([[0.3950, 0.4821, 0.3842, 0.5562, 0.4945],\n",
+       "        [0.4746, 0.5936, 0.3286, 0.4435, 0.3822]], grad_fn=<SigmoidBackward0>)"
+      ]
+     },
+     "execution_count": 108,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# now lets instantiate it and see what it does. \n",
+    "\n",
+    "input = torch.randn(2,5)\n",
+    "\n",
+    "multi_layer_perceptron = MultiLayerPerceptron(5,3)\n",
+    "\n",
+    "#pass our input through our model \n",
+    "multi_layer_perceptron(input)\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 109,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[('model.0.weight',\n",
+       "  Parameter containing:\n",
+       "  tensor([[-0.3665, -0.0611,  0.4206, -0.1927, -0.2036],\n",
+       "          [-0.3317, -0.4113, -0.1653, -0.2854, -0.1773],\n",
+       "          [-0.4202,  0.1413,  0.0547, -0.0287, -0.4163]], requires_grad=True)),\n",
+       " ('model.0.bias',\n",
+       "  Parameter containing:\n",
+       "  tensor([ 0.4075, -0.2855, -0.0500], requires_grad=True)),\n",
+       " ('model.2.weight',\n",
+       "  Parameter containing:\n",
+       "  tensor([[ 0.1899,  0.2779,  0.1600],\n",
+       "          [ 0.1802,  0.2193,  0.5014],\n",
+       "          [-0.2587,  0.1474,  0.2716],\n",
+       "          [-0.3371,  0.4262,  0.0181],\n",
+       "          [-0.4094,  0.2103,  0.2476]], requires_grad=True)),\n",
+       " ('model.2.bias',\n",
+       "  Parameter containing:\n",
+       "  tensor([-0.4264, -0.0715, -0.4720,  0.2259, -0.0221], requires_grad=True))]"
+      ]
+     },
+     "execution_count": 109,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# we can inspect the parameters of our model with `named_parameters()` and `parameters()`\n",
+    "list(multi_layer_perceptron.named_parameters())"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 116,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[Parameter containing:\n",
+       " tensor([[-0.1681,  0.3615, -0.1085,  0.2136, -0.3068],\n",
+       "         [ 0.1105, -0.0750, -0.4361,  0.4284,  0.3173],\n",
+       "         [ 0.1914,  0.1469,  0.4325, -0.1647, -0.3230]], requires_grad=True),\n",
+       " Parameter containing:\n",
+       " tensor([-0.0533, -0.1809, -0.3992], requires_grad=True),\n",
+       " Parameter containing:\n",
+       " tensor([[ 0.5454,  0.0090, -0.0701],\n",
+       "         [ 0.4629, -0.2518, -0.4816],\n",
+       "         [ 0.0641,  0.3194,  0.4202],\n",
+       "         [ 0.5256,  0.1806, -0.4929],\n",
+       "         [ 0.0383,  0.4789, -0.3110]], requires_grad=True),\n",
+       " Parameter containing:\n",
+       " tensor([-0.0959,  0.4256, -0.3175, -0.5408, -0.3457], requires_grad=True)]"
+      ]
+     },
+     "execution_count": 116,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "list(multi_layer_perceptron.parameters())"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Optimization \n",
+    "\n",
+    "We have showed how gradients are calculated with the `backward()` function. Having the gradients isn't enough for our models to learn. We also need to know how to update the parameters of our models. this is where the optimizers comes in. `torch.optim` module contains several optimizers that we can use. Some popular examples are `optim.SGD` and `optim.Adam`. When initalizing optimizers, we pass our model parameters which can be accessed with `model.parameters()`, tellin g the optimizers which values it will optimizing. Optimizers also have a learning rate `lr` parameters, which determines how big of a n update will be made in every step. Different optimizers have different hypterparameters as well. "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "after we have our optimization function, we can defien a loss that we want to optimize for. We can either define the loss ourselve, or use one of th epredefined loss function in `pyTorch`, such as `nn.BCELoss()`. Let's put everything together now! We will start by creating some dummy data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 112,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "tensor([[1.6229, 1.6178, 1.8535, 1.6113, 1.0269],\n",
+       "        [1.2863, 1.9466, 1.5752, 1.0483, 1.0441],\n",
+       "        [1.9156, 1.2305, 1.4923, 1.9213, 1.9353],\n",
+       "        [1.1242, 1.0729, 1.3030, 1.0789, 1.0008],\n",
+       "        [1.9523, 1.0653, 1.9383, 1.7856, 1.3598],\n",
+       "        [1.2001, 1.4937, 1.9633, 1.7623, 1.8388],\n",
+       "        [1.9458, 1.6999, 1.5424, 1.1136, 1.2931],\n",
+       "        [1.3253, 1.6970, 1.1702, 1.5542, 1.9958],\n",
+       "        [1.4209, 1.7721, 1.8824, 1.5063, 1.6478],\n",
+       "        [1.8896, 1.3686, 1.0754, 1.1028, 1.7788]])"
+      ]
+     },
+     "execution_count": 112,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "import torch.optim as optim \n",
+    "\n",
+    "# create label/y/output data \n",
+    "y = torch.ones(10, 5)\n",
+    "\n",
+    "# add some noise to our y to geenrate our x \n",
+    "x = y + torch.rand_like(y)\n",
+    "x\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Define our \n",
+    "1. model \n",
+    "2. optimizer \n",
+    "3. loss function (to optimize)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 114,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "<class 'torch.optim.adam.Adam'>\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "0.7805761098861694"
+      ]
+     },
+     "execution_count": 114,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "multi_layer_perceptron = MultiLayerPerceptron(5,3)\n",
+    "\n",
+    "# optimizer \n",
+    "adam = optim.Adam(multi_layer_perceptron.parameters(), lr=1e-1)\n",
+    "print(type(adam))\n",
+    "\n",
+    "# loss function \n",
+    "loss_function = nn.BCELoss()\n",
+    "\n",
+    "# calculate how our model is doing now \n",
+    "y_pred = multi_layer_perceptron(x)\n",
+    "loss_function(y_pred, y).item()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's see if we can have our model achieve a smaller loss. Now that we have everything we need, we can setup our training loop. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 117,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 0: training loss: 0.7805761098861694\n",
+      "Epoch 1: training loss: 0.465933233499527\n",
+      "Epoch 2: training loss: 0.21142368018627167\n",
+      "Epoch 3: training loss: 0.06858699023723602\n",
+      "Epoch 4: training loss: 0.018102366477251053\n",
+      "Epoch 5: training loss: 0.004522338043898344\n",
+      "Epoch 6: training loss: 0.0011653534602373838\n",
+      "Epoch 7: training loss: 0.00032139578252099454\n",
+      "Epoch 8: training loss: 9.614464215701446e-05\n",
+      "Epoch 9: training loss: 3.128984462819062e-05\n"
+     ]
+    }
+   ],
+   "source": [
+    "# set number of epoch which determines the number of training iterations. \n",
+    "n_epoch = 10 \n",
+    "\n",
+    "for epoch in range(n_epoch):\n",
+    "    #set gradients to zero \n",
+    "    adam.zero_grad()\n",
+    "\n",
+    "    # get model predictions \n",
+    "    y_pred = multi_layer_perceptron(x)\n",
+    "\n",
+    "\n",
+    "    # calculate the loss \n",
+    "    loss = loss_function(y_pred, y)\n",
+    "\n",
+    "    # print stats\n",
+    "    print(f\"Epoch {epoch}: training loss: {loss}\")\n",
+    "\n",
+    "    #computer the gradients \n",
+    "    loss.backward() \n",
+    "\n",
+    "    # take a step to optimize the weights \n",
+    "    adam.step()\n",
+    "\n",
+    "    \n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 118,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[Parameter containing:\n",
+       " tensor([[ 0.5215,  1.0621,  0.5877,  0.9016,  0.3710],\n",
+       "         [ 0.8491,  0.6657,  0.3032,  1.1651,  1.0538],\n",
+       "         [-0.2273, -0.2718,  0.0138, -0.5835, -0.7417]], requires_grad=True),\n",
+       " Parameter containing:\n",
+       " tensor([ 0.6412,  0.5603, -0.8180], requires_grad=True),\n",
+       " Parameter containing:\n",
+       " tensor([[ 1.2475,  0.7545,  0.3486],\n",
+       "         [ 1.2015,  0.5252, -0.0629],\n",
+       "         [ 0.7851,  1.0803,  0.8389],\n",
+       "         [ 1.2163,  0.9155, -0.0741],\n",
+       "         [ 0.7432,  1.2263,  0.1078]], requires_grad=True),\n",
+       " Parameter containing:\n",
+       " tensor([0.5621, 1.1223, 0.3672, 0.1077, 0.3128], requires_grad=True)]"
+      ]
+     },
+     "execution_count": 118,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "list(multi_layer_perceptron.parameters())"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "we can see that our loss is decreasing. Let's check the predictions of our model now and see if they are close to our original y of all ones."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 120,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "tensor([[1.0000, 1.0000, 1.0000, 1.0000, 1.0000],\n",
+       "        [1.0000, 1.0000, 1.0000, 1.0000, 1.0000],\n",
+       "        [1.0000, 1.0000, 1.0000, 1.0000, 1.0000],\n",
+       "        [0.9999, 0.9999, 0.9999, 1.0000, 0.9999],\n",
+       "        [1.0000, 1.0000, 1.0000, 1.0000, 1.0000],\n",
+       "        [1.0000, 1.0000, 1.0000, 1.0000, 1.0000],\n",
+       "        [1.0000, 1.0000, 1.0000, 1.0000, 1.0000],\n",
+       "        [1.0000, 1.0000, 1.0000, 1.0000, 1.0000],\n",
+       "        [1.0000, 1.0000, 1.0000, 1.0000, 1.0000],\n",
+       "        [1.0000, 1.0000, 1.0000, 1.0000, 1.0000]], grad_fn=<SigmoidBackward0>)"
+      ]
+     },
+     "execution_count": 120,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "y_pred = multi_layer_perceptron(x)\n",
+    "y_pred"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Looks our model almost perfectly learned to filter out the noise from the `x` that we passed in!"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# NLP - Word Window Classification \n",
+    "\n",
+    "In this section, our goal will be to train a model that will find the words in a sentence corresponding to a `LOCATION`, which will be always of span `1`. (Meaning tha t`San Fransisco` won't be recognized a s a `LOCATION`). our task is called `Word Window Classification` for a reason. Instead of letting our model to only take a look at one word in each forward pass, we would like tot be able to consider the context of the word in question. That i, for each word, we want our model to be aware of the surrounding words. Let's dive in! "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 121,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Data \n",
+    "\n",
+    "# Our raw corpus. \n",
+    "corpus = [\n",
+    "    \"We always come to Paris\", \n",
+    "    \"The processor is from Australia\", \n",
+    "    \"I live in Stanford\",\n",
+    "    \"He comes from Taiwan\",\n",
+    "    \"The capital of Turkey is Ankara\",\n",
+    "]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Preprocessing\n",
+    "\n",
+    "To make it easier for our models to learn, we usually apply a few preprocessing steps to our data. This is especially important when dealing with text data. Here are some examples of text preprocessing: \n",
+    "\n",
+    "* Tokenization: turning sentence in token representation \n",
+    "* Lowercasing: convert all letters to lowercase \n",
+    "* Noise Removal: removing special characters (such as punctuations)\n",
+    "* Stop words removal: removing commonly used words \n",
+    "\n",
+    "Which preprocessing steps are necessary is determind by the task at hand. For example, although it is useful to remove special characers in some tasks, for other they may be important( for example, if we are dealing with ultiple langugaes). For our task, we will lowercase our words and tokenize. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 123,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[['we', 'always', 'come', 'to', 'paris'],\n",
+       " ['the', 'processor', 'is', 'from', 'australia'],\n",
+       " ['i', 'live', 'in', 'stanford'],\n",
+       " ['he', 'comes', 'from', 'taiwan'],\n",
+       " ['the', 'capital', 'of', 'turkey', 'is', 'ankara']]"
+      ]
+     },
+     "execution_count": 123,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "def preprocess_sentence(sentence):\n",
+    "    return sentence.lower().split()\n",
+    "\n",
+    "# create our training set \n",
+    "train_sentences = [preprocess_sentence(sentence) for sentence in corpus]\n",
+    "train_sentences"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "for each training example we have, we should also have a corresponding label. Recall that the goal of our model was to determine which words correspond to a `LOCATION`. That is, we want our model to output `0` for all the words that are not `LOCATION` and `1` for the ones that are `LOCATION`s. But we need to create a data for our output "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 125,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[[0, 0, 0, 0, 1],\n",
+       " [0, 0, 0, 0, 1],\n",
+       " [0, 0, 0, 1],\n",
+       " [0, 0, 0, 1],\n",
+       " [0, 0, 0, 1, 0, 1]]"
+      ]
+     },
+     "execution_count": 125,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# set of lcoations tha tappear in our corpus \n",
+    "locations = set([\"australia\", \"ankara\", \"paris\", \"stanford\", \"taiwan\", \"turkey\"])\n",
+    "\n",
+    "# our train labels \n",
+    "train_labels = [[1 if word in locations else 0 for word in sentence] for sentence in train_sentences]\n",
+    "train_labels"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Converting Words to Embeddings \n"
+   ]
+  }
+ ],
+ "metadata": {
+  "interpreter": {
+   "hash": "b7e818f66e33c31ac0526ee7f8556503ff93918b8b22809241939dc19e90de0b"
+  },
+  "kernelspec": {
+   "display_name": "Python 3.8.12 64-bit ('pytorch_m1': conda)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.13 | packaged by conda-forge | (default, Mar 25 2022, 06:04:14) \n[Clang 12.0.1 ]"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

code/cs229-scratchpad.ipynb

@@ -0,0 +1,812 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "ename": "ModuleNotFoundError",
+     "evalue": "No module named 'numpy'",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
+      "\u001b[1;32m/Users/johnnydevriese/projects/jupyter/cs229-scratchpad.ipynb Cell 1'\u001b[0m in \u001b[0;36m<cell line: 1>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/cs229-scratchpad.ipynb#ch0000000?line=0'>1</a>\u001b[0m \u001b[39mimport\u001b[39;00m \u001b[39mnumpy\u001b[39;00m \u001b[39mas\u001b[39;00m \u001b[39mnp\u001b[39;00m\n\u001b[1;32m      <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/cs229-scratchpad.ipynb#ch0000000?line=2'>3</a>\u001b[0m x \u001b[39m=\u001b[39m np\u001b[39m.\u001b[39marray([\u001b[39m1\u001b[39m, \u001b[39m2\u001b[39m, \u001b[39m3\u001b[39m])\n\u001b[1;32m      <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/cs229-scratchpad.ipynb#ch0000000?line=3'>4</a>\u001b[0m y \u001b[39m=\u001b[39m np\u001b[39m.\u001b[39marray([\u001b[39m4\u001b[39m, \u001b[39m5\u001b[39m, \u001b[39m6\u001b[39m])\n",
+      "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'numpy'"
+     ]
+    }
+   ],
+   "source": [
+    "import numpy as np\n",
+    "\n",
+    "x = np.array([1, 2, 3])\n",
+    "y = np.array([4, 5, 6])\n",
+    "\n",
+    "print(x.size)\n",
+    "k = 5\n",
+    "for k in range(k+1):\n",
+    "    print(k)\n",
+    "    foo = np.power(x[1], k)\n",
+    "\n",
+    "# todo: need to build a matrix \n",
+    "x[1] ** np.arange(k)\n",
+    "\n",
+    "\n",
+    "# polyn = np.array([1, np.power(x[0], )])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 45,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[[8.41470985e-01 1.00000000e+00 1.00000000e+00 1.00000000e+00\n",
+      "  1.00000000e+00 1.00000000e+00 1.00000000e+00]\n",
+      " [9.09297427e-01 1.00000000e+00 2.00000000e+00 4.00000000e+00\n",
+      "  8.00000000e+00 1.60000000e+01 3.20000000e+01]\n",
+      " [1.41120008e-01 1.00000000e+00 3.00000000e+00 9.00000000e+00\n",
+      "  2.70000000e+01 8.10000000e+01 2.43000000e+02]]\n",
+      "(3, 7)\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "array([   8.75909112,   15.        ,   32.        ,   78.        ,\n",
+       "        206.        ,  570.        , 1622.        ])"
+      ]
+     },
+     "execution_count": 45,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "X = np.array([])\n",
+    "\n",
+    "# Build out our matrix X \n",
+    "for feature in x:\n",
+    "    feature_vec = feature ** np.arange(k+1)\n",
+    "    # print(feature_vec)\n",
+    "    X = np.append(X, np.sin(feature))\n",
+    "    X = np.append(X, feature_vec)\n",
+    "\n",
+    "X = X.reshape(3, k+2)\n",
+    "print(X)\n",
+    "print(X.shape)\n",
+    "\n",
+    "X.T @ y"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# PS1: 3 Logistic Regression: Training Stability  "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 32,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[Text(0.5, 1.0, 'ds1_a.csv data')]"
+      ]
+     },
+     "execution_count": 32,
+     "metadata": {},
+     "output_type": "execute_result"
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABKMElEQVR4nO3dd3hc1bXw4d+eUe+9d0sucrflbjoEMGADIWB6bxcCuSlfCAnkhtzkhnCTCwQSSugl2BA6xvRujDs2trEtV0lW773N/v44Y0kjjWyVqZr1Po8ePHvOzKxj5LPm7LK20lojhBDCd5ncHYAQQgj3kkQghBA+ThKBEEL4OEkEQgjh4yQRCCGEj5NEIIQQPk4SgRBC+DhJBMKrKKWeVkr9t7vjcAel1AGl1KnujkOMPZIIxJiklLpVKbVBKdWulHra3fG4mlJKK6Vy3R2H8A5+7g5ACCc5DPw3cDoQ7OZYhPBockcgPJpSaqZSapNSqlEptQII6vNcnFLqbaVUnVKqRin1hVLKBKC1flVr/TpQPYzPira+X6VSqtb657SjHD9ZKfWB9bPLlVJ3WtvnWu9GGqztf7W2r1ZK3drvPb5VSp0/yPtfrpQ6qJSqVkr9ut9zc5VSX1vPvVQp9ZBSKsD63OfWw75VSjUppS4a7rkJ3yKJQHgs64XtdeA5IAZ4Gfhhn0N+BhQD8UAicCcwmuJZJuApIBPIAFqBhwaJLRz4EFgNpAC5wEfWpx8AHtBaRwDjgJXW9heBi/u8R771s96x8/75wD+Ay63vHwv0vXB3A/8JxAELgFOA/wDQWh9vPWa61jpMa71iOOcmfI8kAuHJ5gP+wP1a606t9SvA+j7PdwLJQKb1+S/0KKooaq2rtdb/1lq3aK0bgT8AJwxy+NlAmdb6L1rrNq11o9b6mz5x5Sql4rTWTVrrtdb214AZSqlM6+NLgVe11u123v8C4G2t9efW5+8CLH1i3ai1Xqu17tJaHwAePUqswz034WMkEQhPlgKU9Lu4H+zz5/uAQuB9pdQ+pdQdo/kwpVSIUupRa3dMA/A5EKWUMts5PB3YO8hbXQuMB75XSq1XSp0NYL0AvwMstx63HHhhkPdIAYqOPNBaN9Onm0spNd7avVNmjfWPGHcHjjg34WMkEQhPVgqkKqVUn7aMI3+wfgv/mdY6BzgH+KlS6pRRfN7PgAnAPGu3zpEuFmXn2CKMbp8BtNZ7tNYXAwnAvcArSqlQ69P/Ai5WSi3AGMT+ZJBYSjGSjRGAUiEY3UNH/AP4HsizxnrnIHGO5NyEj5FEIDzZ10AXcJtSys86qDr3yJNKqbOVUrnWRNGA0W/ebX3OTykVBJgBs1IqSCl1rFly4Rh953VKqRjgt0c59m0gSSn1E6VUoFIqXCk1z/rZlyml4rXWFqDOeny39b+rMPrp7wFWWI+x5xXgbKXUYutYyT3Y/nsNt55zk1JqInBzv9eXAzkjPDfhYyQRCI+lte4AzgeuAmqBi4BX+xyShzFg24SRNP6utf7U+txvMC58dwCXWf/8m2N85P0Y39KrgLUYA8E9lFKPKKUescbWCJyGcSdSBuwBTrIeegawXSnVhDFwvFxr3WZ9Xbv1HE7FGDwe7Ny3A7dYjym1nn9xn0N+DlwCNAKPAyv6vcV/Ac9YZxVdeKxzE75NyQ5lQgjh2+SOQAghfJwkAuFTlFJ3WhdZ9f95192xCeEu0jUkhBA+zutqDcXFxemsrCx3hyGEEF5l48aNVVrreHvPeV0iyMrKYsOGDe4OQwghvIpS6uBgz8kYgRBC+DhJBEII4eOclgiUUk8qpSqUUt8N8rxSSj2olCpUSm1VSs1yVixCCCEG58w7gqcxVlgO5kyMlaF5wA0YtVOEEEK4mNMSgdb6c6DmKIcsA57VhrUYlRCTnRWPEEII+9w5ayiVPmV2MeqopGLUVbGhlLoB466BjIyM/k8fU1VjO9sP11Pe2E5GdAhTUiMJC/K6CVNCCOEU7rwa2it/a3d1m9b6MeAxgIKCgmGtgGto7eTe977n5Q299bp+vWQS1yzOxmySCrxCCDuqC2Hn27D/M5iwBMafDlHD/xLqLdyZCIrpU28dYxu+w47+kD0VjTZJAOB/39/FyZMSGBcf5uiPE0J4u6YKeOVaKN1iPN77Mez/HM79BwSOzWuGO6ePvglcYZ09NB+o11oP6BYarca2rgFt7V0WWtoHtgshBLUHYcIZcMIvYc51YPKDnW9CzT53R+Y0TrsjUEr9CzgRiFNKFWNshOEPoLV+BGODjiUYWw22AFc7I46s2FAKMqM4fnwCHd0W/M0mthXXkhYd4oyPE0J4s9oD8MFv4JB1m+mYHCMhfPIHBum5HhOclgisW/Ud7XmNsfGGU6VEBXH6lGT+uGonWoNJwZ/On0ZksL+zP1oI4W32ftKbBMC4C6jdDwXXGUlhjBrzK4v3VjbzP9YkAGDRcPeb33GgutloqDsE+z6Dw1ugo8VtcQohPEDx+oFt5dth8U8gMNzl4bjKmE8EVU3tWPrd0bV1Wqhp7oDiDfDYCfDsUuO/n/0JWuvcEqcQwgPknDCwbeI5EJU+sH0MGfOJIDUqmCB/29OMDvEnKdQEq34BLX3WvH31AJRtHd0HVu+FrSth3WNQtA66Okb3fkII18k6DmZe1vt43Ckw9Yfui8dFxvyqquy4UP5+6Sx+tvJbals6iQ8P5G/LZ5LmVw+HNw18QX3JyD+seh88dx7UWau9KgWXrIS8H4z8PYUQrhORAmfeB/NuBkuXMS4QFOHuqJxuzCcCpRQnT0zk7R8fR01zO3FhgSRHBUObP6QvgKKvbV8wmkUjhzf1JgEAreGD30LaXAiOGvn7jpbFAqWb4cCXYPKHrMWQPM198QjhyQJCIGmKu6NwqTGfCI5IjQ4mNTq4tyEoAs78E6y4FOqLwWSGE38NSVNH/iH2xheaK6CrfeTv6QjF6+Dps4xvOAD+IXD1KkiZ6d64xNjVWm+sztXdEJsHIdHujkgchc8kArtSZsB1H0LtIWNGQGwe+I1iWmnydFAm0JbetjnXQ1jCqEMdMYsFvnm0NwkAdLbAzrckEYihaaowZtcFRRpdJSbz0Y+vPQTv/gJ2rzYeZyyEZQ9B7DjnxypGZMwPFh9TeDJkzIPE/NElATASyyUrIXEKhMbDCXfAzMuNsQJ30RbjH3J/TZWuj2UomiqgvdHdUYgjSr+FJ34A/zwFHlkEG56Ejuajv6bww94kAHBoDWx/zblxilHx7TsCRzP7Q95pxphAV5txJ+DOJABg9oO518PBL23bp5zvnngGU18C375oXGgi0uDkuyBr0bG/fQrnaa2Ht39qLKgCo4tz1c+NO9/0uYO/bt+nA9t2r4ZFPzF+H4XHkTsCZwiOhPBE9yeBI8adBOc9BvETIWkaXPQCpM9zd1S9tIaNT8HH/w0Nh40xjefPg9JRTuUVo9NcBSUbBrbXDroHuiHnxIFteadLEvBg8n/GFwRFwvSLYPwZYDJ53grJxlJj3UVfli5jRWeqjGO4TXAkxOYag759hScd/XW5p8K4U2Hvh8bjtLmedwcqbEgi8CXBke6OwD6/QAiOhrZ62/YAKQzoVqFxcM6D8OKF0NFktC249dgz66Iz4IInoHqPMUYVkwuhsc6PV4yYJALhfiGxcNrvYeXlvW1RmTKryRNkLYIbP4ea/RASA3Hjh1aTPzgK0uY4PbwxrbsLqncb42fhycbfvV+AUz5KEoHwDHk/gKvfNeo/hcYZYxgx2e6OSoAx7VOmfo5e+Q7Y+xE0lPZOKgkMtX+sxQI7XofXbjS6SU1mOOv/YMYlxqQUB1Nae1eN7YKCAr1hg50BLCGE8FSVu+CpM6GlurftgqcGHzupKoRHF0Nna2+byQ9u/MKY6j4CSqmNWusCe8/JrCEh+utsh7LvjGmQ1XvdHY0YC0o22SYBgI9/b1v0sq+WStskAMadQVO5U8KTriEh+upoNtYyfHC3MdAZGA4XvQg5x7s7MuGpGsqgfKtRYiZuvLGgtP9UWXtlZjpbjHEAe8JTICgK2up62/yDISLVQUHbkjsCIfqq2Anv/6a3TEh7I7x+o/GPXYj+Gkrh9ZvghR/Bq9fD4ycZ4wD9pUwHc7+B3oW3Q/gg5WeiM42uo5AY43FQJPzwCWM6rxPIHYEQfTUctt/WUgURx5g/L3xP2VbY90nvY22Bd34GqbOMMjNHJM+AK98y9jypLzJqkE048+jvnXsy3PC50R0UGgfRWc44A0ASgRC2Iu3sRBWV6d7CgcJz2as43FBidDH2TQRKQcZ8SJkFlk4IGGS2UH9R6S7ZHU26hoToK2EinH1/7218SCyc96gkAmFfXN7AUjL55xrz/u3xCxh6EnAhuSPwFZW7oPw7QBkrQ+Py3B2RZ/IPhllXQOZCY0ZHZNqY369WjELiFKN216qfG6VSJi2Dk35trJb3IpIIfEHpt/DMOb0lHEJijf7KxMnujctTmcwQP8HdUQhv4BcAE88yVlF3NBt3Av5B7o5q2KRryBdsft62jk9LtdSHF8KRwhKMlfBemARAEsHYZ7FAxY6B7ZW7XR+LEMIjSSIY60wmY5e0/nyxLHBXm5EAq/ZAd6e7oxHCY8gYgYfp6OpmZ2kj+6uaiQ0NID8lgtiwUQ48jTsFTv0dfPG/xp7KJ94B2T62Ura+GD77M2x+zvg7mPcfsPBWYwMhIXycJAIP8/72cn780maO1AJcOj2F3y3NJzp0FMkgLB4W3Q5TLwBMEJnikFi9yvfvwKZnjD9rC3z9ICRPg2k/cm9cQngA6RryIIfrWrnrje/oWxD2zW8Ps7PMAZu5K2VMhfTFJNDdBdteHti+613XxyKGpqPZKPjnpCJrwpYkAg/S3N5FbcvAvus6O21iGMx+kD5/YLtsg+mZKnfBiivgb7PgsRONhD1YcTbhEJIIPEhSZBBzsmJs2vxMiuw4z1uJ6HVmXGK72jM2z9jDWXiW9iZ4947e/Y4bDsOKS6Fiu3vjGuNkjMCDhAf584fzpvD7t3bwRWEVqVHB/OG8KYxP9LDN5r1RYj5c8x5Ufm8MFifkQ+QxSvpWFxozjPxDIGGSlJlwhcZS2PexbZul2/h/kTzdPTF5Aku38eONW1Uqpc4AHgDMwD+11n/q93wk8DyQYY3lf7XWTzkzJk83PjGcRy6fTUVjO+GBfsSFe9dSdY8WnWn8DEXJRnj2XGhvMB7nnATLHj528hCjExBmJNymCtv24Bj7x/uCovWw7lGoPQAF1xrbXIbGOfQjnNY1pJQyAw8DZwL5wMVKqf57rN0C7NBaTwdOBP6ilHJOyvMioYF+ZMeFen4S6OqA5mrjm8pY0tECH/+hNwmAUWq4eL37YvIVEclw1l9tC7lNWmrU9PFFpVvhmbONyQ7F6429D777t8M/xpl3BHOBQq31PgCl1EvAMqDvMlcNhCulFBAG1AAyKuQNSrfCl/9n/HJOOsf4phLnnE0zXK69Ecq+Hdhed9D1sfiivNPhuk+gZq9RFytxijEF2heVbjEWQvb15V9h8nkO7ap0ZiJIBYr6PC4G5vU75iHgTeAwEA5cpPWRraF6KaVuAG4AyMjIcEqwY1FdSwfr9tfw2e5KxieGcVxePDnxYaN/49pD8Pz50FxpPF77dyjfDhc9D0ERo39/dwuJNUoJr/+nbXvSNLeE43P8AowZXTKry9iwvj9zgDHO5UDOTATKTpvu9/h0YAtwMjAO+EAp9YXWusHmRVo/BjwGUFBQ0P89hB1aa/617hD3rt7V05YVG8IL180nNTp4dG9evbs3CRyx/zOjDzN5DFwszX4w72ao2W9sO+gXZJQWTpnt7siEr0mZBcHR0Frb23bSrx0+RuDMRFAM9C3knobxzb+vq4E/aa01UKiU2g9MBNY5MS6vYLFovi2u493vymjt6OKsaSnMyogiwM88pNcX1bbywEd7bNoOVLews6xh9InAz87rTX5eV4P9qOJy4cJnoO6QkQiis426TcJ5Kr43Em/dIWNANH0eBPr4jLmEiXDl28ZairpDkH8OZCxw+Mc4MxGsB/KUUtlACbAcuKTfMYeAU4AvlFKJwARgnxNj8hrfFtdx0aNr6eg2esqe/+YQz107j8W5Q/smYLFouroH3jx1dQ/oeRu++IlG/aK+m3QvvB2ic0b/3p4kMFz2bHCV6kJ4dhk0lRmPv3kElv0dZl7q3rg8QdIU48eJnJYItNZdSqlbgfcwpo8+qbXerpS6yfr8I8DvgaeVUtswupJ+qbWuclZM3uS97WU9SQBAa3jii33Mz47Bz3zsb6apUcFcMi+DZ7/uHeCMCPZzzJqE0FhY+iAcWmusAk2dZWzM4ec/+vcWvqn0294kcMRHvzPuDGT9htM5dR2B1noVsKpf2yN9/nwY+IEzY/BW7Z0Dv7m3d3VjGTDMYp+/n4mbThhHZmwI/95YwpTUCK5YkOWYwWIw6hZNvcAx7yWEvbLgna3eUVrC0m3M7ilaZ3SbZswzFiB6EVlZ7KHOmJrE018fsClAd/WiHALMQxsjAEiJCubaxTlcPDeDQD8zZpO98XshPEDiFGO/6M7W3raFtxnrCjzdobXw7FKwWJNWcDRc9Y5XdStKIvBQM9OjeeHaeTzx5X7aOru5elE2C8bFjui9QgLkf7PwcElT4Iq34Ou/GVVHC66BiWfbLizzRF3txrx+S587l9ZaKPzIfiKoLzHG1go/gsyFRtdXjPvH1uQK4aEC/EwszI1jXk4sWushjQsI4dXS50DKk9DdDgF2Ci1W7zW+fTdXGmNSqbOMuwh36u6ExrKB7f2nV4NRWvuD38J31pLoO16HbXPh4hch1L0L5iQReDijO8fDvxUJ4ShmP+Onv5oD8PwFUNtnUuGPnoHJ57oqMvsCw2DejfDmj23b804beGz13t4kcETxOmP7VEkEwtcdqGpma0k9re1dTEqJYHJKpIxnCFulW2yTAMD7v4HMRe4vPzFhCZzVCV89YEw5PulO446lv4FFE47e7kKSCPpo7exie0mDsV9wWCBTUyOJ9/TCb15uX2UTVzzxDcV1Rj0VP5Pi2WvmsnCI6yWEj+hsGdjWWgPdHa6Ppb/QOJhzrVGWxOQHwZH2j4sZZ9RR2vNeb1tCPsSNd0mYRyOJoI93tpby85e39jw+LT+RP50/dfSbx4tBbThQ25MEALosmr9+sJvp6VGEBsqvp7BKyAezv+0007k32G425G6hx5jMERQOZ/4Zso+DHW9AzsnGFOzwRNfEdxQyAmlVXNvCPW/tsGn7YEc53ztiv2AxqOrmgd/oSuvbaOscY6WtxegkTYPLXjPKK0Skwkm/gTnXeV/Zj5gsWPhjuPo9OPlOiHf/3QDIHUGP1o5uGtoGLl5paJP9gp1pVkbUgLbL5mfIXdhY0lJrVKdtroSYbOPb/XB32jKZjG/SKa8YZZkdXHTN5YaxHsgVJBFYJUcFsXBcLGv2Vve0BfqZGOeolbijVFrXypq9VazdV0NBVgyLcmNJiw5xd1ijNiM9ikcvm8X/rP6e2uZOrl6YxXkz09wdlmfqbDW2zmxvMIrgecNuaa31RqmIjdaNB5WCC54y6umPRGCY8TNSjeXG36HJbHwbDxnZ2pyxRmntXVWdCwoK9IYNG5zy3oUVTfzl/V28t72M3IQwfrd0MvNzYlFuXtTS3NbFr17bypvflva0nTA+ngeWzyAqZGxs6Fbb3EFHt4WE8EC3/317pNY6+Op+40drCEuEi1d4fs3+g2vgqTNt24Kj4cYvICrd/mucpXI3vHwlVFi7gLOOh2UPDX37UndqrTOqj/qHGHdVpuHfUSilNmqtC+w9J3cEfeQmhHH/RTOobGonLNDPpRfZxtZOzGZldxXw/upmmyQA8NnuSvZVNjMrc2wkgujQsXEeTlP6rbEj3BFN5bD6V3DZy72lmhtKobkCQhM8pzRDc/XAttZa221AXUFr+PbF3iQAcOBzYwvS2Ve5Npbhqvge3rgFSjYYpd5PvgtmXenQTaC8bKTF+QL9zaRFh7gsCdS1dPDKxiLO/8caLn38Gz75voL2fgOlg5WO7rK4f/6xcJH6ooFtxWuhpcb4877P4fGT4NHjjf/u+8y18Q0mJnvgLlvJ040BX1fqaod9nw5sP/i1a+MYrq52+OxeIwkcefz+b4wvBg4kicDNPv6+gp+/vJU9FU1sLqrj6qfXs7mozuaYrLhQCjKjbdomJoWTHecZ4xfCBSLtbNGasQhCYoxVtysvh0brXWNjqfG4Zr9LQ7QrYRIsfxHCk4zHqQXGPgPBUa6Nwz8Ixp85sD3nBNfGMVzNlbB79cD26kKHfox0DblRS0cXT3418B/rJ99XMD+ndxArKiSA+340ndc2F/PRzgqOy4vjgtnpstjNXeoOQdF6aCiG5BmQOnt0A5hDkTIdjvsFfPkXYyVqRAqc/gejW+jwFmirsz2+rR7qi41v5O5kMsP40+H6T42YwpMHX3DlbFN/ZIxZ7P/U+vhCyPbwRBAUCUnToajfnYuDu/4kEbiRn0kRbacLyl63VHZcKD89bQI3n5hLkJ9JBlTdpeEwrLwSDm/qbTvnQZh9pXM/NygSTvi5UVunvQGis4xkAMZUyv6LrUx+njUjJiLZ/eMWsTlw4bNQs9f4+4nNgQAPv6sODIfTfw/Pnd87rpJ/LiQ7dpKAJAI3CvAzc/OJ4/iqsAqLdfJWWKAfJ4wffI50sP/gswVqmtv5Zn8NH++sYEJSOCdOiCc3wcf3fHW0sm22SQDgg7uNrTujnDzt1S/I/paFsblwxp9h1U+NQVGl4Ix7IS7PufF4o+BIo2qpN0mbAzd8ahStCwwztooNiXHoR0gicLO5WTG8ctMCvtpbTUiAmQXj4shPHv5sAK01L60v4s+rd/W0Pb3mAP+6fj7pMd6/3qCv6qZ29lU1o4Bx8WGunXHUYafmTXuDUTrZXcz+MOMS4wLXUGIMxMZPNNrF2BA7zvhxEkkEbuZnNjErM4ZZmaPL8CW1rfztI9sBpOLaVr4vaxxdIuhqN6Yl+gd7RE2U/VXN/GTFFr61DqjPy47hzxdMIzPWTv16Z0iYMHAnrak/MrbudCf/IEiZYfwIMUwya2iM6Nba7nTS7tFMMa3eB2/cCn+bCY8dbxTK6nTjN19g1dbDPUkAMLrCvq9wXQAJ+XD565B1nLGoa+FtcOKdxvzusaK1AQ58BdteMfbh7Wh2d0TCySQRjBEpUcFcscB2hWRksD8TEkc4RtDVaSxg2rbSmKXSWGasyixz7Pzl4dBa88mugTs/fVVY5dpAMubDJSvgxi/h1N8ZhcTGio5mWHM/PL0E/n0tPHEabH7eOzaRFyMmXUNjhL/ZxPXHjSMjJpR/bypmckokl83PIHuEtZI6Gw7jv22FbaPWULUb0uc6IOLhU0pxWn4iGw7W2rSfMCHB9cEEhNrfTtHbVe2GL/5i2/b+byDnRIif4JaQhPNJIhhDkiKDuHJhFsvnpONvNmEa4S5fZfWtrN3ZwDkRGZhr9tg+GRxt/0UucsaUJL4srOKLPcZdwGn5CZw0wc07VI0VTRXQ1TFwDKS7w1gDIHppDZZOMI+N0iiSCMagwKNMMR2Kr/dW86vVxUw64zdM+OgasFhLXmQsMBZQuVFmbCgPXzKL/VVNKKXIiQslLEhmx4xKY7nRBfj1wxAUZWy1uHWlMVUWjIFweyubfdXhLbDhSSj/DmZdYaxY9oCJFKMhiUAM8PW+ato6LVz/ZRj/ddwKMixF6MAIsibPx99JpY/3VjbxVWEVB6tbWJwXx+yMaCKC7V/gI4L9mZ7u3juTMWXHG0b3DxjlKd6/y1iHsPqXxoYwSx+EiCT3xugpKnfBM+f0Lu4q2QjHl8CJv/K+TXL6kEQgBpiZHs3KDcUcqm3nmvcAErnx+BzuiHJOEjhU3cKVT66juNbojnjiy/3897lTuGy+F5QH9natdbDuETvttfDjzcbCJVfXBfJk5dsHVk79+m/GnYGry2o7kPemMOE0i/PiWDCutzzBuPgwfjQ7zWllLXaUNvQkgSP+/N73lNa3DvIK4TDmAAhLGdgelmCUYBhpEqjYCZueg3WPQ8kmGCuVcu3tA2DyN1ZzezG5IxADpMeE8PdLZlFY0USXxcK4+DASIoKc9nkdXQMvEu2dFrq7vWvTJK8UEAIn/j94bk3vWFBYAmQtHvl7lm+Hp88y7irAWOF8xVuQuWD08bpb4lSjcF5jn/1BTvyV+xcUjpIkAmFXdGgAc7IdW89kMBOSwgkJMNPS0bsPw5ULs0iOCnbJ5/u8zEVwzQdweCP4h0F6AcSNYlP1wg97kwAYxfDWPGhUaR3uXsWeJjYHLn8Ndq2Gqu9h4tmQOYqk6SEkEQi3m5AUzovXz+Oxz/dRWNHERXPSWTI1GfMIp7+KYTKZIW228eMIjeUD2+qLwdIFeHkiAGOPhYRJ7o7CoSQRCI8wIz2aBy6aSXtXt0wH9XbjfwBrH7Ztm3ej0Q0lPJIMFguP4e9nkiQwFqTNhQufM7qXItNgyX0w/gx3RyWOwql3BEqpM4AHADPwT631n+wccyJwP+APVGmtPXzLICHEUQWEQP5SyD7e6A4KHXx/DadpKDVKcgdGAAp0t7Fb21gqDuhATksESikz8DBwGlAMrFdKvam13tHnmCjg78AZWutDSik3FI0Zu7q6LdS3dhIe5EeA3+hWGwsxbO5af1C0Hl6+wthNzj8EFt0O2183igWe8P96d3YTPZzZNTQXKNRa79NadwAvAcv6HXMJ8KrW+hCA1tqF9YRdp6Orm23F9byz9TDf7KumvrXD6Z+5t7KJ3721nXP+9iW/eGUrO0sbjv0iIbxdUyW8er2RBAA6W+CzP8GU82HjU1D4kXvj81DO7BpKBYr6PC4G5vU7Zjzgr5T6FAgHHtBaP9v/jZRSNwA3AGRkeF/Nk9XflXH7ii1o67T4qxZm8bPTxhM+SAmF0apr6eBnK79li7Vu//r9New43MCz18yVKZlibGsqh9r9tm1a9+4gt+MNmHW56+PycM68I7A396//CiE/YDZwFnA6cJdSasAEZq31Y1rrAq11QXy8d1WaLKpp4Tevf9eTBMDYQnJXRaPTPvNgTQtbiuqYlRHNL8+YwMLcOI4fH8/BajvbLAoxloTEGAvi+jNbxwYy5rs2Hi/hzDuCYqBv8Y004LCdY6q01s1As1Lqc2A6sNuJcblUQ1snDW0DN/WoaXJe91Cgn4nIYD9OnBDPvX32MH5zy2FW3DifnBHuUSCEx4tIgWX/gBWXQleb0TbnOmORW2weTFrq3vg8lDMTwXogTymVDZQAyzHGBPp6A3hIKeWHsdJkHvB/TozJ5ZIjgxmfGMbu8qaeNn+zcuoeu9lxofz6rHwe+NB2L4HKpna2ldRLIhBjW+4pcNOXUHsAAsON3dUmLDEWgclAsV1OSwRa6y6l1K3AexjTR5/UWm9XSt1kff4RrfVOpdRqYCtgwZhi+p2zYnKHmNAA/u+iGdzx761sK2kgITyQe384jbwE512MA/3MzM2KpqVj4J2Ivbo+QowpSkFcnvEjhkRp7V2FvQoKCvSGDRvcHcaw1bd0UN7YTmSwP4lOLODW1z8+LbTpGgr0M/HqfyxkckqkSz5fCOE5lFIbtdYF9p6TEhMuEhkSQGSIa+us/HB2GmGB/jy/9iDpMcHcdMI48pMjXBqDOIbag7D7Pdj7idGlkXcaRMs+DMK15I7ABzS3d+JnNhEoi8o8S2stvHw17Pukty33NPjhExAsd23CsY52RyC1hnxAaKC/TySBysY2Khra3B3G0FXtsU0CAIUfQPUe+8cL4STSNSQGOFTTwo7DDXRbLExICic3IdzdIR1VfWsHq7aW8dcPd2OxaG49OZdlM1KICfXwujKD3Y1rGdB3u6ZK6GiCsCQIGPuLMCURCBuFFY1c+eQ6SuqMb9bhgX68cP08pqVFuTewo1i7r4Zfvbat5/Hv3tpBTGgAy2Y4Z49lh4nLhfT5ULS2ty1zEcTmui8mX2fphr0fwzs/g/pDMOEsOPW3o9uoxwtI15Cw8dmuyp4kANDY3sUzaw7QbfHcsaS3tvRfpwj/WncIiwfHDEBILJz7Dzjlv4wVr6f+Fyx7yFgdK9yjYgf86yKoO2jcsX3/Nqy+Ezqa3R2ZU8kdgbCxt3LgL/z3ZY10dFsItrdxtwfIihu4OC8nLgyTN+xwFpsDx/2nUSHTJN/L3K50W+/ezUcUfmAUsRvD6xLkN0/YOGniwFpOFxakE+zvmUkAYMnUJCKCe7/ThAaYWT4n/Siv8ECSBNzv8GZoLBnYHhoPAc6rBOAJ5I5A2JibFcvdZ+fz1w9209lt4epFWZwxOdHdYR1Vfkok/75pIdsP19OtYUpKBBOSZL2EGKYdb0B9EWQdBwe+MNqUMnZYG+OlKSQRCBuRIf5cszibM6Yk0a01KZHBXrGJfF5iOHmJnj27SXi45irY9gpMuxBOuhO6OyAsGfJOd3dkTieJQNiVIvsWCF8z+XzY/BxsXdnb9qNnjK03x7gRdUwqpU5zdCBCCOFWGfPhohcgaRrET4LzH4eck9wdlUuM9I7gCcD7tgoTQojBBITApLMh5wRj6miQ74wzDZoIlFJvDvYUEOuccIQQws0CfW+s6Wh3BMcBlwFN/doVxsb0Qgjhuap2G6uEq/dB7snGKu7gKHdH5ZGOlgjWAi1a68/6P6GU2mXneCGE8Ay1B+D5C4wVwgDrHoUz/wzzbnRrWJ5q0MFirfWZWutPlFL5dp6+24kxCSHE6JRt600CR3zyB6gvdk88Hm4og8UrlVLPAX8Ggqz/LQAWODMwIYajq9vClqI6Pt1VSYCfiRMnxDM1NRKlPH8NhHCC7s6BbZ2tA8tHCGBoiWAecC+wBggHXgAWOTMob9DZZWFfVTPVze2kRgU7dTN6cWwbDtZyyeNrOVJn7uFPCll54wKmp0e5NS7hJgn5xqBve2Nv27ybIDLNfTF5sKEkgk6gFQjGuCPYr7VvF0xv7exi5foi7nl7J90WTXigH49cPptFuXEui6G5vYv9Vc2AJjM2lPAgf5d9tqfptmie+mo/fYuNtndZWP1dmSQCX5UwEa54A77+O1TuhJmXQ/5S8NDCie42lESwHngDmIMxbfRRpdQFWusLnBqZB9tT3sRv39zR87ixvYufrtzCm7csJjHS+RvTH65r5Y+rdvL21lIATp2UwG/PySc9xjfvSrTWNLQO7Aqob7PTPSC8W9k22PWu0dc/6RxjEdhg0z1TZ8N5j0JXGwSGuTZOLzOUlcXXaq3v1lp3aq3LtNbLMBKDzyqrH7gdYnlDO9XNHS75/M92VfYkAYAPd1bw/vbyY75Oa83+qia+3lvFnvJGurrHxo2dn9nEVYuyB7SfPTXZDdEIp6nYCU+fbQz6bnoGXrgAdq8++mvMfpIEhuCYiUBrPWCneK31c84Jxzsk2fnWnxwZRFyYP6V1rXa/nTrSJ7sqBrSt3l52zI1YPt1VyVkPfsnFj3/DWQ9+yWubS+jo8p5k0NDaycHqZupbBybcReNi+fuls5iRHsm87BieumoOszOj3RClcJqSTdBWZ9v28R+gudot4YwlUnRuBCYkhvOHc6fwX29tp7NbExXiz4PLZ/CX93fz1tZSxsWH8puz8pmbHeOUWSvzcmJ5f4ftHcBxefFH3YilqKaF21dspqXDmDXR0W3hjle3MTU1konJnr+UfmtxHXe/sZ0tRXVMSYng9+dOYWZG74U+LMifJVOTOWliAiYg0IP3TxAjZG8mUHc7aJkJNFqyG8YIBPqbuWhOOqtuO44VN87nrVsX8fgX+1mxoZiWjm62lTRwxZPr2F3ef1G2Y5w6MYEZfQZBJyWFc/a0o3eDVDW109DaZdPWbdGUNQzs5vI0pfWtXP/sBrYU1QHw3eEGrntmA8W1LQOODfY3SxIYq1Jngl+gbdvin0JYgnviGUPkjmCE/Mymnvr3eyubBnxDb++ysK+yiQlJjq9bkhkXyhNXFlBY0YRFa3ITwogPP/ogdXx4IJHB/tT36bYym5Tdbi5PU1zTSnlDu01bdXMHRTWtpEWP/RLBwippGlz5Nqx9xNhYvuBayJNCyI4gicABgvzMRAT7DfjGHRrovL/e2LBAYsMCj32gVVp0CA8sn8GtL26mqb2LQD8TfzxvKuPiPX8gLTzID5PCZnqoUthsTyl8gFKQPteYDWTpBr8Ad0c0Zsi/JAdIjQ7mrrPy+cUrW3vaFo6LZaIT7gZG48QJCbzz48WUNrQRGxpATnyYV+w+lhMfyk9PG8//vr+7p+2WE3MZF++b02V9nsks6wEcTBKBg5w1LZnM2BD2VjQTGxbA1LRIEiI8r9slMy6UzDjvuoAG+Jm5amEWc7JiKKlrJSUymPyUcIL85ddXCEeQf0kOEhLgx9zsWOZmy1YNzhAW5M+8HPm7FcIZJBEIIQaoaGijvrWThIhAIoOlL36sk0QghOhhsWi+LKzi/72ylbKGNqamRvA/509jSmqku0MTTuTUdQRKqTOUUruUUoVKqTuOctwcpVS3Uspn6xcJ4Qn2VjZx/bMbetaXbCtp4CcrNlPT3H6MV7pRdSFsfAY++R/Y9xm0N7s7Iq/jtDsCpZQZeBg4DSgG1iul3tRa77Bz3L3Ae86KRQgxNAerm2nvV3aksKKZw3VtxIQOfbqyy9Tsh+d/aOxIdsS5j8CMi90Wkjdy5h3BXKBQa71Pa90BvAQss3Pcj4F/AwML6Ajh4Q5WN7O1uI7KRs9foT0UMaEDxwPCAv2ICPLQXuSyrbZJAODDu6Hx2EUYRS9nJoJUoKjP42JrWw+lVCpwHvDI0d5IKXWDUmqDUmpDZWWlwwMVYrg6uiy8uaWEsx78kqUPfcX5/1jDlqJad4c1auMTw7l6UZZN2z3LJpPhqRsvddpJwG0N9usSiUE5M83bW6nUvzzm/cAvtdbdRyvOprV+DHgMoKCg4OglNsWgqpra2XSwlm0l9UxMCmd2ZoxXlJjwRIUVjfxkxZae1c5FNa38dMUWXrlpITHDWPHtacKC/PnJqXmcMTmJyqZ2MmJCmJDoWQsjbSTmG/WHuvqMYcy7ESJS3BeTF3JmIigG0vs8TgMO9zumAHjJmgTigCVKqS6t9etOjMujlDe08fnuSt7ZWsrMjCjOmppMrhP+4bV1dvG3j/fwzJreDb3PmJzEvRdMIzLYd3c3G6mimlb6V/3eV9VCeWO7VycCgMjgAO9Zs5E4BS5/Az6/D2r3w6wrYeqPwCT1NIfDmYlgPZCnlMoGSoDlwCV9D9Ba9+wmopR6Gnjbl5JAZ5eFRz/by5NfHQDg092VvLyxmBU3LCA1Otihn7W/qplnvz5o07Z6exnXH58jdftHID58YF96XFiAJFVXUwoyF8DyF4xuohD5XR4Jp6VNrXUXcCvGbKCdwEqt9Xal1E1KqZuc9bnepKi2ZcDFubi2ld3ljYO8YuTauyxoO51q7Z1Sy30kxidF8OOTc3seB5hN3PvDaaREOTaBiyHyD5YkMApOnQqgtV4FrOrXZndgWGt9lTNj8VT2Bjy0vSv2KGXGhjI9LYpvi+t62tKig8mWwm0jEhbox80njOOUSQnUNHWQHhPiFZVchbDHQ+eE+Ya06GAun5fJ018f6GlLjQpivBOqlkaHBPCXC6fz7JoDfLyrgvnZMVx3XA7JkfINdqRCAv2YkS7fQoX3U8749ulMBQUFesOGAdsoe63y+lY+3V3JG1sOMzszmqXTU3o2vHGGrm4LDW1dhAf64e/n2J7Brm4LZQ1tBPiZSDjGRjlCCNdSSm3UWhfYfU4SgXCEktpWnvpqP89+fZCIYD9+fdYkFufGsaWojre+PUxmbChnTk0iP1lq1gjhDkdLBNI1JBzilY1F/PPL/QBUNXXw+7d3cvspefz2ze09xzz79UFeuWmBU+94hBDDJ5NtxahVN7WzYn2RTdsZk5N46JNCm7b61k6+O1zvytCEEEMgiUCMWnCAmcxY203kTSZFd/8VV4DFMqBJCOFmkgjEqIUE+PGfp40nwNz767SrtIGbT8jpd5yZyakRrg5P9FNW38Z735Xy1Ff7+XJPJY2tUpfH18kYgXCIOVkxvH7LQnaVNxHsb2JySiShgWbiwwP517oisuNCuXhuBhOTJBG4U3VTO796dSuf7Oot3vjrJZO4dnE2JtPg9b7E2CaJoJ/SulYqG9uJCw+UVaLDoJQiPyWS/BTbWUHnzkzjnOmpmOUiMyQHqpo5VNNCdEgAuQmhBAc49p/o7vJGmyQA8JcPdnFafiJZcbK40FdJIujji92V/GTFFqqbO4gLC+D+i2awOC/e3WF5PUkCQ7N2bzXXPbuBpvYuAG47JZcbjsshLMhx9Yua2weWFGnrtNAqpUZ8mowRWB2saubmFzZR3dwBGFMgb35+EwerZdu7vpraOunokouGo1U3tfPLV7f2JAGABz8q5Psyx9adGhcfSnig7fe/heNiSXdwkUPhXSQRWB2ub7X5RwjQ2N5Faf3Y2HlqtMobWnl6zQHO/8cabn1xM5sO1rg7pDGltqWTg9UtA9od/fuXHR/Gs9fOZcG4GCKC/bioIJ3fL5sytLuOtgao3gct3r8Bj7AlXUNWsWGB+JsVnd29Ux4DzCZi7Wzd52u01rz4TREPfLQHgN3lTXy2u5LXb1nEpGQZ/HWE2LAAJiaFD7gDcMY39ZkZ0fzzyjk0tXURExIwtFIjh7fAqp9D8XpjD4Cz/wrp8xwem3APuSOwyo4L5Z6lUzjSnW1S8Ptlk8mWATTK6tt4/It9Nm3tXRa+L2twU0RjT3RIAPf+cCrJ1h3jAv1M3LNsMhOTnbMKOzTAj8SIoKElgcYyWHGZkQQAyr+DFy+CmgNOiU24ntwRWPmbTZw/K5VpaZGUNbSRFBlEXkIYfmbJlWazIiTATEuH7dhAgPzdONT09Ghev2URJbWtRAT7kx0X6hkD7XWHoN525TittcaOYDFZbglJOJYkgj4C/c1MTo1kcqoURusrITyIX54xkV+8srWnLSkykMkp8vfkaIkRQSRGeFjl1sBwMPmBxXYMjaAot4QjHE8SgRiSM6cmkxQZxOe7K0mLDmFRbqzMO3eAioY29lY2YTaZyE0II8YTx6Ric+GU38IHd/W2Lbod4ie4LybhUJIIxJCEBfpxXF48x8m6CocprGjkxuc2srfSmKI8PyeG+y6YTnpMyDFe6WJmfyi4GtLmGN1EESmQNBUCPCxOMWLSySuEG2itWbmhuCcJAKzdV8OXe6rcGNVRBIYbm8RPvwiyj4PgKHdHJBxIEoEQbtDeZeGrwoEX/U2HZI6+cD1JBEK4QZC/mR/kJw5oXzgu1g3RCF8niUAINzl3RiqLc+N6Hl8wO40FkgiEG8hgsRBukhkXyj8uncX+6mbMJkVOXBjBAWZ3hyV8kCQCIdwoPNifaWlR7g5D+DhJBEIIx+psg5KNcHANhMRA1mJZc+DhJBEIIRxr78fw0sW9j8MS4Mp3IH68+2ISRyWDxUIIx2mtg49+Z9vWVGHcIQiPJYlACOE43R3QVjewvaPJ5aGIoZNEIIalpaOLb4vq+GBHGTtLG+jstrg7JOFJwhJg/i22bSYzpMxyTzxiSGSMQAxZa0cXT321n/ve2w0YexHff9EMzpme4ubIhEeZdpFRn2jdYxCeDCf8ElJmuDuqo+rs7KS4uJi2Nu/fkTAoKIi0tDT8/Ye+17UkAjFkeyqaepIAQLdF86tXtzEtLZLMWKlEKqzCE2H+zTBtOfgFQIDn/24UFxcTHh5OVlYWSnnAHhAjpLWmurqa4uJisrOzh/w66RoSQ1bV1D6gram9i7qWTjdEIzxeSLRXJAGAtrY2YmNjvToJACiliI2NHfadjVMTgVLqDKXULqVUoVLqDjvPX6qU2mr9WaOUmu7MeMTopEWHDNiVLDkyqGd7RSG8mbcngSNGch5OSwRKKTPwMHAmkA9crJTK73fYfuAErfU04PfAY86KR4zeuPgw/n7pLKJDjL7HtOhgHrpkFgmetqOWEGJYnDlGMBco1FrvA1BKvQQsA3YcOUBrvabP8WuBNCfGI0bJbFKcmp/I2z8+jtqWDhIiAkkIlyQghLdzZiJIBfrueF0MzDvK8dcC79p7Qil1A3ADQEZGhqPi8xhtnd3sKW+kpK6V5MhgJiSFEeTvueP4qdHBpEYHuzsMIYamuQqayiE4BiKS3R2NR3Lm1cZeR5W2e6BSJ2EkgsX2ntdaP4a126igoMDue3irzm4LL28s4q7Xt/e03X12PpfNzyDATypRCjEqxRvgtRuhutCYyrrsYRh3MrhwPOCuu+4iLi6O22+/HYBf//rXJCYmctttt7kshmNx5mBxMZDe53EacLj/QUqpacA/gWVa62onxuORDlQ1c89bO2za/rBqJ/uqmgd5hfB1Xd0Wmtu73B2G52sohZWXG0kAoLEUVlza+3i0LBboaIG2BugafJbOtddeyzPPPGN9iYWXXnqJSy+91DExOIgz7wjWA3lKqWygBFgOXNL3AKVUBvAqcLnWevfAtxj7alo66Oy2vcnptmhqmjvcFJHwZN+V1PP0mgN8V1LP+bNSOXtaCilRY7ebrr2zm0M1LZiUIiMmBH+/YXx3bSiBhn7fPTtbofYAxOWNLjBLNzRXGskFQJkgJsfY27mfrKwsYmNj2bx5M+Xl5cycOZPYWM/agMhpiUBr3aWUuhV4DzADT2qttyulbrI+/whwNxAL/N065alLa13grJg8UUpkMFEh/jZz8cMD/Ugbw/+4xcjsr2zikn+upaHVuBv446rvKapp4T9PG8+hmhYC/czkxIUS6D82uhQP17Xyfx/s5pVNxZiU4soFmdx0wrihz1ILjgL/EOhssW0PjbN7+LB0tfUmAQBtgbqDEDfBWFXdz3XXXcfTTz9NWVkZ11xzzeg/38Gcuo5Aa71Kaz1eaz1Oa/0Ha9sj1iSA1vo6rXW01nqG9cenkgBAekwIj142mxTrXPykiCAevWI2GbJSV/Szq7yxJwkc8eK6Il7bVMK5D69hyYNf8Md3d9pd+OeN3t9Rxssbi9HauEt+8qsDrNlbNfQ3iM6BJf9rOx5w4p3GxXq0uu0souzuBIv9LrvzzjuP1atXs379ek4//fTRf76Dee7UFB8yLyeW129ZRHVzB7GhATIvX9jlbx74vS3AbKKi0bjwaw3PrDnIonFx/GBykqvDc6iubgtvbhkwpMgHOyo4d+YQZ5mbTJC5GJb/C+qKICodUmZCQMjoA7TzrR9zIJjsX1IDAgI46aSTiIqKwmz2vDs2SQQeIiEiSBKAOKqJyeGMiwtlb5+JBJfMy+Dd78psjttWUu/1icDPbKIgK4ZNh+ps2mekRw39TRrL4dXroHhdb9vJd8Hi/zQqoo4qwGCITIf6YkCDyR+iM+0nCIxB4rVr1/Lyyy+P7nOdRBKBEC5SVm/MLEkaYUmO1KgQHr+ygC8Lq9hT3sRxeXHsrWziUI1tH/jEpIEDlv21d3ZT09xBRIg/oQGeeRm4YHYab317mFLr31tufBinTEoY+htU7LBNAgCf/xkmnwex40YXnMkEIbEQEAa6C8wBxo8dO3bs4Oyzz+a8884jL2+Ug9RO4pm/AUKMITXN7by2qYQHPt4DwE9OyePcmanEhAYO+71y4sPIiQ/refx9aQMvfHOI4tpWAM6YnMTszJijvseuskbu/3A3n++uZEZ6FL88cyLT0qKGHYuzjU8M5+WbFrCnvAmTSTE+IYzk4Uyi6LQzpbOr3X7//kgoBf7HTur5+fns27fPMZ/pJJIIhHCyLwur+f07O3se3/P2TuLDgxyyj8PE5AhevmkB+yubCfQ3k5sQRmTw4HXoa5rbuf2lzXxf1gjAV3urueqp9bx56yLSoh3Qd+5gadEhI48rfgIER0NrbW/bpKUQNfaqE4yWJAIhnOzVjcUD2zaVOGxDn+TIYJIjh/ZN+VBNa08SOKKmuYP9Vc0emQhGJTYHLn8dvvw/KN0CU34IMy93zGDxGCOJQAgny0sM49PdlTZtuQlhgxztXCEBZvxMii6L7SLG0EDXXwpK61s5UNVCSKCZcXFhhAU5IYaUGXD+Y9DRbNwdjJFS044mG9MI4WTnzkwlIrj3IhcZ7M+5M92zvWd2bCi3npxr0/bDWWnkuTgx7Thcz/l/X8PFj69l2UNfcc/bO6hqdNL6B79ACImRJHAUckfgBepbOlBKEXGUvl/huSanRPLqzQvZfrgBpRT5yeHkJhx7Zo8z+PuZuHpRFgWZ0RyobiElKphpaZGEB7nud6uts4v7P9zTMxsIYOWGIk6fnMgpkxJdFocnWr16Nbfffjvd3d1cd9113HGH7X5eWmtuv/12Vq1aRUhICE8//TSzZs0a9edKIvBgDa0dfLizgoc/KcRsUtx2Sh4nTUhwy228GJ3cBPdd/PuLDA5gcV48i900k7G+pYtv9tcMaD9Q7T2FFl/fXMJ97+3icF0rKVHB/OL0CZw7M3VU79nd3c0tt9zCBx98QFpaGnPmzGHp0qXk5/fu5/Xuu++yZ88e9uzZwzfffMPNN9/MN998M9rTka4hT/bV3mp+uvJb9lY2s7u8iVtf3My6AwP/AQnhTaJC/Tkub2C9n5w494ybDNfrm0v41avbKKlrRQMlda386tVtvL65ZFTvu27dOnJzc8nJySEgIIDly5fzxhtv2BzzxhtvcMUVV6CUYv78+dTV1VFaWjrIOw6dJAIPZbFoXlh7aED7aH/ZhHC3QD8zPz45j3HxRj0tpeC6xdlMT490c2RDc997u2jt7LZpa+3s5r73do3qfUtKSkhP763cn5aWRklJybCPGQnpY/BQJpMiMWLggqNEKUPhcO1d3RTXtmJSkBETitkkg4rONiEpnJU3LuBAdQshAWay40IJ8pKqqYfrWofVPlRaD9xzq/9G9EM5ZiQkEXiwS+dl8vbWUtq7LAAE+5s5e5pstedIJbUtPPjRHl7eWIyfycTNJ47jioWZxI5g1a8YntiwQGLDvO/vOSUqmBI7F/3R7guRlpZGUVHv7r7FxcWkpKQM+5iRkK4hDzYzI4pXblrIb8/J53dLJ/PmrYuYkOgZA45jxTvbSlmxoRiLho5uCw98tId1dgYyhTjiF6dPILjf3Uuwv5lfnD668tZz5sxhz5497N+/n46ODl566SWWLl1qc8zSpUt59tln0Vqzdu1aIiMjSU4e/ZdDuSPwYEoppqZFMjUtkm3FdTz6+T52lTVy0Zx0TstPlG6iUWrp6OL1zQNLHX+2q5Izp8idl7DvyOwgR88a8vPz46GHHuL000+nu7uba665hsmTJ/PII48AcNNNN7FkyRJWrVpFbm4uISEhPPXUU6M+H5BE4BX2lDdy8ePf0GTdp3ZbST1lDW389NTxmKQ/e8QC/cxMT49iR2mDTXt+SoTN4/qWDhrauogLCyQ4wDv6sYVznTszddQXfnuWLFnCkiVLbNpuuummnj8rpXj44Ycd/rnSNeQFdpU19iSBIx7/fB+H60c3OOXrzCbF5fMziA3tLR88PjHMZmrjuv3VXPTYWo6/7xNueWEju/rV6RFiLJA7Ai9gNg/81h9gNsnsFgfIT4nk1f9YyJ7yRvzMJiYkhveUOt5b0cSVT67vmSr48a5KKpvaee7aeUSF2K89L4Q3kkTgBfKTI0iKDKSsvrcWy89+MH7IFSfF0WXGhpJpZ4/oA9XNA+aLbytpoKS2tScRdHRZ2FxUy8r1RVg0XFiQzuzMKAL8pAtJeA9JBF4gMzaU566Zxye7Ktlb2cipExOZmx3r7rDGPHu1nYL9zTYlPjYfqmX542s5Mr379S0lvHjdPBaMG7hyVghP5ZOJoKOrmy6LJsRDt+izJy8xnDyZOupSeQlhnD8zlVf7rOa+c8lEMmN769m/srGYvmt8tIbn1x6SRCC8ivdcCR2gs9vCuv01/OPTvdS3dnDt4hxOnBAv/b3CrqiQAH591iSWzUilsqmNzNhQJqdE2KzktNhZ6Wlv9acQnsynZg19W1TH5U98w5eFVWwraeAnK7bwya7KY79Q+KzYsEBOmBDPBbPTmZMVM+Au8vxZaQPK3F+2INOFEXqn6qZ23t9exn2rv+f1zSWjLs8wFlxzzTUkJCQwZcoUu89rrbntttvIzc1l2rRpbNq0yWGf7VN3BF/sqaLfxkw8/vlefpCfQGig1PoXQ3eoupl/bypmS1Ed910wnc92VaCBS+ZmMDsj2t3hebT2rm4e+Wwfj3/Ru6H7cXlxPLB8JjGhXnJ3vnUlfHQP1BdDZBqccjdMu3BUb3nVVVdx6623csUVV9h93lklqMHHEkFo4MCZHKGBfpiUT90YiVFq6ejij6u+Z/X2MgA+213FonGx/PXC6STKTK5jOljdwhNf7rNp+2JPFXvKG5mX4wWTILauhLdug07rXUx9kfEYRpUMjj/+eA4cODDo84OVoHZEiQmfugIuzo0jtM/KUKXg1pNyZbWol9lX2cRL6w5x/4e7WbO3ipaOrmO/yIEO1bT0JIEjvtpbzY5S31tsVt7QSnnD8Lp1OrosA+7MAdq7LQ6Kysk+uqc3CRzR2Wq0O5GzSlCDj90R5KdEsvLGBXy6u5LGtk5OmpDAjPQod4flldo6uzEpXD5f/mB1M1c8uY7i2t5/iA8sn8GyGY5f7j8Yf5MJf7Ois9v2ahbg5zvfq2qbO3htcwkPfrwHgJ+ckseyGalED6FrJyMmhAU5MXy9r7e4X2JEIOO8ZGMa6ouH1+4gzipBDT6WCAAmp0YyOdU7NsDwRA2tHXy2u5InvtxPeJA/N50wjrnZMfibXXMR3F7SYJMEAP64aieLxsURF+6aksYZsSHceHwOD32yt6dtRnoU4xO95ELmAF8VVnHP2zt6Hv/XWzuICwvk7OnHLokcEezPH8+bygvrDvHe9jLmZMVw3eIcUqO9pFstMs3oDrLX7kTOKkENPpgIxOh8uquS217a0vP4q8IqVt64gIKsGJd8fmtX94C2xrYuOl3YreBvNnH14mympkXxzb4aJiaHsyAnlvhw36kG+8rGgd9+/725ZEiJACA7Pow7z5zELSflEhbgh7833U2dcrftGAGAf7DR7kRLly7loYceYvny5XzzzTcOK0ENkgjEMLR0dPHYF7aDfBYNn+yqcFkimJgUToDZREefC/81i7JcXpI7NjSQ0ycncfrkJJd+rqfITQjj0922U69z4weW6Tgak0kR7Y1reI4MCDt41tDFF1/Mp59+SlVVFWlpafzud7+js7MTcG4JapBEIIbBrBThdqbZhrpwhXZ+cgQvXDePBz7azaGaVi6em865M1OlHLeLnTszlZUbimhoMwbqI4L9WOaEsswea9qFo77w9/evf/3rqM87qwQ1ODkRKKXOAB4AzMA/tdZ/6ve8sj6/BGgBrtJaO26VhHCoQH8zN584jrX7q3vKKgT7mzl+fLzLYlBKMSc7hsevmENbV7d3fqMcA6akRvLvmxey43ADSsGk5AgpgeLFnJYIlFJm4GHgNKAYWK+UelNrvaPPYWcCedafecA/rP8VHmp+Tgwv3TCfD3eUExHkz0kTE5jihsH34ACzTPt1M6l/NXY4845gLlCotd4HoJR6CVgG9E0Ey4BntTEvaq1SKkoplay1LnViXGIUAvzMzMuOZZ5UPxVjjNbaYdMx3Wkkta6cOVSfCvSdY1VsbRvuMSilblBKbVBKbaislNpAQgjHCgoKorq62usLBmqtqa6uJihoeJMnnHlHYC+19v9bHsoxaK0fAx4DKCgo8O7/U0IIj5OWlkZxcTFj4YtmUFAQaWnDW9PgzERQDKT3eZwGHB7BMUII4VT+/v5kZ2e7Owy3cWbX0HogTymVrZQKAJYDb/Y75k3gCmWYD9TL+IAQQriW0+4ItNZdSqlbgfcwpo8+qbXerpS6yfr8I8AqjKmjhRjTR692VjxCCCHsc+o6Aq31KoyLfd+2R/r8WQO3ODMGIYQQR6e8bZRcKVUJHBzBS+OAKgeH4+l88ZzBN8/bF88ZfPO8R3rOmVpru6s/vS4RjJRSaoPWusDdcbiSL54z+OZ5++I5g2+etzPO2YtK/gkhhHAGSQRCCOHjfCkRPObuANzAF88ZfPO8ffGcwTfP2+Hn7DNjBEIIIezzpTsCIYQQdkgiEEIIHzfmEoFS6gyl1C6lVKFS6g47zyul1IPW57cqpWa5I05HGsI5X2o9161KqTVKqenuiNPRjnXefY6bo5TqVkpd4Mr4nGEo56yUOlEptUUptV0p9ZmrY3SGIfyORyql3lJKfWs9b6+vUqCUelIpVaGU+m6Q5x13LdNaj5kfjFIWe4EcIAD4Fsjvd8wS4F2MyqfzgW/cHbcLznkhEG3985nefs5DPe8+x32MscL9AnfH7YL/11EYe35kWB8nuDtuF533ncC91j/HAzVAgLtjH+V5Hw/MAr4b5HmHXcvG2h1Bz2Y4WusO4MhmOH31bIajtV4LRCmlkl0dqAMd85y11mu01rXWh2sxqrx6u6H8vwb4MfBvoMKVwTnJUM75EuBVrfUhAK21r5y3BsKt29+GYSSCLteG6Vha688xzmMwDruWjbVE4LDNcLzIcM/nWoxvEd7umOetlEoFzgMeYWwYyv/r8UC0UupTpdRGpdQVLovOeYZy3g8BkzDK2G8DbtdaW1wTnts47Frm1KJzbuCwzXC8yJDPRyl1EkYiWOzUiFxjKOd9P/BLrXX3WNiCkKGdsx8wGzgFCAa+Vkqt1VrvdnZwTjSU8z4d2AKcDIwDPlBKfaG1bnBybO7ksGvZWEsEvrgZzpDORyk1DfgncKbWutpFsTnTUM67AHjJmgTigCVKqS6t9esuidDxhvr7XaW1bgaalVKfA9MBb04EQznvq4E/aaPzvFAptR+YCKxzTYhu4bBr2VjrGvLFzXCOec5KqQzgVeByL/9m2Ncxz1trna21ztJaZwGvAP/hxUkAhvb7/QZwnFLKTykVAswDdro4TkcbynkfwrgLQimVCEwA9rk0Stdz2LVsTN0RaB/cDGeI53w3EAv83frtuEt7ecXGIZ73mDKUc9Za71RKrQa2Ahbgn1pru9MPvcUQ/1//HnhaKbUNo8vkl1prry5PrZT6F3AiEKeUKgZ+C/iD469lUmJCCCF83FjrGhJCCDFMkgiEEMLHSSIQQggfJ4lACCF8nCQCIYTwcZIIhHAgpdSVSqk91p8r3R2PEEMh00eFcBClVAywAWNFswY2ArP7FPwTwiPJHYEQI2Dd42CrUipIKRWqlNoO3AJ8oLWusV78PwDOcG+kQhzbmFpZLISraK3XK6XeBP4bo7jb80AnY6uyrfARckcgxMjdA5yG0RX0Z8ZeZVvhIyQRCDFyMRiboIQDQYy9yrbCR8hgsRAjZO0aegnIBpIxivttxNheEGATxmDx0XaZEsLtZIxAiBGw7vzVpbV+USllBtYAMzCqYK63HnaPJAHhDeSOQAghfJyMEQghhI+TRCCEED5OEoEQQvg4SQRCCOHjJBEIIYSPk0QghBA+ThKBEEL4uP8PJ0iygoRrfLAAAAAASUVORK5CYII=",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "import pandas as pd \n",
+    "import numpy as np\n",
+    "import matplotlib.pyplot as plt\n",
+    "import seaborn as sns\n",
+    "\n",
+    "df_a = pd.read_csv(\"/Users/johnnydevriese/Documents/cs229-spring-22/XCS229-PS1/stability/ds1_a.csv\")\n",
+    "df_b = pd.read_csv(\"/Users/johnnydevriese/Documents/cs229-spring-22/XCS229-PS1/stability/ds1_b.csv\")\n",
+    "\n",
+    "# print(df_a.head())\n",
+    "# print(df_b.head())\n",
+    "\n",
+    "sns.scatterplot(data=df_a, x=\"x0\", y=\"x1\", hue=\"y\").set(title='ds1_a.csv data')\n",
+    "# sns.scatterplot(data=df_b, x=\"x0\", y=\"x1\", hue=\"y\").set(title='ds1_b.csv data')\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 53,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "array([2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,\n",
+       "       2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,\n",
+       "       2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,\n",
+       "       2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,\n",
+       "       2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,\n",
+       "       2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.])"
+      ]
+     },
+     "execution_count": 53,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "import numpy as np\n",
+    "\n",
+    "size = 3\n",
+    "\n",
+    "theta = np.zeros(size)\n",
+    "# x = np.ones(size)\n",
+    "# x.dot(theta)\n",
+    "sigmoid = lambda x: 1 / 1 + np.exp(- x @ theta)\n",
+    "\n",
+    "sigmoid(x)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 51,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "x = np.array([[ 1.        , -0.3226045 ,  1.44016009],\n",
+    "       [ 1.        ,  1.45213116,  2.26274285],\n",
+    "       [ 1.        ,  0.85117646,  0.87461786],\n",
+    "       [ 1.        , -0.42506633,  0.49448067],\n",
+    "       [ 1.        ,  1.07977541,  4.15222455],\n",
+    "       [ 1.        ,  0.06839514,  3.60693507],\n",
+    "       [ 1.        ,  3.86229163, 25.99079097],\n",
+    "       [ 1.        ,  0.63078087,  0.69715553],\n",
+    "       [ 1.        , -0.64119999,  0.07896407],\n",
+    "       [ 1.        ,  0.53135242,  4.50417546],\n",
+    "       [ 1.        ,  2.19032426,  6.54381687],\n",
+    "       [ 1.        , -0.38465533,  0.42296288],\n",
+    "       [ 1.        ,  0.21679465,  5.01812571],\n",
+    "       [ 1.        ,  0.4846679 ,  1.36387124],\n",
+    "       [ 1.        ,  2.39608694, 10.87208984],\n",
+    "       [ 1.        , -0.51197311,  0.86848666],\n",
+    "       [ 1.        ,  0.61284557,  1.92105571],\n",
+    "       [ 1.        ,  1.20247   ,  1.39818937],\n",
+    "       [ 1.        ,  2.48716225,  4.41633191],\n",
+    "       [ 1.        ,  0.27191322,  2.48805513],\n",
+    "       [ 1.        ,  0.46650294,  3.4560205 ],\n",
+    "       [ 1.        , -0.77155423,  0.70674626],\n",
+    "       [ 1.        ,  0.74548073,  0.78677636],\n",
+    "       [ 1.        ,  1.34588118,  1.84466679],\n",
+    "       [ 1.        ,  0.03825659,  1.37679837],\n",
+    "       [ 1.        ,  0.97144296,  2.33499786],\n",
+    "       [ 1.        ,  0.73578508,  5.36219354],\n",
+    "       [ 1.        , -0.98294485,  0.60613226],\n",
+    "       [ 1.        , -1.06748414,  0.51752952],\n",
+    "       [ 1.        ,  0.16444179,  1.19318199],\n",
+    "       [ 1.        , -0.7898848 ,  1.0424866 ],\n",
+    "       [ 1.        , -0.14329976,  0.56735751],\n",
+    "       [ 1.        , -0.82024735,  0.49960062],\n",
+    "       [ 1.        ,  0.66933574,  2.03132583],\n",
+    "       [ 1.        ,  2.18580501,  4.79542536],\n",
+    "       [ 1.        ,  1.24524603,  2.22477477],\n",
+    "       [ 1.        , -0.38608243,  1.17125926],\n",
+    "       [ 1.        ,  1.97265966,  0.94942542],\n",
+    "       [ 1.        ,  0.11536834,  0.41934752],\n",
+    "       [ 1.        , -0.62494875,  0.36937505],\n",
+    "       [ 1.        ,  0.73560961,  2.00940088],\n",
+    "       [ 1.        ,  1.79850535,  1.03246785],\n",
+    "       [ 1.        , -0.14833684,  0.46835912],\n",
+    "       [ 1.        , -0.09403357,  1.07422495],\n",
+    "       [ 1.        ,  1.95126591, 11.27001531],\n",
+    "       [ 1.        , -0.76667657,  0.48703998],\n",
+    "       [ 1.        ,  0.53937355,  0.91782734],\n",
+    "       [ 1.        , -0.22698133,  1.18536598],\n",
+    "       [ 1.        , -0.39058305,  1.03621699],\n",
+    "       [ 1.        ,  0.96225877,  1.0804548 ],\n",
+    "       [ 1.        ,  2.87853267, 32.40471731],\n",
+    "       [ 1.        ,  0.63533685,  1.31647006],\n",
+    "       [ 1.        ,  2.44918624, 32.26914179],\n",
+    "       [ 1.        ,  1.55800428,  1.41965492],\n",
+    "       [ 1.        ,  1.58703176, 10.27789785],\n",
+    "       [ 1.        ,  2.92302652,  6.32084127],\n",
+    "       [ 1.        ,  1.16179168,  8.79339693],\n",
+    "       [ 1.        ,  1.01926912,  7.27956453],\n",
+    "       [ 1.        ,  1.64074981,  5.30590931],\n",
+    "       [ 1.        ,  2.1625751 , 12.70579893],\n",
+    "       [ 1.        ,  2.20355391,  6.93546526],\n",
+    "       [ 1.        ,  1.80281512,  9.08987516],\n",
+    "       [ 1.        ,  0.38185288,  2.08733116],\n",
+    "       [ 1.        ,  2.73898669,  6.43298261],\n",
+    "       [ 1.        ,  2.33229757, 13.46785592],\n",
+    "       [ 1.        ,  2.1515753 ,  3.30175484],\n",
+    "       [ 1.        ,  2.08755952,  7.46399068],\n",
+    "       [ 1.        ,  2.42536537, 26.4483108 ],\n",
+    "       [ 1.        ,  1.22952296,  2.63268864],\n",
+    "       [ 1.        ,  2.29617283, 12.01655075],\n",
+    "       [ 1.        ,  3.30064721, 11.28460368],\n",
+    "       [ 1.        ,  0.98606951,  2.43854265],\n",
+    "       [ 1.        ,  1.71111559,  4.78419937],\n",
+    "       [ 1.        ,  1.43791364,  2.54852746],\n",
+    "       [ 1.        ,  2.06546113,  6.82796588],\n",
+    "       [ 1.        ,  2.6042525 , 18.17196626],\n",
+    "       [ 1.        ,  2.56193154,  2.67573402],\n",
+    "       [ 1.        ,  1.67306091,  2.28230367],\n",
+    "       [ 1.        ,  3.07760672, 13.17400532],\n",
+    "       [ 1.        ,  0.80343565,  1.95356398],\n",
+    "       [ 1.        ,  0.29409223,  1.8304515 ],\n",
+    "       [ 1.        ,  4.17743477, 28.43916382],\n",
+    "       [ 1.        ,  2.20240356, 14.48263971],\n",
+    "       [ 1.        ,  2.75638736,  9.03904178],\n",
+    "       [ 1.        ,  1.59509206,  3.23851668],\n",
+    "       [ 1.        ,  2.31360137,  8.12523688],\n",
+    "       [ 1.        ,  2.25188961,  6.10723549],\n",
+    "       [ 1.        ,  1.74848706,  4.97610269],\n",
+    "       [ 1.        ,  1.94191761,  6.17617417],\n",
+    "       [ 1.        ,  3.23970139, 36.2488673 ],\n",
+    "       [ 1.        ,  0.77654359,  1.719952  ],\n",
+    "       [ 1.        ,  1.53339444,  7.11126102],\n",
+    "       [ 1.        ,  1.44950114,  7.08491125],\n",
+    "       [ 1.        ,  1.62476639,  8.74420153],\n",
+    "       [ 1.        ,  1.96366438,  8.17721712],\n",
+    "       [ 1.        ,  1.79561844,  2.61247159],\n",
+    "       [ 1.        ,  1.95348024,  1.6333998 ],\n",
+    "       [ 1.        ,  1.86352418,  5.88989936],\n",
+    "       [ 1.        ,  2.91729395, 22.77683958],\n",
+    "       [ 1.        ,  1.31972857,  2.84750546]])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# PS3: Working on Poisson Regression and reshaping the gradient calcuation"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "3\n",
+      "(3,)\n",
+      "(2, 2)\n",
+      "[[1 2]\n",
+      " [3 4]]\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "array([4, 6])"
+      ]
+     },
+     "execution_count": 17,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# our code looks so funky because the problem says we have to do\n",
+    "#  **full batch gradient ascent**\n",
+    "\n",
+    "foo = np.array([1 ,2, 3,])  \n",
+    "\n",
+    "# just an array that is three long\n",
+    "print(len(foo))\n",
+    "print(foo.shape)\n",
+    "\n",
+    "# three by 1 matrix \n",
+    "# three rows by 1 column \n",
+    "foo.reshape(3, 1).shape\n",
+    "\n",
+    "bar = np.array([1, 2, 3, 4])\n",
+    "\n",
+    "bar_reshaped = bar.reshape(2, 2)\n",
+    "\n",
+    "print(bar_reshaped.shape)\n",
+    "print(bar_reshaped)\n",
+    "\n",
+    "# np.sum(bar_reshaped, axis=0)\n",
+    "\n",
+    "# sum *down* the columns\n",
+    "bar_reshaped.sum(axis=0)\n",
+    "\n",
+    "# np.exp(bar, out=(2,2))\n",
+    "\n",
+    "# print(foo)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 24,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "-0.9262096826685897"
+      ]
+     },
+     "execution_count": 24,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from numpy import linalg as LA\n",
+    "\n",
+    "LA.norm(np.array([1,2])) - LA.norm(np.array([1, 3]))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 22,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "1.0"
+      ]
+     },
+     "execution_count": 22,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "LA.norm(np.array([1,2]) - np.array([1,3]))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 31,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "state = []\n",
+    "\n",
+    "for x in state:\n",
+    "    # print(\"this is coef\", coef)\n",
+    "    print(\"this is x\", x)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 34,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "1"
+      ]
+     },
+     "execution_count": 34,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# print(\"hello world\")\n",
+    "\n",
+    "\n",
+    "def sign(a):\n",
+    "    \"\"\"Gets the sign of a scalar input.\"\"\"\n",
+    "    if a >= 0:\n",
+    "        return 1\n",
+    "    else:\n",
+    "        return 0\n",
+    "\n",
+    "sign(22)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Fashion MNIST Problem scratchpad"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "array([[0.00626879, 0.01704033, 0.04632042, 0.93037047],\n",
+       "       [0.01203764, 0.08894682, 0.24178252, 0.65723302],\n",
+       "       [0.00446236, 0.66227241, 0.24363641, 0.08962882]])"
+      ]
+     },
+     "execution_count": 16,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "import numpy as np\n",
+    "\n",
+    "def softmax(x):\n",
+    "    x = x - np.max(x,axis=1)[:,np.newaxis]\n",
+    "    exp = np.exp(x)\n",
+    "    s = exp / np.sum(exp,axis=1)[:,np.newaxis]\n",
+    "    return s\n",
+    "\n",
+    "\n",
+    "overflow_test = np.array([[10000, 10010, 10]])\n",
+    "softmax([[22, 14, 16]])\n",
+    "\n",
+    "softmax(overflow_test)\n",
+    "\n",
+    "# correct solution:\n",
+    "def softmax_from_stackoverflow(x):\n",
+    "    \"\"\"Compute softmax values for each sets of scores in x.\"\"\"\n",
+    "    e_x = np.exp(x - np.max(x))\n",
+    "    return e_x / e_x.sum(axis=0) # only difference\n",
+    "\n",
+    "# def softmax_2(x):\n",
+    "# my (correct) solution:\n",
+    "def softmax_2(z):\n",
+    "    assert len(z.shape) == 2\n",
+    "    s = np.max(z, axis=1)\n",
+    "    s = s[:, np.newaxis] # necessary step to do broadcasting\n",
+    "    e_x = np.exp(z - s)\n",
+    "    div = np.sum(e_x, axis=1)\n",
+    "    div = div[:, np.newaxis] # dito\n",
+    "    return e_x / div\n",
+    "\n",
+    "# BOOM -- this seems like it has the same result and just needs keepdims argument! \n",
+    "def softmax_3(z):\n",
+    "    return np.exp(z) / np.sum(np.exp(z), axis=1, keepdims=True)\n",
+    "\n",
+    "\n",
+    "\n",
+    "# softmax_from_stackoverflow(overflow_test)\n",
+    "\n",
+    "scores2D = np.array([[1, 2, 3, 6],\n",
+    "                     [2, 4, 5, 6],\n",
+    "                     [3, 8, 7, 6]])\n",
+    "\n",
+    "# softmax_from_stackoverflow(scores2D)\n",
+    "softmax(scores2D)\n",
+    "\n",
+    "softmax_2(scores2D)\n",
+    "\n",
+    "softmax_3(scores2D)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 31,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[[0. 0.]]\n",
+      "[0. 0.]\n"
+     ]
+    }
+   ],
+   "source": [
+    "def foo():\n",
+    "    return (\"foo\", \"bar\", \"baz\")\n",
+    "\n",
+    "foo, bar, baz = foo()\n",
+    "\n",
+    "# print(baz)\n",
+    "\n",
+    "input_size = 2\n",
+    "num_hidden = 2\n",
+    "num_output = 2\n",
+    "\n",
+    "\n",
+    "W1 = np.random.normal(size = (input_size, num_hidden))\n",
+    "b1 = np.zeros((1, num_hidden)) \n",
+    "W2 = np.random.normal(size = (num_hidden, num_output)),  \n",
+    "b2= np.zeros((1, num_output))\n",
+    "dict(W1=W1, b1=b1, W2=W2, b2=b2)\n",
+    "\n",
+    "working = {\n",
+    "        'W1': np.random.normal(size = (input_size, num_hidden)),\n",
+    "        'b1': np.zeros(num_hidden),\n",
+    "        'W2': np.random.normal(size = (num_hidden, 10)),\n",
+    "        'b2': np.zeros(10)\n",
+    "    }\n",
+    "\n",
+    "# print(working)\n",
+    "\n",
+    "b1 = np.zeros((1, 2))\n",
+    "print(b1)\n",
+    "\n",
+    "b1_again = np.zeros(2)\n",
+    "print(b1_again)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Naive Bayes Classifier "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 74,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'puttin': 1, 'fone': 2}"
+      ]
+     },
+     "execution_count": 74,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "ham = \"THANX 4 PUTTIN DA FONE DOWN ON ME!\"\n",
+    "ham2 = \"So how are you really. What are you up to. How's the masters. And so on.\"\n",
+    "\n",
+    "ham_split = ham.lower().split()\n",
+    "\n",
+    "# would have to be a dict so we can lookup words \n",
+    "ham_split\n",
+    "word_count_dict = {}\n",
+    "\n",
+    "for word in ham_split:\n",
+    "    # temp = dictionary.get(word, 0)\n",
+    "    # dictionary[word] += 1\n",
+    "\n",
+    "    if word in word_count_dict:\n",
+    "        word_count_dict[word] += 1\n",
+    "    else:\n",
+    "        word_count_dict[word] = 1\n",
+    "    \n",
+    "    # dictionary.setdefault(word, 1)\n",
+    "\n",
+    "# filter this dict if count > 5 \n",
+    "# for key, value in dictionary.items():\n",
+    "#     print(key)\n",
+    "#     print(value)\n",
+    "\n",
+    "word_count_dict.items()\n",
+    "\n",
+    "word_count_dict[\"puttin\"] = 22\n",
+    "word_count_dict[\"fone\"] = 12\n",
+    "\n",
+    "\n",
+    "word_count_dict_filtered = dict(filter(lambda item: item[1] > 10, word_count_dict.items()))\n",
+    "# foo = list(filter(lambda item: item[1] > 10, dictionary.items()))\n",
+    "\n",
+    "word_dict = {}\n",
+    "idx = 0\n",
+    "for key, _ in word_count_dict_filtered.items():\n",
+    "    idx += 1\n",
+    "    word_dict[key] = idx\n",
+    "\n",
+    "# dictionary\n",
+    "word_dict"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 55,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Filter dictionary: {8: 'u', 9: 'z', 10: 'j'}\n"
+     ]
+    }
+   ],
+   "source": [
+    "my_dict = {8:'u',4:'t',9:'z',10:'j',5:'k',3:'s'}\n",
+    "\n",
+    "new_filt = dict(filter(lambda val: val[0] > 5, my_dict.items()))\n",
+    "print(\"Filter dictionary:\",new_filt)\n",
+    "\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "word_dict = {}\n",
+    "for message in messages:\n",
+    "    # word_list = get_words(message)\n",
+    "    for word in word_list:\n",
+    "        if word in word_dict:\n",
+    "            word_dict[word] += 1\n",
+    "        else:\n",
+    "            word_dict[word] = 1"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "N, V = len(messages), len(word_dictionary)\n",
+    "data = np.zeros((N, V))\n",
+    "for i, m in enumerate(messages):\n",
+    "    for w in get_words(m):\n",
+    "        if w in word_dictionary:\n",
+    "            data[i, word_dictionary[w]] += 1\n",
+    "return data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 78,
+   "metadata": {},
+   "outputs": [
+    {
+     "ename": "NameError",
+     "evalue": "name 'word_count' is not defined",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
+      "\u001b[1;32m/Users/johnnydevriese/projects/jupyter/cs229-scratchpad.ipynb Cell 22'\u001b[0m in \u001b[0;36m<cell line: 7>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/cs229-scratchpad.ipynb#ch0000024?line=2'>3</a>\u001b[0m np\u001b[39m.\u001b[39mzeros((\u001b[39m1\u001b[39m, word_num))\n\u001b[1;32m      <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/cs229-scratchpad.ipynb#ch0000024?line=4'>5</a>\u001b[0m word_array \u001b[39m=\u001b[39m np\u001b[39m.\u001b[39marray([])\u001b[39m.\u001b[39mreshape(\u001b[39m0\u001b[39m, word_num)\n\u001b[0;32m----> <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/cs229-scratchpad.ipynb#ch0000024?line=6'>7</a>\u001b[0m word_array \u001b[39m=\u001b[39m np\u001b[39m.\u001b[39mvstack([word_array, word_count])\n",
+      "\u001b[0;31mNameError\u001b[0m: name 'word_count' is not defined"
+     ]
+    }
+   ],
+   "source": [
+    "word_num = 10\n",
+    "\n",
+    "np.zeros((1, word_num))\n",
+    "\n",
+    "word_array = np.array([]).reshape(0, word_num)\n",
+    "\n",
+    "word_array = np.vstack([word_array, word_count])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 86,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "array([[1, 2, 3]])"
+      ]
+     },
+     "execution_count": 86,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "np.ones((2, 3)).sum(axis=0)\n",
+    "\n",
+    "\n",
+    "np.array([[1 ,2, 3], [4, 5, 6]]).sum(axis=0) # sums column wise. \n",
+    "\n",
+    "np.array([[1 ,2, 3], [4, 5, 6]]).sum(axis=1) # sums row wise. \n",
+    "\n",
+    "\n",
+    "foo = np.array([[1 ,2, 3], [4, 5, 6]])\n",
+    "\n",
+    "foo[0:1,:]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
+       " [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],\n",
+       " [0, 2, 4, 6, 8, 10, 12, 14, 16, 18],\n",
+       " [0, 3, 6, 9, 12, 15, 18, 21, 24, 27],\n",
+       " [0, 4, 8, 12, 16, 20, 24, 28, 32, 36],\n",
+       " [0, 5, 10, 15, 20, 25, 30, 35, 40, 45],\n",
+       " [0, 6, 12, 18, 24, 30, 36, 42, 48, 54],\n",
+       " [0, 7, 14, 21, 28, 35, 42, 49, 56, 63],\n",
+       " [0, 8, 16, 24, 32, 40, 48, 56, 64, 72],\n",
+       " [0, 9, 18, 27, 36, 45, 54, 63, 72, 81]]"
+      ]
+     },
+     "execution_count": 8,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# [i for i in range(10) for j in range(10)] \n",
+    "\n",
+    "[[i*j for i in range(10)] for j in range(10)]\n",
+    "# p_x_given_z(x[i], mu[j], sigma[j]) * phi[j] \n",
+    "\n",
+    "#     for i in range(n):\n",
+    "#         for j in range(k):\n",
+    "\n",
+    "[[p_x_given_z(x[i], mu[j], sigma[j]) * phi[j] for i in range(n) for j in range(k)]]"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3.10.5 ('pytorch-nightly')",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.5"
+  },
+  "orig_nbformat": 4,
+  "vscode": {
+   "interpreter": {
+    "hash": "8a8bcccfb183d1298694efece6cf41240378bc61621e95c864629a40c5876542"
+   }
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

code/cs229_random_gems.md

@@ -0,0 +1,13 @@
+# Random Gems they point out 
+
+### lecture 10 
+
+* Boosting is an ensembling method to decrease **bias** in trees
+
+decision tree has low bias and high variance 
+
+ensembling a decision tree reduces the variance? 
+
+https://towardsdatascience.com/decision-tree-ensembles-bagging-and-boosting-266a8ba60fd9
+
+

code/data/cifar-10-batches-py/data_batch_1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:54636561a3ce25bd3e19253c6b0d8538147b0ae398331ac4a2d86c6d987368cd
+size 31035704

code/data/cifar-10-batches-py/data_batch_2

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:766b2cef9fbc745cf056b3152224f7cf77163b330ea9a15f9392beb8b89bc5a8
+size 31035320

code/data/cifar-10-batches-py/data_batch_3

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f00d98ebfb30b3ec0ad19f9756dc2630b89003e10525f5e148445e82aa6a1f9
+size 31035999

code/data/cifar-10-batches-py/data_batch_4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3f7bb240661948b8f4d53e36ec720d8306f5668bd0071dcb4e6c947f78e9682b
+size 31035696

code/data/cifar-10-batches-py/data_batch_5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d91802434d8376bbaeeadf58a737e3a1b12ac839077e931237e0dcd43adcb154
+size 31035623

code/data/cifar-10-batches-py/readme.html

@@ -0,0 +1 @@
+<meta HTTP-EQUIV="REFRESH" content="0; url=http://www.cs.toronto.edu/~kriz/cifar.html">

code/data/cifar-10-batches-py/test_batch

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f53d8d457504f7cff4ea9e021afcf0e0ad8e24a91f3fc42091b8adef61157831
+size 31035526

code/data/cifar-10-python.tar.gz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6d958be074577803d12ecdefd02955f39262c83c16fe9348329d7fe0b5c001ce
+size 170498071

code/diffusers/hf_diffusers.ipynb

@@ -0,0 +1,508 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "tensor([1., 1., 1., 1., 1.], device='mps:0')\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages/torch/_tensor_str.py:103: UserWarning: The operator 'aten::bitwise_and.Tensor_out' is not currently supported on the MPS backend and will fall back to run on the CPU. This may have performance implications. (Triggered internally at  /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1659484612588/work/aten/src/ATen/mps/MPSFallback.mm:11.)\n",
+      "  nonzero_finite_vals = torch.masked_select(tensor_view, torch.isfinite(tensor_view) & tensor_view.ne(0))\n"
+     ]
+    }
+   ],
+   "source": [
+    "import torch\n",
+    "\n",
+    "import torch\n",
+    "\n",
+    "\n",
+    "# Check that MPS is available\n",
+    "if not torch.backends.mps.is_available():\n",
+    "    if not torch.backends.mps.is_built():\n",
+    "        print(\"MPS not available because the current PyTorch install was not \"\n",
+    "              \"built with MPS enabled.\")\n",
+    "    else:\n",
+    "        print(\"MPS not available because the current MacOS version is not 12.3+ \"\n",
+    "              \"and/or you do not have an MPS-enabled device on this machine.\")\n",
+    "\n",
+    "else:\n",
+    "    mps_device = torch.device(\"mps\")\n",
+    "\n",
+    "    # Create a Tensor directly on the mps device\n",
+    "    x = torch.ones(5, device=mps_device)\n",
+    "    # Or\n",
+    "    # x = torch.ones(5, device=\"mps\")\n",
+    "    print(x)\n",
+    "\n",
+    "    # # Any operation happens on the GPU\n",
+    "    # y = x * 2\n",
+    "\n",
+    "    # # Move your model to mps just like any other device\n",
+    "    # model = YourFavoriteNet()\n",
+    "    # model.to(mps_device)\n",
+    "\n",
+    "    # # Now every call runs on the GPU\n",
+    "    # pred = model(x)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Collecting diffusers==0.2.4\n",
+      "  Downloading diffusers-0.2.4-py3-none-any.whl (112 kB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m113.0/113.0 kB\u001b[0m \u001b[31m1.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m\n",
+      "\u001b[?25hRequirement already satisfied: numpy in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from diffusers==0.2.4) (1.23.2)\n",
+      "Collecting filelock\n",
+      "  Downloading filelock-3.8.0-py3-none-any.whl (10 kB)\n",
+      "Collecting importlib-metadata\n",
+      "  Downloading importlib_metadata-4.12.0-py3-none-any.whl (21 kB)\n",
+      "Requirement already satisfied: torch>=1.4 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from diffusers==0.2.4) (1.12.1)\n",
+      "Requirement already satisfied: Pillow in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from diffusers==0.2.4) (9.2.0)\n",
+      "Collecting huggingface-hub<1.0,>=0.8.1\n",
+      "  Downloading huggingface_hub-0.9.0-py3-none-any.whl (120 kB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m120.5/120.5 kB\u001b[0m \u001b[31m4.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+      "\u001b[?25hCollecting regex!=2019.12.17\n",
+      "  Downloading regex-2022.8.17-cp310-cp310-macosx_11_0_arm64.whl (282 kB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m283.0/283.0 kB\u001b[0m \u001b[31m6.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m\n",
+      "\u001b[?25hRequirement already satisfied: requests in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from diffusers==0.2.4) (2.28.1)\n",
+      "Requirement already satisfied: typing-extensions>=3.7.4.3 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.8.1->diffusers==0.2.4) (4.3.0)\n",
+      "Requirement already satisfied: packaging>=20.9 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.8.1->diffusers==0.2.4) (21.3)\n",
+      "Collecting pyyaml>=5.1\n",
+      "  Downloading PyYAML-6.0-cp310-cp310-macosx_11_0_arm64.whl (173 kB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m174.0/174.0 kB\u001b[0m \u001b[31m6.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+      "\u001b[?25hCollecting tqdm\n",
+      "  Downloading tqdm-4.64.0-py2.py3-none-any.whl (78 kB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m78.4/78.4 kB\u001b[0m \u001b[31m2.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+      "\u001b[?25hCollecting zipp>=0.5\n",
+      "  Downloading zipp-3.8.1-py3-none-any.whl (5.6 kB)\n",
+      "Requirement already satisfied: certifi>=2017.4.17 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from requests->diffusers==0.2.4) (2022.6.15)\n",
+      "Requirement already satisfied: charset-normalizer<3,>=2 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from requests->diffusers==0.2.4) (2.1.1)\n",
+      "Requirement already satisfied: idna<4,>=2.5 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from requests->diffusers==0.2.4) (3.3)\n",
+      "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from requests->diffusers==0.2.4) (1.26.11)\n",
+      "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from packaging>=20.9->huggingface-hub<1.0,>=0.8.1->diffusers==0.2.4) (3.0.9)\n",
+      "Installing collected packages: zipp, tqdm, regex, pyyaml, filelock, importlib-metadata, huggingface-hub, diffusers\n",
+      "Successfully installed diffusers-0.2.4 filelock-3.8.0 huggingface-hub-0.9.0 importlib-metadata-4.12.0 pyyaml-6.0 regex-2022.8.17 tqdm-4.64.0 zipp-3.8.1\n",
+      "Collecting transformers\n",
+      "  Downloading transformers-4.21.2-py3-none-any.whl (4.7 MB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m4.7/4.7 MB\u001b[0m \u001b[31m20.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m00:01\u001b[0m\n",
+      "\u001b[?25hCollecting scipy\n",
+      "  Downloading scipy-1.9.0-cp310-cp310-macosx_12_0_arm64.whl (29.9 MB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m29.9/29.9 MB\u001b[0m \u001b[31m34.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m00:01\u001b[0m\n",
+      "\u001b[?25hCollecting ftfy\n",
+      "  Downloading ftfy-6.1.1-py3-none-any.whl (53 kB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m53.1/53.1 kB\u001b[0m \u001b[31m2.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+      "\u001b[?25hRequirement already satisfied: huggingface-hub<1.0,>=0.1.0 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from transformers) (0.9.0)\n",
+      "Collecting tokenizers!=0.11.3,<0.13,>=0.11.1\n",
+      "  Downloading tokenizers-0.12.1.tar.gz (220 kB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m220.7/220.7 kB\u001b[0m \u001b[31m8.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+      "\u001b[?25h  Installing build dependencies ... \u001b[?25ldone\n",
+      "\u001b[?25h  Getting requirements to build wheel ... \u001b[?25ldone\n",
+      "\u001b[?25h  Preparing metadata (pyproject.toml) ... \u001b[?25ldone\n",
+      "\u001b[?25hRequirement already satisfied: packaging>=20.0 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from transformers) (21.3)\n",
+      "Requirement already satisfied: numpy>=1.17 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from transformers) (1.23.2)\n",
+      "Requirement already satisfied: tqdm>=4.27 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from transformers) (4.64.0)\n",
+      "Requirement already satisfied: requests in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from transformers) (2.28.1)\n",
+      "Requirement already satisfied: regex!=2019.12.17 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from transformers) (2022.8.17)\n",
+      "Requirement already satisfied: pyyaml>=5.1 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from transformers) (6.0)\n",
+      "Requirement already satisfied: filelock in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from transformers) (3.8.0)\n",
+      "Requirement already satisfied: wcwidth>=0.2.5 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from ftfy) (0.2.5)\n",
+      "Requirement already satisfied: typing-extensions>=3.7.4.3 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.1.0->transformers) (4.3.0)\n",
+      "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from packaging>=20.0->transformers) (3.0.9)\n",
+      "Requirement already satisfied: idna<4,>=2.5 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from requests->transformers) (3.3)\n",
+      "Requirement already satisfied: certifi>=2017.4.17 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from requests->transformers) (2022.6.15)\n",
+      "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from requests->transformers) (1.26.11)\n",
+      "Requirement already satisfied: charset-normalizer<3,>=2 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from requests->transformers) (2.1.1)\n",
+      "Building wheels for collected packages: tokenizers\n",
+      "  Building wheel for tokenizers (pyproject.toml) ... \u001b[?25lerror\n",
+      "  \u001b[1;31merror\u001b[0m: \u001b[1msubprocess-exited-with-error\u001b[0m\n",
+      "  \n",
+      "  \u001b[31m×\u001b[0m \u001b[32mBuilding wheel for tokenizers \u001b[0m\u001b[1;32m(\u001b[0m\u001b[32mpyproject.toml\u001b[0m\u001b[1;32m)\u001b[0m did not run successfully.\n",
+      "  \u001b[31m│\u001b[0m exit code: \u001b[1;36m1\u001b[0m\n",
+      "  \u001b[31m╰─>\u001b[0m \u001b[31m[51 lines of output]\u001b[0m\n",
+      "  \u001b[31m   \u001b[0m running bdist_wheel\n",
+      "  \u001b[31m   \u001b[0m running build\n",
+      "  \u001b[31m   \u001b[0m running build_py\n",
+      "  \u001b[31m   \u001b[0m creating build\n",
+      "  \u001b[31m   \u001b[0m creating build/lib.macosx-11.0-arm64-cpython-310\n",
+      "  \u001b[31m   \u001b[0m creating build/lib.macosx-11.0-arm64-cpython-310/tokenizers\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers\n",
+      "  \u001b[31m   \u001b[0m creating build/lib.macosx-11.0-arm64-cpython-310/tokenizers/models\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/models/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers/models\n",
+      "  \u001b[31m   \u001b[0m creating build/lib.macosx-11.0-arm64-cpython-310/tokenizers/decoders\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/decoders/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers/decoders\n",
+      "  \u001b[31m   \u001b[0m creating build/lib.macosx-11.0-arm64-cpython-310/tokenizers/normalizers\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/normalizers/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers/normalizers\n",
+      "  \u001b[31m   \u001b[0m creating build/lib.macosx-11.0-arm64-cpython-310/tokenizers/pre_tokenizers\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/pre_tokenizers/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers/pre_tokenizers\n",
+      "  \u001b[31m   \u001b[0m creating build/lib.macosx-11.0-arm64-cpython-310/tokenizers/processors\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/processors/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers/processors\n",
+      "  \u001b[31m   \u001b[0m creating build/lib.macosx-11.0-arm64-cpython-310/tokenizers/trainers\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/trainers/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers/trainers\n",
+      "  \u001b[31m   \u001b[0m creating build/lib.macosx-11.0-arm64-cpython-310/tokenizers/implementations\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/implementations/byte_level_bpe.py -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers/implementations\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/implementations/sentencepiece_unigram.py -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers/implementations\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/implementations/sentencepiece_bpe.py -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers/implementations\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/implementations/base_tokenizer.py -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers/implementations\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/implementations/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers/implementations\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/implementations/char_level_bpe.py -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers/implementations\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/implementations/bert_wordpiece.py -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers/implementations\n",
+      "  \u001b[31m   \u001b[0m creating build/lib.macosx-11.0-arm64-cpython-310/tokenizers/tools\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/tools/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers/tools\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/tools/visualizer.py -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers/tools\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/__init__.pyi -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/models/__init__.pyi -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers/models\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/decoders/__init__.pyi -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers/decoders\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/normalizers/__init__.pyi -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers/normalizers\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/pre_tokenizers/__init__.pyi -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers/pre_tokenizers\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/processors/__init__.pyi -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers/processors\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/trainers/__init__.pyi -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers/trainers\n",
+      "  \u001b[31m   \u001b[0m copying py_src/tokenizers/tools/visualizer-styles.css -> build/lib.macosx-11.0-arm64-cpython-310/tokenizers/tools\n",
+      "  \u001b[31m   \u001b[0m running build_ext\n",
+      "  \u001b[31m   \u001b[0m running build_rust\n",
+      "  \u001b[31m   \u001b[0m error: can't find Rust compiler\n",
+      "  \u001b[31m   \u001b[0m \n",
+      "  \u001b[31m   \u001b[0m If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler.\n",
+      "  \u001b[31m   \u001b[0m \n",
+      "  \u001b[31m   \u001b[0m To update pip, run:\n",
+      "  \u001b[31m   \u001b[0m \n",
+      "  \u001b[31m   \u001b[0m     pip install --upgrade pip\n",
+      "  \u001b[31m   \u001b[0m \n",
+      "  \u001b[31m   \u001b[0m and then retry package installation.\n",
+      "  \u001b[31m   \u001b[0m \n",
+      "  \u001b[31m   \u001b[0m If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler toolchain.\n",
+      "  \u001b[31m   \u001b[0m \u001b[31m[end of output]\u001b[0m\n",
+      "  \n",
+      "  \u001b[1;35mnote\u001b[0m: This error originates from a subprocess, and is likely not a problem with pip.\n",
+      "\u001b[?25h\u001b[31m  ERROR: Failed building wheel for tokenizers\u001b[0m\u001b[31m\n",
+      "\u001b[0mFailed to build tokenizers\n",
+      "\u001b[31mERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects\u001b[0m\u001b[31m\n",
+      "\u001b[0mCollecting ipywidgets<8,>=7\n",
+      "  Downloading ipywidgets-7.7.2-py2.py3-none-any.whl (123 kB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m123.4/123.4 kB\u001b[0m \u001b[31m2.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m\n",
+      "\u001b[?25hRequirement already satisfied: ipython>=4.0.0 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from ipywidgets<8,>=7) (8.4.0)\n",
+      "Requirement already satisfied: ipykernel>=4.5.1 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from ipywidgets<8,>=7) (6.15.1)\n",
+      "Collecting widgetsnbextension~=3.6.0\n",
+      "  Downloading widgetsnbextension-3.6.1-py2.py3-none-any.whl (1.6 MB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.6/1.6 MB\u001b[0m \u001b[31m13.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m00:01\u001b[0m\n",
+      "\u001b[?25hCollecting ipython-genutils~=0.2.0\n",
+      "  Downloading ipython_genutils-0.2.0-py2.py3-none-any.whl (26 kB)\n",
+      "Collecting jupyterlab-widgets<3,>=1.0.0\n",
+      "  Downloading jupyterlab_widgets-1.1.1-py3-none-any.whl (245 kB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m245.3/245.3 kB\u001b[0m \u001b[31m9.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+      "\u001b[?25hRequirement already satisfied: traitlets>=4.3.1 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from ipywidgets<8,>=7) (5.3.0)\n",
+      "Requirement already satisfied: nest-asyncio in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from ipykernel>=4.5.1->ipywidgets<8,>=7) (1.5.5)\n",
+      "Requirement already satisfied: psutil in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from ipykernel>=4.5.1->ipywidgets<8,>=7) (5.9.1)\n",
+      "Requirement already satisfied: matplotlib-inline>=0.1 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from ipykernel>=4.5.1->ipywidgets<8,>=7) (0.1.6)\n",
+      "Requirement already satisfied: jupyter-client>=6.1.12 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from ipykernel>=4.5.1->ipywidgets<8,>=7) (7.3.4)\n",
+      "Requirement already satisfied: debugpy>=1.0 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from ipykernel>=4.5.1->ipywidgets<8,>=7) (1.6.3)\n",
+      "Requirement already satisfied: pyzmq>=17 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from ipykernel>=4.5.1->ipywidgets<8,>=7) (23.2.1)\n",
+      "Requirement already satisfied: appnope in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from ipykernel>=4.5.1->ipywidgets<8,>=7) (0.1.3)\n",
+      "Requirement already satisfied: tornado>=6.1 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from ipykernel>=4.5.1->ipywidgets<8,>=7) (6.2)\n",
+      "Requirement already satisfied: packaging in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from ipykernel>=4.5.1->ipywidgets<8,>=7) (21.3)\n",
+      "Requirement already satisfied: pexpect>4.3 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from ipython>=4.0.0->ipywidgets<8,>=7) (4.8.0)\n",
+      "Requirement already satisfied: pickleshare in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from ipython>=4.0.0->ipywidgets<8,>=7) (0.7.5)\n",
+      "Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from ipython>=4.0.0->ipywidgets<8,>=7) (3.0.30)\n",
+      "Requirement already satisfied: decorator in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from ipython>=4.0.0->ipywidgets<8,>=7) (5.1.1)\n",
+      "Requirement already satisfied: jedi>=0.16 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from ipython>=4.0.0->ipywidgets<8,>=7) (0.18.1)\n",
+      "Requirement already satisfied: setuptools>=18.5 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from ipython>=4.0.0->ipywidgets<8,>=7) (65.3.0)\n",
+      "Requirement already satisfied: stack-data in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from ipython>=4.0.0->ipywidgets<8,>=7) (0.4.0)\n",
+      "Requirement already satisfied: pygments>=2.4.0 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from ipython>=4.0.0->ipywidgets<8,>=7) (2.13.0)\n",
+      "Requirement already satisfied: backcall in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from ipython>=4.0.0->ipywidgets<8,>=7) (0.2.0)\n",
+      "Collecting notebook>=4.4.1\n",
+      "  Downloading notebook-6.4.12-py3-none-any.whl (9.9 MB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m9.9/9.9 MB\u001b[0m \u001b[31m31.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m00:01\u001b[0m\n",
+      "\u001b[?25hRequirement already satisfied: parso<0.9.0,>=0.8.0 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from jedi>=0.16->ipython>=4.0.0->ipywidgets<8,>=7) (0.8.3)\n",
+      "Requirement already satisfied: python-dateutil>=2.8.2 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from jupyter-client>=6.1.12->ipykernel>=4.5.1->ipywidgets<8,>=7) (2.8.2)\n",
+      "Requirement already satisfied: entrypoints in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from jupyter-client>=6.1.12->ipykernel>=4.5.1->ipywidgets<8,>=7) (0.4)\n",
+      "Requirement already satisfied: jupyter-core>=4.9.2 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from jupyter-client>=6.1.12->ipykernel>=4.5.1->ipywidgets<8,>=7) (4.11.1)\n",
+      "Collecting terminado>=0.8.3\n",
+      "  Downloading terminado-0.15.0-py3-none-any.whl (16 kB)\n",
+      "Collecting nbformat\n",
+      "  Downloading nbformat-5.4.0-py3-none-any.whl (73 kB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m73.3/73.3 kB\u001b[0m \u001b[31m2.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+      "\u001b[?25hCollecting jinja2\n",
+      "  Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m133.1/133.1 kB\u001b[0m \u001b[31m4.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+      "\u001b[?25hCollecting prometheus-client\n",
+      "  Downloading prometheus_client-0.14.1-py3-none-any.whl (59 kB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m59.5/59.5 kB\u001b[0m \u001b[31m2.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+      "\u001b[?25hCollecting nbconvert>=5\n",
+      "  Downloading nbconvert-7.0.0-py3-none-any.whl (271 kB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m271.3/271.3 kB\u001b[0m \u001b[31m10.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+      "\u001b[?25hCollecting argon2-cffi\n",
+      "  Downloading argon2_cffi-21.3.0-py3-none-any.whl (14 kB)\n",
+      "Collecting Send2Trash>=1.8.0\n",
+      "  Downloading Send2Trash-1.8.0-py3-none-any.whl (18 kB)\n",
+      "Requirement already satisfied: ptyprocess>=0.5 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from pexpect>4.3->ipython>=4.0.0->ipywidgets<8,>=7) (0.7.0)\n",
+      "Requirement already satisfied: wcwidth in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->ipython>=4.0.0->ipywidgets<8,>=7) (0.2.5)\n",
+      "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from packaging->ipykernel>=4.5.1->ipywidgets<8,>=7) (3.0.9)\n",
+      "Requirement already satisfied: executing in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from stack-data->ipython>=4.0.0->ipywidgets<8,>=7) (0.10.0)\n",
+      "Requirement already satisfied: asttokens in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from stack-data->ipython>=4.0.0->ipywidgets<8,>=7) (2.0.8)\n",
+      "Requirement already satisfied: pure-eval in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from stack-data->ipython>=4.0.0->ipywidgets<8,>=7) (0.2.2)\n",
+      "Collecting mistune<3,>=2.0.3\n",
+      "  Downloading mistune-2.0.4-py2.py3-none-any.whl (24 kB)\n",
+      "Collecting beautifulsoup4\n",
+      "  Downloading beautifulsoup4-4.11.1-py3-none-any.whl (128 kB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m128.2/128.2 kB\u001b[0m \u001b[31m5.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+      "\u001b[?25hCollecting tinycss2\n",
+      "  Downloading tinycss2-1.1.1-py3-none-any.whl (21 kB)\n",
+      "Collecting defusedxml\n",
+      "  Downloading defusedxml-0.7.1-py2.py3-none-any.whl (25 kB)\n",
+      "Collecting markupsafe>=2.0\n",
+      "  Downloading MarkupSafe-2.1.1-cp310-cp310-macosx_10_9_universal2.whl (17 kB)\n",
+      "Collecting bleach\n",
+      "  Downloading bleach-5.0.1-py3-none-any.whl (160 kB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m160.9/160.9 kB\u001b[0m \u001b[31m5.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+      "\u001b[?25hCollecting nbclient>=0.5.0\n",
+      "  Downloading nbclient-0.6.7-py3-none-any.whl (71 kB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m71.8/71.8 kB\u001b[0m \u001b[31m2.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+      "\u001b[?25hCollecting pandocfilters>=1.4.1\n",
+      "  Downloading pandocfilters-1.5.0-py2.py3-none-any.whl (8.7 kB)\n",
+      "Collecting jupyterlab-pygments\n",
+      "  Downloading jupyterlab_pygments-0.2.2-py2.py3-none-any.whl (21 kB)\n",
+      "Collecting lxml\n",
+      "  Downloading lxml-4.9.1.tar.gz (3.4 MB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.4/3.4 MB\u001b[0m \u001b[31m27.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m00:01\u001b[0m\n",
+      "\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25ldone\n",
+      "\u001b[?25hCollecting fastjsonschema\n",
+      "  Downloading fastjsonschema-2.16.1-py3-none-any.whl (22 kB)\n",
+      "Collecting jsonschema>=2.6\n",
+      "  Downloading jsonschema-4.14.0-py3-none-any.whl (82 kB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━���━━━━━━━━━━━\u001b[0m \u001b[32m82.4/82.4 kB\u001b[0m \u001b[31m3.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+      "\u001b[?25hRequirement already satisfied: six>=1.5 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.12->ipykernel>=4.5.1->ipywidgets<8,>=7) (1.16.0)\n",
+      "Collecting argon2-cffi-bindings\n",
+      "  Downloading argon2_cffi_bindings-21.2.0-cp38-abi3-macosx_10_9_universal2.whl (53 kB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m53.1/53.1 kB\u001b[0m \u001b[31m2.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+      "\u001b[?25hCollecting attrs>=17.4.0\n",
+      "  Downloading attrs-22.1.0-py2.py3-none-any.whl (58 kB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.8/58.8 kB\u001b[0m \u001b[31m2.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+      "\u001b[?25hCollecting pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0\n",
+      "  Downloading pyrsistent-0.18.1-cp310-cp310-macosx_10_9_universal2.whl (81 kB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m81.4/81.4 kB\u001b[0m \u001b[31m3.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+      "\u001b[?25hRequirement already satisfied: cffi>=1.0.1 in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from argon2-cffi-bindings->argon2-cffi->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8,>=7) (1.15.1)\n",
+      "Collecting soupsieve>1.2\n",
+      "  Downloading soupsieve-2.3.2.post1-py3-none-any.whl (37 kB)\n",
+      "Collecting webencodings\n",
+      "  Downloading webencodings-0.5.1-py2.py3-none-any.whl (11 kB)\n",
+      "Requirement already satisfied: pycparser in /Users/johnnydevriese/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages (from cffi>=1.0.1->argon2-cffi-bindings->argon2-cffi->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets<8,>=7) (2.21)\n",
+      "Building wheels for collected packages: lxml\n",
+      "  Building wheel for lxml (setup.py) ... \u001b[?25ldone\n",
+      "\u001b[?25h  Created wheel for lxml: filename=lxml-4.9.1-cp310-cp310-macosx_11_0_arm64.whl size=1423116 sha256=1e181d1aacf06a988fc9ff742b400e46f8f773abdef087dab0ac94e2435f8e60\n",
+      "  Stored in directory: /Users/johnnydevriese/Library/Caches/pip/wheels/a4/ec/7b/8acde6da24b5aabeee049213d5bec12d1e9214d3cae276387b\n",
+      "Successfully built lxml\n",
+      "Installing collected packages: webencodings, Send2Trash, mistune, ipython-genutils, fastjsonschema, tinycss2, terminado, soupsieve, pyrsistent, prometheus-client, pandocfilters, markupsafe, lxml, jupyterlab-widgets, jupyterlab-pygments, defusedxml, bleach, attrs, jsonschema, jinja2, beautifulsoup4, argon2-cffi-bindings, nbformat, argon2-cffi, nbclient, nbconvert, notebook, widgetsnbextension, ipywidgets\n",
+      "Successfully installed Send2Trash-1.8.0 argon2-cffi-21.3.0 argon2-cffi-bindings-21.2.0 attrs-22.1.0 beautifulsoup4-4.11.1 bleach-5.0.1 defusedxml-0.7.1 fastjsonschema-2.16.1 ipython-genutils-0.2.0 ipywidgets-7.7.2 jinja2-3.1.2 jsonschema-4.14.0 jupyterlab-pygments-0.2.2 jupyterlab-widgets-1.1.1 lxml-4.9.1 markupsafe-2.1.1 mistune-2.0.4 nbclient-0.6.7 nbconvert-7.0.0 nbformat-5.4.0 notebook-6.4.12 pandocfilters-1.5.0 prometheus-client-0.14.1 pyrsistent-0.18.1 soupsieve-2.3.2.post1 terminado-0.15.0 tinycss2-1.1.1 webencodings-0.5.1 widgetsnbextension-3.6.1\n"
+     ]
+    }
+   ],
+   "source": [
+    "!pip install diffusers==0.2.4\n",
+    "!pip install transformers scipy ftfy\n",
+    "!pip install \"ipywidgets>=7,<8\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "8f312208bf4744df84d15d15e956afb2",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "VBox(children=(HTML(value='<center> <img\\nsrc=https://huggingface.co/front/assets/huggingface_logo-noborder.sv…"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "from huggingface_hub import notebook_login\n",
+    "\n",
+    "notebook_login()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "GREP_COLOR=1;33\n",
+      "MANPATH=/opt/homebrew/share/man::\n",
+      "SHELL=/bin/zsh\n",
+      "HOMEBREW_REPOSITORY=/opt/homebrew\n",
+      "TMPDIR=/var/folders/4k/y4ljh2217c57vl68z1zkl0440000gn/T/\n",
+      "CONDA_SHLVL=2\n",
+      "PYTHONUNBUFFERED=1\n",
+      "CONDA_PROMPT_MODIFIER=(pytorch-1-12) \n",
+      "ORIGINAL_XDG_CURRENT_DESKTOP=undefined\n",
+      "MallocNanoZone=0\n",
+      "ZSH=/Users/johnnydevriese/.oh-my-zsh\n",
+      "PYTHONIOENCODING=utf-8\n",
+      "USER=johnnydevriese\n",
+      "LS_COLORS=di=34;40:ln=35;40:so=32;40:pi=33;40:ex=31;40:bd=34;46:cd=34;43:su=0;41:sg=0;46:tw=0;42:ow=0;43:\n",
+      "COMMAND_MODE=unix2003\n",
+      "CONDA_EXE=/Users/johnnydevriese/miniforge3/bin/conda\n",
+      "SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.QcecMP1ruy/Listeners\n",
+      "__CF_USER_TEXT_ENCODING=0x1F5:0x0:0x0\n",
+      "PAGER=cat\n",
+      "VSCODE_AMD_ENTRYPOINT=vs/workbench/api/node/extensionHostProcess\n",
+      "ELECTRON_RUN_AS_NODE=1\n",
+      "_CE_CONDA=\n",
+      "LSCOLORS=exfxcxdxbxegedabagacad\n",
+      "CONDA_PREFIX_1=/Users/johnnydevriese/miniforge3\n",
+      "PATH=/Users/johnnydevriese/miniforge3/envs/pytorch-1-12/bin:/Users/johnnydevriese/miniforge3/envs/pytorch-1-12/bin:/Users/johnnydevriese/miniforge3/condabin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/VMware Fusion Tech Preview.app/Contents/Public\n",
+      "LaunchInstanceID=82C5D7F6-FDB0-43D6-88D3-5FBAA6CD505A\n",
+      "CONDA_PREFIX=/Users/johnnydevriese/miniforge3/envs/pytorch-1-12\n",
+      "__CFBundleIdentifier=com.microsoft.VSCode\n",
+      "PWD=/Users/johnnydevriese/projects/machine_learning/code/diffusers\n",
+      "VSCODE_HANDLES_UNCAUGHT_ERRORS=true\n",
+      "XPC_FLAGS=0x0\n",
+      "_CE_M=\n",
+      "XPC_SERVICE_NAME=0\n",
+      "SHLVL=2\n",
+      "HOME=/Users/johnnydevriese\n",
+      "VSCODE_NLS_CONFIG={\"locale\":\"en-us\",\"availableLanguages\":{},\"_languagePackSupport\":true}\n",
+      "HOMEBREW_PREFIX=/opt/homebrew\n",
+      "CONDA_PYTHON_EXE=/Users/johnnydevriese/miniforge3/bin/python\n",
+      "LESS=-R\n",
+      "LOGNAME=johnnydevriese\n",
+      "VSCODE_IPC_HOOK=/Users/johnnydevriese/Library/Application Support/Code/1.70.2-main.sock\n",
+      "VSCODE_CODE_CACHE_PATH=/Users/johnnydevriese/Library/Application Support/Code/CachedData/e4503b30fc78200f846c62cf8091b76ff5547662\n",
+      "CONDA_DEFAULT_ENV=pytorch-1-12\n",
+      "VSCODE_PID=583\n",
+      "INFOPATH=/opt/homebrew/share/info:\n",
+      "HOMEBREW_CELLAR=/opt/homebrew/Cellar\n",
+      "VSCODE_CWD=/\n",
+      "SECURITYSESSIONID=186b3\n",
+      "LC_CTYPE=UTF-8\n",
+      "PYTHONPATH=/Users/johnnydevriese/.vscode/extensions/ms-toolsai.jupyter-2022.7.1102252217/pythonFiles:/Users/johnnydevriese/.vscode/extensions/ms-toolsai.jupyter-2022.7.1102252217/pythonFiles/lib/python\n",
+      "JUPYTER_PATH=/Users/johnnydevriese/.vscode/extensions/ms-toolsai.jupyter-2022.7.1102252217/temp/jupyter\n",
+      "PYDEVD_USE_FRAME_EVAL=NO\n",
+      "OLDPWD=/Users/johnnydevriese/projects/machine_learning/code/diffusers\n",
+      "CONDA_ROOT=/Users/johnnydevriese/miniforge3\n",
+      "JPY_PARENT_PID=7629\n",
+      "TERM=xterm-color\n",
+      "CLICOLOR=1\n",
+      "GIT_PAGER=cat\n",
+      "MPLBACKEND=module://matplotlib_inline.backend_inline\n",
+      "_=/usr/bin/env\n",
+      "PYTORCH_ENABLE_MPS_FALLBACK=1\n"
+     ]
+    }
+   ],
+   "source": [
+    "! env PYTORCH_ENABLE_MPS_FALLBACK=1"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "ftfy or spacy is not installed using BERT BasicTokenizer instead of ftfy.\n"
+     ]
+    },
+    {
+     "ename": "NotImplementedError",
+     "evalue": "The operator 'aten::index.Tensor' is not current implemented for the MPS device. If you want this op to be added in priority during the prototype phase of this feature, please comment on https://github.com/pytorch/pytorch/issues/77764. As a temporary fix, you can set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mNotImplementedError\u001b[0m                       Traceback (most recent call last)",
+      "\u001b[1;32m/Users/johnnydevriese/projects/machine_learning/code/diffusers/hf_diffusers.ipynb Cell 5\u001b[0m in \u001b[0;36m<cell line: 21>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/machine_learning/code/diffusers/hf_diffusers.ipynb#W1sZmlsZQ%3D%3D?line=18'>19</a>\u001b[0m prompt \u001b[39m=\u001b[39m \u001b[39m\"\u001b[39m\u001b[39ma photo of an astronaut riding a horse on mars\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[1;32m     <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/machine_learning/code/diffusers/hf_diffusers.ipynb#W1sZmlsZQ%3D%3D?line=19'>20</a>\u001b[0m \u001b[39m# with autocast(\"mps\"):\u001b[39;00m\n\u001b[0;32m---> <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/machine_learning/code/diffusers/hf_diffusers.ipynb#W1sZmlsZQ%3D%3D?line=20'>21</a>\u001b[0m image \u001b[39m=\u001b[39m pipe(prompt)[\u001b[39m\"\u001b[39m\u001b[39msample\u001b[39m\u001b[39m\"\u001b[39m][\u001b[39m0\u001b[39m]  \n\u001b[1;32m     <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/machine_learning/code/diffusers/hf_diffusers.ipynb#W1sZmlsZQ%3D%3D?line=22'>23</a>\u001b[0m image\u001b[39m.\u001b[39msave(\u001b[39m\"\u001b[39m\u001b[39mastronaut_rides_horse.png\u001b[39m\u001b[39m\"\u001b[39m)\n",
+      "File \u001b[0;32m~/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages/torch/autograd/grad_mode.py:27\u001b[0m, in \u001b[0;36m_DecoratorContextManager.__call__.<locals>.decorate_context\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m     24\u001b[0m \u001b[39m@functools\u001b[39m\u001b[39m.\u001b[39mwraps(func)\n\u001b[1;32m     25\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39mdecorate_context\u001b[39m(\u001b[39m*\u001b[39margs, \u001b[39m*\u001b[39m\u001b[39m*\u001b[39mkwargs):\n\u001b[1;32m     26\u001b[0m     \u001b[39mwith\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mclone():\n\u001b[0;32m---> 27\u001b[0m         \u001b[39mreturn\u001b[39;00m func(\u001b[39m*\u001b[39;49margs, \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n",
+      "File \u001b[0;32m~/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py:82\u001b[0m, in \u001b[0;36mStableDiffusionPipeline.__call__\u001b[0;34m(self, prompt, height, width, num_inference_steps, guidance_scale, eta, generator, output_type, **kwargs)\u001b[0m\n\u001b[1;32m     74\u001b[0m \u001b[39m# get prompt text embeddings\u001b[39;00m\n\u001b[1;32m     75\u001b[0m text_input \u001b[39m=\u001b[39m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mtokenizer(\n\u001b[1;32m     76\u001b[0m     prompt,\n\u001b[1;32m     77\u001b[0m     padding\u001b[39m=\u001b[39m\u001b[39m\"\u001b[39m\u001b[39mmax_length\u001b[39m\u001b[39m\"\u001b[39m,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m     80\u001b[0m     return_tensors\u001b[39m=\u001b[39m\u001b[39m\"\u001b[39m\u001b[39mpt\u001b[39m\u001b[39m\"\u001b[39m,\n\u001b[1;32m     81\u001b[0m )\n\u001b[0;32m---> 82\u001b[0m text_embeddings \u001b[39m=\u001b[39m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mtext_encoder(text_input\u001b[39m.\u001b[39;49minput_ids\u001b[39m.\u001b[39;49mto(\u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mdevice))[\u001b[39m0\u001b[39m]\n\u001b[1;32m     84\u001b[0m \u001b[39m# here `guidance_scale` is defined analog to the guidance weight `w` of equation (2)\u001b[39;00m\n\u001b[1;32m     85\u001b[0m \u001b[39m# of the Imagen paper: https://arxiv.org/pdf/2205.11487.pdf . `guidance_scale = 1`\u001b[39;00m\n\u001b[1;32m     86\u001b[0m \u001b[39m# corresponds to doing no classifier free guidance.\u001b[39;00m\n\u001b[1;32m     87\u001b[0m do_classifier_free_guidance \u001b[39m=\u001b[39m guidance_scale \u001b[39m>\u001b[39m \u001b[39m1.0\u001b[39m\n",
+      "File \u001b[0;32m~/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages/torch/nn/modules/module.py:1130\u001b[0m, in \u001b[0;36mModule._call_impl\u001b[0;34m(self, *input, **kwargs)\u001b[0m\n\u001b[1;32m   1126\u001b[0m \u001b[39m# If we don't have any hooks, we want to skip the rest of the logic in\u001b[39;00m\n\u001b[1;32m   1127\u001b[0m \u001b[39m# this function, and just call forward.\u001b[39;00m\n\u001b[1;32m   1128\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39mnot\u001b[39;00m (\u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_backward_hooks \u001b[39mor\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_forward_hooks \u001b[39mor\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_forward_pre_hooks \u001b[39mor\u001b[39;00m _global_backward_hooks\n\u001b[1;32m   1129\u001b[0m         \u001b[39mor\u001b[39;00m _global_forward_hooks \u001b[39mor\u001b[39;00m _global_forward_pre_hooks):\n\u001b[0;32m-> 1130\u001b[0m     \u001b[39mreturn\u001b[39;00m forward_call(\u001b[39m*\u001b[39;49m\u001b[39minput\u001b[39;49m, \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n\u001b[1;32m   1131\u001b[0m \u001b[39m# Do not call functions when jit is used\u001b[39;00m\n\u001b[1;32m   1132\u001b[0m full_backward_hooks, non_full_backward_hooks \u001b[39m=\u001b[39m [], []\n",
+      "File \u001b[0;32m~/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py:721\u001b[0m, in \u001b[0;36mCLIPTextModel.forward\u001b[0;34m(self, input_ids, attention_mask, position_ids, output_attentions, output_hidden_states, return_dict)\u001b[0m\n\u001b[1;32m    693\u001b[0m \u001b[39m@add_start_docstrings_to_model_forward\u001b[39m(CLIP_TEXT_INPUTS_DOCSTRING)\n\u001b[1;32m    694\u001b[0m \u001b[39m@replace_return_docstrings\u001b[39m(output_type\u001b[39m=\u001b[39mBaseModelOutputWithPooling, config_class\u001b[39m=\u001b[39mCLIPTextConfig)\n\u001b[1;32m    695\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39mforward\u001b[39m(\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    702\u001b[0m     return_dict: Optional[\u001b[39mbool\u001b[39m] \u001b[39m=\u001b[39m \u001b[39mNone\u001b[39;00m,\n\u001b[1;32m    703\u001b[0m ) \u001b[39m-\u001b[39m\u001b[39m>\u001b[39m Union[Tuple, BaseModelOutputWithPooling]:\n\u001b[1;32m    704\u001b[0m     \u001b[39mr\u001b[39m\u001b[39m\"\"\"\u001b[39;00m\n\u001b[1;32m    705\u001b[0m \u001b[39m    Returns:\u001b[39;00m\n\u001b[1;32m    706\u001b[0m \n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    719\u001b[0m \u001b[39m    >>> pooled_output = outputs.pooler_output  # pooled (EOS token) states\u001b[39;00m\n\u001b[1;32m    720\u001b[0m \u001b[39m    ```\"\"\"\u001b[39;00m\n\u001b[0;32m--> 721\u001b[0m     \u001b[39mreturn\u001b[39;00m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mtext_model(\n\u001b[1;32m    722\u001b[0m         input_ids\u001b[39m=\u001b[39;49minput_ids,\n\u001b[1;32m    723\u001b[0m         attention_mask\u001b[39m=\u001b[39;49mattention_mask,\n\u001b[1;32m    724\u001b[0m         position_ids\u001b[39m=\u001b[39;49mposition_ids,\n\u001b[1;32m    725\u001b[0m         output_attentions\u001b[39m=\u001b[39;49moutput_attentions,\n\u001b[1;32m    726\u001b[0m         output_hidden_states\u001b[39m=\u001b[39;49moutput_hidden_states,\n\u001b[1;32m    727\u001b[0m         return_dict\u001b[39m=\u001b[39;49mreturn_dict,\n\u001b[1;32m    728\u001b[0m     )\n",
+      "File \u001b[0;32m~/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages/torch/nn/modules/module.py:1130\u001b[0m, in \u001b[0;36mModule._call_impl\u001b[0;34m(self, *input, **kwargs)\u001b[0m\n\u001b[1;32m   1126\u001b[0m \u001b[39m# If we don't have any hooks, we want to skip the rest of the logic in\u001b[39;00m\n\u001b[1;32m   1127\u001b[0m \u001b[39m# this function, and just call forward.\u001b[39;00m\n\u001b[1;32m   1128\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39mnot\u001b[39;00m (\u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_backward_hooks \u001b[39mor\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_forward_hooks \u001b[39mor\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_forward_pre_hooks \u001b[39mor\u001b[39;00m _global_backward_hooks\n\u001b[1;32m   1129\u001b[0m         \u001b[39mor\u001b[39;00m _global_forward_hooks \u001b[39mor\u001b[39;00m _global_forward_pre_hooks):\n\u001b[0;32m-> 1130\u001b[0m     \u001b[39mreturn\u001b[39;00m forward_call(\u001b[39m*\u001b[39;49m\u001b[39minput\u001b[39;49m, \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n\u001b[1;32m   1131\u001b[0m \u001b[39m# Do not call functions when jit is used\u001b[39;00m\n\u001b[1;32m   1132\u001b[0m full_backward_hooks, non_full_backward_hooks \u001b[39m=\u001b[39m [], []\n",
+      "File \u001b[0;32m~/miniforge3/envs/pytorch-1-12/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py:656\u001b[0m, in \u001b[0;36mCLIPTextTransformer.forward\u001b[0;34m(self, input_ids, attention_mask, position_ids, output_attentions, output_hidden_states, return_dict)\u001b[0m\n\u001b[1;32m    652\u001b[0m last_hidden_state \u001b[39m=\u001b[39m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mfinal_layer_norm(last_hidden_state)\n\u001b[1;32m    654\u001b[0m \u001b[39m# text_embeds.shape = [batch_size, sequence_length, transformer.width]\u001b[39;00m\n\u001b[1;32m    655\u001b[0m \u001b[39m# take features from the eot embedding (eot_token is the highest number in each sequence)\u001b[39;00m\n\u001b[0;32m--> 656\u001b[0m pooled_output \u001b[39m=\u001b[39m last_hidden_state[torch\u001b[39m.\u001b[39;49marange(last_hidden_state\u001b[39m.\u001b[39;49mshape[\u001b[39m0\u001b[39;49m]), input_ids\u001b[39m.\u001b[39;49margmax(dim\u001b[39m=\u001b[39;49m\u001b[39m-\u001b[39;49m\u001b[39m1\u001b[39;49m)]\n\u001b[1;32m    658\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39mnot\u001b[39;00m return_dict:\n\u001b[1;32m    659\u001b[0m     \u001b[39mreturn\u001b[39;00m (last_hidden_state, pooled_output) \u001b[39m+\u001b[39m encoder_outputs[\u001b[39m1\u001b[39m:]\n",
+      "\u001b[0;31mNotImplementedError\u001b[0m: The operator 'aten::index.Tensor' is not current implemented for the MPS device. If you want this op to be added in priority during the prototype phase of this feature, please comment on https://github.com/pytorch/pytorch/issues/77764. As a temporary fix, you can set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS."
+     ]
+    }
+   ],
+   "source": [
+    "# make sure you're logged in with `huggingface-cli login`\n",
+    "from torch import autocast\n",
+    "from diffusers import StableDiffusionPipeline, LMSDiscreteScheduler\n",
+    "\n",
+    "lms = LMSDiscreteScheduler(\n",
+    "    beta_start=0.00085, \n",
+    "    beta_end=0.012, \n",
+    "    beta_schedule=\"scaled_linear\"\n",
+    ")\n",
+    "\n",
+    "pipe = StableDiffusionPipeline.from_pretrained(\n",
+    "    \"CompVis/stable-diffusion-v1-3\", \n",
+    "    scheduler=lms,\n",
+    "    torch_dtype=torch.float16, \n",
+    "    revision=\"fp16\",\n",
+    "    use_auth_token=True\n",
+    ").to(\"mps\")\n",
+    "\n",
+    "prompt = \"a photo of an astronaut riding a horse on mars\"\n",
+    "# with autocast(\"mps\"):\n",
+    "image = pipe(prompt)[\"sample\"][0]  \n",
+    "    \n",
+    "image.save(\"astronaut_rides_horse.png\")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3.10.6 ('pytorch-1-12')",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.6"
+  },
+  "orig_nbformat": 4,
+  "vscode": {
+   "interpreter": {
+    "hash": "d91b751b6cafe1e473109edab0583e459c2e471c181546b21e3fef0fb0f3aa3b"
+   }
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

code/eval_results.json

@@ -0,0 +1,8 @@
+{
+    "epoch": 1.0,
+    "eval_accuracy": 0.9774436090225563,
+    "eval_loss": 0.10330713540315628,
+    "eval_runtime": 10.9904,
+    "eval_samples_per_second": 12.101,
+    "eval_steps_per_second": 1.547
+}

code/fin_hackerrank.ipynb

@@ -0,0 +1,115 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "foo = [1, 2, 3, 3]\n",
+    "\n",
+    "for idx, value in enumerate(foo):\n",
+    "    print(\"value\", value)\n",
+    "    print(\"foo\", foo[:idx])\n",
+    "    print(\"index\", idx)\n",
+    "    print(\"sum\", sum(foo[:idx+1]))\n",
+    "\n",
+    "\n",
+    "sum(foo[:1])\n",
+    "\n",
+    "#!/bin/python3\n",
+    "\n",
+    "import math\n",
+    "import os\n",
+    "import random\n",
+    "import re\n",
+    "import sys\n",
+    "\n",
+    "\n",
+    "\n",
+    "#\n",
+    "# Complete the 'balancedSum' function below.\n",
+    "#\n",
+    "# The function is expected to return an INTEGER.\n",
+    "# The function accepts INTEGER_ARRAY arr as parameter.\n",
+    "#\n",
+    "\n",
+    "def balancedSum(arr):\n",
+    "    # Write your code here\n",
+    "    left_sum = 0 \n",
+    "    right_sum = 0\n",
+    "    for idx, value in enumerate(arr):\n",
+    "        # print(idx, value)\n",
+    "        # left_sum += value\n",
+    "        # print(\"this is left_sum\", left_sum)\n",
+    "        # print(\"this is the sum\", sum(arr[idx+2::]))\n",
+    "        # print(sum(arr[:idx+1]) ==  sum(arr[idx+2::]))\n",
+    "        # this way isn't fast enough. I suppose you could also just keep a running total \n",
+    "        if sum(arr[:idx+1]) == sum(arr[idx+2::]):\n",
+    "            return idx+1\n",
+    "        # if left_sum == sum(arr[idx+2::]):\n",
+    "            # print(\"success\")\n",
+    "            # return idx+1    \n",
+    "            \n",
+    "        \n",
+    "\n",
+    "if __name__ == '__main__':\n",
+    "    fptr = open(os.environ['OUTPUT_PATH'], 'w')\n",
+    "\n",
+    "    arr_count = int(input().strip())\n",
+    "\n",
+    "    arr = []\n",
+    "\n",
+    "    for _ in range(arr_count):\n",
+    "        arr_item = int(input().strip())\n",
+    "        arr.append(arr_item)\n",
+    "\n",
+    "    result = balancedSum(arr)\n",
+    "\n",
+    "    fptr.write(str(result) + '\\n')\n",
+    "\n",
+    "    fptr.close()\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "arr = [1, 2, 3, 3]\n",
+    "left_sum = 0 \n",
+    "right_sum = 0 \n",
+    "\n",
+    "for idx, value in enumerate(arr):\n",
+    "    if idx == 0: \n",
+    "        # initialize \n",
+    "        right_sum = sum(arr[idx+2::])\n",
+    "\n",
+    "    left_sum += value\n",
+    "    right_sum -= arr[idx+1]\n",
+    "\n",
+    "    # compare\n",
+    "\n",
+    "    if left_sum == right_sum:\n",
+    "        print(\"success\")\n",
+    "        print(idx)\n",
+    "\n",
+    "\n",
+    "    \n",
+    "    # print(\"value\", value)\n",
+    "    # print(\"foo\", foo[:idx])\n",
+    "    # print(\"index\", idx)\n",
+    "    # print(\"sum\", sum(foo[:idx+1]))\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "language_info": {
+   "name": "python"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

code/gcp_ml_engineer_questions.md

@@ -0,0 +1,226 @@
+### 1. You work for a textile manufacturer and have been asked to build a model to detect and classify fabric defects. You trained a machine learning model with high recall based on high resolution images taken at the end of the production line. You want quality control inspectors to gain trust in your model. Which technique should you use to understand the rationale of your classifier?
+
+
+A. Use K-fold cross validation to understand how the model performs on different test datasets.
+
+B. Use the Integrated Gradients method to efficiently compute feature attributions for each predicted image.
+
+C. Use PCA (Principal Component Analysis) to reduce the original feature set to a smaller set of easily understood features.
+ 
+D. Use k-means clustering to group similar images together, and calculate the Davies-Bouldin index to evaluate the separation between clusters.
+
+<details>
+  <summary>Answer</summary>
+
+    A is not correct because K-fold cross validation offers no explanation on the predictions made by the model.
+    B is correct because it identifies the pixel of the input image that leads to the classification of the image itself.
+    C is not correct because PCA simplifies higher dimensional datasets but offers no added benefit to the scenario.
+    D is not correct because clustering images does not provide any insight into why the classification model made the predictions that it did.
+</details>
+
+
+### 2. You need to write a generic test to verify whether Dense Neural Network (DNN) models automatically released by your team have a sufficient number of parameters to learn the task for which they were built. What should you do?
+
+
+A. Train the model for a few iterations, and check for NaN values.
+
+B. Train the model for a few iterations, and verify that the loss is constant.
+
+C. Train a simple linear model, and determine if the DNN model outperforms it.
+ 
+D. Train the model with no regularization, and verify that the loss function is close to zero.
+
+<details> 
+    <summary>Answer</summary>
+
+    A is not correct because the test does not check that the model has enough parameters to learn the task.
+    B is not correct because the loss should decrease if you have enough parameters to learn the task.
+    C is not correct because outperforming the linear model does not guarantee that the model has enough parameters to learn tasks with non-linear data representations. The option also doesn’t quantify a metric to give an indication of how well the model performed.
+    D is correct because the test can check that the model has enough parameters to memorize the task.
+</details>
+
+
+### 3. Your team is using a TensorFlow Inception-v3 CNN model pretrained on ImageNet for an image classification prediction challenge on 10,000 images. You will use AI Platform to perform the model training. What TensorFlow distribution strategy and AI Platform training job configuration should you use to train the model and optimize for wall-clock time?
+
+
+A. Default Strategy; Custom tier with a single master node and four v100 GPUs.
+
+B. One Device Strategy; Custom tier with a single master node and four v100 GPUs.
+
+C. One Device Strategy; Custom tier with a single master node and eight v100 GPUs.
+ 
+D. MirroredStrategy; Custom tier with a single master node and four v100 GPUs.
+
+
+<details>
+  <summary>Answer</summary>
+
+    A is not correct because Default Strategy does not distribute training across multiple devices.
+    B is not correct because One Device Strategy does not distribute training across multiple devices.
+    C is not correct because One Device Strategy does not distribute training across multiple devices.
+    D is correct because this is the only strategy that can perform distributed training; albeit there is only a single copy of the variables on the CPU host.
+</details>
+
+
+### 4. You work on a team where the process for deploying a model into production starts with data scientists training different versions of models in a Kubeflow pipeline. The workflow then stores the new model artifact into the corresponding Cloud Storage bucket. You need to build the next steps of the pipeline after the submitted model is ready to be tested and deployed in production on AI Platform. How should you configure the architecture before deploying the model to production?
+
+
+A. Deploy model in test environment -> Evaluate and test model -> Create a new AI Platform model version
+ 
+B. Validate model -> Deploy model in test environment -> Create a new AI Platform model version
+
+C. Create a new AI Platform model version -> Evaluate and test model -> Deploy model in test environment
+
+D. Create a new AI Platform model version - > Deploy model in test environment -> Validate model
+
+
+<details>
+  <summary>Answer</summary>
+    A is correct because the model can be validated after it is deployed to the test environment, and the release version is established before the model is deployed in production.
+    B is not correct because the model cannot be validated before being deployed to the test environment.
+    C is not correct because the model version is being set up for the release candidate before the model is validated. Moreover, the model cannot be validated before being deployed to the test environment.
+    D is not correct because the model version is being set up for the release candidate before the model is validated.
+</details>
+
+ 
+### 5. You work for a maintenance company and have built and trained a deep learning model that identifies defects based on thermal images of underground electric cables. Your dataset contains 10,000 images, 100 of which contain visible defects. How should you evaluate the performance of the model on a test dataset?
+
+
+A. Calculate the Area Under the Curve (AUC) value.
+ 
+B. Calculate the number of true positive results predicted by the model.
+
+C. Calculate the fraction of images predicted by the model to have a visible defect.
+
+D. Calculate the Cosine Similarity to compare the model’s performance on the test dataset to the model’s performance on the training dataset.
+
+
+<details>
+  <summary>Answer</summary>
+
+    A is correct because it is scale-invariant. AUC measures how well predictions are ranked, rather than their absolute values. AUC is also classification-threshold invariant. It measures the quality of the model's predictions irrespective of what classification threshold is chosen.
+    B is incorrect because calculating the number of true positives without considering false positives can lead to misleading results. For instance, the model could classify nearly every image as a defect. This would result in many true positives, but the model would in fact be a very poor discriminator.
+    C is incorrect because merely calculating the fraction of images that contain defects doesn’t indicate whether your model is accurate or not.
+    D is incorrect because this metric is more commonly used in distance-based models (e.g., K Nearest Neighbors). This isn’t an appropriate metric for checking the performance of an image classification model.
+</details>
+
+
+
+### 6. You work for a manufacturing company that owns a high-value machine which has several machine settings and multiple sensors. A history of the machine’s hourly sensor readings and known failure event data are stored in BigQuery. You need to predict if the machine will fail within the next 3 days in order to schedule maintenance before the machine fails. Which data preparation and model training steps should you take?
+
+
+A. Data preparation: Daily max value feature engineering; Model training: AutoML classification with BQML
+ 
+B. Data preparation: Daily min value feature engineering; Model training: Logistic regression with BQML and AUTO_CLASS_WEIGHTS set to True
+
+C. Data preparation: Rolling average feature engineering; Model training: Logistic regression with BQML and AUTO_CLASS_WEIGHTS set to False
+
+D. Data preparation: Rolling average feature engineering; Model training: Logistic regression with BQML and AUTO_CLASS_WEIGHTS set to True
+
+<details>
+  <summary>Answer</summary>
+
+    A is not correct because a rolling average is a better feature engineering technique, as it will smooth out the noise and fluctuation in the data to demonstrate whether there is a trend. Using the max value could be an artifact of some noise and may not capture the trend accurately.
+    B is not correct because a rolling average is a better feature engineering technique, as it will smooth out the noise and fluctuation in the data to demonstrate whether there is a trend. Using the min value could be an artifact of some noise and may not capture the trend accurately.
+    C is not correct because the model training does not balance class labels for an imbalanced dataset.
+    D is correct because it uses the rolling average of the sensor data and balances the weights using the BQML auto class weight balance parameter.
+</details>
+
+
+### 7. You are an ML engineer at a media company. You want to use machine learning to analyze video content, identify objects, and alert users if there is inappropriate content. Which Google Cloud products should you use to build this project?
+
+A. Pub/Sub, Cloud Function, Cloud Vision API
+
+B. Pub/Sub, Cloud IoT, Dataflow, Cloud Vision API, Cloud Logging
+ 
+C. Pub/Sub, Cloud Function, Video Intelligence API, Cloud Logging
+
+D. Pub/Sub, Cloud Function, AutoML Video Intelligence, Cloud Logging
+
+<details>
+  <summary>Answer</summary>
+
+    A is not correct as there is no tool for alerting and notifying.
+    B is not correct as it uses Cloud Vision API for processing videos.
+    C is correct as Video Intelligence API can find inappropriate components and other components satisfy the requirements of real-time processing and notification. (https://cloud.google.com/video-intelligence)
+    D is not correct as AutoML Video intelligence should be only used in case of customization.
+</details>
+
+### 8.  You work for a large retailer. You want to use ML to forecast future sales leveraging 10 years of historical sales data. The historical data is stored in Cloud Storage in Avro format. You want to rapidly experiment with all the available data. How should you build and train your model for the sales forecast?
+
+
+A. Load data into BigQuery and use the ARIMA model type on BigQuery ML.
+ 
+B. Convert the data into CSV format and create a regression model on AutoML Tables.
+
+C. Convert the data into TFRecords and create an RNN model on TensorFlow on AI Platform Notebooks.
+
+D. Convert and refactor the data into CSV format and use the built-in XGBoost algorithm on AI Platform Training.
+
+<details>
+  <summary>Answer</summary>
+
+    A is correct because BigQuery ML is designed for fast and rapid experimentation and it is possible to use federated queries to read data directly from Cloud Storage. Moreover, ARIMA is considered one of the best in class for time series forecasting.
+    B is not correct because AutoML Tables is not ideal for fast iteration and rapid experimentation. Even if it does not require data cleanup and hyperparameter tuning, it takes at least one hour to create a model.
+    C is not correct because in order to build a custom TensorFlow model, you would still need to do data cleanup and hyperparameter tuning.
+    D is not correct because using AI Platform Training requires preprocessing your data in a particular CSV structure and it is not ideal for fast iteration, as training times can take a long time because it cannot be distributed on multiple machines.
+</details>
+
+
+
+### 9. You need to build an object detection model for a small startup company to identify if and where the company’s logo appears in an image. You were given a large repository of images, some with logos and some without. These images are not yet labelled. You need to label these pictures, and then train and deploy the model. What should you do?
+
+A. Use Google Cloud’s Data Labelling Service to label your data. Use AutoML Object Detection to train and deploy the model.
+
+B. Use Vision API to detect and identify logos in pictures and use it as a label. Use AI Platform to build and train a convolutional neural network.
+
+C. Create two folders: one where the logo appears and one where it doesn’t. Manually place images in each folder. Use AI Platform to build and train a convolutional neural network.
+ 
+D. Create two folders: one where the logo appears and one where it doesn’t. Manually place images in each folder. Use AI Platform to build and train a real time object detection model.
+
+<details>
+  <summary>Answer</summary>
+
+    A is correct as this will allow you to easily create a request for a labelling task and deploy a high-performance model.
+    B is not correct because Vision API is not guaranteed to work with any company logos, and in the statement it explicitly mentions a small startup, which will further decrease the chance of success.
+    C is not correct because the task of manually labelling the data is time consuming and should be avoided if possible.
+    D is not correct because the task of labelling object detection data is very tedious, and real time object detection is designed detecting objects in videos rather than in images.
+</details>
+
+### 10. You work for a large financial institution that is planning to use Dialogflow to create a chatbot for the company’s mobile app. You have reviewed old chat logs and tagged each conversation for intent based on each customer’s stated intention for contacting customer service. About 70% of customer inquiries are simple requests that are solved within 10 intents. The remaining 30% of inquiries require much longer and more complicated requests. Which intents should you automate first?
+
+A. Automate a blend of the shortest and longest intents to be representative of all intents.
+
+B. Automate the more complicated requests first because those require more of the agents’ time.
+
+C. Automate the 10 intents that cover 70% of the requests so that live agents can handle the more complicated requests.
+ 
+D. Automate intents in places where common words such as “payment” only appear once to avoid confusing the software.
+
+<details>
+  <summary>Answer</summary>
+
+    A is incorrect because you should not automate the higher value requests.
+    B is incorrect because live agents are better suited to handle these complicated requests.
+    C is correct because it enables a machine to handle the most simple requests and gives the live agents more opportunity to handle higher value requests.
+    D is incorrect because Dialogflow can handle the same word in multiple intents.
+</details>
+
+### 11. You work for a gaming company that develops and manages a popular massively multiplayer online (MMO) game. The game’s environment is open-ended, and a large number of positions and moves can be taken by a player. Your team has developed an ML model with TensorFlow that predicts the next move of each player. Edge deployment is not possible, but low-latency serving is required. How should you configure the deployment?
+
+A. Use a Cloud TPU to optimize model training speed.
+
+B. Use AI Platform Prediction with a NVIDIA GPU to make real-time predictions.
+ 
+C. Use AI Platform Prediction with a high-CPU machine type to get a batch prediction for the players.
+
+D. Use AI Platform Prediction with a high-memory machine type to get a batch prediction for the players.
+
+<details>
+  <summary>Answer</summary>
+
+    A is not correct because changing the training will not improve the prediction latency.
+    B is correct because using a VM with a GPU and NVIDIA drivers enables you to use TensorRT. NVIDIA has developed TensorRT (an inference optimization library) for high-performance inference on GPUs. Google Cloud’s Deep Learning VMs are ideal for this case because they have everything you need pre-installed.
+    B is not correct because batch jobs do not satisfy the low-latency requirements for an online multiplayer game.
+    D is not correct because batch jobs do not satisfy the low-latency requirements for an online multiplayer game.
+</details>

code/gcp_ml_engineer_tips.md

@@ -0,0 +1,626 @@
+### Tips 
+
+from https://towardsdatascience.com/how-i-passed-the-gcp-professional-ml-engineer-certification-47104f40bec5
+
+General
+
+Typical Big Data pipeline for streaming data:
+
+Pub/Sub -> Dataflow -> BigQuery or Cloud Storage
+
+Typical Big Data pipeline for batch data:
+
+Pub/Sub -> Cloud Run or Cloud Functions -> Dataflow -> BigQuery or Cloud Storage
+
+* Use the general use APIs by default (Vision, Video Intelligence, Natural Language…). Only use AutoML if you have custom needs (custom labels, etc.)
+* To de-identify sensible data, you can redact, tokenize or hash, using BigQuery, Cloud Storage, Datastore, or Data Loss Protection (DLP)
+* Difference between TensorBoard and TensorFlow Model Analysis: the former evaluates during training, based on mini-batches, while the latter evalutes after training, can be done in slices of data and is based on the full data
+* AI Explanations: with tabular data, you can use Shapely or integrated ingredients for large feature spaces; with images, you can use integrated gradients for pixel-level explanations or XRAI for region-level explanations.
+* When to use Kubeflow over TFX? When you need PyTorch, XGBoost or if you want to dockerize every step of the flow
+* Keras: use the Sequential API by default. If you have multiple inputs or outputs, layer sharing or a non-linear topology, change to the Functional API, unless you have a RNN. If that is the case, Keras Subclasses instead
+* 3 methods for optimizing TensorFlow pipelines: prefetch, interleave and cache
+
+
+### BigQuery ML
+
+It supports the following types of model: linear regression, binary and multiclass logistic regression, k-means, matrix factorization, time series, boosted trees, deep neural networks, AutoML models and imported TensorFlow models
+Use it for quick and easy models, prototyping etc.
+
+### Storage
+
+Choosing storage for analytics:
+
+Structured data: Bigtable for millisecond latency, BigQuery for latency in seconds
+Unstructured: use Cloud Storage by default, and Firebase storage for mobile
+
+### Accelerators
+
+Choosing between CPUs, TPUs and GPUs:
+
+Use CPUs for quick prototypes, simple/small models or if you have many C++ custom operations; use GPU if you have some custom C++ operations and/or medium to large models; use TPUs for big matrix computations, no custom TensorFlow operations and/or very large models that train for weeks or months
+
+To improve performance on TPUs: if data pre-processing is a bottleneck, do it offline as a one-time cost; choose the larges batch size that fits in memory; keep the per-core batch size the same
+
+### Neural networks
+
+Common pitfalls in backpropagation and their solutions:
+
+vanishing gradients -> use ReLu
+exploding gradients -> use batch normalization
+ReLu layers are dying -> lower learning rates
+
+For multiclass classification, if:
+
+labels and probabilities are mutually exclusive, use softmax_cross_entropy_with_logits_v2
+labels are mutually exclusive, but not probabilities, use sparse_softmax_cross_entropy_with_logits
+labels are not mutually exclusive, use sigmoid_cross_entropy_with_logits
+
+# Learning Stuff 
+
+
+### Labs 
+
+ Recommending Products Using Cloud SQL and Spark 
+https://www.cloudskillsboost.google/course_sessions/554292/labs/102245
+
+
+
+```bash
+echo "Authorizing Cloud Dataproc to connect with Cloud SQL"
+CLUSTER=rentals
+CLOUDSQL=rentals
+ZONE=us-central1-f
+NWORKERS=2
+machines="$CLUSTER-m"
+for w in `seq 0 $(($NWORKERS - 1))`; do
+   machines="$machines $CLUSTER-w-$w"
+done
+echo "Machines to authorize: $machines in $ZONE ... finding their IP addresses"
+ips=""
+for machine in $machines; do
+    IP_ADDRESS=$(gcloud compute instances describe $machine --zone=$ZONE --format='value(networkInterfaces.accessConfigs[].natIP)' | sed "s/\['//g" | sed "s/'\]//g" )/32
+    echo "IP address of $machine is $IP_ADDRESS"
+    if [ -z  $ips ]; then
+       ips=$IP_ADDRESS
+    else
+       ips="$ips,$IP_ADDRESS"   
+    fi
+done
+echo "Authorizing [$ips] to access cloudsql=$CLOUDSQL"
+gcloud sql instances patch $CLOUDSQL --authorized-networks $ips
+```
+
+
+### Recommending Products Using Cloud SQL and Spark -- Module Test
+
+1. True or False: Cloud SQL is a big data analytics warehouse
+
+Answer: False -- Correct - Cloud SQL is a transaction RDBMS or relational database management system. It is designed for many more WRITES than READS.Whereas BigQuery is a big data analytics warehouse which is optimized for reporting READS.
+
+2. 
+Cloud SQL and Cloud Dataproc offer familiar tools (MySQL and Hadoop/Pig/Hive/Spark). What is the value-add provided by Google Cloud Platform? (Select the 2 correct options below )
+ 
+
+* Google-proprietary extensions and bug fixes to MySQL, Hadoop, and so on
+
+* It’s the same API, but Google implements it better
+
+
+* Fully-managed versions of the software offer no-ops
+Yes. No-ops is the main value-add here.
+
+* Running it on Google infrastructure offers reliability and cost savings
+Yes. You pay only for the resources you use. Cloud SQL can be shut down when it’s not being used. Hadoop clusters can be of preemptible nodes, and so on.
+
+3. You are thinking about migrating your Hadoop workloads to the cloud and you have a few workloads that are fault-tolerant (they can handle interruptions of individual VMs gracefully). What are some architecture considerations you should explore in the cloud? Choose all that apply
+
+* You are thinking about migrating your Hadoop workloads to the cloud and you have a few workloads that are fault-tolerant (they can handle interruptions of individual VMs gracefully). What are some architecture considerations you should explore in the cloud? Choose all that apply
+
+
+* Migrate your storage from on-cluster HDFS to off-cluster Google Cloud Storage (GCS)
+Correct!
+
+* Use PVMs or Preemptible Virtual Machines
+Correct!
+
+* Consider having multiple Cloud Dataproc instances for each priority workload and then turning them down when not in use
+Correct!
+
+
+4. True or False: If you are migrating your Hadoop workload to the cloud, you must first rewrite all your Spark jobs to be compliant with the cloud.
+
+Answer: False -- Correct - you can run your same Spark job code running on the same Hadoop software but running on cloud hardware with Cloud Dataproc.
+
+
+5. Complete the following: You should feed your machine learning model your _______ and not your _______. It will learn those for itself!
+
+data, rules 
+
+6. Relational databases are a good choice when you need:
+
+* Fast queries on terabytes of data
+
+* Streaming, high-throughput writes
+
+* Aggregations on unstructured data
+
+* Transactional updates on relatively small datasets -- correct 
+
+7. Google Cloud Storage is a good option for storing data that: (Select the 2 correct options below).
+
+* Will be accessed frequently and updated constantly with new transactions from a front-end and needs to be stored in a relational database
+
+* Is ingested in real-time from sensors and other devices and supports SQL-based queries
+
+
+* May be required to be read at some later time (i.e. load a CSV file into BigQuery) -- correct 
+
+* May be imported from a bucket into a Hadoop cluster for analysis -- correct 
+
+
+
+
+### Lab -- Creating a Streaming Data Pipeline for a Real-Time Dashboard with Dataflow 
+
+
+Task 1. Create a Pub/Sub topic and BigQuery dataset
+Task 2. Create a Cloud Storage bucket
+Task 3. Set up a Dataflow Pipeline
+Task 4. Analyze the taxi data using BigQuery
+Task 5. Perform aggregations on the stream for reporting
+Task 6. Create a real-time dashboard
+Task 7. Create a time series dashboard
+Task 8. Stop the Dataflow job
+
+
+biggest thing is creating datflow pipeline from template and creating aggregate in bigquery 
+
+Task 3. Set up a Dataflow Pipeline
+
+Dataflow is a serverless way to carry out data analysis. In this lab, you set up a streaming data pipeline to read sensor data from Pub/Sub, compute the maximum temperature within a time window, and write this out to BigQuery.
+
+    In the Cloud Console, go to Navigation menu > Dataflow.
+
+    In the top menu bar, click CREATE JOB FROM TEMPLATE.
+
+    Enter streaming-taxi-pipeline as the Job name for your Dataflow job.
+
+    Under Dataflow template, select the Pub/Sub Topic to BigQuery template.
+
+    Under Input Pub/Sub topic, enter projects/pubsub-public-data/topics/taxirides-realtime
+
+    Under BigQuery output table, enter <myprojectid>:taxirides.realtime
+    
+    Under Temporary location, enter gs://<mybucket>/tmp/
+
+
+And then use this SQL query to make aggregates
+
+
+```sql 
+WITH streaming_data AS (
+SELECT
+  timestamp,
+  TIMESTAMP_TRUNC(timestamp, HOUR, 'UTC') AS hour,
+  TIMESTAMP_TRUNC(timestamp, MINUTE, 'UTC') AS minute,
+  TIMESTAMP_TRUNC(timestamp, SECOND, 'UTC') AS second,
+  ride_id,
+  latitude,
+  longitude,
+  meter_reading,
+  ride_status,
+  passenger_count
+FROM
+  taxirides.realtime
+WHERE ride_status = 'dropoff'
+ORDER BY timestamp DESC
+LIMIT 100000
+)
+# calculate aggregations on stream for reporting:
+SELECT
+ ROW_NUMBER() OVER() AS dashboard_sort,
+ minute,
+ COUNT(DISTINCT ride_id) AS total_rides,
+ SUM(meter_reading) AS total_revenue,
+ SUM(passenger_count) AS total_passengers
+FROM streaming_data
+GROUP BY minute, timestamp
+```
+
+
+### Perform Foundational Data, ML, and AI Tasks in Google Cloud: Challenge Lab  (Expert) Lab
+
+
+Create a simple Dataproc job
+Create a simple DataFlow job
+Create a simple Dataprep job
+Perform one of the three Google machine learning backed API tasks
+
+Task 4: AI
+
+Complete one of the tasks below, YOUR_PROJECT must be replaced with your lab project name.
+
+    Use Google Cloud Speech API to analyze the audio file gs://cloud-training/gsp323/task4.flac. Once you have analyzed the file you can upload the resulting analysis to gs://YOUR_PROJECT-marking/task4-gcs.result.
+
+    Use the Cloud Natural Language API to analyze the sentence from text about Odin. The text you need to analyze is "Old Norse texts portray Odin as one-eyed and long-bearded, frequently wielding a spear named Gungnir and wearing a cloak and a broad hat." Once you have analyzed the text you can upload the resulting analysis to gs://YOUR_PROJECT-marking/task4-cnl.result.
+
+    Use Google Video Intelligence and detect all text on the video gs://spls/gsp154/video/train.mp4. Once you have completed the processing of the video, pipe the output into a file and upload to gs://YOUR_PROJECT-marking/task4-gvi.result. Ensure the progress of the operation is complete and the service account you're uploading the output with has the Storage Object Admin role.
+
+
+
+### Invoking ML APIs from AI Platform Notebooks (jupyter notebook) labs 
+
+https://www.cloudskillsboost.google/course_sessions/570479/labs/102982
+
+REAlly cool to see basic usage of some crazy powerful APIs! 
+
+Also noticed there is a new book out (put in amazon cart) for learning about AI on GCP.
+
+
+### cloud natural language 
+
+
+score of the sentiment ranges between -1.0 (negative) and 1.0 (positive) and corresponds to the overall emotional leaning of the text.
+
+magnitude indicates the overall strength of emotion (both positive and negative) within the given text, between 0.0 and +inf. Unlike score, magnitude is not normalized; each expression of emotion within the text (both positive and negative) contributes to the text's magnitude (so longer text blocks may have greater magnitudes). 
+
+### LAB  Analyzing data using AI Platform Notebooks and BigQuery
+
+In this lab, you analyze a large (70 million rows, 8 GB) airline dataset using BigQuery and AI Platform Notebooks.
+
+Looking at flights and presenter points out how powerful it is to be able to make aggregates in bigquery and then analyze them later in notebooks. 
+
+for example we have 70M (8GB) records in big query that we then create an aggregate of and can actually plot these in our little Jupyter notebook for much cheaper. 
+
+### LAB Improving Data Quality 
+
+Machine learning models can only consume numeric data, and that numeric data should be 1s or 0s. Data is said to be messy or untidy if it is missing attribute values, contains noise or outliers, has duplicates, wrong data, or upper/lower case column names, or is essentially not ready for ingestion by a machine learning algorithm.
+
+In this lab, you will present and solve some of the most common issues of untidy data. Note that different problems will require different methods, and they are beyond the scope of this notebook.
+
+What you learn
+
+In this lab, you will:
+
+    Resolve missing values.
+
+    Convert the Date feature column to a datetime format.
+
+    Rename a feature column, remove a value from a feature column.
+
+    Create one-hot encoding features.
+
+    Understand temporal feature conversions.
+
+
+In the notebook interface, navigate to **training-data-analyst > courses > machine_learning > deepdive2 > launching_into_ml > labs, and open improve_data_quality.ipynb** 
+
+
+Solutions notebook 
+
+https://github.com/GoogleCloudPlatform/training-data-analyst/blob/master/courses/machine_learning/deepdive2/launching_into_ml/solutions/improve_data_quality.ipynb
+
+
+
+#### Data Quality Issue #5:
+Temporal Feature Columns
+
+
+Our dataset now contains year, month, and day feature columns. Let's convert the month and day feature columns to meaningful representations as a way to get us thinking about changing temporal features -- as they are sometimes overlooked.
+
+Note that the Feature Engineering course in this Specialization will provide more depth on methods to handle year, month, day, and hour feature columns.
+
+
+```python
+# Here we map each temporal variable onto a circle such that the lowest value for that variable appears right next to the largest value. We compute the x- and y- component of that point using the sin and cos trigonometric functions.
+df['day_sin'] = np.sin(df.day*(2.*np.pi/31))
+df['day_cos'] = np.cos(df.day*(2.*np.pi/31))
+df['month_sin'] = np.sin((df.month-1)*(2.*np.pi/12))
+df['month_cos'] = np.cos((df.month-1)*(2.*np.pi/12))
+
+# Let's drop month, and day
+# TODO 5
+df = df.drop(['month','day','year'], axis=1)
+```
+
+
+###  Exploratory Data Analysis Using Python and BigQuery (LAB)
+
+In the notebook interface, navigate to training-data-analyst > courses > machine_learning > deepdive2 > launching_into_ml > labs and open python.BQ_explore_data.ipynb.
+
+
+### Improve Data Quality - Quiz
+
+1. Which of the following refers to the Orderliness of data?
+
+
+The data record with specific details appears only once in the database
+The data represents reality within a reasonable period
+None of the above
+x - The data entered has the required format and structure
+
+2. Which of the following are categories of data quality tools?
+
+Cleaning tools 
+Monitoring tools
+Both A and B 
+None of the Above
+
+3. What are the features of low data quality?
+Unreliable info
+Duplicated data
+Incomplete data 
+All of the above
+
+4. Which of the following are best practices for data quality management?
+Resolving missing values
+Automating data entry
+Preventing duplicates 
+All of the above
+
+
+5. Which of the following is not a Data Quality attribute?
+Consistency
+Auditability
+Accuracy
+x - redundancy 
+
+
+### Exploratory Data Analysis Using Python and BigQuery - LAB 
+
+https://github.com/GoogleCloudPlatform/training-data-analyst/blob/master/courses/machine_learning/deepdive2/launching_into_ml/solutions/python.BQ_explore_data.ipynb
+
+### Quiz: Exploratory Data Analysis 
+
+1. Which of the following is not true about Exploratory Data Analysis?
+
+
+Discovers new knowledge.
+Generates a posteriori hypothesis.
+Does not provide insight into the data. - x
+Deals with unknowns. 
+
+
+2. Exploratory Data Analysis is majorly performed using the following methods: 
+Bivariate
+Univariate
+both A & B -x
+None of the above
+
+3. What are the objectives of exploratory data analysis?
+
+Gain maximum insight into the data set and its underlying structure.
+Check for missing data and other mistakes.
+Uncover a parsimonious model, one which explains the data with a minimum number of predictor variables.
+All of the above - x
+
+
+
+4. Which of the following is not a component of Exploratory Data Analysis?
+
+Anomaly Detection
+Accounting and Summarizing
+Statistical Analysis and Clustering
+Hyperparameter tuning - x 
+
+5. Which is the correct sequence of steps in data analysis and data visualisation of Exploratory Data Analysis?  
+
+Data Exploration -> Data Cleaning -> Model Building -> Present Results - x
+Data Exploration -> Data Cleaning -> Present Results -> Model Building
+Data Exploration -> Model Building -> Present Results -> Data Cleaning
+Data Exploration -> Model Building -> Data Cleaning -> Present Results
+
+### Quiz: Supervised Learning 
+
+1. Which model would you use if your problem required a discrete number of values or classes?
+
+Regression Model 
+Classification Model - x
+Supervised Model 
+Unsupervised Model 
+
+
+2. Which of the following machine learning models have labels, or in other words, the correct answers to whatever it is that we want to learn to predict? 
+
+Unsupervised Model
+None of the above.
+Reinforcement Model
+Supervised Model - x
+
+3. Which statement is true?
+
+Depending on the problem you are trying to solve, the data you have, explainability, etc. will not determine which machine learning methods you use to find a solution.
+None of the above
+Determining which machine learning methods you use to find a solution depends only on the problem or hypothesis.
+Depending on the problem you are trying to solve, the data you have, explainability, etc. will determine which machine learning methods you use to find a solution. - x
+
+4. What is a type of Supervised machine learning model?
+
+Regression model 
+Classification model 
+Both A & B - x
+None of the above
+
+5. When the data isn’t labelled, what is an alternative way of predicting the output? 
+
+Clustering Algorithms -x 
+Logistic Regression 
+Linear Regression 
+None of the above 
+
+
+
+
+###  Introduction to Linear Regression 
+
+training-data-analyst > courses > machine_learning > deepdive2 > launching_into_ml > Labs and open intro_linear_regression.ipynb.
+
+https://github.com/GoogleCloudPlatform/training-data-analyst/blob/master/courses/machine_learning/deepdive2/launching_into_ml/solutions/intro_linear_regression.ipynb
+
+###  Quiz: Neural Networks 
+
+1. Which activation functions are needed to get the complex chain functions that allow neural networks to learn data distributions.
+
+Nonlinear activation functions - x
+Linear activation functions 
+All of the above 
+none of the above
+
+2. A single unit for a non-input neuron has ____________________ a/an
+
+Output of the activation function
+Activation function
+Weighted Sum
+all of the above - x
+
+3. Which of the following activation functions are used for nonlinearity?
+
+Tanh
+Hyperbolic tangent
+Sigmoid
+All of the above - x
+
+
+4. Which activation function has a range between zero and Infinity?
+
+ReLU - x
+Tanh
+Sigmoid
+ELU
+
+5. If we wanted our outputs to be in the form of probabilities, which activation function should I choose in the final layer?
+
+ReLU
+Tanh
+Sigmoid - x  
+ELU 
+
+### Decision trees and Random Forests LAB 
+
+https://github.com/GoogleCloudPlatform/training-data-analyst/blob/master/courses/machine_learning/deepdive2/launching_into_ml/solutions/decision_trees_and_random_Forests_in_Python.ipynb
+
+###  Quiz: Decision Trees AND Random Forests 
+
+1. In a decision classification tree, what does each decision or node consist of? 
+
+Euclidean distance minimizer
+Mean squared error minimizer
+Linear classifier of one feature - x
+ Linear classifier of all features 
+
+2. Which of the following statements is true?
+
+Mean squared error minimizer and euclidean distance minimizer are used in classification, not regression. 
+Mean squared error minimizer and euclidean distance minimizer are used in regression, not classification. - x
+Mean squared error minimizer and euclidean distance minimizer are not used in regression and classification.
+Mean squared error minimizer and euclidean distance minimizer are used in regression and classification.
+
+3. Decision trees are one of the most intuitive machine learning algorithms. They can be used for which of the following?
+
+Regression
+Classification
+Both A & B -x 
+None of the above 
+
+
+4. A random forest is usually more complex than an individual decision tree; this makes it harder to visually interpret ? 
+
+True - x
+False 
+
+
+### Optimization Quiz 
+
+1. For the formula used to model the relationship i.e. y = mx + b, what does ‘m’ stand for? 
+
+
+It refers to a bias term which can be used for regression.
+It captures the amount of change we've observed in our label in response to a small change in our feature. - x
+Both a & b
+None of the above
+
+2. What are the basic steps in an ML workflow (or process)?
+
+Check for anomalies, missing data and clean the data
+Perform statistical analysis and initial visualization
+Collect data 
+All of the above - x
+
+3. Which of the following statements is true?
+
+To calculate the Prediction y for any Input value x we have three unknowns, the m = slope(Gradient), b = y-intercept(also called bias) and z = third degree polynomial.
+To calculate the Prediction y for any Input value x we have two unknowns, the m = slope(Gradient) and b = y-intercept(also called bias).  - x
+None of the above 
+To calculate the Prediction y for any Input value x we have three unknowns, the m = slope(Gradient), b = y-intercept(also called bias) and z = hyperplane.
+
+### Optimization Quiz 2 
+
+1. Fill in the blanks: Simply speaking, __________ is the workhorse of basic loss functions. ______ is the sum of squared distances between our target variable and predicted values. 
+
+
+Log loss
+Likelihood
+Mean Squared Error - x 
+None of the above 
+
+
+2. Which of the following loss functions is used for classification problems?
+
+MSE 
+cross entropy - x
+Both A & B 
+None of the above 
+
+3. Fill in the blanks: At its core, a ________ is a method of evaluating how well your algorithm models your dataset. If your predictions are totally off, your _________ will output a higher number. If they’re pretty good, it will output a lower number. As you change pieces of your algorithm to try and improve your model, your ______ will tell you if you’re getting anywhere. 
+
+Loss function - x
+Bias term
+Activation functions
+Linear model
+
+4. Loss functions can be broadly categorized into 2 types: Classification and Regression Loss. _____ is typically used for regression and ______ is typically used for classification.
+
+Log Loss, Focus Loss
+Mean Squared Error, Cross Entropy - x
+Cross Entropy, Log Loss
+None of the above
+
+### Optimization Quiz - Gradients 
+
+1. Which of the following gradient descent methods is used to compute the entire dataset? 
+
+Mini-batch gradient descent 
+Gradient descent
+None of the above 
+Batch gradient descent  -x
+
+
+2. Fill in the blanks. ________________: Parameters are updated after computing the gradient of error with respect to the entire training set ________________: Parameters are updated after computing the gradient of error with respect to a single training example ________________: Parameters are updated after computing the gradient of error with respect to a subset of the training set
+
+Mini Batch Gradient Descent, Batch Gradient Descent, Stochastic Gradient Descent
+Mini-Batch Gradient Descent, Stochastic Gradient Descent, Batch Gradient Descent
+Batch Gradient Descent, Stochastic Gradient Descent, Mini-Batch Gradient Descent - x
+None of the above
+
+3. Select which statement is true.
+
+Batch gradient descent, also called vanilla gradient descent, calculates the error for each example within the training dataset, but only after all training examples have been evaluated does the model get updated. This whole process is like a cycle and it's called a training epoch. - x
+
+Batch gradient descent, also called vanilla gradient descent, calculates the gain for each example within the training dataset, but only before all training examples have been evaluated does the model get updated. This whole process is like a cycle and it's called a training epoch.
+
+Batch gradient descent, also called vanilla gradient descent, calculates the error for each example within the training dataset, but only before all training examples have been evaluated does the model get updated.
+
+None of the above 
+
+4. Select the correct statement(s) regarding gradient descent.
+
+In machine learning, we use gradient descent to determine if our model labels needs to be de-optimized.
+
+Gradient descent is an optimization algorithm used to minimize some function by iteratively moving in the direction of steepest descent as defined by the negative of the gradient. In machine learning, we use gradient descent to update the parameters of our model.
+
+Gradient descent is an optimization algorithm used to maximize some function by iteratively moving in the direction of steepest descent as defined by the negative of the gradient. In machine learning, we use gradient descent to update the parameters of our model.
+
+ All of the above 
+
+
+

code/gcp_sample_questions.md

@@ -0,0 +1,64 @@
+Professional Machine Learning Engineer Exam Objectives
+Frame ML problems
+Architect ML solutions
+Prepare and process data
+Develop ML models
+Automate & orchestrate ML pipelines
+Monitor, optimize, and maintain ML solutions
+
+Share Google Professional Machine Learning Engineer Sample Questions
+NO.1 You are an ML engineer at a global shoe store. You manage the ML models for the company's website. You are asked to build a model that will recommend new products to the user based on their purchase behavior and similarity with other users. What should you do?
+A. Build a collaborative-based filtering model
+B. Build a classification model
+C. Build a regression model using the features as predictors
+D. Build a knowledge-based filtering model
+Answer: A
+
+NO.2 You have been asked to develop an input pipeline for an ML training model that processes images from disparate sources at a low latency. You discover that your input data does not fit in memory. How should you create a dataset following Google-recommended best practices?
+A. Convert the images to tf .Tensor Objects, and then run tf. data. Dataset. from_tensors ().
+B. Convert the images to tf .Tensor Objects, and then run Dataset. from_tensor_slices{).
+C. Convert the images Into TFRecords, store the images in Cloud Storage, and then use the tf. data API to read the images for training
+D. Create a tf.data.Dataset.prefetch transformation
+Answer: C
+
+
+NO.3 You work for an online retail company that is creating a visual search engine. You have set up an end-to-end ML pipeline on Google Cloud to classify whether an image contains your company's product. Expecting the release of new products in the near future, you configured a retraining functionality in the pipeline so that new data can be fed into your ML models. You also want to use Al Platform's continuous evaluation service to ensure that the models have high accuracy on your test data set. What should you do?
+A. Keep the original test dataset unchanged even if newer products are incorporated into retraining
+B. Extend your test dataset with images of the newer products when they are introduced to retraining
+C. Replace your test dataset with images of the newer products when they are introduced to retraining.
+D. Update your test dataset with images of the newer products when your evaluation metrics drop below a pre-decided threshold.
+Answer: C
+
+NO.4 You are developing a Kubeflow pipeline on Google Kubernetes Engine. The first step in the pipeline is to issue a query against BigQuery. You plan to use the results of that query as the input to the next step in your pipeline. You want to achieve this in the easiest way possible. What should you do?
+A. Use the BigQuery console to execute your query and then save the query results Into a new BigQuery table.
+B. Write a Python script that uses the BigQuery API to execute queries against BigQuery Execute this script as the first step in your Kubeflow pipeline
+C. Locate the Kubeflow Pipelines repository on GitHub Find the BigQuery Query Component, copy that component's URL, and use it to load the component into your pipeline. Use the component to execute queries against BigQuery
+D. Use the Kubeflow Pipelines domain-specific language to create a custom component that uses the Python BigQuery client library to execute queries
+Answer: A
+
+NO.5 You manage a team of data scientists who use a cloud-based backend system to submit training jobs. This system has become very difficult to administer, and you want to use a managed service instead. The data scientists you work with use many different frameworks, including Keras, PyTorch, theano. Scikit-team, and custom libraries. What should you do?
+A. Set up Slurm workload manager to receive jobs that can be scheduled to run on your cloud infrastructure.
+B. Create a library of VM images on Compute Engine; and publish these images on a centralized repository
+C. Configure Kubeflow to run on Google Kubernetes Engine and receive training jobs through TFJob
+D. Use the Al Platform custom containers feature to receive training jobs using any framework
+Answer: A
+
+def LRUCache(strArr):
+  CACHE_SIZE = 5
+  CACHE_DELIMITER = '-'
+
+  cache = []
+  for element in strArr:
+    if element in cache: 
+      cache.append(element)
+      # remove first occurance of value in list
+      cache.remove(element)
+    else:
+      cache.append(element)
+
+  cacheString = CACHE_DELIMITER.join(cache[-CACHE_SIZE:])
+  return cacheString
+
+
+# keep this function call here 
+print(LRUCache(input()))

code/interview_questions.md

@@ -0,0 +1,135 @@
+https://brainstation.io/career-guides/machine-learning-engineer-interview-questions
+
+## What’s the trade-off between bias and variance?
+
+https://machinelearningmastery.com/gentle-introduction-to-the-bias-variance-trade-off-in-machine-learning/
+
+low bias ML algos: decision trees, k-nearest neighbors and support vector machines 
+high bias ML aglos: linear regression, linear discriminant analysis and logistic regression 
+
+
+
+http://cs229.stanford.edu/summer2020/BiasVarianceAnalysis.pdf
+
+Andrew Ng talks about this in CS229. (need to double check)
+
+**bias** is about what the model is assuming about the data 
+
+**variance** is about number of features in the data?? idk this doesnt seem right. 
+
+## How is KNN different from k-means clustering?
+
+k means
+https://dzone.com/articles/10-interesting-use-cases-for-the-k-means-algorithm
+https://www.wikiwand.com/en/K-means_clustering
+https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html
+
+
+k-nearest neighbors
+https://towardsdatascience.com/k-nearest-neighbors-knn-algorithm-23832490e3f4
+https://www.wikiwand.com/en/K-nearest_neighbors_algorithm
+https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html
+
+## What is cross validation and what are different methods of using it? 
+
+https://towardsdatascience.com/understanding-8-types-of-cross-validation-80c935a4976d
+
+Leave p out cross-validation
+Leave one out cross-validation
+Holdout cross-validation
+Repeated random subsampling validation
+k-fold cross-validation
+Stratified k-fold cross-validation
+Time Series cross-validation
+Nested cross-validation
+
+## Explain how a ROC curve works.
+
+Aside: it says that it is for all thresholds of classifications but it's not like that's a parameter of logistic regression model. This answers that you can just change it after the fact. 
+
+https://towardsdatascience.com/classification-metrics-thresholds-explained-caff18ad2747
+
+```python
+# Adjusting the threshold down from 0.5 to 0.25
+# Any data point with a probability  of 0.25 or higher will be 
+#  classified as 1. clf = LogisticRegression()
+clf.fit(X_train, y_train)
+THRESHOLD = 0.25
+y_pred = np.where(clf.predict_proba(X_test)[:,1] >= THRESHOLD, 1, 0)
+y_pred_proba_new_threshold = (clf.predict_proba(X_test)[:,1] >= THRESHOLD).astype(int)
+```
+
+The Receiver Operator Characteristic (ROC) curve is an evaluation metric for binary classification problems. It is a probability curve that plots the TPR against FPR at various threshold values and essentially separates the ‘signal’ from the ‘noise’. The Area Under the Curve (AUC) is the measure of the ability of a classifier to distinguish between classes and is used as a summary of the ROC curve.
+
+https://www.analyticsvidhya.com/blog/2020/06/auc-roc-curve-machine-learning/
+
+## What's the difference between "likelihood" and "probability"
+
+Probability quantifies anticipation (of outcome), likelihood quantifies trust (in model).
+
+Suppose somebody challenges us to a 'profitable gambling game'. Then, probabilities will serve us to compute things like the expected profile of your gains and loses (mean, mode, median, variance, information ratio, value at risk, gamblers ruin, and so on). In contrast, likelihood will serve us to quantify whether we trust those probabilities in the first place; or whether we 'smell a rat'.
+
+Incidentally -- since somebody above mentioned the religions of statistics -- I believe likelihood ratio to be an integral part of the Bayesian world as well as of the frequentist one: In the Bayesian world, Bayes formula just combines prior with likelihood to produce posterior.
+
+https://stats.stackexchange.com/questions/2641/what-is-the-difference-between-likelihood-and-probability
+
+
+## How to prune decision trees? 
+
+1. preprocessing - early stopping 
+2. post processing - fit tree perfectly and then prune it back 
+
+https://www.kaggle.com/arunmohan003/pruning-decision-trees-tutorial
+
+
+## how can you choose a  classifier based on a training set size ? 
+
+https://www.researchgate.net/post/How-to-decide-the-best-classifier-based-on-the-data-set-provided 
+
+As far as I know there is no a well defined rule for such task. In general, it depends on the kind of data and amount of samples x features. For instance, I would recommend to use naive Bayes or linear SVM for text classification/categorization. For datasets with numerical attributes: I would suggest linear SVM, neural networks or logistic regression if the amount of features is much greater than the number of samples. On the other hand, I would recommend neural networks or SVM with RBF or polynomial kernel if the amount of samples is not too large and greater than the number of features. Otherwise, if the number of samples is huge I would suggest to use neural networks or linear SVM, and so on. Obviously, there are other options for each scenario than those I have mentioned.
+
+## What methods for dimensionality reduction do you know and how do they compare with each other?
+
+https://towardsdatascience.com/11-dimensionality-reduction-techniques-you-should-know-in-2021-dcb9500d388b
+
+big ones seem to be: 
+The Principal Component Analysis (PCA) procedure is a dimension reduction technique that projects the data on kkk dimensions by maximizing the variance of the data as follows: 
+
+https://stanford.edu/~shervine/teaching/cs-229/cheatsheet-unsupervised-learning#dimension-reduction
+
+Independent Component Analysis (ICA) 
+It is a technique meant to find the underlying generating sources.
+
+LDA 
+https://www.wikiwand.com/en/Linear_discriminant_analysis
+
+LDA is also closely related to principal component analysis (PCA) and factor analysis in that they both look for linear combinations of variables which best explain the data.[4] LDA explicitly attempts to model the difference between the classes of data. PCA, in contrast, does not take into account any difference in class, and factor analysis builds the feature combinations based on differences rather than similarities. Discriminant analysis is also different from factor analysis in that it is not an interdependence technique: a distinction between independent variables and dependent variables (also called criterion variables) must be made. 
+
+## What’s an imbalanced dataset? Can you list some ways to deal with it?
+
+Any dataset with an unequal class distribution is technically imbalanced.
+
+Here are some techniques to handle imbalanced data:
+
+Resample the training set: There are two approaches to make a balanced dataset out of an imbalanced one are under-sampling and over-sampling.
+Generate synthetic samples: Using SMOTE (Synthetic Minority Oversampling Technique) to generate new and synthetic data to train the model.
+
+# More Questions 
+
+https://elitedatascience.com/machine-learning-interview-questions-answers#:~:text=21%20Machine%20Learning%20Interview%20Questions%20and%20Answers%201,learning%20can%20help%20different%20types%20of%20businesses.%20
+
+
+## Explain the Bias-Variance Tradeoff.
+
+https://elitedatascience.com/bias-variance-tradeoff
+
+Predictive models have a tradeoff between bias (how well the model fits the data) and variance (how much the model changes based on changes in the inputs).
+
+Simpler models are stable (low variance) but they don't get close to the truth (high bias).
+
+More complex models are more prone to being overfit (high variance) but they are expressive enough to get close to the truth (low bias).
+
+The best model for a given problem usually lies somewhere in the middle.
+
+
+

code/leetcode/blind_75.ipynb

@@ -0,0 +1,768 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 217. Contains Duplicate\n",
+    "\n",
+    "Given an integer array nums, return true if any value appears at least twice in the array, and return false if every element is distinct.\n",
+    "\n",
+    "Example 1:\n",
+    "\n",
+    "Input: nums = [1,2,3,1]\n",
+    "Output: true\n",
+    "\n",
+    "Example 2:\n",
+    "\n",
+    "Input: nums = [1,2,3,4]\n",
+    "Output: false\n",
+    "\n",
+    "Example 3:\n",
+    "\n",
+    "Input: nums = [1,1,1,3,3,4,3,2,4,2]\n",
+    "Output: true\n",
+    "\n",
+    "Constraints:\n",
+    "\n",
+    "    1 <= nums.length <= 105\n",
+    "    -109 <= nums[i] <= 109\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from typing import List\n",
+    "\n",
+    "\n",
+    "class Solution:\n",
+    "    def containsDuplicate(self, nums: List[int]) -> bool:\n",
+    "        return not (len(set(nums)) == len(nums))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class SolutionNeetCode:\n",
+    "    def containsDuplicate(self, nums: List[int]) -> bool:\n",
+    "        hashset = set()\n",
+    "\n",
+    "        for n in nums:\n",
+    "            if n in hashset:\n",
+    "                return True\n",
+    "            hashset.add(n)\n",
+    "        return False"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "True"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "nums = [1, 2, 3, 1]\n",
+    "\n",
+    "not (len(set(nums)) == len(nums))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "True"
+      ]
+     },
+     "execution_count": 10,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "nums = [1, 2, 3, 1]\n",
+    "\n",
+    "Solution().containsDuplicate(nums)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 242. Valid Anagram\n",
+    "\n",
+    "Given two strings s and t, return true if t is an anagram of s, and false otherwise.\n",
+    "\n",
+    "An Anagram is a word or phrase formed by rearranging the letters of a different word or phrase, typically using all the original letters exactly once.\n",
+    "\n",
+    "Example 1:\n",
+    "\n",
+    "Input: s = \"anagram\", t = \"nagaram\"\n",
+    "Output: true\n",
+    "\n",
+    "Example 2:\n",
+    "\n",
+    "Input: s = \"rat\", t = \"car\"\n",
+    "Output: false\n",
+    "\n",
+    "Constraints:\n",
+    "\n",
+    "    1 <= s.length, t.length <= 5 * 104\n",
+    "    s and t consist of lowercase English letters.\n",
+    "\n",
+    "Follow up: What if the inputs contain Unicode characters? How would you adapt your solution to such a case?\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 29,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "s = \"anagram\"\n",
+    "t = \"nagaramdsafhjsadlfhj\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 47,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "{'a': 3, 'n': 1, 'g': 1, 'r': 1, 'm': 1}\n",
+      "{'n': 1, 'a': 3, 'g': 1, 'r': 1, 'm': 1}\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "True"
+      ]
+     },
+     "execution_count": 47,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from itertools import zip_longest\n",
+    "\n",
+    "s = \"anagram\"\n",
+    "t = \"nagaram\"\n",
+    "\n",
+    "s_dict = dict()\n",
+    "t_dict = dict()\n",
+    "\n",
+    "for s_char, t_char in zip_longest(s, t):\n",
+    "    s_dict[s_char] = s_dict.get(s_char, 0) + 1\n",
+    "    t_dict[t_char] = t_dict.get(t_char, 0) + 1\n",
+    "\n",
+    "print(s_dict)\n",
+    "print(t_dict)\n",
+    "\n",
+    "s_dict == t_dict"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 23,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "None\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(s_dict.get(None))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 42,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'nagaramdsafhjsadlfhj'"
+      ]
+     },
+     "execution_count": 42,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "t"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 1. Two Sum\n",
+    "\n",
+    "Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.\n",
+    "\n",
+    "You may assume that each input would have exactly one solution, and you may not use the same element twice.\n",
+    "\n",
+    "You can return the answer in any order.\n",
+    "\n",
+    "Example 1:\n",
+    "\n",
+    "Input: nums = [2,7,11,15], target = 9\n",
+    "Output: [0,1]\n",
+    "Explanation: Because nums[0] + nums[1] == 9, we return [0, 1].\n",
+    "\n",
+    "Example 2:\n",
+    "\n",
+    "Input: nums = [3,2,4], target = 6\n",
+    "Output: [1,2]\n",
+    "\n",
+    "Example 3:\n",
+    "\n",
+    "Input: nums = [3,3], target = 6\n",
+    "Output: [0,1]\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 50,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from typing import List\n",
+    "\n",
+    "\n",
+    "class Solution:\n",
+    "    def twoSum(self, nums: List[int], target: int) -> List[int]:\n",
+    "        prevMap = {}  # val -> index\n",
+    "\n",
+    "        for i, n in enumerate(nums):\n",
+    "            diff = target - n\n",
+    "            if diff in prevMap:\n",
+    "                return [prevMap[diff], i]\n",
+    "            prevMap[n] = i"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 52,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[0, 1]"
+      ]
+     },
+     "execution_count": 52,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "nums = [2, 7, 11, 5]\n",
+    "\n",
+    "target = 9\n",
+    "\n",
+    "Solution().twoSum(nums, target)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 68,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "{3: 0, 2: 1, 4: 2}\n",
+      "[1, 2]\n"
+     ]
+    }
+   ],
+   "source": [
+    "nums_2 = [3, 2, 4]\n",
+    "target_2 = 6\n",
+    "\n",
+    "data = {k: v for v, k in enumerate(nums_2)}\n",
+    "print(data)\n",
+    "\n",
+    "\n",
+    "def find_two_sum(data, target):\n",
+    "    for key, current_index in data.items():\n",
+    "        difference = target - key\n",
+    "        try:\n",
+    "            found_index = data[difference]\n",
+    "            if found_index == current_index:\n",
+    "                # no match found -- continue looking\n",
+    "                continue\n",
+    "            return [current_index, found_index]\n",
+    "        except KeyError:\n",
+    "            # i guess uncessary because we have been told that a solution is guaranteed. \n",
+    "            print(\"no index found\")\n",
+    "        except Exception as e:\n",
+    "            print(\"some other error\")\n",
+    "\n",
+    "\n",
+    "print(find_two_sum(data, target_2))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 49. Group Anagrams\n",
+    "\n",
+    "Given an array of strings strs, group the anagrams together. You can return the answer in any order.\n",
+    "\n",
+    "An Anagram is a word or phrase formed by rearranging the letters of a different word or phrase, typically using all the original letters exactly once.\n",
+    "\n",
+    " \n",
+    "\n",
+    "Example 1:\n",
+    "\n",
+    "Input: strs = [\"eat\",\"tea\",\"tan\",\"ate\",\"nat\",\"bat\"]\n",
+    "Output: [[\"bat\"],[\"nat\",\"tan\"],[\"ate\",\"eat\",\"tea\"]]\n",
+    "\n",
+    "Example 2:\n",
+    "\n",
+    "Input: strs = [\"\"]\n",
+    "Output: [[\"\"]]\n",
+    "\n",
+    "Example 3:\n",
+    "\n",
+    "Input: strs = [\"a\"]\n",
+    "Output: [[\"a\"]]\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 72,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[['eat'], ['tea'], ['tan'], ['ate'], ['nat'], ['bat']]"
+      ]
+     },
+     "execution_count": 72,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "strs = [\"eat\",\"tea\",\"tan\",\"ate\",\"nat\",\"bat\"]\n",
+    "\n",
+    "output = []\n",
+    "for word in strs:\n",
+    "    # check if anagram in output and if so append to that array \n",
+    "    for \n",
+    "    # can't think of how to do this without doing another double for loop! \n",
+    "\n",
+    "    # if no anagram found then append\n",
+    "    output.append([word])\n",
+    "\n",
+    "# [\"eat\"] in output\n",
+    "output"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 76,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "dict_values([['eat', 'tea', 'ate'], ['tan', 'nat'], ['bat']])"
+      ]
+     },
+     "execution_count": 76,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from collections import defaultdict\n",
+    "\n",
+    "class Solution:\n",
+    "    def groupAnagrams(self, strs: List[str]) -> List[List[str]]:\n",
+    "        ans = defaultdict(list)\n",
+    "\n",
+    "        for s in strs:\n",
+    "            count = [0] * 26\n",
+    "            for c in s:\n",
+    "                count[ord(c) - ord(\"a\")] += 1\n",
+    "            # this maps the characters to values to check for anagram \n",
+    "            ans[tuple(count)].append(s)\n",
+    "        return ans.values()\n",
+    "\n",
+    "Solution().groupAnagrams(strs)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 347. Top K Frequent Elements\n",
+    "\n",
+    "Given an integer array nums and an integer k, return the k most frequent elements. You may return the answer in any order.\n",
+    "\n",
+    " \n",
+    "\n",
+    "Example 1:\n",
+    "\n",
+    "Input: nums = [1,1,1,2,2,3], k = 2\n",
+    "Output: [1,2]\n",
+    "\n",
+    "Example 2:\n",
+    "\n",
+    "Input: nums = [1], k = 1\n",
+    "Output: [1]\n",
+    "\n",
+    " \n",
+    "\n",
+    "Constraints:\n",
+    "\n",
+    "    1 <= nums.length <= 105\n",
+    "    -104 <= nums[i] <= 104\n",
+    "    k is in the range [1, the number of unique elements in the array].\n",
+    "    It is guaranteed that the answer is unique.\n",
+    "\n",
+    " \n",
+    "\n",
+    "Follow up: Your algorithm's time complexity must be better than O(n log n), where n is the array's size.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 80,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "nums = [1,1,1,2,2,3] \n",
+    "k = 2\n",
+    "\n",
+    "from collections import OrderedDict\n",
+    "\n",
+    "# foo = OrderedDict()\n",
+    "foo = {}\n",
+    "\n",
+    "for number in nums:\n",
+    "    foo[number] = foo.get(number, 0) + 1\n",
+    "\n",
+    "# neetcode says using a heap and popping off the top k elements would be k log(n) time too \n",
+    "    \n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 81,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{1: 3, 2: 2, 3: 1}"
+      ]
+     },
+     "execution_count": 81,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "foo"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 238. Product of Array Except Self\n",
+    "\n",
+    "Given an integer array nums, return an array answer such that answer[i] is equal to the product of all the elements of nums except nums[i].\n",
+    "\n",
+    "The product of any prefix or suffix of nums is guaranteed to fit in a 32-bit integer.\n",
+    "\n",
+    "You must write an algorithm that runs in O(n) time and without using the division operation.\n",
+    "\n",
+    " \n",
+    "\n",
+    "Example 1:\n",
+    "\n",
+    "Input: nums = [1,2,3,4]\n",
+    "Output: [24,12,8,6]\n",
+    "\n",
+    "Example 2:\n",
+    "\n",
+    "Input: nums = [-1,1,0,-3,3]\n",
+    "Output: [0,0,9,0,0]\n",
+    "\n",
+    " \n",
+    "\n",
+    "Constraints:\n",
+    "\n",
+    "    2 <= nums.length <= 105\n",
+    "    -30 <= nums[i] <= 30\n",
+    "    The product of any prefix or suffix of nums is guaranteed to fit in a 32-bit integer.\n",
+    "\n",
+    " \n",
+    "\n",
+    "Follow up: Can you solve the problem in O(1) extra space complexity? (The output array does not count as extra space for space complexity analysis.)\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 88,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "None\n",
+      "None\n",
+      "None\n",
+      "None\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "[24, 12, 8, 6]"
+      ]
+     },
+     "execution_count": 88,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "import math \n",
+    "\n",
+    "nums = [1,2,3,4]\n",
+    "output = [24,12,8,6]\n",
+    "\n",
+    "math.prod(nums) \n",
+    "\n",
+    "output_2 = []\n",
+    "for value in nums: \n",
+    "    print(output_2.append(int(math.prod(nums) * (1.0/value))))\n",
+    "\n",
+    "output_2"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 93,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[1, 3, 4]"
+      ]
+     },
+     "execution_count": 93,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# could concatenate arrays like this and then add them up? \n",
+    "nums[0:1] + nums[2:]\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# MLE Flashcards problem \n",
+    "\n",
+    "Solve fib(n) using dynamic programming of memoization and tabulation\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 111,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "The 5-th Fibonacci number is: 5\n"
+     ]
+    }
+   ],
+   "source": [
+    "# def fib_memoized(n, memo={}):\n",
+    "#     if n == 0 or n == 1:\n",
+    "#         return n \n",
+    "    \n",
+    "#     if n not in memo:\n",
+    "#         memo[n] = fib_memoized(n-1, memo) + fib_memoized(n-2, memo)\n",
+    "\n",
+    "#     return memo\n",
+    "    \n",
+    "# # memoization = {}\n",
+    "# fib_memoized(22)\n",
+    "# memoization\n",
+    "\n",
+    "# this shit is crazyyyy ! \n",
+    "def fib_n(n, memo={}):\n",
+    "    # Check if the result is already memoized\n",
+    "    if n in memo:\n",
+    "        return memo[n]\n",
+    "    \n",
+    "    # Base cases\n",
+    "    if n == 0:\n",
+    "        result = 0\n",
+    "    elif n == 1:\n",
+    "        result = 1\n",
+    "    else:\n",
+    "        # Recursive calls with memoization\n",
+    "        result = fib_n(n - 1, memo) + fib_n(n - 2, memo)\n",
+    "    \n",
+    "    # Memoize the result before returning\n",
+    "    memo[n] = result\n",
+    "    return result\n",
+    "\n",
+    "# # Example usage\n",
+    "n = 5\n",
+    "result = fib_n(n)\n",
+    "print(f\"The {n}-th Fibonacci number is: {result}\")\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 110,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "The 5-th Fibonacci number is: 5\n"
+     ]
+    }
+   ],
+   "source": [
+    "def fib_n(n):\n",
+    "    # Base cases\n",
+    "    if n == 0:\n",
+    "        return 0\n",
+    "    elif n == 1:\n",
+    "        return 1\n",
+    "    \n",
+    "    # Create a table to store Fibonacci numbers\n",
+    "    fib_table = [0] * (n + 1)\n",
+    "    \n",
+    "    # Initialize the base cases\n",
+    "    fib_table[0] = 0\n",
+    "    fib_table[1] = 1\n",
+    "    \n",
+    "    # Fill in the table using bottom-up approach\n",
+    "    for i in range(2, n + 1):\n",
+    "        fib_table[i] = fib_table[i - 1] + fib_table[i - 2]\n",
+    "    \n",
+    "    # The result is the value at index n\n",
+    "    return fib_table[n]\n",
+    "\n",
+    "# Example usage\n",
+    "n = 5\n",
+    "result = fib_n(n)\n",
+    "print(f\"The {n}-th Fibonacci number is: {result}\")\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 112,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "13 is a prime number.\n"
+     ]
+    }
+   ],
+   "source": [
+    "def is_prime(n):\n",
+    "    if n <= 1:\n",
+    "        return False\n",
+    "    elif n == 2:\n",
+    "        return True\n",
+    "    elif n % 2 == 0:\n",
+    "        return False\n",
+    "    else:\n",
+    "        # Check for factors up to the square root of n\n",
+    "        for i in range(3, int(n**0.5) + 1, 2):\n",
+    "            if n % i == 0:\n",
+    "                return False\n",
+    "        return True\n",
+    "\n",
+    "# Example usage\n",
+    "number = 13\n",
+    "if is_prime(number):\n",
+    "    print(f\"{number} is a prime number.\")\n",
+    "else:\n",
+    "    print(f\"{number} is not a prime number.\")\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "pytorch_m1",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.13"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

code/leetcode/two_sum_grind_75.ipynb

@@ -0,0 +1,2195 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[1, 0]"
+      ]
+     },
+     "execution_count": 13,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from typing import List\n",
+    "# best solution according to LEETCODE\n",
+    "class Solution:\n",
+    "    def twoSum(self, nums: List[int], target: int) -> List[int]:\n",
+    "        hashmap = {}\n",
+    "        for i in range(len(nums)):\n",
+    "            complement = target - nums[i]\n",
+    "            if complement in hashmap:\n",
+    "                return [i, hashmap[complement]]\n",
+    "            hashmap[nums[i]] = i\n",
+    "        \n",
+    "Solution().twoSum(case_1, 9)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[1, 0]"
+      ]
+     },
+     "execution_count": 4,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "case_1 = [2,7,11,15]\n",
+    "\n",
+    "Solution().twoSum(case_1, 9)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[0, 1]"
+      ]
+     },
+     "execution_count": 6,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "class Solution:\n",
+    "    def twoSum(self, nums: List[int], target: int) -> List[int]:\n",
+    "        for i in range(len(nums)):\n",
+    "            for j in range(i + 1, len(nums)):\n",
+    "                if nums[j] == target - nums[i]:\n",
+    "                    return [i, j]\n",
+    "\n",
+    "\n",
+    "Solution().twoSum(case_1, 9)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[2, 7, 11, 15]\n",
+      "7\n",
+      "2\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "[0, 1]"
+      ]
+     },
+     "execution_count": 11,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "class Solution:\n",
+    "    def twoSum(self, nums, target):\n",
+    "        \"\"\"\n",
+    "        :type nums: List[int]\n",
+    "        :type target: int\n",
+    "        :rtype: List[int]\n",
+    "        \"\"\"\n",
+    "        print(nums)\n",
+    "        hashmap = {}\n",
+    "        for i, num in enumerate(nums):\n",
+    "            n = target - num\n",
+    "            print(n)\n",
+    "            if n not in hashmap:\n",
+    "                hashmap[num] = i\n",
+    "            else:\n",
+    "                return [hashmap[n], i]\n",
+    "\n",
+    "Solution().twoSum(case_1, 9)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "      1\n",
+      "     1 1\n",
+      "    1 2 1\n",
+      "   1 3 3 1\n",
+      "  1 4 6 4 1\n",
+      " 1 6 1 0 5 1\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Print Pascal's Triangle in Python\n",
+    "\n",
+    "# input n\n",
+    "n = 6\n",
+    "\n",
+    "# iterarte upto n\n",
+    "for i in range(n):\n",
+    "\t# adjust space\n",
+    "\tprint(' '*(n-i), end='')\n",
+    "\n",
+    "\t# compute power of 11\n",
+    "\tprint(' '.join(map(str, str(11**i))))\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "1, 5, 10, 10, 5, 1"
+     ]
+    }
+   ],
+   "source": [
+    "# Python3 program to implement the above approach\n",
+    "\n",
+    "# Print the N-th row of the\n",
+    "# Pascal's Triangle\n",
+    "def generateNthRow (N):\n",
+    "\n",
+    "\t# nC0 = 1\n",
+    "\tprev = 1\n",
+    "\tprint(prev, end = '')\n",
+    "\n",
+    "\tfor i in range(1, N + 1):\n",
+    "\n",
+    "\t\t# nCr = (nCr-1 * (n - r + 1))/r\n",
+    "\t\tcurr = (prev * (N - i + 1)) // i\n",
+    "\t\tprint(\",\", curr, end = '')\n",
+    "\t\tprev = curr\n",
+    "\n",
+    "# Driver code\n",
+    "N = 5\n",
+    "\n",
+    "# Function calling\n",
+    "generateNthRow(N)\n",
+    "\n",
+    "# This code is contributed by himanshu77\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 21,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "22"
+      ]
+     },
+     "execution_count": 21,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "\"foo\".__hash__()\n",
+    "\n",
+    "bar = 22.0\n",
+    "\n",
+    "bar.__hash__()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 22,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[][][][][][][][][][][][][][][][][][][][][][][][][][('gfg@example.com', 'some value')][][][][][][][][][][][][][][][][][][][][][][][][]\n",
+      "\n",
+      "[][][][][][][][][][][][][][][][][][('portal@example.com', 'some other value')][][][][][][][][('gfg@example.com', 'some value')][][][][][][][][][][][][][][][][][][][][][][][][]\n",
+      "\n",
+      "some other value\n",
+      "\n",
+      "[][][][][][][][][][][][][][][][][][][][][][][][][][('gfg@example.com', 'some value')][][][][][][][][][][][][][][][][][][][][][][][][]\n"
+     ]
+    }
+   ],
+   "source": [
+    "class HashTable:\n",
+    "\n",
+    "\t# Create empty bucket list of given size\n",
+    "\tdef __init__(self, size):\n",
+    "\t\tself.size = size\n",
+    "\t\tself.hash_table = self.create_buckets()\n",
+    "\n",
+    "\tdef create_buckets(self):\n",
+    "\t\treturn [[] for _ in range(self.size)]\n",
+    "\n",
+    "\t# Insert values into hash map\n",
+    "\tdef set_val(self, key, val):\n",
+    "\t\t\n",
+    "\t\t# Get the index from the key\n",
+    "\t\t# using hash function\n",
+    "\t\thashed_key = hash(key) % self.size\n",
+    "\t\t\n",
+    "\t\t# Get the bucket corresponding to index\n",
+    "\t\tbucket = self.hash_table[hashed_key]\n",
+    "\n",
+    "\t\tfound_key = False\n",
+    "\t\tfor index, record in enumerate(bucket):\n",
+    "\t\t\trecord_key, record_val = record\n",
+    "\t\t\t\n",
+    "\t\t\t# check if the bucket has same key as\n",
+    "\t\t\t# the key to be inserted\n",
+    "\t\t\tif record_key == key:\n",
+    "\t\t\t\tfound_key = True\n",
+    "\t\t\t\tbreak\n",
+    "\n",
+    "\t\t# If the bucket has same key as the key to be inserted,\n",
+    "\t\t# Update the key value\n",
+    "\t\t# Otherwise append the new key-value pair to the bucket\n",
+    "\t\tif found_key:\n",
+    "\t\t\tbucket[index] = (key, val)\n",
+    "\t\telse:\n",
+    "\t\t\tbucket.append((key, val))\n",
+    "\n",
+    "\t# Return searched value with specific key\n",
+    "\tdef get_val(self, key):\n",
+    "\t\t\n",
+    "\t\t# Get the index from the key using\n",
+    "\t\t# hash function\n",
+    "\t\thashed_key = hash(key) % self.size\n",
+    "\t\t\n",
+    "\t\t# Get the bucket corresponding to index\n",
+    "\t\tbucket = self.hash_table[hashed_key]\n",
+    "\n",
+    "\t\tfound_key = False\n",
+    "\t\tfor index, record in enumerate(bucket):\n",
+    "\t\t\trecord_key, record_val = record\n",
+    "\t\t\t\n",
+    "\t\t\t# check if the bucket has same key as\n",
+    "\t\t\t# the key being searched\n",
+    "\t\t\tif record_key == key:\n",
+    "\t\t\t\tfound_key = True\n",
+    "\t\t\t\tbreak\n",
+    "\n",
+    "\t\t# If the bucket has same key as the key being searched,\n",
+    "\t\t# Return the value found\n",
+    "\t\t# Otherwise indicate there was no record found\n",
+    "\t\tif found_key:\n",
+    "\t\t\treturn record_val\n",
+    "\t\telse:\n",
+    "\t\t\treturn \"No record found\"\n",
+    "\n",
+    "\t# Remove a value with specific key\n",
+    "\tdef delete_val(self, key):\n",
+    "\t\t\n",
+    "\t\t# Get the index from the key using\n",
+    "\t\t# hash function\n",
+    "\t\thashed_key = hash(key) % self.size\n",
+    "\t\t\n",
+    "\t\t# Get the bucket corresponding to index\n",
+    "\t\tbucket = self.hash_table[hashed_key]\n",
+    "\n",
+    "\t\tfound_key = False\n",
+    "\t\tfor index, record in enumerate(bucket):\n",
+    "\t\t\trecord_key, record_val = record\n",
+    "\t\t\t\n",
+    "\t\t\t# check if the bucket has same key as\n",
+    "\t\t\t# the key to be deleted\n",
+    "\t\t\tif record_key == key:\n",
+    "\t\t\t\tfound_key = True\n",
+    "\t\t\t\tbreak\n",
+    "\t\tif found_key:\n",
+    "\t\t\tbucket.pop(index)\n",
+    "\t\treturn\n",
+    "\n",
+    "\t# To print the items of hash map\n",
+    "\tdef __str__(self):\n",
+    "\t\treturn \"\".join(str(item) for item in self.hash_table)\n",
+    "\n",
+    "\n",
+    "hash_table = HashTable(50)\n",
+    "\n",
+    "# insert some values\n",
+    "hash_table.set_val('gfg@example.com', 'some value')\n",
+    "print(hash_table)\n",
+    "print()\n",
+    "\n",
+    "hash_table.set_val('portal@example.com', 'some other value')\n",
+    "print(hash_table)\n",
+    "print()\n",
+    "\n",
+    "# search/access a record with key\n",
+    "print(hash_table.get_val('portal@example.com'))\n",
+    "print()\n",
+    "\n",
+    "# delete or remove a value\n",
+    "hash_table.delete_val('portal@example.com')\n",
+    "print(hash_table)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Longest Substring Without Repeating Characters (medium) -- blind 75 \n",
+    "\n",
+    "Given a string s, find the length of the longest\n",
+    "substring\n",
+    "without repeating characters.\n",
+    "\n",
+    "Input: s = \"abcabcbb\"\n",
+    "Output: 3\n",
+    "Explanation: The answer is \"abc\", with the length of 3."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 25,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "3"
+      ]
+     },
+     "execution_count": 25,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "class Solution:\n",
+    "    def lengthOfLongestSubstring(self, s: str) -> int:\n",
+    "        def check(start, end):\n",
+    "            chars = set()\n",
+    "            for i in range(start, end + 1):\n",
+    "                c = s[i]\n",
+    "                if c in chars:\n",
+    "                    return False\n",
+    "                chars.add(c)\n",
+    "            return True\n",
+    "\n",
+    "        n = len(s)\n",
+    "\n",
+    "        res = 0\n",
+    "        for i in range(n):\n",
+    "            for j in range(i, n):\n",
+    "                if check(i, j):\n",
+    "                    res = max(res, j - i + 1)\n",
+    "        return res\n",
+    "\n",
+    "s = \"abcabcbb\"\n",
+    "\n",
+    "Solution().lengthOfLongestSubstring(s)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 27,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "3"
+      ]
+     },
+     "execution_count": 27,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "class Solution:\n",
+    "    def lengthOfLongestSubstring(self, s: str) -> int:\n",
+    "        n = len(s)\n",
+    "        ans = 0\n",
+    "        # mp stores the current index of a character\n",
+    "        mp = {}\n",
+    "\n",
+    "        i = 0\n",
+    "        # try to extend the range [i, j]\n",
+    "        for j in range(n):\n",
+    "            if s[j] in mp:\n",
+    "                i = max(mp[s[j]], i)\n",
+    "\n",
+    "            ans = max(ans, j - i + 1)\n",
+    "            mp[s[j]] = j + 1\n",
+    "\n",
+    "        return ans\n",
+    "\n",
+    "s = \"abcabcbb\"\n",
+    "\n",
+    "Solution().lengthOfLongestSubstring(s)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 5. Longest Palindromic Substring\n",
+    "\n",
+    "Given a string s, return the longest palindromic substring in s.\n",
+    "\n",
+    "Input: s = \"babad\"\n",
+    "Output: \"bab\"\n",
+    "Explanation: \"aba\" is also a valid answer."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class Solution:\n",
+    "    def longestPalindrome(self, s: str) -> str:\n",
+    "        m = ''  # Memory to remember a palindrome\n",
+    "        for i in range(len(s)):  # i = start, O = n\n",
+    "            for j in range(len(s), i, -1):  # j = end, O = n^2\n",
+    "                if len(m) >= j-i:  # To reduce time\n",
+    "                    break\n",
+    "                elif s[i:j] == s[i:j][::-1]:\n",
+    "                    m = s[i:j]\n",
+    "                    break\n",
+    "        return m"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 11. Container With Most Water \n",
+    "\n",
+    "You are given an integer array height of length n. There are n vertical lines drawn such that the two endpoints of the ith line are (i, 0) and (i, height[i]).\n",
+    "\n",
+    "Find two lines that together with the x-axis form a container, such that the container contains the most water.\n",
+    "\n",
+    "Return the maximum amount of water a container can store.\n",
+    "\n",
+    "Notice that you may not slant the container.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 29,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "49"
+      ]
+     },
+     "execution_count": 29,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "class Solution:\n",
+    "    def maxArea(self, height: List[int]) -> int:\n",
+    "        maxarea = 0\n",
+    "        left = 0\n",
+    "        right = len(height) - 1\n",
+    "        \n",
+    "        while left < right:\n",
+    "            width = right - left\n",
+    "            maxarea = max(maxarea, min(height[left], height[right]) * width)\n",
+    "            if height[left] <= height[right]:\n",
+    "                left += 1\n",
+    "            else:\n",
+    "                right -= 1\n",
+    "                \n",
+    "        return maxarea\n",
+    "\n",
+    "height = [1,8,6,2,5,4,8,3,7]\n",
+    "\n",
+    "Solution().maxArea(height)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 15. 3Sum \n",
+    "\n",
+    "Given an integer array nums, return all the triplets [nums[i], nums[j], nums[k]] such that i != j, i != k, and j != k, and nums[i] + nums[j] + nums[k] == 0.\n",
+    "\n",
+    "Notice that the solution set must not contain duplicate triplets.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 32,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[(-1, -1, 2), (-1, 0, 1)]"
+      ]
+     },
+     "execution_count": 32,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "class Solution:\n",
+    "    def threeSum(self, nums):\n",
+    "        res = []\n",
+    "        nums.sort()\n",
+    "        for i in range(len(nums)-2):\n",
+    "            if i > 0 and nums[i] == nums[i-1]:\n",
+    "                continue\n",
+    "            l, r = i+1, len(nums)-1\n",
+    "            while l < r:\n",
+    "                s = nums[i] + nums[l] + nums[r]\n",
+    "                if s < 0:\n",
+    "                    l +=1 \n",
+    "                elif s > 0:\n",
+    "                    r -= 1\n",
+    "                else:\n",
+    "                    res.append((nums[i], nums[l], nums[r]))\n",
+    "                    while l < r and nums[l] == nums[l+1]:\n",
+    "                        l += 1\n",
+    "                    while l < r and nums[r] == nums[r-1]:\n",
+    "                        r -= 1\n",
+    "                    l += 1; r -= 1\n",
+    "        return res\n",
+    "\n",
+    "nums = [-1,0,1,2,-1,-4]\n",
+    "\n",
+    "Solution().threeSum(nums)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 20. Valid Parenthesis \n",
+    "\n",
+    "Given a string s containing just the characters '(', ')', '{', '}', '[' and ']', determine if the input string is valid.\n",
+    "\n",
+    "An input string is valid if:\n",
+    "\n",
+    "    Open brackets must be closed by the same type of brackets.\n",
+    "    Open brackets must be closed in the correct order.\n",
+    "    Every close bracket has a corresponding open bracket of the same type.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 118,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "False"
+      ]
+     },
+     "execution_count": 118,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "class Solution:\n",
+    "    # @return a boolean\n",
+    "    def isValid(self, s):\n",
+    "        stack = []\n",
+    "        dict = {\"]\":\"[\", \"}\":\"{\", \")\":\"(\"}\n",
+    "        for char in s:\n",
+    "            if char in dict.values():\n",
+    "                stack.append(char)\n",
+    "            elif char in dict.keys():\n",
+    "                if stack == [] or dict[char] != stack.pop():\n",
+    "                    return False\n",
+    "            else:\n",
+    "                return False\n",
+    "        return stack == []\n",
+    "\n",
+    "\n",
+    "s = \"()[]{}\"\n",
+    "s = \"()[]{}\"\n",
+    "Solution().isValid(s)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 21. Merge Two Sorted List \n",
+    "\n",
+    "You are given the heads of two sorted linked lists list1 and list2.\n",
+    "\n",
+    "Merge the two lists in a one sorted list. The list should be made by splicing together the nodes of the first two lists.\n",
+    "\n",
+    "Return the head of the merged linked list."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 37,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from typing import Optional\n",
+    "\n",
+    "class ListNode(object):\n",
+    "    def __init__(self, x):\n",
+    "        self.val = x\n",
+    "        self.next = None\n",
+    "\n",
+    "\n",
+    "class Solution:\n",
+    "    def mergeTwoLists(self, list1: Optional[ListNode], list2: Optional[ListNode]) -> Optional[ListNode]:\n",
+    "        cur = dummy = ListNode()\n",
+    "        while list1 and list2:               \n",
+    "            if list1.val < list2.val:\n",
+    "                cur.next = list1\n",
+    "                list1, cur = list1.next, list1\n",
+    "            else:\n",
+    "                cur.next = list2\n",
+    "                list2, cur = list2.next, list2\n",
+    "                \n",
+    "        if list1 or list2:\n",
+    "            cur.next = list1 if list1 else list2\n",
+    "            \n",
+    "        return dummy.next"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 23. Merge k Sorted Lists (hard)\n",
+    "\n",
+    "You are given an array of k linked-lists lists, each linked-list is sorted in ascending order.\n",
+    "\n",
+    "Merge all the linked-lists into one sorted linked-list and return it.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 39,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# brute force \n",
+    "\n",
+    "class Solution(object):\n",
+    "    def mergeKLists(self, lists):\n",
+    "        \"\"\"\n",
+    "        :type lists: List[ListNode]\n",
+    "        :rtype: ListNode\n",
+    "        \"\"\"\n",
+    "        self.nodes = []\n",
+    "        head = point = ListNode(0)\n",
+    "        for l in lists:\n",
+    "            while l:\n",
+    "                self.nodes.append(l.val)\n",
+    "                l = l.next\n",
+    "        for x in sorted(self.nodes):\n",
+    "            point.next = ListNode(x)\n",
+    "            point = point.next\n",
+    "        return head.next"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 38,
+   "metadata": {},
+   "outputs": [
+    {
+     "ename": "ModuleNotFoundError",
+     "evalue": "No module named 'Queue'",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
+      "Cell \u001b[0;32mIn[38], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[39mfrom\u001b[39;00m \u001b[39mQueue\u001b[39;00m \u001b[39mimport\u001b[39;00m PriorityQueue\n\u001b[1;32m      3\u001b[0m \u001b[39mclass\u001b[39;00m \u001b[39mSolution\u001b[39;00m(\u001b[39mobject\u001b[39m):\n\u001b[1;32m      4\u001b[0m     \u001b[39mdef\u001b[39;00m \u001b[39mmergeKLists\u001b[39m(\u001b[39mself\u001b[39m, lists):\n",
+      "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'Queue'"
+     ]
+    }
+   ],
+   "source": [
+    "from Queue import PriorityQueue\n",
+    "\n",
+    "class Solution(object):\n",
+    "    def mergeKLists(self, lists):\n",
+    "        \"\"\"\n",
+    "        :type lists: List[ListNode]\n",
+    "        :rtype: ListNode\n",
+    "        \"\"\"\n",
+    "        head = point = ListNode(0)\n",
+    "        q = PriorityQueue()\n",
+    "        for l in lists:\n",
+    "            if l:\n",
+    "                q.put((l.val, l))\n",
+    "        while not q.empty():\n",
+    "            val, node = q.get()\n",
+    "            point.next = ListNode(val)\n",
+    "            point = point.next\n",
+    "            node = node.next\n",
+    "            if node:\n",
+    "                q.put((node.val, node))\n",
+    "        return head.next\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 33. Search in Rotated Sorted Array (medium)\n",
+    "\n",
+    "There is an integer array nums sorted in ascending order (with distinct values).\n",
+    "\n",
+    "Prior to being passed to your function, nums is possibly rotated at an unknown pivot index k (1 <= k < nums.length) such that the resulting array is [nums[k], nums[k+1], ..., nums[n-1], nums[0], nums[1], ..., nums[k-1]] (0-indexed). For example, [0,1,2,4,5,6,7] might be rotated at pivot index 3 and become [4,5,6,7,0,1,2].\n",
+    "\n",
+    "Given the array nums after the possible rotation and an integer target, return the index of target if it is in nums, or -1 if it is not in nums.\n",
+    "\n",
+    "You must write an algorithm with O(log n) runtime complexity."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 41,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "4"
+      ]
+     },
+     "execution_count": 41,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "class Solution:\n",
+    "    def search(self, A: List[int], target: int) -> int:\n",
+    "        n = len(A)\n",
+    "        left, right = 0, n - 1\n",
+    "        if n == 0: return -1\n",
+    "        \n",
+    "        while left <= right:\n",
+    "            mid = left + (right - left) // 2\n",
+    "            if A[mid] == target: return mid\n",
+    "            \n",
+    "            # inflection point to the right. Left is strictly increasing\n",
+    "            if A[mid] >= A[left]:\n",
+    "                if A[left] <= target < A[mid]:\n",
+    "                    right = mid - 1\n",
+    "                else:\n",
+    "                    left = mid + 1\n",
+    "                    \n",
+    "            # inflection point to the left of me. Right is strictly increasing\n",
+    "            else:\n",
+    "                if A[mid] < target <= A[right]:\n",
+    "                    left = mid + 1\n",
+    "                else:\n",
+    "                    right = mid - 1\n",
+    "            \n",
+    "        return -1\n",
+    "\n",
+    "A = [4,5,6,7,0,1,2]\n",
+    "target = 0 \n",
+    "\n",
+    "# return index of pivot \n",
+    "Solution().search(A, target)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 39. Combination Sum (medium)\n",
+    "\n",
+    "Given an array of distinct integers candidates and a target integer target, return a list of all unique combinations of candidates where the chosen numbers sum to target. You may return the combinations in any order.\n",
+    "\n",
+    "The same number may be chosen from candidates an unlimited number of times. Two combinations are unique if the\n",
+    "frequency\n",
+    "of at least one of the chosen numbers is different.\n",
+    "\n",
+    "The test cases are generated such that the number of unique combinations that sum up to target is less than 150 combinations for the given input.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 42,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[[2, 2, 3], [7]]"
+      ]
+     },
+     "execution_count": 42,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "class Solution(object):\n",
+    "    def combinationSum(self, candidates, target):\n",
+    "        ret = []\n",
+    "        self.dfs(candidates, target, [], ret)\n",
+    "        return ret\n",
+    "    \n",
+    "    def dfs(self, nums, target, path, ret):\n",
+    "        if target < 0:\n",
+    "            return \n",
+    "        if target == 0:\n",
+    "            ret.append(path)\n",
+    "            return \n",
+    "        for i in range(len(nums)):\n",
+    "            self.dfs(nums[i:], target-nums[i], path+[nums[i]], ret)\n",
+    "\n",
+    "candidates = [2,3,6,7]\n",
+    "targets = 7 \n",
+    "\n",
+    "Solution().combinationSum(candidates, targets)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 48. Rotate Image (medium)\n",
+    "\n",
+    "You are given an n x n 2D matrix representing an image, rotate the image by 90 degrees (clockwise).\n",
+    "\n",
+    "You have to rotate the image in-place, which means you have to modify the input 2D matrix directly. DO NOT allocate another 2D matrix and do the rotation.\n",
+    "\n",
+    "\n",
+    "\n",
+    "**Bonus Question:** If you're not too confident with matrices and linear algebra, get some more practice by also coding a method that transposes the matrix on the other diagonal, and another that reflects from top to bottom. You can test your functions by printing out the matrix before and after each operation. Finally, use your functions to find three more solutions to this problem. Each solution uses two matrix operations.\n",
+    "\n",
+    "\n",
+    "**Interview Tip:** Terrified of being asked this question in an interview? Many people are: it can be intimidating due to the fiddly logic. Unfortunately, if you do a lot of interviewing, the probability of seeing it at least once is high, and some people have claimed to have seen it multiple times! This is one of the few questions where I recommend practicing until you can confidently code it and explain it without thinking too much.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class Solution:\n",
+    "    def rotate(self, matrix: List[List[int]]) -> None:\n",
+    "        self.transpose(matrix)\n",
+    "        self.reflect(matrix)\n",
+    "    \n",
+    "    def transpose(self, matrix):\n",
+    "        n = len(matrix)\n",
+    "        for i in range(n):\n",
+    "            for j in range(i + 1, n):\n",
+    "                matrix[j][i], matrix[i][j] = matrix[i][j], matrix[j][i]\n",
+    "\n",
+    "    def reflect(self, matrix):\n",
+    "        n = len(matrix)\n",
+    "        for i in range(n):\n",
+    "            for j in range(n // 2):\n",
+    "                matrix[i][j], matrix[i][-j - 1] = matrix[i][-j - 1], matrix[i][j]\n",
+    "\n",
+    "matrix = [[1,2,3],[4,5,6],[7,8,9]]\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "49. Group Anagrams\n",
+    "\n",
+    "Given an array of strings strs, group the anagrams together. You can return the answer in any order.\n",
+    "\n",
+    "An Anagram is a word or phrase formed by rearranging the letters of a different word or phrase, typically using all the original letters exactly once.\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 45,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "dict_values([['eat', 'tea', 'ate'], ['tan', 'nat'], ['bat']])"
+      ]
+     },
+     "execution_count": 45,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "import collections\n",
+    "\n",
+    "class Solution:\n",
+    "    def groupAnagrams(self, strs):\n",
+    "        ans = collections.defaultdict(list)\n",
+    "        for s in strs:\n",
+    "            count = [0] * 26\n",
+    "            for c in s:\n",
+    "                count[ord(c) - ord('a')] += 1\n",
+    "            ans[tuple(count)].append(s)\n",
+    "        return ans.values()\n",
+    "\n",
+    "strs = [\"eat\",\"tea\",\"tan\",\"ate\",\"nat\",\"bat\"]\n",
+    "\n",
+    "Solution().groupAnagrams(strs)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 53. Maximum Subarray \n",
+    "iven an integer array nums, find the\n",
+    "subarray\n",
+    "which has the largest sum and return its sum.\n",
+    "\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# kadane algo \n",
+    "for i in range(1, len(nums)):\n",
+    "        if nums[i-1] > 0:\n",
+    "            nums[i] += nums[i-1]\n",
+    "    return max(nums)\n",
+    "\n",
+    "\n",
+    "from itertools import accumulate\n",
+    "\n",
+    "return max(accumulate(nums, lambda x, y: x+y if x > 0 else y))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# spiral matrix\n",
+    "\n",
+    "Given an m x n matrix, return all elements of the matrix in spiral order.\n",
+    "\n",
+    "matrix = [[1,2,3],[4,5,6],[7,8,9]]\n",
+    "output [1,2,3,6,9,8,7,4,5]\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 46,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class Solution:\n",
+    "    def spiralOrder(self, matrix: List[List[int]]) -> List[int]:\n",
+    "        res = []\n",
+    "        if len(matrix) == 0:\n",
+    "            return res\n",
+    "        row_begin = 0\n",
+    "        col_begin = 0\n",
+    "        row_end = len(matrix)-1 \n",
+    "        col_end = len(matrix[0])-1\n",
+    "        while (row_begin <= row_end and col_begin <= col_end):\n",
+    "            for i in range(col_begin,col_end+1):\n",
+    "                res.append(matrix[row_begin][i])\n",
+    "            row_begin += 1\n",
+    "            for i in range(row_begin,row_end+1):\n",
+    "                res.append(matrix[i][col_end])\n",
+    "            col_end -= 1\n",
+    "            if (row_begin <= row_end):\n",
+    "                for i in range(col_end,col_begin-1,-1):\n",
+    "                    res.append(matrix[row_end][i])\n",
+    "                row_end -= 1\n",
+    "            if (col_begin <= col_end):\n",
+    "                for i in range(row_end,row_begin-1,-1):\n",
+    "                    res.append(matrix[i][col_begin])\n",
+    "                col_begin += 1\n",
+    "        return res\n",
+    "    \n",
+    "                \n",
+    "        "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Jump Game (medium) \n",
+    "\n",
+    "You are given an integer array nums. You are initially positioned at the array's first index, and each element in the array represents your maximum jump length at that position.\n",
+    "\n",
+    "Return true if you can reach the last index, or false otherwise."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 50,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "False"
+      ]
+     },
+     "execution_count": 50,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "class Solution:\n",
+    "    def canJump(self, nums: List[int]) -> bool:\n",
+    "        m = 0\n",
+    "        for i, n in enumerate(nums):\n",
+    "            if i > m:\n",
+    "                return False\n",
+    "            m = max(m, i+n)\n",
+    "        return True\n",
+    "\n",
+    "nums = [2,3,1,1,4]\n",
+    "\n",
+    "nums = [3,2,1,0,4]\n",
+    "\n",
+    "Solution().canJump(nums)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 56. Merge Intervals\n",
+    "\n",
+    "Given an array of intervals where intervals[i] = [starti, endi], merge all overlapping intervals, and return an array of the non-overlapping intervals that cover all the intervals in the input.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 54,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[[1, 6], [8, 10], [15, 18]]"
+      ]
+     },
+     "execution_count": 54,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# time complexity analysis O(n log n )\n",
+    "class Solution:\n",
+    "    def merge(self, intervals: List[List[int]]) -> List[List[int]]:\n",
+    "\n",
+    "        intervals.sort(key=lambda x: x[0])\n",
+    "\n",
+    "        merged = []\n",
+    "        for interval in intervals:\n",
+    "            # if the list of merged intervals is empty or if the current\n",
+    "            # interval does not overlap with the previous, simply append it.\n",
+    "            if not merged or merged[-1][1] < interval[0]:\n",
+    "                merged.append(interval)\n",
+    "            else:\n",
+    "            # otherwise, there is overlap, so we merge the current and previous\n",
+    "            # intervals.\n",
+    "                merged[-1][1] = max(merged[-1][1], interval[1])\n",
+    "\n",
+    "        return merged\n",
+    "\n",
+    "intervals = [[1,3],[2,6],[8,10],[15,18]]\n",
+    "\n",
+    "Solution().merge(intervals)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 57. Insert Interval\n",
+    "\n",
+    "You are given an array of non-overlapping intervals intervals where intervals[i] = [starti, endi] represent the start and the end of the ith interval and intervals is sorted in ascending order by starti. You are also given an interval newInterval = [start, end] that represents the start and end of another interval.\n",
+    "\n",
+    "Insert newInterval into intervals such that intervals is still sorted in ascending order by starti and intervals still does not have any overlapping intervals (merge overlapping intervals if necessary).\n",
+    "\n",
+    "Return intervals after the insertion."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 59,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[[1, 5], [6, 9]]"
+      ]
+     },
+     "execution_count": 59,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# class Solution:\n",
+    "#     def insert(self, intervals: List[List[int]], newInterval: List[int]) -> List[List[int]]:\n",
+    "#         # s, e = newInterval.start, newInterval.end\n",
+    "#         s, e = newInterval[0], newInterval[1]\n",
+    "#         left = [i for i in intervals if i.end < s]\n",
+    "#         right = [i for i in intervals if i.start > e]\n",
+    "#         if left + right != intervals:\n",
+    "#             s = min(s, intervals[len(left)].start)\n",
+    "#             e = max(e, intervals[~len(right)].end)\n",
+    "#         return left + [Interval(s, e)] + right\n",
+    "\n",
+    "\n",
+    "class Solution:\n",
+    "    def insert(self, intervals: List[List[int]], newInterval: List[int]) -> List[List[int]]:\n",
+    "        s, e = newInterval[0], newInterval[1]\n",
+    "        left, right = [], []\n",
+    "        for i in intervals:\n",
+    "            if i[1] < s:\n",
+    "                left += i,\n",
+    "            elif i[0] > e:\n",
+    "                right += i,\n",
+    "            else:\n",
+    "                s = min(s, i[0])\n",
+    "                e = max(e, i[1])\n",
+    "        return left + [[s, e]] + right\n",
+    "\n",
+    "intervals = [[1,3],[6,9]]\n",
+    "newInterval = [2,5]\n",
+    "\n",
+    "Solution().insert(intervals, newInterval)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 70. Climbing Stairs\n",
+    "\n",
+    "You are climbing a staircase. It takes n steps to reach the top.\n",
+    "\n",
+    "Each time you can either climb 1 or 2 steps. In how many distinct ways can you climb to the top?"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 60,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "2"
+      ]
+     },
+     "execution_count": 60,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "class Solution:\n",
+    "    def climbStairs(self, n: int) -> int:\n",
+    "        a = b = 1\n",
+    "        for _ in range(n):\n",
+    "            a, b = b, a + b\n",
+    "        return a\n",
+    "\n",
+    "n = 2\n",
+    "Solution().climbStairs(n)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 73. Set Matrix Zeros \n",
+    "\n",
+    "Given an m x n integer matrix matrix, if an element is 0, set its entire row and column to 0's.\n",
+    "\n",
+    "You must do it in place."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 63,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[[1, 0, 1], [0, 0, 0], [1, 0, 1]]\n"
+     ]
+    }
+   ],
+   "source": [
+    "class Solution(object):\n",
+    "    def setZeroes(self, matrix):\n",
+    "        \"\"\"\n",
+    "        :type matrix: List[List[int]]\n",
+    "        :rtype: void Do not return anything, modify matrix in-place instead.\n",
+    "        \"\"\"\n",
+    "        is_col = False\n",
+    "        R = len(matrix)\n",
+    "        C = len(matrix[0])\n",
+    "        for i in range(R):\n",
+    "            # Since first cell for both first row and first column is the same i.e. matrix[0][0]\n",
+    "            # We can use an additional variable for either the first row/column.\n",
+    "            # For this solution we are using an additional variable for the first column\n",
+    "            # and using matrix[0][0] for the first row.\n",
+    "            if matrix[i][0] == 0:\n",
+    "                is_col = True\n",
+    "            for j in range(1, C):\n",
+    "                # If an element is zero, we set the first element of the corresponding row and column to 0\n",
+    "                if matrix[i][j]  == 0:\n",
+    "                    matrix[0][j] = 0\n",
+    "                    matrix[i][0] = 0\n",
+    "\n",
+    "        # Iterate over the array once again and using the first row and first column, update the elements.\n",
+    "        for i in range(1, R):\n",
+    "            for j in range(1, C):\n",
+    "                if not matrix[i][0] or not matrix[0][j]:\n",
+    "                    matrix[i][j] = 0\n",
+    "\n",
+    "        # See if the first row needs to be set to zero as well\n",
+    "        if matrix[0][0] == 0:\n",
+    "            for j in range(C):\n",
+    "                matrix[0][j] = 0\n",
+    "\n",
+    "        # See if the first column needs to be set to zero as well        \n",
+    "        if is_col:\n",
+    "            for i in range(R):\n",
+    "                matrix[i][0] = 0\n",
+    "\n",
+    "matrix = [[1,1,1],[1,0,1],[1,1,1]]\n",
+    "\n",
+    "Solution().setZeroes(matrix)\n",
+    "print(matrix)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 76. Minimum Window Substring (hard)\n",
+    "\n",
+    "Given two strings s and t of lengths m and n respectively, return the minimum window\n",
+    "substring\n",
+    "of s such that every character in t (including duplicates) is included in the window. If there is no such substring, return the empty string \"\".\n",
+    "\n",
+    "The testcases will be generated such that the answer is unique.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# solution from leetcode \n",
+    "\n",
+    "def minWindow(self, s, t):\n",
+    "    \"\"\"\n",
+    "    :type s: str\n",
+    "    :type t: str\n",
+    "    :rtype: str\n",
+    "    \"\"\"\n",
+    "\n",
+    "    if not t or not s:\n",
+    "        return \"\"\n",
+    "\n",
+    "    # Dictionary which keeps a count of all the unique characters in t.\n",
+    "    dict_t = Counter(t)\n",
+    "\n",
+    "    # Number of unique characters in t, which need to be present in the desired window.\n",
+    "    required = len(dict_t)\n",
+    "\n",
+    "    # left and right pointer\n",
+    "    l, r = 0, 0\n",
+    "\n",
+    "    # formed is used to keep track of how many unique characters in t are present in the current window in its desired frequency.\n",
+    "    # e.g. if t is \"AABC\" then the window must have two A's, one B and one C. Thus formed would be = 3 when all these conditions are met.\n",
+    "    formed = 0\n",
+    "\n",
+    "    # Dictionary which keeps a count of all the unique characters in the current window.\n",
+    "    window_counts = {}\n",
+    "\n",
+    "    # ans tuple of the form (window length, left, right)\n",
+    "    ans = float(\"inf\"), None, None\n",
+    "\n",
+    "    while r < len(s):\n",
+    "\n",
+    "        # Add one character from the right to the window\n",
+    "        character = s[r]\n",
+    "        window_counts[character] = window_counts.get(character, 0) + 1\n",
+    "\n",
+    "        # If the frequency of the current character added equals to the desired count in t then increment the formed count by 1.\n",
+    "        if character in dict_t and window_counts[character] == dict_t[character]:\n",
+    "            formed += 1\n",
+    "\n",
+    "        # Try and contract the window till the point where it ceases to be 'desirable'.\n",
+    "        while l <= r and formed == required:\n",
+    "            character = s[l]\n",
+    "\n",
+    "            # Save the smallest window until now.\n",
+    "            if r - l + 1 < ans[0]:\n",
+    "                ans = (r - l + 1, l, r)\n",
+    "\n",
+    "            # The character at the position pointed by the `left` pointer is no longer a part of the window.\n",
+    "            window_counts[character] -= 1\n",
+    "            if character in dict_t and window_counts[character] < dict_t[character]:\n",
+    "                formed -= 1\n",
+    "\n",
+    "            # Move the left pointer ahead, this would help to look for a new window.\n",
+    "            l += 1    \n",
+    "\n",
+    "        # Keep expanding the window once we are done contracting.\n",
+    "        r += 1    \n",
+    "    return \"\" if ans[0] == float(\"inf\") else s[ans[1] : ans[2] + 1]\n",
+    "\n",
+    "s = \"ADOBECODEBANC\", t = \"ABC\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# soltuion from comments \n",
+    "\n",
+    "class Solution:\n",
+    "    def minWindow(s, t):\n",
+    "        need = collections.Counter(t)            #hash table to store char frequency\n",
+    "        missing = len(t)                         #total number of chars we care\n",
+    "        start, end = 0, 0\n",
+    "        i = 0\n",
+    "        for j, char in enumerate(s, 1):          #index j from 1\n",
+    "            if need[char] > 0:\n",
+    "                missing -= 1\n",
+    "            need[char] -= 1\n",
+    "            if missing == 0:                     #match all chars\n",
+    "                while i < j and need[s[i]] < 0:  #remove chars to find the real start\n",
+    "                    need[s[i]] += 1\n",
+    "                    i += 1\n",
+    "                need[s[i]] += 1                  #make sure the first appearing char satisfies need[char]>0\n",
+    "                missing += 1                     #we missed this first char, so add missing by 1\n",
+    "                if end == 0 or j-i < end-start:  #update window\n",
+    "                    start, end = i, j\n",
+    "                i += 1                           #update i to start+1 for next window\n",
+    "        return s[start:end]\n",
+    "\n",
+    "    "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 79. Word Search\n",
+    "\n",
+    "Given an m x n grid of characters board and a string word, return true if word exists in the grid.\n",
+    "\n",
+    "The word can be constructed from letters of sequentially adjacent cells, where adjacent cells are horizontally or vertically neighboring. The same letter cell may not be used more than once."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 64,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# depth first search \n",
+    "\n",
+    "\n",
+    "def exist(self, board, word):\n",
+    "    if not board:\n",
+    "        return False\n",
+    "    for i in xrange(len(board)):\n",
+    "        for j in xrange(len(board[0])):\n",
+    "            if self.dfs(board, i, j, word):\n",
+    "                return True\n",
+    "    return False\n",
+    "\n",
+    "# check whether can find word, start at (i,j) position    \n",
+    "def dfs(self, board, i, j, word):\n",
+    "    if len(word) == 0: # all the characters are checked\n",
+    "        return True\n",
+    "    if i<0 or i>=len(board) or j<0 or j>=len(board[0]) or word[0]!=board[i][j]:\n",
+    "        return False\n",
+    "    tmp = board[i][j]  # first character is found, check the remaining part\n",
+    "    board[i][j] = \"#\"  # avoid visit agian \n",
+    "    # check whether can find \"word\" along one direction\n",
+    "    res = self.dfs(board, i+1, j, word[1:]) or self.dfs(board, i-1, j, word[1:]) \\\n",
+    "    or self.dfs(board, i, j+1, word[1:]) or self.dfs(board, i, j-1, word[1:])\n",
+    "    board[i][j] = tmp\n",
+    "    return res\n",
+    "\n",
+    "board = [[\"A\",\"B\",\"C\",\"E\"],[\"S\",\"F\",\"C\",\"S\"],[\"A\",\"D\",\"E\",\"E\"]]\n",
+    "word = \"ABCCED\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 98. Validate Binary Search Tree\n",
+    "\n",
+    "Given the root of a binary tree, determine if it is a valid binary search tree (BST).\n",
+    "\n",
+    "A valid BST is defined as follows:\n",
+    "\n",
+    "The left\n",
+    "subtree\n",
+    "of a node contains only nodes with keys less than the node's key.\n",
+    "The right subtree of a node contains only nodes with keys greater than the node's key.\n",
+    "Both the left and right subtrees must also be binary search trees.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 69,
+   "metadata": {},
+   "outputs": [
+    {
+     "ename": "AttributeError",
+     "evalue": "'list' object has no attribute 'val'",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
+      "Cell \u001b[0;32mIn[69], line 19\u001b[0m\n\u001b[1;32m     15\u001b[0m         \u001b[39mreturn\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39misValidBST(root\u001b[39m.\u001b[39mleft, floor, root\u001b[39m.\u001b[39mval) \u001b[39mand\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39misValidBST(root\u001b[39m.\u001b[39mright, root\u001b[39m.\u001b[39mval, ceiling)\n\u001b[1;32m     17\u001b[0m root \u001b[39m=\u001b[39m [\u001b[39m2\u001b[39m,\u001b[39m1\u001b[39m,\u001b[39m3\u001b[39m]\n\u001b[0;32m---> 19\u001b[0m Solution()\u001b[39m.\u001b[39;49misValidBST(root)\n",
+      "Cell \u001b[0;32mIn[69], line 12\u001b[0m, in \u001b[0;36mSolution.isValidBST\u001b[0;34m(self, root, floor, ceiling)\u001b[0m\n\u001b[1;32m     10\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39mnot\u001b[39;00m root: \n\u001b[1;32m     11\u001b[0m     \u001b[39mreturn\u001b[39;00m \u001b[39mTrue\u001b[39;00m\n\u001b[0;32m---> 12\u001b[0m \u001b[39mif\u001b[39;00m root\u001b[39m.\u001b[39;49mval \u001b[39m<\u001b[39m\u001b[39m=\u001b[39m floor \u001b[39mor\u001b[39;00m root\u001b[39m.\u001b[39mval \u001b[39m>\u001b[39m\u001b[39m=\u001b[39m ceiling:\n\u001b[1;32m     13\u001b[0m     \u001b[39mreturn\u001b[39;00m \u001b[39mFalse\u001b[39;00m\n\u001b[1;32m     14\u001b[0m \u001b[39m# in the left branch, root is the new ceiling; contrarily root is the new floor in right branch\u001b[39;00m\n",
+      "\u001b[0;31mAttributeError\u001b[0m: 'list' object has no attribute 'val'"
+     ]
+    }
+   ],
+   "source": [
+    "# Definition for a binary tree node.\n",
+    "# class TreeNode:\n",
+    "#     def __init__(self, val=0, left=None, right=None):\n",
+    "#         self.val = val\n",
+    "#         self.left = left\n",
+    "#         self.right = right\n",
+    "\n",
+    "class Solution:\n",
+    "    def isValidBST(self, root, floor=float('-inf'), ceiling=float('inf')):\n",
+    "        if not root: \n",
+    "            return True\n",
+    "        if root.val <= floor or root.val >= ceiling:\n",
+    "            return False\n",
+    "        # in the left branch, root is the new ceiling; contrarily root is the new floor in right branch\n",
+    "        return self.isValidBST(root.left, floor, root.val) and self.isValidBST(root.right, root.val, ceiling)\n",
+    "\n",
+    "root = [2,1,3]\n",
+    "\n",
+    "Solution().isValidBST(root)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 100. Same Tree\n",
+    "\n",
+    "Given the roots of two binary trees p and q, write a function to check if they are the same or not.\n",
+    "\n",
+    "Two binary trees are considered the same if they are structurally identical, and the nodes have the same value."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 70,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from collections import deque\n",
+    "class Solution:\n",
+    "    def isSameTree(self, p, q):\n",
+    "        \"\"\"\n",
+    "        :type p: TreeNode\n",
+    "        :type q: TreeNode\n",
+    "        :rtype: bool\n",
+    "        \"\"\"    \n",
+    "        def check(p, q):\n",
+    "            # if both are None\n",
+    "            if not p and not q:\n",
+    "                return True\n",
+    "            # one of p and q is None\n",
+    "            if not q or not p:\n",
+    "                return False\n",
+    "            if p.val != q.val:\n",
+    "                return False\n",
+    "            return True\n",
+    "        \n",
+    "        deq = deque([(p, q),])\n",
+    "        while deq:\n",
+    "            p, q = deq.popleft()\n",
+    "            if not check(p, q):\n",
+    "                return False\n",
+    "            \n",
+    "            if p:\n",
+    "                deq.append((p.left, q.left))\n",
+    "                deq.append((p.right, q.right))\n",
+    "                    \n",
+    "        return True\n",
+    "\n",
+    "        "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 102. Binary Tree Level Order Traversal\n",
+    "\n",
+    "Given the root of a binary tree, return the level order traversal of its nodes' values. (i.e., from left to right, level by level)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 71,
+   "metadata": {},
+   "outputs": [
+    {
+     "ename": "NameError",
+     "evalue": "name 'null' is not defined",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
+      "Cell \u001b[0;32mIn[71], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m root \u001b[39m=\u001b[39m [\u001b[39m3\u001b[39m,\u001b[39m9\u001b[39m,\u001b[39m20\u001b[39m,null,null,\u001b[39m15\u001b[39m,\u001b[39m7\u001b[39m]\n",
+      "\u001b[0;31mNameError\u001b[0m: name 'null' is not defined"
+     ]
+    }
+   ],
+   "source": [
+    "def levelOrder(self, root):\n",
+    "    ans, level = [], [root]\n",
+    "    while root and level:\n",
+    "        ans.append([node.val for node in level])\n",
+    "        LRpair = [(node.left, node.right) for node in level]\n",
+    "        level = [leaf for LR in LRpair for leaf in LR if leaf]\n",
+    "    return ans\n",
+    "\n",
+    "root = [3,9,20,null,null,15,7]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 72,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from collections import deque\n",
+    "def levelOrder(self, root):\n",
+    "    # traverse in order level, keeping track of (row number, current node)\n",
+    "    queue = deque([(0, root)])\n",
+    "    # keep track of the nodes in each row\n",
+    "    d = {}\n",
+    "\n",
+    "    while queue:\n",
+    "        row, node = queue.popleft()\n",
+    "        if node:\n",
+    "            d[row] = d.get(row, []) + [node.val]\n",
+    "            queue += (row+1, node.left), (row+1, node.right)\n",
+    "\n",
+    "    # return a list of lists containing node values in increasing order with respect to the row number\n",
+    "    return [d[row] for row in sorted(d.keys())]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 104. Maximum Depth of Binary Tree\n",
+    "\n",
+    "Given the root of a binary tree, return its maximum depth.\n",
+    "\n",
+    "A binary tree's maximum depth is the number of nodes along the longest path from the root node down to the farthest leaf node."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 76,
+   "metadata": {},
+   "outputs": [
+    {
+     "ename": "NameError",
+     "evalue": "name 'null' is not defined",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
+      "Cell \u001b[0;32mIn[76], line 17\u001b[0m\n\u001b[1;32m     12\u001b[0m             \u001b[39mreturn\u001b[39;00m \u001b[39mmax\u001b[39m(dfs(root\u001b[39m.\u001b[39mleft, depth \u001b[39m+\u001b[39m \u001b[39m1\u001b[39m), dfs(root\u001b[39m.\u001b[39mright, depth \u001b[39m+\u001b[39m \u001b[39m1\u001b[39m))\n\u001b[1;32m     14\u001b[0m         \u001b[39mreturn\u001b[39;00m dfs(root, \u001b[39m0\u001b[39m)\n\u001b[0;32m---> 17\u001b[0m root \u001b[39m=\u001b[39m [\u001b[39m3\u001b[39m,\u001b[39m9\u001b[39m,\u001b[39m20\u001b[39m,null,null,\u001b[39m15\u001b[39m,\u001b[39m7\u001b[39m]\n\u001b[1;32m     19\u001b[0m Solution()\u001b[39m.\u001b[39mmaxDepth(root)\n",
+      "\u001b[0;31mNameError\u001b[0m: name 'null' is not defined"
+     ]
+    }
+   ],
+   "source": [
+    "# Definition for a binary tree node.\n",
+    "\n",
+    "# https://leetcode.com/problems/maximum-depth-of-binary-tree/solutions/1769367/python3-recursive-dfs-explained/?orderBy=most_votes\n",
+    "class TreeNode:\n",
+    "    def __init__(self, val=0, left=None, right=None):\n",
+    "        self.val = val\n",
+    "        self.left = left\n",
+    "        self.right = right\n",
+    "\n",
+    "class Solution:\n",
+    "    def maxDepth(self, root: Optional[TreeNode]) -> int:\n",
+    "        def dfs(root, depth):\n",
+    "            if not root: return depth\n",
+    "            return max(dfs(root.left, depth + 1), dfs(root.right, depth + 1))\n",
+    "                       \n",
+    "        return dfs(root, 0)\n",
+    "\n",
+    "\n",
+    "root = [3,9,20,null,null,15,7]\n",
+    "\n",
+    "Solution().maxDepth(root)\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Here, \"Object references are passed by value.\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 78,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[0, 1]\n",
+      "[0, 1]\n"
+     ]
+    }
+   ],
+   "source": [
+    "listA = [0]\n",
+    "listB = listA\n",
+    "listB.append(1)\n",
+    "print (listA)\n",
+    "print(listB)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 83,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[0]\n",
+      "[0, 1]\n",
+      "[0]\n",
+      "[0, 1]\n",
+      "[0, 1]\n"
+     ]
+    }
+   ],
+   "source": [
+    "def reassign(list):\n",
+    "  print(list)\n",
+    "  list = [0, 1]\n",
+    "  print(list)\n",
+    "\n",
+    "def append(list):\n",
+    "  print(list)\n",
+    "  list.append(1)\n",
+    "  print(list)\n",
+    "\n",
+    "list = [0]\n",
+    "reassign(list)\n",
+    "append(list)\n",
+    "\n",
+    "print(list)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 85,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "some value\n"
+     ]
+    }
+   ],
+   "source": [
+    "def foo(x):\n",
+    "    print(x)\n",
+    "\n",
+    "bar = 'some value'\n",
+    "foo(bar)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 88,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "another value\n",
+      "some value\n"
+     ]
+    }
+   ],
+   "source": [
+    "def foo(x):\n",
+    "    x = 'another value'\n",
+    "    print (x)\n",
+    "\n",
+    "bar = 'some value'\n",
+    "foo(bar)\n",
+    "print(bar)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 93,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[0]\n",
+      "[0, 1]\n",
+      "[0, 1]\n"
+     ]
+    }
+   ],
+   "source": [
+    "def append_one(li):\n",
+    "    print(li)\n",
+    "    li.append(1)\n",
+    "    print(li)\n",
+    "    return li\n",
+    "\n",
+    "x = [0]\n",
+    "x = append_one(x)\n",
+    "print(x)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 105. Construct Binary Tree From Preorder and Inorder Traversal \n",
+    "\n",
+    "Given two integer arrays preorder and inorder where preorder is the preorder traversal of a binary tree and inorder is the inorder traversal of the same tree, construct and return the binary tree."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 102,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "3\n"
+     ]
+    }
+   ],
+   "source": [
+    "class TreeNode:\n",
+    "    def __init__(self, val=0, left=None, right=None):\n",
+    "        self.val = val\n",
+    "        self.left = left\n",
+    "        self.right = right\n",
+    "\n",
+    "class Solution:\n",
+    "    def buildTree(self, preorder: List[int], inorder: List[int]) -> TreeNode:\n",
+    "\n",
+    "        def array_to_tree(left, right):\n",
+    "            nonlocal preorder_index\n",
+    "            # if there are no elements to construct the tree\n",
+    "            if left > right: return None\n",
+    "\n",
+    "            # select the preorder_index element as the root and increment it\n",
+    "            root_value = preorder[preorder_index]\n",
+    "            root = TreeNode(root_value)\n",
+    "\n",
+    "\n",
+    "            preorder_index += 1\n",
+    "\n",
+    "            # build left and right subtree\n",
+    "            # excluding inorder_index_map[root_value] element because it's the root\n",
+    "            root.left = array_to_tree(left, inorder_index_map[root_value] - 1)\n",
+    "            root.right = array_to_tree(inorder_index_map[root_value] + 1, right)\n",
+    "\n",
+    "            return root\n",
+    "\n",
+    "        preorder_index = 0\n",
+    "\n",
+    "        # build a hashmap to store value -> its index relations\n",
+    "        inorder_index_map = {}\n",
+    "        for index, value in enumerate(inorder):\n",
+    "            inorder_index_map[value] = index\n",
+    "\n",
+    "        return array_to_tree(0, len(preorder) - 1)\n",
+    "\n",
+    "preorder = [3,9,20,15,7]\n",
+    "inorder = [9,3,15,20,7]\n",
+    "\n",
+    "# preorder = [-1]\n",
+    "# inorder = [-1]\n",
+    "\n",
+    "foo = Solution().buildTree(preorder, inorder)\n",
+    "print(foo.val)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 121. Best Time to Buy and Sell Stock\n",
+    "\n",
+    "You are given an array prices where prices[i] is the price of a given stock on the ith day.\n",
+    "\n",
+    "You want to maximize your profit by choosing a single day to buy one stock and choosing a different day in the future to sell that stock.\n",
+    "\n",
+    "Return the maximum profit you can achieve from this transaction. If you cannot achieve any profit, return 0."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 109,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "inf\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "5"
+      ]
+     },
+     "execution_count": 109,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# could brute force with O(n^2) \n",
+    "class Solution:\n",
+    "    def maxProfit(self, prices: List[int]) -> int:\n",
+    "        min_price = float('inf')\n",
+    "        print(min_price)\n",
+    "        max_profit = 0\n",
+    "        # for i in range(len(prices)):\n",
+    "        for price in prices:\n",
+    "            diff = price - min_price\n",
+    "\n",
+    "            if price < min_price:\n",
+    "                min_price = price\n",
+    "            elif diff > max_profit:\n",
+    "                max_profit = diff\n",
+    "                \n",
+    "        return max_profit\n",
+    "\n",
+    "prices = [7,1,5,3,6,4]\n",
+    "\n",
+    "min(prices)\n",
+    "\n",
+    "Solution().maxProfit(prices)\n",
+    "# Buy on day 2 (price = 1) and sell on day 5 (price = 6), profit = 6-1 = 5.\n",
+    "# Note that buying on day 2 and selling on day 1 is not allowed because you must buy before you sell."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 114,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "1"
+      ]
+     },
+     "execution_count": 114,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "prices = [7,1,5,3,6,4]\n",
+    "\n",
+    "# using the builtins but doesn't really work very nicely. \n",
+    "min(prices)\n",
+    "max(prices)\n",
+    "prices.index(max(prices))\n",
+    "prices.index(min(prices))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 124. Binary Tree Maximum Path Sum (hard)\n",
+    "\n",
+    "A path in a binary tree is a sequence of nodes where each pair of adjacent nodes in the sequence has an edge connecting them. A node can only appear in the sequence at most once. Note that the path does not need to pass through the root.\n",
+    "\n",
+    "The path sum of a path is the sum of the node's values in the path.\n",
+    "\n",
+    "Given the root of a binary tree, return the maximum path sum of any non-empty path."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 125. Valid Palindrome (easy)\n",
+    "\n",
+    "A phrase is a palindrome if, after converting all uppercase letters into lowercase letters and removing all non-alphanumeric characters, it reads the same forward and backward. Alphanumeric characters include letters and numbers.\n",
+    "\n",
+    "Given a string s, return true if it is a palindrome, or false otherwise."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 116,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "amanaplanacanalpanama\n",
+      "amanaplanacanalpanama\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "True"
+      ]
+     },
+     "execution_count": 116,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "def isPalindrome(self, s):\n",
+    "    l, r = 0, len(s)-1\n",
+    "    while l < r:\n",
+    "        while l < r and not s[l].isalnum():\n",
+    "            l += 1\n",
+    "        while l <r and not s[r].isalnum():\n",
+    "            r -= 1\n",
+    "        if s[l].lower() != s[r].lower():\n",
+    "            return False\n",
+    "        l +=1; r -= 1\n",
+    "    return True\n",
+    "\n",
+    "# so dope bro \n",
+    "class Solution:\n",
+    "    def isPalindrome(self, s):\n",
+    "        s = ''.join(e for e in s if e.isalnum()).lower()\n",
+    "        print(s)\n",
+    "        print(s[::-1])\n",
+    "        return s==s[::-1]\n",
+    "\n",
+    "s = \"A man, a plan, a canal: Panama\"\n",
+    "\n",
+    "Solution().isPalindrome(s)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "4.2"
+      ]
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "float(\"4.2/5\".split('/')[0].strip())\n",
+    "\n",
+    "float(\"4.2/5\".split(\"/\")[0].strip())"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Balanced sum \n",
+    "\n",
+    "Given an array of integers nums, calculate the pivot index of this array.\n",
+    "\n",
+    "The pivot index is the index where the sum of all the numbers strictly to the left of the index is equal to the sum of all the numbers strictly to the index's right.\n",
+    "\n",
+    "If the index is on the left edge of the array, then the left sum is 0 because there are no elements to the left. This also applies to the right edge of the array.\n",
+    "\n",
+    "Return the leftmost pivot index. If no such index exists, return -1."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Time Complexity : O(n)\n",
+    "# Space Complexity : O(1)\n",
+    "class Solution(object):\n",
+    "    def pivotIndex(self, nums):\n",
+    "        # Initialize leftSum & rightSum to store the sum of all the numbers strictly to the index's left & right respectively...\n",
+    "        leftSum, rightSum = 0, sum(nums)\n",
+    "        # Traverse elements through the loop...\n",
+    "        for idx, ele in enumerate(nums):\n",
+    "            rightSum -= ele\n",
+    "            # If the sum of all the numbers strictly to the left of the index is equal to the sum of all the numbers strictly to the index's right...\n",
+    "            if leftSum == rightSum:\n",
+    "                return idx      # Return the pivot index...\n",
+    "            leftSum += ele\n",
+    "        return -1       # If there is no index that satisfies the conditions in the problem statement..."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "3"
+      ]
+     },
+     "execution_count": 2,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "nums = [1,7,3,6,5,6]\n",
+    "Solution().pivotIndex(nums)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.4"
+  },
+  "orig_nbformat": 4,
+  "vscode": {
+   "interpreter": {
+    "hash": "b0fa6594d8f4cbf19f97940f81e996739fb7646882a419484c72d19e05852a7e"
+   }
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

code/lru_cache_leetcode.ipynb

@@ -0,0 +1,286 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "OrderedDict([(1, 1)])\n",
+      "OrderedDict([(1, 1), (2, 2)])\n",
+      "OrderedDict([(2, 2), (1, 1)])\n",
+      "OrderedDict([(1, 1), (3, 3)])\n",
+      "OrderedDict([(1, 1), (3, 3)])\n",
+      "OrderedDict([(3, 3), (4, 4)])\n",
+      "OrderedDict([(3, 3), (4, 4)])\n",
+      "OrderedDict([(4, 4), (3, 3)])\n",
+      "OrderedDict([(3, 3), (4, 4)])\n"
+     ]
+    }
+   ],
+   "source": [
+    "from collections import OrderedDict\n",
+    "\n",
+    "class LRUCache:\n",
+    "\n",
+    "  # initialising capacity\n",
+    "  def __init__(self, capacity: int):\n",
+    "    self.cache = OrderedDict()\n",
+    "    self.capacity = capacity\n",
+    "\n",
+    "  # we return the value of the key\n",
+    "  # that is queried in O(1) and return -1 if we\n",
+    "  # don't find the key in out dict / cache.\n",
+    "  # And also move the key to the end\n",
+    "  # to show that it was recently used.\n",
+    "  def get(self, key: int) -> int:\n",
+    "    if key not in self.cache:\n",
+    "      return -1\n",
+    "    else:\n",
+    "      self.cache.move_to_end(key)\n",
+    "      return self.cache[key]\n",
+    "\n",
+    "  # first, we add / update the key by conventional methods.\n",
+    "  # And also move the key to the end to show that it was recently used.\n",
+    "  # But here we will also check whether the length of our\n",
+    "  # ordered dictionary has exceeded our capacity,\n",
+    "  # If so we remove the first key (least recently used)\n",
+    "  def put(self, key: int, value: int) -> None:\n",
+    "    self.cache[key] = value\n",
+    "    self.cache.move_to_end(key)\n",
+    "    if len(self.cache) > self.capacity:\n",
+    "      self.cache.popitem(last = False)\n",
+    "\n",
+    "\n",
+    "# RUNNER\n",
+    "# initializing our cache with the capacity of 2\n",
+    "cache = LRUCache(2)\n",
+    "\n",
+    "\n",
+    "cache.put(1, 1)\n",
+    "print(cache.cache)\n",
+    "cache.put(2, 2)\n",
+    "print(cache.cache)\n",
+    "cache.get(1)\n",
+    "print(cache.cache)\n",
+    "cache.put(3, 3)\n",
+    "print(cache.cache)\n",
+    "cache.get(2)\n",
+    "print(cache.cache)\n",
+    "cache.put(4, 4)\n",
+    "print(cache.cache)\n",
+    "cache.get(1)\n",
+    "print(cache.cache)\n",
+    "cache.get(3)\n",
+    "print(cache.cache)\n",
+    "cache.get(4)\n",
+    "print(cache.cache)\n",
+    "\n",
+    "#This code was contributed by Sachin Negi\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Stop Words (EASY) - 15 minutes\n",
+    "\n",
+    "Implement a function that takes a string text and an integer k, and returns the list of words that occur in the text at least k times. The words must be returned in the order of their first occurrence in the text."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "{'the': [2, 0], 'brown': [3, 1], 'fox': [1, 2], 'jumps': [1, 3], 'over': [1, 4], 'dog': [1, 7], 'and': [1, 8], 'runs': [1, 9], 'away': [1, 10], 'to': [1, 11], 'a': [1, 12], 'house': [1, 14]}\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "['the',\n",
+       " 'brown',\n",
+       " 'fox',\n",
+       " 'jumps',\n",
+       " 'over',\n",
+       " 'dog',\n",
+       " 'and',\n",
+       " 'runs',\n",
+       " 'away',\n",
+       " 'to',\n",
+       " 'a',\n",
+       " 'house']"
+      ]
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "text = \"the brown fox jumps over the brown dog and runs away to a brown house\"\n",
+    "k = 2 \n",
+    "\n",
+    "def stop_words(text, k):\n",
+    "    cnt = {}\n",
+    "    for i, w in enumerate(text.split(\" \")):\n",
+    "        if w not in cnt:\n",
+    "            cnt[w] = [0, i]\n",
+    "        cnt[w][0] += 1\n",
+    "    print(cnt)\n",
+    "    stop_words = [(i, w) for w, (c, i) in cnt.items() if c >= k]\n",
+    "    res = [w for _, w in sorted(stop_words)]\n",
+    "    return res\n",
+    "\n",
+    "\n",
+    "stop_words(text, 1)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# DPI Software Traffic Deviation Report - Medium SQL - 35 minutes\n",
+    "\n",
+    "As part of HackerSniff's DPI (Deep Packet Inspection) software analytics, a team needs a list of all the clients and traffic protocols they have used.\n",
+    "\n",
+    " \n",
+    "\n",
+    "The result should be in the following format: client, protocol.\n",
+    "\n",
+    "    protocol is a comma-separated list of all the protocols for particular client, ordered descending by total traffic, which calculated as sum of traffic_in and traffic_out.\n",
+    "    Results should be sorted ascending by client.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "sql = \"\"\"\n",
+    "WITH _data AS (\n",
+    "SELECT client,\n",
+    "    protocol,\n",
+    "    SUM(traffic_in) + SUM(traffic_out) traffic\n",
+    "FROM traffic\n",
+    "GROUP BY client, protocol\n",
+    ")\n",
+    "\n",
+    "SELECT client,\n",
+    "    GROUP_CONCAT(protocol ORDER BY traffic DESC) AS protocol\n",
+    "FROM _data\n",
+    "GROUP BY client\n",
+    "ORDER BY client\n",
+    "\"\"\"\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Balanced Sales Array (EASY) \n",
+    "\n",
+    "Given has an array of sales numbers, what is the index of the smallest index element for which the sums of all elements to the left and to the right are equal. The array may not be reordered.\n",
+    "For example, given the array sales = [1, 2, 3, 4, 6],we see that 1+2+3=6，Using zero based indexing,sales[3] = 4 is the value sought.The index is 3."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "16\n",
+      "16\n",
+      "[1, 3, 6, 10, 16]\n",
+      "[6, 10, 13, 15, 16]\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "3"
+      ]
+     },
+     "execution_count": 12,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "data = [1, 2, 3, 4, 6]\n",
+    "\n",
+    "class Solution:\n",
+    "    \"\"\"\n",
+    "    @param sales: a integer array\n",
+    "    @return: return a Integer\n",
+    "    \"\"\"\n",
+    "    import itertools\n",
+    "    def BalancedSalesArray(self, sales):\n",
+    "        # write your code here\n",
+    "        left,right = [],[]\n",
+    "        leftsum = rightsum = 0\n",
+    "        n = len(sales)\n",
+    "        for i in range(n):\n",
+    "            leftsum += sales[i] \n",
+    "            left.append(leftsum)\n",
+    "            rightsum += sales[n-1-i]\n",
+    "            right.append(rightsum)\n",
+    "            \n",
+    "        print(leftsum)\n",
+    "        print(rightsum)\n",
+    "\n",
+    "        print(left)\n",
+    "        print(right)\n",
+    "\n",
+    "        n = len(sales)\n",
+    "        for i in range(n):\n",
+    "            if left[i] == right[n-1-i]:\n",
+    "                return i \n",
+    "        return -1\n",
+    "        \n",
+    "        \n",
+    "Solution().BalancedSalesArray(data)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3.10.5 ('pytorch-nightly')",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.5"
+  },
+  "orig_nbformat": 4,
+  "vscode": {
+   "interpreter": {
+    "hash": "8a8bcccfb183d1298694efece6cf41240378bc61621e95c864629a40c5876542"
+   }
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

code/m1_gpu_pytorch.ipynb

@@ -0,0 +1,472 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "ename": "ModuleNotFoundError",
+     "evalue": "No module named 'torch'",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
+      "\u001b[1;32m/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb Cell 1'\u001b[0m in \u001b[0;36m<cell line: 1>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb#ch0000000?line=0'>1</a>\u001b[0m \u001b[39mimport\u001b[39;00m \u001b[39mtorch\u001b[39;00m\n\u001b[1;32m      <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb#ch0000000?line=1'>2</a>\u001b[0m \u001b[39mfrom\u001b[39;00m \u001b[39mtorch\u001b[39;00m \u001b[39mimport\u001b[39;00m nn\n\u001b[1;32m      <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb#ch0000000?line=2'>3</a>\u001b[0m \u001b[39mfrom\u001b[39;00m \u001b[39mtorch\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mutils\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mdata\u001b[39;00m \u001b[39mimport\u001b[39;00m DataLoader\n",
+      "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'torch'"
+     ]
+    }
+   ],
+   "source": [
+    "import torch\n",
+    "from torch import nn\n",
+    "from torch.utils.data import DataLoader\n",
+    "from torchvision import datasets\n",
+    "from torchvision.transforms import ToTensor"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/nightly/cpu\n",
+      "Collecting torch\n",
+      "  Downloading https://download.pytorch.org/whl/nightly/cpu/torch-1.13.0.dev20220703-cp310-none-macosx_11_0_arm64.whl (50.1 MB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.1/50.1 MB\u001b[0m \u001b[31m30.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m00:01\u001b[0m\n",
+      "\u001b[?25hCollecting torchvision\n",
+      "  Downloading https://download.pytorch.org/whl/nightly/cpu/torchvision-0.14.0.dev20220703-cp310-cp310-macosx_11_0_arm64.whl (1.4 MB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.4/1.4 MB\u001b[0m \u001b[31m21.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n",
+      "\u001b[?25hCollecting torchaudio\n",
+      "  Downloading https://download.pytorch.org/whl/nightly/cpu/torchaudio-0.14.0.dev20220603-cp310-cp310-macosx_11_0_arm64.whl (2.7 MB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.7/2.7 MB\u001b[0m \u001b[31m9.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m0:01\u001b[0m\n",
+      "\u001b[?25hCollecting typing-extensions\n",
+      "  Using cached typing_extensions-4.3.0-py3-none-any.whl (25 kB)\n",
+      "Collecting pillow!=8.3.*,>=5.3.0\n",
+      "  Downloading Pillow-9.2.0-cp310-cp310-macosx_11_0_arm64.whl (2.8 MB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.8/2.8 MB\u001b[0m \u001b[31m14.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m00:01\u001b[0m\n",
+      "\u001b[?25hCollecting requests\n",
+      "  Using cached requests-2.28.1-py3-none-any.whl (62 kB)\n",
+      "Collecting numpy\n",
+      "  Downloading numpy-1.23.0-cp310-cp310-macosx_11_0_arm64.whl (13.3 MB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m13.3/13.3 MB\u001b[0m \u001b[31m44.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m00:01\u001b[0m\n",
+      "\u001b[?25hCollecting certifi>=2017.4.17\n",
+      "  Downloading certifi-2022.6.15-py3-none-any.whl (160 kB)\n",
+      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m160.2/160.2 kB\u001b[0m \u001b[31m6.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
+      "\u001b[?25hCollecting idna<4,>=2.5\n",
+      "  Using cached idna-3.3-py3-none-any.whl (61 kB)\n",
+      "Collecting charset-normalizer<3,>=2\n",
+      "  Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)\n",
+      "Collecting urllib3<1.27,>=1.21.1\n",
+      "  Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)\n",
+      "Installing collected packages: urllib3, typing-extensions, pillow, numpy, idna, charset-normalizer, certifi, torch, requests, torchvision, torchaudio\n",
+      "Successfully installed certifi-2022.6.15 charset-normalizer-2.1.0 idna-3.3 numpy-1.23.0 pillow-9.2.0 requests-2.28.1 torch-1.13.0.dev20220703 torchaudio-0.14.0.dev20220603 torchvision-0.14.0.dev20220703 typing-extensions-4.3.0 urllib3-1.26.9\n"
+     ]
+    }
+   ],
+   "source": [
+    "! pip install --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cpu"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "torch 1.13.0.dev20220703\n",
+      "device mps\n",
+      "Downloading https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz to data/cifar-10-python.tar.gz\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "100.0%\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Extracting data/cifar-10-python.tar.gz to data\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Downloading: \"https://github.com/pytorch/vision/zipball/v0.11.0\" to /Users/johnnydevriese/.cache/torch/hub/v0.11.0.zip\n",
+      "/Users/johnnydevriese/miniforge3/envs/pytorch-nightly/lib/python3.10/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and will be removed in 0.15, please use 'weights' instead.\n",
+      "  warnings.warn(\n",
+      "/Users/johnnydevriese/miniforge3/envs/pytorch-nightly/lib/python3.10/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and will be removed in 0.15. The current behavior is equivalent to passing `weights=None`.\n",
+      "  warnings.warn(msg)\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch: 001/001 | Batch 0000/1406 | Loss: 2.5887\n",
+      "Epoch: 001/001 | Batch 0100/1406 | Loss: 2.4339\n",
+      "Epoch: 001/001 | Batch 0200/1406 | Loss: 2.0386\n",
+      "Epoch: 001/001 | Batch 0300/1406 | Loss: 2.0561\n",
+      "Epoch: 001/001 | Batch 0400/1406 | Loss: 2.1730\n",
+      "Epoch: 001/001 | Batch 0500/1406 | Loss: 2.1067\n"
+     ]
+    },
+    {
+     "ename": "KeyboardInterrupt",
+     "evalue": "",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
+      "\u001b[1;32m/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb Cell 3'\u001b[0m in \u001b[0;36m<cell line: 205>\u001b[0;34m()\u001b[0m\n\u001b[1;32m    <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb#ch0000001?line=256'>257</a>\u001b[0m model \u001b[39m=\u001b[39m model\u001b[39m.\u001b[39mto(DEVICE)\n\u001b[1;32m    <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb#ch0000001?line=258'>259</a>\u001b[0m optimizer \u001b[39m=\u001b[39m torch\u001b[39m.\u001b[39moptim\u001b[39m.\u001b[39mAdam(model\u001b[39m.\u001b[39mparameters(), lr\u001b[39m=\u001b[39m\u001b[39m0.0005\u001b[39m)\n\u001b[0;32m--> <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb#ch0000001?line=260'>261</a>\u001b[0m minibatch_loss_list, train_acc_list, valid_acc_list \u001b[39m=\u001b[39m train_classifier_simple_v2(\n\u001b[1;32m    <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb#ch0000001?line=261'>262</a>\u001b[0m     model\u001b[39m=\u001b[39;49mmodel,\n\u001b[1;32m    <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb#ch0000001?line=262'>263</a>\u001b[0m     num_epochs\u001b[39m=\u001b[39;49mNUM_EPOCHS,\n\u001b[1;32m    <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb#ch0000001?line=263'>264</a>\u001b[0m     train_loader\u001b[39m=\u001b[39;49mtrain_loader,\n\u001b[1;32m    <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb#ch0000001?line=264'>265</a>\u001b[0m     valid_loader\u001b[39m=\u001b[39;49mvalid_loader,\n\u001b[1;32m    <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb#ch0000001?line=265'>266</a>\u001b[0m     test_loader\u001b[39m=\u001b[39;49mtest_loader,\n\u001b[1;32m    <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb#ch0000001?line=266'>267</a>\u001b[0m     optimizer\u001b[39m=\u001b[39;49moptimizer,\n\u001b[1;32m    <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb#ch0000001?line=267'>268</a>\u001b[0m     best_model_save_path\u001b[39m=\u001b[39;49m\u001b[39mNone\u001b[39;49;00m,\n\u001b[1;32m    <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb#ch0000001?line=268'>269</a>\u001b[0m     device\u001b[39m=\u001b[39;49mDEVICE,\n\u001b[1;32m    <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb#ch0000001?line=269'>270</a>\u001b[0m     scheduler_on\u001b[39m=\u001b[39;49m\u001b[39m\"\u001b[39;49m\u001b[39mvalid_acc\u001b[39;49m\u001b[39m\"\u001b[39;49m,\n\u001b[1;32m    <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb#ch0000001?line=270'>271</a>\u001b[0m     logging_interval\u001b[39m=\u001b[39;49m\u001b[39m100\u001b[39;49m,\n\u001b[1;32m    <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb#ch0000001?line=271'>272</a>\u001b[0m )\n",
+      "\u001b[1;32m/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb Cell 3'\u001b[0m in \u001b[0;36mtrain_classifier_simple_v2\u001b[0;34m(model, num_epochs, train_loader, valid_loader, test_loader, optimizer, device, logging_interval, best_model_save_path, scheduler, skip_train_acc, scheduler_on)\u001b[0m\n\u001b[1;32m     <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb#ch0000001?line=66'>67</a>\u001b[0m targets \u001b[39m=\u001b[39m targets\u001b[39m.\u001b[39mto(device)\n\u001b[1;32m     <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb#ch0000001?line=68'>69</a>\u001b[0m \u001b[39m# ## FORWARD AND BACK PROP\u001b[39;00m\n\u001b[0;32m---> <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb#ch0000001?line=69'>70</a>\u001b[0m logits \u001b[39m=\u001b[39m model(features)\n\u001b[1;32m     <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb#ch0000001?line=70'>71</a>\u001b[0m loss \u001b[39m=\u001b[39m torch\u001b[39m.\u001b[39mnn\u001b[39m.\u001b[39mfunctional\u001b[39m.\u001b[39mcross_entropy(logits, targets)\n\u001b[1;32m     <a href='vscode-notebook-cell:/Users/johnnydevriese/projects/jupyter/m1_gpu_pytorch.ipynb#ch0000001?line=71'>72</a>\u001b[0m optimizer\u001b[39m.\u001b[39mzero_grad()\n",
+      "File \u001b[0;32m~/miniforge3/envs/pytorch-nightly/lib/python3.10/site-packages/torch/nn/modules/module.py:1186\u001b[0m, in \u001b[0;36mModule._call_impl\u001b[0;34m(self, *input, **kwargs)\u001b[0m\n\u001b[1;32m   1182\u001b[0m \u001b[39m# If we don't have any hooks, we want to skip the rest of the logic in\u001b[39;00m\n\u001b[1;32m   1183\u001b[0m \u001b[39m# this function, and just call forward.\u001b[39;00m\n\u001b[1;32m   1184\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39mnot\u001b[39;00m (\u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_backward_hooks \u001b[39mor\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_forward_hooks \u001b[39mor\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_forward_pre_hooks \u001b[39mor\u001b[39;00m _global_backward_hooks\n\u001b[1;32m   1185\u001b[0m         \u001b[39mor\u001b[39;00m _global_forward_hooks \u001b[39mor\u001b[39;00m _global_forward_pre_hooks):\n\u001b[0;32m-> 1186\u001b[0m     \u001b[39mreturn\u001b[39;00m forward_call(\u001b[39m*\u001b[39;49m\u001b[39minput\u001b[39;49m, \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n\u001b[1;32m   1187\u001b[0m \u001b[39m# Do not call functions when jit is used\u001b[39;00m\n\u001b[1;32m   1188\u001b[0m full_backward_hooks, non_full_backward_hooks \u001b[39m=\u001b[39m [], []\n",
+      "File \u001b[0;32m~/miniforge3/envs/pytorch-nightly/lib/python3.10/site-packages/torchvision/models/vgg.py:66\u001b[0m, in \u001b[0;36mVGG.forward\u001b[0;34m(self, x)\u001b[0m\n\u001b[1;32m     65\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39mforward\u001b[39m(\u001b[39mself\u001b[39m, x: torch\u001b[39m.\u001b[39mTensor) \u001b[39m-\u001b[39m\u001b[39m>\u001b[39m torch\u001b[39m.\u001b[39mTensor:\n\u001b[0;32m---> 66\u001b[0m     x \u001b[39m=\u001b[39m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mfeatures(x)\n\u001b[1;32m     67\u001b[0m     x \u001b[39m=\u001b[39m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mavgpool(x)\n\u001b[1;32m     68\u001b[0m     x \u001b[39m=\u001b[39m torch\u001b[39m.\u001b[39mflatten(x, \u001b[39m1\u001b[39m)\n",
+      "File \u001b[0;32m~/miniforge3/envs/pytorch-nightly/lib/python3.10/site-packages/torch/nn/modules/module.py:1186\u001b[0m, in \u001b[0;36mModule._call_impl\u001b[0;34m(self, *input, **kwargs)\u001b[0m\n\u001b[1;32m   1182\u001b[0m \u001b[39m# If we don't have any hooks, we want to skip the rest of the logic in\u001b[39;00m\n\u001b[1;32m   1183\u001b[0m \u001b[39m# this function, and just call forward.\u001b[39;00m\n\u001b[1;32m   1184\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39mnot\u001b[39;00m (\u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_backward_hooks \u001b[39mor\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_forward_hooks \u001b[39mor\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_forward_pre_hooks \u001b[39mor\u001b[39;00m _global_backward_hooks\n\u001b[1;32m   1185\u001b[0m         \u001b[39mor\u001b[39;00m _global_forward_hooks \u001b[39mor\u001b[39;00m _global_forward_pre_hooks):\n\u001b[0;32m-> 1186\u001b[0m     \u001b[39mreturn\u001b[39;00m forward_call(\u001b[39m*\u001b[39;49m\u001b[39minput\u001b[39;49m, \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n\u001b[1;32m   1187\u001b[0m \u001b[39m# Do not call functions when jit is used\u001b[39;00m\n\u001b[1;32m   1188\u001b[0m full_backward_hooks, non_full_backward_hooks \u001b[39m=\u001b[39m [], []\n",
+      "File \u001b[0;32m~/miniforge3/envs/pytorch-nightly/lib/python3.10/site-packages/torch/nn/modules/container.py:141\u001b[0m, in \u001b[0;36mSequential.forward\u001b[0;34m(self, input)\u001b[0m\n\u001b[1;32m    139\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39mforward\u001b[39m(\u001b[39mself\u001b[39m, \u001b[39minput\u001b[39m):\n\u001b[1;32m    140\u001b[0m     \u001b[39mfor\u001b[39;00m module \u001b[39min\u001b[39;00m \u001b[39mself\u001b[39m:\n\u001b[0;32m--> 141\u001b[0m         \u001b[39minput\u001b[39m \u001b[39m=\u001b[39m module(\u001b[39minput\u001b[39;49m)\n\u001b[1;32m    142\u001b[0m     \u001b[39mreturn\u001b[39;00m \u001b[39minput\u001b[39m\n",
+      "File \u001b[0;32m~/miniforge3/envs/pytorch-nightly/lib/python3.10/site-packages/torch/nn/modules/module.py:1186\u001b[0m, in \u001b[0;36mModule._call_impl\u001b[0;34m(self, *input, **kwargs)\u001b[0m\n\u001b[1;32m   1182\u001b[0m \u001b[39m# If we don't have any hooks, we want to skip the rest of the logic in\u001b[39;00m\n\u001b[1;32m   1183\u001b[0m \u001b[39m# this function, and just call forward.\u001b[39;00m\n\u001b[1;32m   1184\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39mnot\u001b[39;00m (\u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_backward_hooks \u001b[39mor\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_forward_hooks \u001b[39mor\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_forward_pre_hooks \u001b[39mor\u001b[39;00m _global_backward_hooks\n\u001b[1;32m   1185\u001b[0m         \u001b[39mor\u001b[39;00m _global_forward_hooks \u001b[39mor\u001b[39;00m _global_forward_pre_hooks):\n\u001b[0;32m-> 1186\u001b[0m     \u001b[39mreturn\u001b[39;00m forward_call(\u001b[39m*\u001b[39;49m\u001b[39minput\u001b[39;49m, \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n\u001b[1;32m   1187\u001b[0m \u001b[39m# Do not call functions when jit is used\u001b[39;00m\n\u001b[1;32m   1188\u001b[0m full_backward_hooks, non_full_backward_hooks \u001b[39m=\u001b[39m [], []\n",
+      "File \u001b[0;32m~/miniforge3/envs/pytorch-nightly/lib/python3.10/site-packages/torch/nn/modules/batchnorm.py:150\u001b[0m, in \u001b[0;36m_BatchNorm.forward\u001b[0;34m(self, input)\u001b[0m\n\u001b[1;32m    147\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mtraining \u001b[39mand\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mtrack_running_stats:\n\u001b[1;32m    148\u001b[0m     \u001b[39m# TODO: if statement only here to tell the jit to skip emitting this when it is None\u001b[39;00m\n\u001b[1;32m    149\u001b[0m     \u001b[39mif\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mnum_batches_tracked \u001b[39mis\u001b[39;00m \u001b[39mnot\u001b[39;00m \u001b[39mNone\u001b[39;00m:  \u001b[39m# type: ignore[has-type]\u001b[39;00m\n\u001b[0;32m--> 150\u001b[0m         \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mnum_batches_tracked\u001b[39m.\u001b[39;49madd_(\u001b[39m1\u001b[39;49m)  \u001b[39m# type: ignore[has-type]\u001b[39;00m\n\u001b[1;32m    151\u001b[0m         \u001b[39mif\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mmomentum \u001b[39mis\u001b[39;00m \u001b[39mNone\u001b[39;00m:  \u001b[39m# use cumulative moving average\u001b[39;00m\n\u001b[1;32m    152\u001b[0m             exponential_average_factor \u001b[39m=\u001b[39m \u001b[39m1.0\u001b[39m \u001b[39m/\u001b[39m \u001b[39mfloat\u001b[39m(\u001b[39mself\u001b[39m\u001b[39m.\u001b[39mnum_batches_tracked)\n",
+      "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
+     ]
+    }
+   ],
+   "source": [
+    "#!/usr/bin/env python\n",
+    "# coding: utf-8\n",
+    "\n",
+    "import argparse\n",
+    "import os\n",
+    "import random\n",
+    "import time\n",
+    "\n",
+    "import numpy as np\n",
+    "import torch\n",
+    "import torchvision\n",
+    "from torch.utils.data import DataLoader\n",
+    "from torch.utils.data import SubsetRandomSampler\n",
+    "from torchvision import datasets, transforms\n",
+    "\n",
+    "\n",
+    "def set_all_seeds(seed):\n",
+    "    os.environ[\"PL_GLOBAL_SEED\"] = str(seed)\n",
+    "    random.seed(seed)\n",
+    "    np.random.seed(seed)\n",
+    "    torch.manual_seed(seed)\n",
+    "    torch.cuda.manual_seed_all(seed)\n",
+    "\n",
+    "\n",
+    "def compute_accuracy(model, data_loader, device):\n",
+    "    model.eval()\n",
+    "    with torch.no_grad():\n",
+    "        correct_pred, num_examples = 0, 0\n",
+    "        for i, (features, targets) in enumerate(data_loader):\n",
+    "\n",
+    "            features = features.to(device)\n",
+    "            targets = targets.to(device)\n",
+    "\n",
+    "            logits = model(features)\n",
+    "            _, predicted_labels = torch.max(logits, 1)\n",
+    "            num_examples += targets.size(0)\n",
+    "            correct_pred += (predicted_labels.cpu() == targets.cpu()).sum()\n",
+    "    return correct_pred.float() / num_examples * 100\n",
+    "\n",
+    "\n",
+    "def train_classifier_simple_v2(\n",
+    "    model,\n",
+    "    num_epochs,\n",
+    "    train_loader,\n",
+    "    valid_loader,\n",
+    "    test_loader,\n",
+    "    optimizer,\n",
+    "    device,\n",
+    "    logging_interval=50,\n",
+    "    best_model_save_path=None,\n",
+    "    scheduler=None,\n",
+    "    skip_train_acc=False,\n",
+    "    scheduler_on=\"valid_acc\",\n",
+    "):\n",
+    "\n",
+    "    start_time = time.time()\n",
+    "    minibatch_loss_list, train_acc_list, valid_acc_list = [], [], []\n",
+    "    best_valid_acc, best_epoch = -float(\"inf\"), 0\n",
+    "\n",
+    "    for epoch in range(num_epochs):\n",
+    "\n",
+    "        epoch_start_time = time.time()\n",
+    "        model.train()\n",
+    "        for batch_idx, (features, targets) in enumerate(train_loader):\n",
+    "\n",
+    "            features = features.to(device)\n",
+    "            targets = targets.to(device)\n",
+    "\n",
+    "            # ## FORWARD AND BACK PROP\n",
+    "            logits = model(features)\n",
+    "            loss = torch.nn.functional.cross_entropy(logits, targets)\n",
+    "            optimizer.zero_grad()\n",
+    "\n",
+    "            loss.backward()\n",
+    "\n",
+    "            # ## UPDATE MODEL PARAMETERS\n",
+    "            optimizer.step()\n",
+    "\n",
+    "            # ## LOGGING\n",
+    "            minibatch_loss_list.append(loss.item())\n",
+    "            if not batch_idx % logging_interval:\n",
+    "                print(\n",
+    "                    f\"Epoch: {epoch+1:03d}/{num_epochs:03d} \"\n",
+    "                    f\"| Batch {batch_idx:04d}/{len(train_loader):04d} \"\n",
+    "                    f\"| Loss: {loss:.4f}\"\n",
+    "                )\n",
+    "\n",
+    "        model.eval()\n",
+    "\n",
+    "        elapsed = (time.time() - epoch_start_time) / 60\n",
+    "        print(f\"Time / epoch without evaluation: {elapsed:.2f} min\")\n",
+    "        with torch.no_grad():  # save memory during inference\n",
+    "            if not skip_train_acc:\n",
+    "                train_acc = compute_accuracy(model, train_loader, device=device).item()\n",
+    "            else:\n",
+    "                train_acc = float(\"nan\")\n",
+    "            valid_acc = compute_accuracy(model, valid_loader, device=device).item()\n",
+    "            train_acc_list.append(train_acc)\n",
+    "            valid_acc_list.append(valid_acc)\n",
+    "\n",
+    "            if valid_acc > best_valid_acc:\n",
+    "                best_valid_acc, best_epoch = valid_acc, epoch + 1\n",
+    "                if best_model_save_path:\n",
+    "                    torch.save(model.state_dict(), best_model_save_path)\n",
+    "\n",
+    "            print(\n",
+    "                f\"Epoch: {epoch+1:03d}/{num_epochs:03d} \"\n",
+    "                f\"| Train: {train_acc :.2f}% \"\n",
+    "                f\"| Validation: {valid_acc :.2f}% \"\n",
+    "                f\"| Best Validation \"\n",
+    "                f\"(Ep. {best_epoch:03d}): {best_valid_acc :.2f}%\"\n",
+    "            )\n",
+    "\n",
+    "        elapsed = (time.time() - start_time) / 60\n",
+    "        print(f\"Time elapsed: {elapsed:.2f} min\")\n",
+    "\n",
+    "        if scheduler is not None:\n",
+    "\n",
+    "            if scheduler_on == \"valid_acc\":\n",
+    "                scheduler.step(valid_acc_list[-1])\n",
+    "            elif scheduler_on == \"minibatch_loss\":\n",
+    "                scheduler.step(minibatch_loss_list[-1])\n",
+    "            else:\n",
+    "                raise ValueError(\"Invalid `scheduler_on` choice.\")\n",
+    "\n",
+    "    elapsed = (time.time() - start_time) / 60\n",
+    "    print(f\"Total Training Time: {elapsed:.2f} min\")\n",
+    "\n",
+    "    test_acc = compute_accuracy(model, test_loader, device=device)\n",
+    "    print(f\"Test accuracy {test_acc :.2f}%\")\n",
+    "\n",
+    "    elapsed = (time.time() - start_time) / 60\n",
+    "    print(f\"Total Time: {elapsed:.2f} min\")\n",
+    "\n",
+    "    return minibatch_loss_list, train_acc_list, valid_acc_list\n",
+    "\n",
+    "\n",
+    "def get_dataloaders_cifar10(\n",
+    "    batch_size,\n",
+    "    num_workers=0,\n",
+    "    validation_fraction=None,\n",
+    "    train_transforms=None,\n",
+    "    test_transforms=None,\n",
+    "):\n",
+    "\n",
+    "    if train_transforms is None:\n",
+    "        train_transforms = transforms.ToTensor()\n",
+    "\n",
+    "    if test_transforms is None:\n",
+    "        test_transforms = transforms.ToTensor()\n",
+    "\n",
+    "    train_dataset = datasets.CIFAR10(\n",
+    "        root=\"data\", train=True, transform=train_transforms, download=True\n",
+    "    )\n",
+    "\n",
+    "    valid_dataset = datasets.CIFAR10(root=\"data\", train=True, transform=test_transforms)\n",
+    "\n",
+    "    test_dataset = datasets.CIFAR10(root=\"data\", train=False, transform=test_transforms)\n",
+    "\n",
+    "    if validation_fraction is not None:\n",
+    "        num = int(validation_fraction * 50000)\n",
+    "        train_indices = torch.arange(0, 50000 - num)\n",
+    "        valid_indices = torch.arange(50000 - num, 50000)\n",
+    "\n",
+    "        train_sampler = SubsetRandomSampler(train_indices)\n",
+    "        valid_sampler = SubsetRandomSampler(valid_indices)\n",
+    "\n",
+    "        valid_loader = DataLoader(\n",
+    "            dataset=valid_dataset,\n",
+    "            batch_size=batch_size,\n",
+    "            num_workers=num_workers,\n",
+    "            sampler=valid_sampler,\n",
+    "        )\n",
+    "\n",
+    "        train_loader = DataLoader(\n",
+    "            dataset=train_dataset,\n",
+    "            batch_size=batch_size,\n",
+    "            num_workers=num_workers,\n",
+    "            drop_last=True,\n",
+    "            sampler=train_sampler,\n",
+    "        )\n",
+    "\n",
+    "    else:\n",
+    "        train_loader = DataLoader(\n",
+    "            dataset=train_dataset,\n",
+    "            batch_size=batch_size,\n",
+    "            num_workers=num_workers,\n",
+    "            drop_last=True,\n",
+    "            shuffle=True,\n",
+    "        )\n",
+    "\n",
+    "    test_loader = DataLoader(\n",
+    "        dataset=test_dataset,\n",
+    "        batch_size=batch_size,\n",
+    "        num_workers=num_workers,\n",
+    "        shuffle=False,\n",
+    "    )\n",
+    "\n",
+    "    if validation_fraction is None:\n",
+    "        return train_loader, test_loader\n",
+    "    else:\n",
+    "        return train_loader, valid_loader, test_loader\n",
+    "\n",
+    "\n",
+    "if __name__ == \"__main__\":\n",
+    "\n",
+    "    # parser = argparse.ArgumentParser()\n",
+    "    # parser.add_argument(\n",
+    "    #     \"--device\", type=str, required=True, help=\"Which GPU device to use.\"\n",
+    "    # )\n",
+    "\n",
+    "    # args = parser.parse_args()\n",
+    "\n",
+    "\n",
+    "    RANDOM_SEED = 123\n",
+    "    BATCH_SIZE = 32\n",
+    "    NUM_EPOCHS = 1\n",
+    "    # DEVICE = torch.device(args.device)\n",
+    "    # Apple’s Metal Performance Shaders (MPS)\n",
+    "    DEVICE = \"mps\"\n",
+    "\n",
+    "    print('torch', torch.__version__)\n",
+    "    print('device', DEVICE)\n",
+    "\n",
+    "    train_transforms = torchvision.transforms.Compose(\n",
+    "        [\n",
+    "            torchvision.transforms.Resize((256, 256)),\n",
+    "            torchvision.transforms.RandomCrop((224, 224)),\n",
+    "            torchvision.transforms.ToTensor(),\n",
+    "            torchvision.transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),\n",
+    "        ]\n",
+    "    )\n",
+    "\n",
+    "    test_transforms = torchvision.transforms.Compose(\n",
+    "        [\n",
+    "            torchvision.transforms.Resize((256, 256)),\n",
+    "            torchvision.transforms.CenterCrop((224, 224)),\n",
+    "            torchvision.transforms.ToTensor(),\n",
+    "            torchvision.transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),\n",
+    "        ]\n",
+    "    )\n",
+    "\n",
+    "    train_loader, valid_loader, test_loader = get_dataloaders_cifar10(\n",
+    "        batch_size=BATCH_SIZE,\n",
+    "        validation_fraction=0.1,\n",
+    "        train_transforms=train_transforms,\n",
+    "        test_transforms=test_transforms,\n",
+    "        num_workers=2,\n",
+    "    )\n",
+    "\n",
+    "    model = torch.hub.load(\n",
+    "        \"pytorch/vision:v0.11.0\", \"vgg16_bn\", pretrained=False\n",
+    "    )\n",
+    "\n",
+    "    model.classifier[-1] = torch.nn.Linear(\n",
+    "        in_features=4096, out_features=10  # as in original\n",
+    "    )  # number of class labels in Cifar-10)\n",
+    "\n",
+    "    model = model.to(DEVICE)\n",
+    "\n",
+    "    optimizer = torch.optim.Adam(model.parameters(), lr=0.0005)\n",
+    "\n",
+    "    minibatch_loss_list, train_acc_list, valid_acc_list = train_classifier_simple_v2(\n",
+    "        model=model,\n",
+    "        num_epochs=NUM_EPOCHS,\n",
+    "        train_loader=train_loader,\n",
+    "        valid_loader=valid_loader,\n",
+    "        test_loader=test_loader,\n",
+    "        optimizer=optimizer,\n",
+    "        best_model_save_path=None,\n",
+    "        device=DEVICE,\n",
+    "        scheduler_on=\"valid_acc\",\n",
+    "        logging_interval=100,\n",
+    "    )"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "True"
+      ]
+     },
+     "execution_count": 10,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "import torch\n",
+    "\n",
+    "torch.has_mps"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3.10.5 ('pytorch-nightly')",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.5"
+  },
+  "orig_nbformat": 4,
+  "vscode": {
+   "interpreter": {
+    "hash": "8a8bcccfb183d1298694efece6cf41240378bc61621e95c864629a40c5876542"
+   }
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

code/main.py

@@ -0,0 +1,137 @@
+from __future__ import print_function
+import argparse
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+import torch.optim as optim
+from torchvision import datasets, transforms
+from torch.optim.lr_scheduler import StepLR
+
+
+class Net(nn.Module):
+    def __init__(self):
+        super(Net, self).__init__()
+        self.conv1 = nn.Conv2d(1, 32, 3, 1)
+        self.conv2 = nn.Conv2d(32, 64, 3, 1)
+        self.dropout1 = nn.Dropout(0.25)
+        self.dropout2 = nn.Dropout(0.5)
+        self.fc1 = nn.Linear(9216, 128)
+        self.fc2 = nn.Linear(128, 10)
+
+    def forward(self, x):
+        x = self.conv1(x)
+        x = F.relu(x)
+        x = self.conv2(x)
+        x = F.relu(x)
+        x = F.max_pool2d(x, 2)
+        x = self.dropout1(x)
+        x = torch.flatten(x, 1)
+        x = self.fc1(x)
+        x = F.relu(x)
+        x = self.dropout2(x)
+        x = self.fc2(x)
+        output = F.log_softmax(x, dim=1)
+        return output
+
+
+def train(args, model, device, train_loader, optimizer, epoch):
+    model.train()
+    for batch_idx, (data, target) in enumerate(train_loader):
+        data, target = data.to(device), target.to(device)
+        optimizer.zero_grad()
+        output = model(data)
+        loss = F.nll_loss(output, target)
+        loss.backward()
+        optimizer.step()
+        if batch_idx % args.log_interval == 0:
+            print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(
+                epoch, batch_idx * len(data), len(train_loader.dataset),
+                100. * batch_idx / len(train_loader), loss.item()))
+            if args.dry_run:
+                break
+
+
+def test(model, device, test_loader):
+    model.eval()
+    test_loss = 0
+    correct = 0
+    with torch.no_grad():
+        for data, target in test_loader:
+            data, target = data.to(device), target.to(device)
+            output = model(data)
+            test_loss += F.nll_loss(output, target, reduction='sum').item()  # sum up batch loss
+            pred = output.argmax(dim=1, keepdim=True)  # get the index of the max log-probability
+            correct += pred.eq(target.view_as(pred)).sum().item()
+
+    test_loss /= len(test_loader.dataset)
+
+    print('\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format(
+        test_loss, correct, len(test_loader.dataset),
+        100. * correct / len(test_loader.dataset)))
+
+
+def main():
+    # Training settings
+    parser = argparse.ArgumentParser(description='PyTorch MNIST Example')
+    parser.add_argument('--batch-size', type=int, default=64, metavar='N',
+                        help='input batch size for training (default: 64)')
+    parser.add_argument('--test-batch-size', type=int, default=1000, metavar='N',
+                        help='input batch size for testing (default: 1000)')
+    parser.add_argument('--epochs', type=int, default=14, metavar='N',
+                        help='number of epochs to train (default: 14)')
+    parser.add_argument('--lr', type=float, default=1.0, metavar='LR',
+                        help='learning rate (default: 1.0)')
+    parser.add_argument('--gamma', type=float, default=0.7, metavar='M',
+                        help='Learning rate step gamma (default: 0.7)')
+    parser.add_argument('--no-cuda', action='store_true', default=False,
+                        help='disables CUDA training')
+    parser.add_argument('--dry-run', action='store_true', default=False,
+                        help='quickly check a single pass')
+    parser.add_argument('--seed', type=int, default=1, metavar='S',
+                        help='random seed (default: 1)')
+    parser.add_argument('--log-interval', type=int, default=10, metavar='N',
+                        help='how many batches to wait before logging training status')
+    parser.add_argument('--save-model', action='store_true', default=False,
+                        help='For Saving the current Model')
+    args = parser.parse_args()
+    use_cuda = not args.no_cuda and torch.cuda.is_available()
+
+    torch.manual_seed(args.seed)
+
+    device = torch.device("cuda" if use_cuda else "cpu")
+
+    train_kwargs = {'batch_size': args.batch_size}
+    test_kwargs = {'batch_size': args.test_batch_size}
+    if use_cuda:
+        cuda_kwargs = {'num_workers': 1,
+                       'pin_memory': True,
+                       'shuffle': True}
+        train_kwargs.update(cuda_kwargs)
+        test_kwargs.update(cuda_kwargs)
+
+    transform=transforms.Compose([
+        transforms.ToTensor(),
+        transforms.Normalize((0.1307,), (0.3081,))
+        ])
+    dataset1 = datasets.MNIST('../data', train=True, download=True,
+                       transform=transform)
+    dataset2 = datasets.MNIST('../data', train=False,
+                       transform=transform)
+    train_loader = torch.utils.data.DataLoader(dataset1,**train_kwargs)
+    test_loader = torch.utils.data.DataLoader(dataset2, **test_kwargs)
+
+    model = Net().to(device)
+    optimizer = optim.Adadelta(model.parameters(), lr=args.lr)
+
+    scheduler = StepLR(optimizer, step_size=1, gamma=args.gamma)
+    for epoch in range(1, args.epochs + 1):
+        train(args, model, device, train_loader, optimizer, epoch)
+        test(model, device, test_loader)
+        scheduler.step()
+
+    if args.save_model:
+        torch.save(model.state_dict(), "mnist_cnn.pt")
+
+
+if __name__ == '__main__':
+    main()

code/netflix_price_increase.ipynb

@@ -0,0 +1,63 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 34,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAESCAYAAAD+GW7gAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAABEkUlEQVR4nO3dd2BUZb7/8feZMy2ZzCSZEBAFFFBBBRVpCkgTBBE2tADrqov7W+td14IICoi7lrWgd13cqyvXVS+yAqE3EQQRpShFUbEhKt2QZFJmJpl25vz+mGSkJKTOkPJ9/aOcnJnvc4B8ODnzPN9H0XVdRwghRKNnONsDEEIIER8S+EII0URI4AshRBMhgS+EEE2EBL4QQjQREvhCCNFESOA3IYcPH6ZDhw5kZWWddPz1119n6tSplb4+KyuLefPmAeDxeJgwYQI33ngjb7zxBl26dAHgnXfe4bXXXqvWuDRN48477yQ3N7fCcw4dOsS9995brfetTJcuXTh8+HCt3uPOO+9kyZIlADz00EPs37//tHO2bt1KRkYGGRkZ9O7dm6uvvjr666VLlzJhwoRajSFWBg4cyJAhQ8jIyGDkyJHceOONPP3004TD4dPOzc7OrrfXIX5lPNsDEPFlMBh49tln6dq1K+3atavWa3ft2sVFF10EwDfffENeXh7r16/n8OHD/OMf/wDgt7/9bbXH9O9//5sePXrQrFmzCs85evQoP/30U7XfO57uu+8+Jk2axIIFC1AUJXq8V69eLF++HIDZs2eTn5/PY489Fv36qFGj4j7Wqpo1axadO3cGIBAIcMstt/Cf//yHm2+++aTzWrRowfz588/GEEU1SOA3MVarldtuu42HHnqI+fPnYzabT/p6IBBg1qxZ7NixA03TuPTSS5k+fTrbtm1j48aNbNmyBaPRyNtvv012djYZGRm8+OKL0deXBdo999zDyJEjeeqpp+jXrx9///vf2bNnD6+//joGw68/WJaUlPDWW2+xcuVKAPbv38+0adMIBALous7YsWOZMGEC06dPJzs7m//3//4fr7/+Oq+++iobNmzA5/NRUlLClClTGDx4MLNnz+bIkSPk5ORw5MgRWrRowfPPP0/z5s3ZuXMnTzzxBIqi0Llz5+idajgc5umnn2bPnj14vV50XefJJ5+ka9euTJ06lYKCAg4dOkT//v259dZbmTp1KsePH+fcc88lLy8vei2tW7fGbrezYcMGBg0aVKU/j8OHDzNixAg+++wzZs+ezcGDB8nOziYnJ4fLLruMnj17smzZMg4fPszkyZMZPnw4AK+88grr1q0jHA5z3nnnMXPmTFq0aHHSe0+YMIHbbruNIUOGAPD8888DMHHiRKZMmUJ+fj4A/fr14/777690rGazma5du/Ljjz9y+PBhfve739G+fXuOHDnCM888wx/+8Ac+++wzQqEQzz//PJs2bUJVVbp06cLMmTMxm81VGreIHXmk0wTdfffdJCYm8t///d+nfe21115DVVWWLFnCihUraN68ObNmzWLw4MEMHDiQiRMn8vvf/54nn3ySNm3asHz5ciwWy2nv06xZM5555hlmzJjB+vXrWbZsGS+88MJJYQ+wfft22rZtS2pqKhB5vDRw4ECWLFnCa6+9xs6dO1EUJVrv9ddf58iRI2zdupW5c+eycuVKHnjggehPGAA7d+7kpZdeYu3atSQkJDB//nwCgQD33XcfU6dOZdmyZfTs2ROfzwfAnj17OH78OAsWLGDNmjWMGjWKOXPmRN/P5/OxevVqJk+ezF//+leuuOIKVq9ezfTp00/7qaNPnz6sX7++xn82u3bt4p///CdLly5l8+bN7N+/n3nz5jFjxgxmz54NwLJly/j+++/Jyspi+fLl9OvXj+nTp5/2XpmZmdHHTZqmsWLFCjIzM1m4cCGtWrVi6dKlzJs3jwMHDuB2uysdW3Z2Nh988AE9e/YE4JdffuGee+7hvffeIz09PXref/7zH/bu3cvy5ctZtWoVXq+XNWvWVHncInbkDr8JMhgMPP/884wcOZI+ffqc9LVNmzbhdrvZunUrAMFgkLS0tBrV6dOnD8OGDePee+/l7bffxul0nnbOjz/+SJs2baK/Hjx4MFOmTOGLL77gmmuuYfr06af9I3Heeefx3HPPsXLlSg4cOBC9My/To0cPkpKSALj00kspLCzk+++/x2g0cs011wAwfPjw6GOVLl26kJyczPz58zl06BCffPIJNpst+n5du3aN/v/WrVuZMmUKAOeff340/Mq0atWKd999t0a/XxB5/GO32wFo3rw51157LQBt2rShoKAAgA8++IAvv/ySMWPGAJGfUEpKSk57r2HDhvHcc8+Rk5PD119/zQUXXMAFF1zAtddeyx133MGxY8fo1asXkyZNitY81UMPPYTVaiUcDmMymcjMzGTIkCEcPnwYo9HIlVdeedpryj6zsFqtAPz9738HIo+8qjJuETsS+E1Uy5Yt+ctf/sKUKVMYOXJk9Hg4HObRRx+lX79+AHi9Xvx+f41q6LrO/v37adasGZ9//jndunU77RxFUU76EHDAgAG89957bN26lW3btvHPf/4zepdaZu/evdxzzz1MnDiR3r170717d/7yl79Ev14WNGXvX9Yu6tS2UUZj5K//pk2beOqpp7jtttu47rrraNeuHStWrIiel5iYWO77nfgeJ/761H+gquPUR2ynvj9E/oz++Mc/ctNNNwGRx3CFhYWnnZeQkMCQIUNYtWoVn332GZmZmQBcfvnlbNiwgW3btrF9+3YyMzOZM2cOnTp1Ou09TnyGX95Yyxvfqcdyc3MJh8NVHreIHXmk04QNHTqUvn378tZbb0WP9enTh3nz5hEIBAiHw8yYMSP6jF5VVUKhUJXf/80336S4uJjFixfz5ptv8sUXX5x2Ttu2bTl06FD015MmTWLNmjXceOONzJw5k6SkJA4ePIiqqgSDQQB27NhBp06duO222+jRowcbNmxA07QzjqVDhw7ous6HH34IwIYNG6Jhs2XLFgYMGMBNN91Ep06deP/99yt8v2uvvZYFCxYAkQ+SP/nkk5O+fvjw4Wp/GF5dffr0YdGiRXg8HgBeeuklHn744XLPHTduHEuXLmX37t3RZ/mzZs3if/7nfxg0aBDTpk3jwgsvZN++fXU2vmuuuYZVq1ZF/w49/vjjrF69ulrjFrEhd/hN3PTp09m1a1f01/fccw/PPvsso0aNQtM0LrnkkuiUzb59+/LMM88AlPuj/Im+/vprXn31VRYtWkSLFi149NFHmTRpEkuXLo0+boHII4xp06ZRVFSEw+HgnnvuYdq0aSxYsABVVRk0aBDdu3ensLAQi8XC2LFjefXVV1m3bh033HAD4XCYAQMGUFhYGA2S8phMJv75z3/y+OOP8+KLL3LJJZdEH1VNmDCBSZMmMWLECEKhEL17945+sHiqmTNn8sgjj3DDDTdwzjnn0LFjx5O+/tFHH502g6WuZWZmkp2dzbhx41AUhZYtW0b/XE7VqVMnVFVl6NCh0c9afv/73zN16lSGDx+O2WymQ4cO3HjjjXU2vgkTJnDkyBFGjx6Nruv06NGDW265BYPBUOVxi9hQpD2yONteffVVVFXl9ttvP9tDqZWDBw/y0EMPnTYtU4j6Qh7piLPuD3/4A9u3bycnJ+dsD6VW/v73v/Pkk09K2It6S+7whRCiiZA7fCGEaCJiFvh79uzhlltuASAvL4+7776b3/3ud0yYMIGDBw/GqqwQQogKxGSWzpw5c1ixYgUJCQlAZEn3iBEjGDZsGNu3bz9tsU15wuEwmlbzp02qqtTq9fW1VmOv15ivLd71GvO1xbteQ7o2k0mt8GsxCfw2bdowe/bs6Bzb3bt306FDByZOnMh5553HtGnTKn0PTdMpKCiu8RhSUhJr9fr6Wqux12vM1xbveo352uJdryFdW3p6+aumIUaBX7b0usyRI0dwOBy8+eabvPzyy8yZM4f77rvvjO+hqgopKYlnPOfMrzfU6vX1tVZjr9eYry3e9RrztcW7XmO5trgsvEpJSWHgwIFApMd2eU27TiV3+E2zXmO+tnjXa8zXFu96DenaznSHH5dZOl27do0uad+xYwcXXnhhPMoKIYQ4QVwCf8qUKSxfvpwJEybw0Ucfcdddd8WjrBBCiBPE7JFOq1atWLhwIRBpZ/vGG2/EqpQQQogqkIVXQgjRREjgCyFEEyGBL4QQTYT0wxdCiHrCvXMH+SuWcrDYi2624Bydib1b9zp7f7nDF0KIesC9cwe5895C0cOY0tIIuVxkv/oy7p076qyGBL4QQtQD7g/WY1CNBPLzUdAxWK0oRhOuJVl1VkMe6QghxFmk6GG0oiJK9u1DV40nbaCjWCwEs3+ps1pyhy+EEGeBogC+EoLHjxP0eDDYHRAInHSO7vdjanFOndWUwBdCiDhTtBCay4Xf5UILhQBIHjwUXQuh+/3oQNjnQw8FcY7OrLO68khHCCHiRkcvLiZYVERY0076iq1zZ+BmCtevJewtxuh01vksHQl8IYSIh1CQUGEhms9X4SmJl16KXlKMo2U6pqv71vkQJPCFECKWdB292EOwyI0eDld4WvFXX5K3aCHBY0c5Dlw8/yrUpKQ6HYoEvhBCxErAT6ioEM0fqPiUI4fJW7SQkq/3Rg4oCi1GZmCw2ep8OBL4QghRxxQ9jOZ2E/J60MPl700bKiwkf8Uy3Fs+Aj1yjrVDR9Iyx3Nu7x54NKXc19WGBL4QQtQRRQG9pIRgYWF09s2pwoEAhe+vp2DtanS/HwBTi3Nwjskk8fIrTpqHX9ck8IUQog4oWgjNXUSwuCR6x34iPRzGu/NT8pYsRst3AWCwJZE6/Dc4+vVDUWMfxxL4QghRKzq610vQ7T5tqmUZ3w/7yMtagP/nnyIHVJXkgYNIGTYcNTF+m6NL4AshRE0FA5GplqWPZk77ck4OriVZeHfvih6zXdUV5+ixmNKbx2uUURL4QghRXbqO7nUTdHvKnWqpFRdTsGYlhR9shNJn+ZYL2pKWOR7rhRfFe7RREvhCCFEdZ5hqqWshijZ/SP7KFYS9HgDUVCfOUWNI6t4DxXB2u9lI4AshRBVEp1p6POinfCir6zrFX36Ba9HCaHdLxWIh5YYbSb5uMAaz+WwM+TQS+EIIcQaKEnlEEzx+vNyplv5DB8nLWoDvu2+jL7D36UvqbzIwOpLjPNozk8AXQogKlE21DKj6aWEfKsjHtXwpnm1bo9MwEy65lLTM8ZjPa3U2hlspCXwhhDiNDiXFBAojXS0T7NboV8J+P4Xr36PgvXfRS/vXm1qeS9rYcSRc1immC6dqSwJfCCFOpIXQCgsJ+UrghEf1ejiM55PtuJYtQSvIB8Bgt+McMRJ7n2tRVPUsDbjqJPCFEALO2NXS/fXXHPm/twkcPACAYjSSPOh6UoYOw5CQcDZGWyMS+EIIUcECqmB2NnlLsij+/LPoMVv3HjhHjsHUrFm8R1lrEvhCiCZL0cNontKplid0tdS8HvJXraRo0wcQjrRLsLRrH1k41a792RpurUngCyGaHEUB3eeLdLUMBqPH9VCIok0fkL96BeHiYgCMzZpx3k2/xXBpbDtZxoMEvhCiSSmvq6Wu6xR//hl5S7IIHT8eOc+aQOqw4TgGXkeK006Ru+KtCRsKCXwhRJOhez2ndbX0H/g5snBq3/eRAwYDjr79SR3+G1S7/SyNNDYk8IUQjV85H8qG8l24li3Bs31b9Fhi58txjsnE3PLcszHKmJPAF0I0WuV9KBv2+ShYt5bCde+hByMLp8ytWuEcO57ESy49m8ONOQl8IUSjU96Hsno4jHvrFvKXL0UrKgRAdThIzRiFvVefs97JsoxqNGIwmUArfy/c2ohZ4O/Zs4dZs2Yxd+7c6LGVK1fy9ttvs2DBgliVFUI0cu6dO3AtyeLHnOOo6c1xjs7E3q179OvlfShb8s3X5C1aQODw4cg5JhPJg4eQMmQoBms9WDilKKhmE8akJLAkRMbkK67zMjEJ/Dlz5rBixQoSTliB9s0337Bo0aLT2ooKIURVuXfuIPvVl1GMJox2OyGXi+xXX4a7/oS9azf04pO3GgwcO4prcRbFX34RfY+kq6/BmTEao9N5ti4jSjEoqBYLalISmC1AbKd9xuRnmDZt2jB79uzor/Pz85k1axaPPvpoLMoJIZoI15IsFKMJg9WKoigYrFbUJDueTe8TysshUFBAWNPQ3G5y35nH4b/OjIa99cKLOO+RGTS/7Y9nPewVgwGTzYa5WTrGtGZgthLrsIcY3eEPGTKEw6U/OmmaxrRp03j00UexWCxVfg9VVUhJqfnmvqpqqNXr62utxl6vMV9bvOs1xmv7Mec4Rrs9EvZGFWNKCno4TMm+fdjMBsKKSu7a9/hl2TLCJSUAmFu04Nzf/pbk7t1qvHBKVQ04TuiYWWOqitGWiGqzYTBVvClKrH4vY/6h7d69ezlw4ACPP/44fr+fH374gaeeeopp06ad8XWaplNQUPNnWCkpibV6fX2t1djrNeZri3e9xnhtanpzQvn5mNLSMCZY8eUXEMrPx5CczNFNH+FasohQbi4AhsREUoYNJ3nAdShGI25P+RuNV4XDbq3VwivVaERNsmFItOHHAN4QcPpmKmVq83uZnl7x2oGYB/7ll1/O6tWrATh8+DAPPvhgpWEvhBDlaTbut+Qt/A96SQklhQVoxSWEfSWgKBx/7dXISQYVR/8BpN44IvJs/Gwp+yDWZgNrIigKZ/sTTJmWKYRoGEJBrG1aY+8/iIL31hAqLCQcDBF2F6EVRqZZJl5xZWThVItzztow4/1BbHXELPBbtWrFwoULKz0mhBBn8uviKS96OIy1XTusbdtR+ON+9NI59ubWbUjLHE9Ch45nb5wGA8bERAyJiShmM/VxQqLc4Qsh6ikdfCUEC4vQQiF0TcO95SNcK5YRdrsBUFNScI4cTVLPa87awilVLX0+n5CArkYitT6GPUjgCyHqo1AQraiQkM8HOhR/9SV5ixcSPHoUAMVspsWI4Vj7DcJQjdl/deak5/MJoBjO+vP5qpDAF0LUG4oeJuz1EHR70MNhAkcOk7coi5Kvvyo9QSHpml44M0bhbN0y7i2LFYOh9Pm8rd49n68KCXwhxFmnKKCXlER634RChIoKyV+5HPdHm6PPR6wdOpI2dhyWNufHf4Cqislhx5BoA7XhxmbDHbkQosFTFND9foJuN5rfT9jvp3DDegrWrkH3Re7eTS1a4BwzjsTL47/jVNn8eWuLNPyeQFxrx4IEvhDi7ChtchYqLiGsaXh37sC1dBEhlwsAg81G6vAMHP36ocTzrloB1VTayKz0+bxiNAIS+EIIUS1KWEPzeqLTLH0/7CMvawH+n3+KnKCqJA+8jpQbhqPabHEcmIJqMWNMsoOlDtoo1EMS+EKI+NDDpd0sPYQ1jWBODq4lWXh374qeYruqK87RYzGlN4/bsCILpayoSUkoFku9nVJZFyTwhRAxFplPHypyowWDaMXFFKxZReEHGyAU6SdjuaAtaZnjsV54UdxGFV0oZbOhmEzoev2dP19XJPCFEDER+UDWFwn6gB89FKJo82byVy4n7PUAoKY6SRs9Blu3HnFbOGVQVYxJNgwJifV+oVRdk8AXQtQ9LYRWVESwpAQ9HKb4yy9wLVpIMPsXABSLhZShw0gedD0Gc8VtguvSiR0r9QayUKquSeALIerMqR/I+g8dwrVoASXfflN6goK997Wk/mYkxuTkOAyo/nWsPJsk8IUQtaeHCRUVEjieS1jTCBUWkL98Ke6tW6LPSxIuuZS0zPGYz2sV+/GcNOOm4a2IjRUJfCFELeh4d+/CtWIZ/kMHMTiSMTrTKP58N7o/suGIqWVL0saOJ+GyTjFfONWUZtzUhAS+EKLaylbIFn26neNv/hs9rKOENfz7f8C/73sADHY7zhEZ2Pv0RVHV2I7HYMCYkBCZcVPamljC/nQS+EKI6gkGCLndhHw+8pYuIewPEPZ60AO/rkRVnU5aP/YXDAmx3eNWUQ0YbbaTetxI0FdMAl8IUTVaiLDHTai4GD2sE8zOxv/TjycFvSExEUNyCgT8MQ17VVUxJjuwJKnohtj+9NCYSOALIc7o1Jk3mtdD/upVFH2wEcJa5ByzGVNaGpjM6H4/arP0mIxFNZlQbTaUxERMyUnocdykvTGQwBdClO+UVgh6KETRhx+Qv2oF4eJI0Brs9sh/ExIxWCxoJT50LUTy4KF1Nw5FQTWbI83MLNbIBwiiRiTwhRAn03X0kmI0tzuytaCuU7znM/IWZxE6fhwAxZpA6rAbcQwcRMm331K4fi2aKw/VmUby4KHYOneu9TBkxk3dk8AXQpTSwe8n5C5C80eey/sP/EzeooX4vv8ucorBgOPavqSOGIlaendv69wZW+fOOOzWOtmBqrzNwCXs64YEvhBNXNkUy2jQ6zqhfBeuZUvxbN8aPS+hU2fSxozDfO65MRlHU+5xEy8S+EI0ZSdsQqLrOmGfj4J1aylc9x56MHKXbz6vFc6xmSRe2ikmQ4j0uEnCkJjYZHvcxIsEvhBNkKKF0LxeQt7IzBs9HMa9bQv5y5aiFRUCoDocpGaMwt6rT913sizrcZOUBJaEJt/jJl4k8IVoSk6ZeQNQ8s3X5C1aQODwYQAUk4nkwUNIGTIUgzWhTstHPoi1oCYlgVl63MSbBL4QTcEpM28AAr8cw7Uoi+Iv90RPS+pxNc5RYzA6nXVavrzWByL+JPCFaNR08PkiH8gGggBoHjf5K1dQtHkThMMAWC+8CGfmeKwXtK3T6mUfxCoJidL6oB6QwBeiEVIUCJeUEMrLRfP7QQc9GKTwgw0UrFlFuKQEAGN6c9JGjyWxy1V12sny1M1GRP0ggS9EY1Pa3MxvAs3nR9d1vLt34lqyiFBuLhDpeZMybDjJ/QeimEx1Vjr6Qaw1AWTGTb0jgS9EY6GFCHs8hIq96GEdjFZ8P/1IXtYC/Pt/iJxjUHH070/qjb+JfHBaF0o3GzE3a4Y/UPnp4uyRwBeigTu1uRlAyJXHgbeWkb/114VTiVdciXNMJuYW59RN3VNaH6iJiRCQZmb1mQS+EA1VOVMswyUlFKxdQ+H769BLZ+OYW7cmLXMCCR061knZaOsDmw3FZJLWBw2IBL4QDY4OvhJChUXRKZa6puHe8hH5K5ajuYsAMKakkJoxiqSre9XJwinFYIi0PpDNRhosCXwhGohIzxsfoaJfm5sBFO/9irxFCwgePRo5z2wm+fqhtB6dgTdYB3VVA6akpJN63IiGKWZ/env27GHWrFnMnTuXb775hieeeAJVVTGbzTz77LM0a9YsVqWFaHxO2Faw7LY6cPQIeYsWUrL3q8g5ikLS1b1wZozCmJqKarVCsObdK6PNzBJt6AZVZtw0AjEJ/Dlz5rBixQoSEiLLsp966ilmzJjBJZdcwvz585kzZw6PPPJILEoL0bhoIcLuIkIlJZGZN4BWVIRr5TLcH22Ohr+1Q0fSxo7D0ub8WpdUVRXVnhSdQy9B33jEJPDbtGnD7NmzefjhhwF48cUXad68OQCapmGxWGJRVojGIzrFsjg68yYcCFC4YT0Fa9eg+yJ37qYWLXCOGUfi5VfUeuHUqYulJOgbnzMGvq7rbNq0iU8//ZSCggKcTifXXHMNvXv3PuNfriFDhnC4tBETEA373bt38/bbbzNv3rxKB6aqCikpNd8EWVUNtXp9fa3V2Os15murSr1wKITmdhMq8YCig82MrusUbNvG0fnzCebmRd4nKYlzxoym2XXXoRjL/zZWVQMOu7XSMSlGI8akpMhesWrNNwSvb7+XDbVWLOtVGPjbtm3jlVde4bLLLqNDhw6kp6dTWFjIxx9/zJw5c7jzzjvp1atXlQutWbOGV155hddeew1nFRozaZpOQS02KE5JSazV6+trrcZerzFf2xnrlU6xDJ0wxRLA98O+yMKpn3+KHFBVkgdcR8qw4ag2G+6SEBAqt9YZd6Aqa09ss4E1EcIKuP2xubYYkb+X5UtPt1f4tQoD/8CBA7zxxhuop/yLP2zYMDRNY8GCBVUO/OXLl7NgwQLmzp1LSkpK1UYtRFNQThdLgGBODq6li/Du2hk9ltjlKtLGZGJKb17jctKeuGmrMPAnTJgAQCAQwO12k5aWFv2aqqrcdNNNVSqgaRpPPfUULVu25N577wWge/fu/PnPf67NuIVo0BQF9LIulidMsQyXFJO/ZjWFG9+H0n8ALOdfgDNzPAkXXVzzeuXsEyuangoDv7CwkBkzZrB3714cDge5ublcc801PPbYYyRVoQdHq1atWLhwIQCffvpp3Y1YiIYu4Cfodpd2sYwkr66FKNq8mfxVywl7PACoqak4R44hqUfPGi+ckvbE4kQVBv5TTz3F4MGD+cc//hE9lpWVxV//+leee+65uAxOiEZFCxHIzSWQm49eFvS6TslXX5C3KIvgL8cAUCwWUoYOI3nQYAzmms1oU00mTCkpmO0GaU8soioM/EOHDjFixIiTjmVmZrJy5cqYD0qIRuWEKZaW0lk3AP5Dh3AtWkDJt99EzlMU7L2vJfU3IzEmJ1e/jqKgms2l+8RaMTps6HH8oFHUfxUGvqmCHtl1uUmCEI2ZooXQir0ndbEECBUWkL98Ge6tH0efryRccinOseOwtGpd/TrRrpU2FItVHtmIClUY+D6fj59//jl6N1KmpHSnHCFE+RQ9TNjrJeg5eYplOODnlyXvkr1yJbo/MgXS1LIlaWPHk3BZp2rfTCmqAWOiLfJBrHStFFVQYeBbLBZmzJhR7nEhRDn0MHpxMSG3G+2EoNfDYTyfbMe1fAlafj4ABrsd54gM7H36Vnuxk6oaUe22SDMzQ+S1EvSiKioM/Llz58ZzHEI0bCXeSNAHT14EVfL9d+RlLSBw8AAQWdXquG4wqTcMw5BQvZWUsn2gqK0KP74/dOgQ//Vf/0UoFGLHjh307t2bwYMH8/nnn8dxeELUZ6V96XOP43flnxT2wexsfnnlnxx74blo2Nu6dafjrOdJGz22WmGvWsxY0pwY01tAgg1k1o2ooQrv8J9++mnGjh2L0WjkmWee4bnnnuPCCy/koYcekrt/0aRF+tL7I33pA35OvNXWvF4KVq+kcNNGKH2sY2nbjrTM8VjbX4jFbsVfUbuDU6hGI8ZkR6T1gRB1oMLADwQCXHfddeTn5/PLL7/Qu3dvAMInzDYQoskppy89gB4KUfThB+SvWkG4ODIV0piWhnPUWGzdulfrA1nFoESamSXZZQ69qFOVtkfetm0bV199NRAJe7fbHfNBCVHvaCHCbjehkuJoX3qILJwq3vMZrsWLCB7PBkCxWkm94UYc1w3GUMH05vIoBgWjNQFDUhKYzPKMXtS5CgP/oosu4sEHH2Tv3r088cQTHD9+nBdffDEa/kI0CVqIsNdLyHvyXHoA/8ED5GUtwPf9d5EDioKjbz9Sh2egOhxVLqEYlMj0SpsNjFX/B0KI6qow8KdMmcLmzZu56667uPjii/nuu+/o2LEjt9xySzzHJ8RZUdGiKYBQfj6u5UvwbN/268KpTp1JG5OJ+dzzqlFEwWi1otrtYDLX5fCFKFeFgX/s2DEuuuii6P+np6czceLEeI1LiLNCCWuEi70EPB507eSgD/t8FKxbS+G699CDkQ6XpnPPIy1zHImXdqpGkdIWCA67rIwVcVVh4D/wwAMoihJdaVtcXEwgEOC5557jiiuuiNsAhYiHilbHQmThlHvbFvKXL0UrLARAdThI/c1I7L2vrVInS++XX1L0wftkh4KYWrbEft0QkrpcJWEv4qrCwF+wYMFpxw4ePMgjjzxSpS0KhWgI9FAI3esm6D496AFKvv2GvEULCBw6BIBiMpE86HpSht6AwZpQpRrFe/dStG5NZAtBk5HiH/bj/fYlWtz1J+zdutfp9QhxJtXaxLxNmzbSPE00DqVbCvq9IQIF3tO+HPjlGK7FWRR/sSd6LKnH1ThHjcboTDvt/HIpoJrN+L74DB2FkNuDqhowWK2EfeBakiWBL+KqWoGvaZpMyxQNm66jF3vRPB60UAjrKZt8ax43+StXULR5E5R+WGu98CKcmeOxXtC2ajVKg95od4DFgvfbbzDYkk66WVIsFoLZv9TVVQlRJVV+pBMIBNi4cSODBw+O+aCEqHMV7B0b/XIwSOEHGyhYs4pwaUdYY7N0nGPGYuvStco/2f66OjaBsv1iTc1bEHK5UKy//uOi+/2YWpxT++sSohoqDPycnJyTfm2xWLj99turvHG5EPVGiZeQ24MWDJ72JV3X8ezaiWvJIkK5kb/zhoQEUm4cQXL/gShVXDilGAyY7EkYbEmnrY51js4k+9WXCfvAkJhA2OdDDwVxjs6s/bUJUQ0VBn5mZiYtWrSo8IXZ2dln/LoQZ5cOZZuEB04PegDfTz+SvSQL7/ffRw4YVBz9+pM6fARqkr3KlVSrBWNyChhN5a6OtXfrDnf9CdeSLLSc4xjTm+McnSnP70XcVRj4//u//4vRaGTEiBFcdNFFmEwmdF1n7969LF++nHA4XG6/fCHOLh38/tKgD1BeAodcebiWLsbz6SfRY4lXXIlzdCbmc6r+mMWgqhgdDpTERMoe31TE3q079m7dSUlJpEC2HRRnSYWBP23aNPbs2cPrr7/Ojh07CIfDWK1WrrrqKm666SauvPLKOA5TiDOLdLD0ESpyn9bBsky4pISCtWso3LAevfTxjvX8NqSOHkdCx0uqUUvBaEuMNDdTqzXvQYiz6ox/W6+44gpeeOGFeI1FiGora1UcdBeh+QPlbv2kaxruLR+Tv2IZmrsIADU5GWfGaM69fiBub6DK9VSLGaMjGcwWaW4mGhy5PRENVwWtik9U/PVX5GUtJHj0CACKyUzy9UNIuX4oBqu1SqtkofRDWYcdJTEp8q+MEA2QBL5oeCpoVXyiwNEj5C3OouSrLyMHFIWkq6/BmTEaY2pqtcoZE6yoySkgj29EA1fp3+APPviAAQMGRH+9Zs0ahg0bFtNBCVEeRQuhVdCquIxWVIRr5XLcH30Yveu3XtyBtMzxWNqcX6161flQVoiGoMLA/+CDD9i9ezerV6/ms88+AyIrbTdu3CiBL+KqrLFZwOM+rYNlmXAwSNGG9eS/uxrdF9lC0NS8Bc4xmSRecWX1WoIoCqaEBFSHQz6UFY1KhX+bO3bsSEFBARaLhbZtI0vKFUVh+PDhcRucaOJK+91U1NgMIgunvDs+xbVsMaG8PAAMiYmkjsjA0bc/irF6ga0ajRhTUlCs0rZYND4Vfje0bNmSUaNGkZGRAUS2N/z8889p37593AYnmqhT+t1UxLf/B/KyFuD/6cfIAVUluf9AUm4cjmpLqlbJU/eRlbAXjVGltz8vvPACrVu35ujRo+zdu5dmzZrx7LPPxmNsoqnRdUIeD6GcnHLbIJQJ5ubgWrIY764d0WOJXa4ibfRYTM2rv/pbtVgwJifLPrKi0as08Hft2sXkyZO55ZZbmDt3Lr///e/jMS7R1PiKCbndBC1qhWEfLikmf81qCje+D6V3/uY255OWOZ6EiztUu6RiMGBKScYYln1kRdNQaeCHw2G++OILWrVqRSAQwOVyxWNcoinQ9UjQezy/9ruxqKefpmm4P96Ma8Uywh4PAGpqKs6RY0jq0bPKc+lPVDbV0uhwgLQ6EE1EpYGfkZHBE088wdNPP83zzz/PrbfeGo9xicasLOgr6GD562k6JV99Sd7ihQSPHQMifeRThtxA8uDrMZgt1aurgGo0oSYlyVRL0SRVGvi/+93vGDZsGIcOHeLuu+/G6XTGY1yiMapi0AP4Dx/CtWghJd98HTmgKNh79SE1Y2SkM2V1KEqkJUKSHSzWys8XopGqNPDXrFnDSy+9RPv27dm3bx9/+tOfojN3zmTPnj3MmjWLuXPncuDAAaZOnYqiKFx00UXMnDkTQw1+DBcNVeWtir1ffknh+rUczM1BD4XQioqiC6cSOl6Cc+x4LK1bV7uyajZhdCTLNEshqELgv/XWWyxZsgSbzYbH4+H3v/99pYE/Z84cVqxYQUJCZJPnv/3tb9x///307NmTxx57jA0bNsjOWU1AVTpYQiTsc9+Zi+7zEy72RoNeTU0l/Xe3kNDp8mrvpWxQVYz2pGjvGwl7IaDS22xFUbDZbAAkJSVhsVT+3LRNmzbMnj07+uu9e/fSo0cPAPr27cvWrVtrOl7RACgKEPARzM3Bn5uH5q847PVwGNeiBWj5+YS9nkjYGwwY7A6M6c1J7HxFtcLeoKqYU5Ixt2iBYrNLozMhTlDpHX6bNm145pln6NatGzt37qRNmzaVvumQIUM4fPhw9Ne6rke/aW02W5U2QldVhZSUxErPq/j1hlq9vr7Wqu/1wj4fIbcbze/DYlLAVPENguebbzgy7z8EfzkWPWZMTsaYkgKKgpbvwmGv4jN3RcFoT8KYZK/W6tr6/HvZkGo19nqN5doq/c54+umnWbBgAVu3bqV9+/ZMmjSp2kVOfF7v9XpxOByVvkbT9FrtDBTPnYXivYtRvaxXhQ6WZYLHs8lbsojiz3ZHjykWK8Y0J0aLBU0Lo/t8qM40ity+M9dVFIxWK6rDgR8TeAJA1fvb18vfywZYq7HXa0jXlp5e8faclT7S2bt3L5qm8dhjj7F792727dtX7QFceumlfPJJZDu5zZs3061bt2q/h6intBDhwgICx48T9HrPGPaa10vewvkcenxGNOwtbduROmYcqj0JtDC6rqP7/ehaiOTBQ89YWjUasTidGNPSwCiLp4SoTKV3+E888QTPPPMMAPfffz9Tp05l3rx51SoyZcoUZsyYwYsvvki7du0YMmRIzUYr6g8tRNjjIVRcXGGr4jK6FqLow03kr1pB2OsFwOh04hw9Flu3HiiKgrnluRSuX4vmykN1ppE8eCi2zp3LfT/FYMBkT0KxJYH0vRGiyioNfKPRyIUXXghA69atqzydslWrVixcuBCAtm3b8vbbb9dimKLe0EKEK+lJX0bXdYr3fI5rSRbB7GwAFKuV1BtuxDFwEAazOXqurXNnbJ0747Bbz/gY58S+N0KI6qk08M8991xefPFFrrzySr744guaN28ej3GJekbRQmjFXkKeyoMewH/wAHlZC/B9/13pGyjYr+2Lc8RI1Cp8hnOqyDRLe+SuXghRI5UG/t/+9jfeeecdPvzwQ9q3b88999wTj3GJekIJawQL8vHn5FW4+ciJQvn5uJYvxbN9668Lpy7rRNrYcZjPPa/69Q0KxkQbalKSbEYiRC1V+h1ksViYOHFiHIYi6hMlrBEuLibo8WBONFUa9mG/n4J1aylctxY9EJklYzr3PNLGjiPxsk41GACoFmukuZm0LRaiTsgtkzhJ2XaCQc+Ju0xVPANGD4fxbNuKa/kStMJCAFS7g9SMkdh79UFRT+9+WRlVVTEmOyBBGpwJUZck8AVw8h19RdsJnqrk22/IW7SAwKFDkfcwmUgedD0pQ2/AYE2owSAUVJsNU5IZ3VD9fyiEEGcmgd/ElX9Hf2aBX37BtSSL4j2fR48l9eiJc9QYjM60Go1DNZkwpiRjTnNSLP3phYgJCfwmStHDhCvZIPxUmsdD/qoVFH24CcKR11jaX0ha5nisbdvVbByle8kakuygSAdVIWJJAr+JqUnQh4NBCta/R8GaVYSLI3ffxmbNcI7OxHZV12p3siwjc+qFiC8J/CaiLOhDbg9aFYNe13W8u3dxeNliAsePA2BISCDlxhEk9x+IYqpZOwPFYMDksEe6WQoh4kYCv7HTw+jVDHoA308/kpe1AP/+HyIHDAYcffuTOuI3qEk1D+qyvWSROfVCxJ181zVWpUGvub1oWqjKLwu58nAtXYLn0+3RY44uXXCMHIP5nJY1Ho5BVTE6HLKXrBBnkQR+Y6Pr6CXFaG43WqjqQR/2lVCw9l0K31+HXrrfrLlVa9LGjqNFj6sqb1NcEUXBlJCA6nDISlkhzjL5DmxMfMWR7QQr2SD8RHo4jHvLR+QvX4bmLgJATU4mNWMU9mt6o9Ri72HVaMSYnIySkCAdLYWoByTwGwNfcWSXqQo2CK9I8ddf4VqUReBIZHcyxWQm+fohpFw/FIPVGt1Y/FAVWhafRFEw2WyRu3ppXyxEvSGB32Dp4PMR8rjR/FXf4QkgcPQoeYsXUvLVl5EDikLS1dfgzBiFMdUJRDYWz5v/NopqRLXZ0AoKyJv/NnDzGUNfNZswJqeA2SL9b4SoZyTwGxwdzesllJuDFghUuDl4eTS3m/yVyyj6aDOUtji2XtyBtMzxWNqcf9K5hevXoqhGFIsFRVFQLBbwR46XF/iKQcFU1r5YFlAJUS9J4DcYOvj9hNxFBMyGat3Vh4NBijasJ//dNei+EgBMzVvgHDOWxCu6lLtwKpSbg5JoO/mg2UwoN+e0c2UBlRANgwR+PacooPt9kQ9jA4FIj3mztUqv1XUd784duJYuJpSXC4AhMZHU4b/B0W8AirHiP35js3S0ggKwWH49GAhgbJb+69jKFlAlJkUGKoSo1yTw66myoA+6S5/RV/OTT9/+HyILp376MXLAoOIYMJDUG4ejVmHXqOTBQyPP7P2gJ1hP3lhcAaM1ATU5WRZQCdGAyHdrPRMJej9Bd1GNgj6Ym4tr6SK8O3dEjyVeeRVpo8diatGiyu8TeU5/82kbi9uvvBKT9KoXokGSwK8nahv04ZJi8t9dQ9GG9eilC67Mbc4nLXM8CRd3qNGYTtpY3OPHlJiAapcFVEI0VPKde5bVNuh1TcP98WZcK5YR9ngAUFNScY4cTVLPq2u1cCo6RqMRi9MmC6iEaOAk8M+S2j6j13Wdkq++JG/xQoLHjkXe02IhZcgNJA++HoPZUsk7VGGMBgWjLQlL83R8noCEvRANnAR+nNU26AFKDh7kl7fmUvLN19E3tffqQ2rGyMiipzqgWswYHclgtpTO5qne4i4hRP0jgR8ntX10AxAqLCR/xTLcWz6Kvj6h4yU4x47H0rp13YxTploK0WhJ4MdYXdzRhwMBCt9fR8HaNeh+PwCmc1qSNjaThE6X13jHqVNJr3ohGjf5zo6Rsjv6kMdNyOevUdDr4TCeTz/BtWwJWr4LAIMtiZZjx2Du2QuljoJZetUL0TRI4Nexugh6gJJ93+NatBD/zz9FDhiNJA8cRMoNN5Lawlnz/vSnDFamWgrRdMh3eR1RFNADgUibYp8PvYZBH8w5jmvJIry7d0WP2bp2xzl6DKYT2hrUlvSqF6LpkcCvJUWBsM+H5nIRKimpcdBrXi8Fa1ZR+MEGKN171tK2HWmZ47G2v7AOx6tglF71QjRJEvg1VHZHr3k8+I06weKSGr2ProUo+vBD8lctJ+z1AmB0OnGOGoute486+0AWpFe9EE2dBH5NBHyEPF40vw89rJNgr1r3yhPpuk7xF3twLc4imP0LAIrVSsrQYSRfNxiDue5aDUuveiEESOBXnR5GLylB83rQgqEafxgL4D94gLxFC/F9923kgKJgv7YvqSMyIoud6pD0qhdClJHAr4weRi8uRvN40EqbktVUKD8f1/KleLZv/XXh1GWdSBuTifm8VnUx2ihZQCWEOFWjC3z3zh24lmTxY85x1PTmOEdnYu/Wvdrvo+hhwsVeNLcXTatd0If9fgrXraVg3Vr0QKRFgencc0kbM47ETlXYFLxUVTcVV61WjCkpsoBKCHGSuCVCMBhk6tSpHDlyBIPBwBNPPEH79u3rtIZ75w6yX30ZxWjCaLcTcrnIfvVluOtPVQ99LYReUkzQ4yVcOlumpvRwGM/2rbiWL43sHgWodgepv8nA3vtaFFWt8ntVZVNxWUAlhDiTuAX+hx9+SCgUYv78+WzZsoW///3vzJ49u05ruJZkoRhNGKxWFEXBYLUS9kWOnynwy2bchL3eyNTK0g2+a6Pku2/Jy1pA4NDBSA2jkeRB15MydBiGhIRqv98ZNxW//HJZQCWEqFTc0qFt27ZomkY4HMbj8WA8w36qAKqqkJKSWK0aP+Ycx2i3RwJRAVU1YEhMQMs5XuF7hUtKCHncaD4/GHSwVf/DTVU14CidqeM7doyj/3mHol2/LpxK7dWLluPHYU6v+cKpQ648VJvtpGvTE6wYQgGanX8uqs1W+ZvUkKoaqv1n0RBqNfZ6jfna4l2vsVxb3AI/MTGRI0eOcMMNN5Cfn8+rr756xvM1TaegoLhaNdT05oRcLgxWK6pqQNPChH0+jOnNT3qvaPuDWnSuPJHDbiX/WC75q1dQtGkThEsXTrW/MLJwqm07fICvFu0QVGcaWkEBisUSCXvViNGWiMFuxx1UoJq/V9WRkpJY7T+LhlCrsddrzNcW73oN6drS0+0Vfi1uk7LffPNN+vTpw3vvvcfy5cuZOnUq/tLOj3XFOToTPRQkXNraIOzzoYeCOEdnlp6hQ8BPMDcHf25u5K6+lmGvB4McX72GQzMeoWjjBghrGJs1o/kdd3Pu5KlY27ar/YUR2VRc10LowSBGhwOjPYlgQQGOQUPr5P2FEI1f3O7wHQ4HJpMJgOTkZEKhEFotPxQ9lb1bd7jrT7iWZKHlHMeY3py0seOxX3EF4WIPmtdb6zn0ZXRdx/vZLlyLFxHKzQHAkJBAyrDhJA+4DqX0WuuKrXNnFONEfLs/JZCdDdYE0m+7vUYzkIQQTVPcAn/ixIk8+uij3HTTTQSDQR544AESE+v+GZW9W3fs3bpHfiTKKyLs9eI/frxOPogt4/v5J1xZC/D9sC9ywGDA0bc/qSN+g5pU8Y9TtaFaLaT26weDBsf9x0shROMQt8C32Wy89NJL8SpHqKiQwPGcOg36kMuFa9liPJ9sjx5L7HwFbW69mYDDWWd1TmRQ1cgH0TYbMtVSCFEbjXIOn6KAVuKrs7AP+0ooWPsuhe+vQw8GATC3aoVz7HgSL7kUq91KoC76059IUTAlJGBwOGQBlRCiTkiSnIEeDuPe8hH5K5ahFRUBoDqSSR05Cvs1vVEMsfnMW3rVCyFiQQK/AsVf78W1aCGBI4cBUExmkq8fQsr1QzFYq98dsyqkV70QIpYk8E8ROHqUvMULKfnqy+ixpKt74Rw5CmNqbJ7Tg/SqF0LEngR+Kc3tJn/lMoo+2gylz/6tF11M2tjxWC64IGZ1FYMBkz1JetULIWKuyQd+OBikaOP75K9Zje6L7FplbN6ctDGZJF7RpU53nDqVarVE+t9Lr3ohRBw02cDXdR3vzh24li4mlJcLgCExkdThv8HRbwBKJb1+akOmWgohzoYmGfi+/T+Qt2gh/h/3Rw4YVBz9B5A6fASqLSl2hWWqpRDiLGpSqRPMzcW1dBHenTuixxKv7IJz9FjMLc6JaW2ZaimEONuaROCHS4rJf3cNRRvWo5duU2hucz5pY8eR0KFjTGvLVEshRH3RqANf1zTcH3+Ea+Uywm43AGpKKs6Ro0nqeXXMFk6VkamWQoj6pNEGfuHu3Rz+95sEjx0FQDGbSRlyA8mDh2CwWGJaW6ZaCiHqo0YX+FpxMUeefQrvrp2RA4pC0jW9cGaMwpiSGvP6qtUSuas31m17ZCGEqK1GF/ie7VujYW/t0JG0zPFYWreJeV2ZaimEqO8aXeDbr+lN+i3HcbQ+j3CbdjFdOAWAoqAmJGKymWWqpRCiXmt0CWVISCB9wk1YfW4KcgtjWqtsqqU5PY1i2ZBECFHPNbrAj4dTp1oKIURDIIFfTarFHOl/I1MthRANjAR+Ff061dIe2VJLCCEaGAn8KlAtFozJ0tVSCNGwSeCfgWIwYHLYI3f1QgjRwEngV8CYYEVNTpGplkKIRkPS7BQGVcXocKAkJiILqIQQjYkEfpnSXvWqw4Eud/VCiEZIko2yBVQOlIREaV8shGi0mnbgKwomWyKq3YFuUCXshRCNWpMNfNVkwpiSDGarLKASQjQJTS7wFYOCMSkJNckubRGEEE1Kkwp8aYsghGjKmkTgS1sEIYRoAoEvbRGEECKi0Qa+YjBgTkmWtghCCFGqUQa+roMxNRXFEzjbQxFCiHqj0U5TMRgb5b9lQghRY3FNxX/9619s3LiRYDDIb3/7WzIzM+NZXgghmrS4Bf4nn3zCZ599xjvvvENJSQn//ve/41VaCCEEoOh6fBoKvPDCCyiKwr59+/B4PDz88MN07ty5wvPD4TCaVvOhqaoBTQvX+PX1tVZjr9eYry3e9RrztcW7XkO6NpNJrfBrcbvDz8/P5+jRo7z66qscPnyYu+++m7Vr16JUMC9e03QKCoprXC8lJbFWr6+vtRp7vcZ8bfGu15ivLd71GtK1padXPDMxboGfkpJCu3btMJvNtGvXDovFgsvlIi0tLV5DEEKIJi1us3S6du3KRx99hK7rZGdnU1JSQkpKSrzKCyFEkxe3O/wBAwawY8cOxo4di67rPPbYY6hqxc+ahBBC1K24Tst8+OGH41lOCCHECeI2S0cIIcTZ1WhX2gohhDiZBL4QQjQREvhCCNFESOALIUQTIYEvhBBNhAS+EEI0ERL4QgjRRDSoXUKCwSCPPvooR44cIRAIcPfdd3PhhRcydepUFEXhoosuYubMmRgMkX/HXC4XEyZMYOXKlVgsFgAOHDjAf/3Xf7Fq1aqY1nK73UyePBmPx0MwGGTq1Kl06dIlZvWKi4uZNGkShYWFJCQk8Pzzz+N0OmP6ewmwf/9+xo0bx9atW086Xpe1dF2nb9++XHDBBQBceeWVTJo0KWbXpmkaf/vb3/jqq68IBALce++9DBgwIGb1XnvtNT766CMAioqKyM3NZcuWLTGp5Xa7eeCBBygpKcFkMvH888+Tnp4es2srKCiIfh+kpKTw5JNPnrF/VnXqvfnmm6xevRqAfv368ac//Qmfz8fkyZPJy8vDZrPx7LPPVvh9UNtaZdavX8/atWt54YUXKryuuqhX3Uwpl96ALFq0SH/yySd1Xdd1l8ul9+vXT7/zzjv17du367qu6zNmzNDXrVun67qub968Wc/IyNC7dOmi+3w+Xdd1fenSpfqoUaP0Xr16xbzWSy+9pL/xxhu6ruv6/v379ZEjR8a03htvvKHPnj1b13VdX7x4sf7EE0/EtJ6u67rb7dZvv/12/eqrrz7peF3X+vnnn/U777zzjNdTl/UWL16sz5w5U9d1Xf/ll1+if46xqneiO+64Q9+8eXPMar355pv6s88+q+u6ri9YsED/29/+FtNre+aZZ/RXXnlF13Vd37Jli/7oo4/WSb2DBw/qo0aN0kOhkK5pmj5+/Hj9m2++0f/973/r//jHP3Rd1/VVq1ad8fugtrV0XdefeOIJfciQIfr9999/xuuqi3rVzZTyNKhHOkOHDuW+++6L/lpVVfbu3UuPHj0A6Nu3L1u3bgXAYDDwxhtvnNSgLTk5mbfffjsutSZOnMiECRMA0DStwrvfuqx39913A3D06FGaNWsW03q6rjNjxgwefPBBEhISYlpr7969ZGdnc8stt3D77bfz448/xrTexx9/zDnnnMMdd9zB9OnTGThwYEzrlVm3bh0Oh4Nrr702ZrUuvvhivF4vAB6PB2MlW4HWtt4PP/xA3759AbjqqqvYtWtXndQ755xz+N///V9UVcVgMBAKhbBYLOzatSv6+9e3b1+2bdsWs1pl1/T444+f8Zrqql51M6U8DSrwbTYbSUlJeDwe/vznP3P//fej63q0p77NZsPtdgPQu3dvUlNTT3r9gAEDSExMjEsth8OB1WolJyeHyZMn8+CDD8a0HkT+At166628/fbb9OvXL6b1Xn75Zfr160fHjh3PWKcuaqWnp3PHHXcwd+5c7rzzTiZPnhzTevn5+Rw4cIB//etf3H777TzyyCMxrVfmX//610mPCmJRKzU1lS1btjBs2DBef/11xo4dG9N6l1xyCRs3bgRg48aN+Hy+OqlnMplwOp3ous6zzz7LpZdeStu2bfF4PNjt9tPGFotaAMOGDatwT4+6rlfdTClPgwp8gGPHjnHrrbeSkZHBiBEjos8OAbxeLw6Ho97U+u6775g4cSIPPPBA9F/xWNYD+L//+z/mzZvHvffeG9N6K1asYPHixdxyyy3k5OTwhz/8IWa1OnXqxHXXXQdAt27dyM7ORq+kBVRt6qWkpNC/f38URaFHjx78/PPPZ6xV23oQuRN2OBycf/75Ma318ssv88c//pE1a9bw+uuvx/zvyR133MGRI0eYOHEix44d45xzzqmzen6/n4ceegiv18vMmTMBSEpKiv4EU5Xf99rUqona1qtuppyqQQV+bm4uf/jDH5g8eXL0zuTSSy/lk08+AWDz5s1069atXtT64YcfuO+++3jhhRcqvduui3r/+te/WLZsGQCJiYmVtp6ubb3169czd+5c5s6dS3p6+hn3KK5trZdffpm33noLgG+//ZZzzz33jHdVta3XtWtXPvzww2i9li1bVnhuXdQD2Lp1a/TRRyxrORyO6B1wWlpaNBxjVW/nzp1kZGTw5ptv0qpVK6666qo6qafrOvfccw8dOnTgr3/9a/Tv+1VXXRX9s9u8eTNdu3aNWa3qqm296mZKeRpUt8wnn3ySd999l3bt2kWPTZs2jSeffJJgMEi7du148sknT/oDGThwIO++++5Jz7t69+5d4SyIuqp19913891333HeeecBkTuPV155JWb1cnNzmTJlCoFAAE3TmDRp0hn/stfV7+WZjtdVrcLCQiZPnkxxcTGqqvLYY4/Rvn37mF1bIBBg5syZ7N+/H13Xefzxx7nsssti+nv5l7/8hd69ezNo0KAK69RFrezsbKZPn05xcTGhUIg///nP9O7dO2b1Dhw4wJQpUwBo3rw5Tz/9NElJSbWut3HjRh588EGuvPLK6HkPPvggHTt2ZMqUKeTk5GAymXjhhRcqnIVU21plM2Q++eQT5s+fz3//939XeF11Ue+1116rVqaUp0EFvhBCiJprUI90hBBC1JwEvhBCNBES+EII0URI4AshRBMhgS+EEE2EBL4QQjQREvhCCNFENKj2yELE2qRJkxgxYgT9+/dn//79PPvsszRr1owDBw4QDoe5//776dmzJ2vXrmXevHnR17300kvs27ePWbNmYTKZGDduHCNHjjx7FyJEOSTwhThBZmYm77zzDv3792fRokV06dIFj8fD008/TX5+PjfffDOrV6/m559/5rXXXiMhIYHHHnuMjz/+mBYtWuD3+8nKyjrblyFEuSTwhThBz549eeqpp8jLy2PLli106dKF3bt388UXXwAQCoXIz88nLS2NKVOmYLPZ+PHHH6PL4Mu6KApRH0ngC3ECRVEYMWIETz31FL1796Zly5a0bNmSu+66C5/PxyuvvILRaOQf//gHmzZtAuC2226Ldu88sfuhEPWNBL4Qpxg9ejT9+/dn+fLltG7dmunTp3PzzTfj8Xi46aabSEpK4qqrrmLUqFEkJibicDg4fvw4rVq1OttDF+KMpHmaEKfIzs7m4YcfjrZkFqKxkJ8/hTjBe++9xx//+MczbpIuREMld/hCCNFEyB2+EEI0ERL4QgjRREjgCyFEEyGBL4QQTYQEvhBCNBH/H20WFJO/rltTAAAAAElFTkSuQmCC",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "import matplotlib.pyplot as plt\n",
+    "import seaborn as sns\n",
+    "# import matplotlib \n",
+    "\n",
+    "\n",
+    "dates = [2011, 2013, 2014, 2015, 2017, 2019, 2020, 2022]\n",
+    "values = [7.99, 7.99, 8.99, 9.99, 10.99, 12.99, 13.99, 15.49]\n",
+    "\n",
+    "# plt.scatter(dates,values)\n",
+    "# plt.scatter(dates, values)\n",
+    "# sns.set_style('darkgrid')\n",
+    "sns.set_palette(\"RdBu\")\n",
+    "# sns.color_palette(\"flare\", as_cmap=True)\n",
+    "sns.regplot(x=dates, y=values)\n",
+    "# plt.plot(dates, values)\n",
+    "plt.xlabel('year')\n",
+    "plt.ylabel('cost (USD)') \n",
+    "plt.xticks(range(2011, 2023))\n",
+    "plt.title('Netflix (standard) Time vs Price')\n",
+    "# plt.legend()\n",
+    "\n",
+    "plt.savefig('netflix.jpg')\n",
+    "plt.show()\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "interpreter": {
+   "hash": "b7e818f66e33c31ac0526ee7f8556503ff93918b8b22809241939dc19e90de0b"
+  },
+  "kernelspec": {
+   "display_name": "Python 3.8.12 64-bit ('pytorch_m1': conda)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python",
+   "version": "3.8.12"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

code/penguin_classifier.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36bab9fe0b430613c34c25bca7a87e8ec059c61b40119d71da00153ef95ed3cb
+size 2839

code/penguins_nn.ipynb

@@ -0,0 +1,866 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 22,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "<div>\n",
+       "<style scoped>\n",
+       "    .dataframe tbody tr th:only-of-type {\n",
+       "        vertical-align: middle;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe tbody tr th {\n",
+       "        vertical-align: top;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe thead th {\n",
+       "        text-align: right;\n",
+       "    }\n",
+       "</style>\n",
+       "<table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       "    <tr style=\"text-align: right;\">\n",
+       "      <th></th>\n",
+       "      <th>CulmenLength</th>\n",
+       "      <th>CulmenDepth</th>\n",
+       "      <th>FlipperLength</th>\n",
+       "      <th>BodyMass</th>\n",
+       "      <th>Species</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <th>187</th>\n",
+       "      <td>48.4</td>\n",
+       "      <td>16.3</td>\n",
+       "      <td>22.0</td>\n",
+       "      <td>54.00</td>\n",
+       "      <td>1</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>303</th>\n",
+       "      <td>49.5</td>\n",
+       "      <td>19.0</td>\n",
+       "      <td>20.0</td>\n",
+       "      <td>38.00</td>\n",
+       "      <td>2</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>196</th>\n",
+       "      <td>50.5</td>\n",
+       "      <td>15.9</td>\n",
+       "      <td>22.2</td>\n",
+       "      <td>55.50</td>\n",
+       "      <td>1</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>16</th>\n",
+       "      <td>38.7</td>\n",
+       "      <td>19.0</td>\n",
+       "      <td>19.5</td>\n",
+       "      <td>34.50</td>\n",
+       "      <td>0</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>85</th>\n",
+       "      <td>41.3</td>\n",
+       "      <td>20.3</td>\n",
+       "      <td>19.4</td>\n",
+       "      <td>35.50</td>\n",
+       "      <td>0</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>234</th>\n",
+       "      <td>47.4</td>\n",
+       "      <td>14.6</td>\n",
+       "      <td>21.2</td>\n",
+       "      <td>47.25</td>\n",
+       "      <td>1</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>145</th>\n",
+       "      <td>39.0</td>\n",
+       "      <td>18.7</td>\n",
+       "      <td>18.5</td>\n",
+       "      <td>36.50</td>\n",
+       "      <td>0</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>21</th>\n",
+       "      <td>37.7</td>\n",
+       "      <td>18.7</td>\n",
+       "      <td>18.0</td>\n",
+       "      <td>36.00</td>\n",
+       "      <td>0</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>30</th>\n",
+       "      <td>39.5</td>\n",
+       "      <td>16.7</td>\n",
+       "      <td>17.8</td>\n",
+       "      <td>32.50</td>\n",
+       "      <td>0</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>330</th>\n",
+       "      <td>42.5</td>\n",
+       "      <td>17.3</td>\n",
+       "      <td>18.7</td>\n",
+       "      <td>33.50</td>\n",
+       "      <td>2</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table>\n",
+       "</div>"
+      ],
+      "text/plain": [
+       "     CulmenLength  CulmenDepth  FlipperLength  BodyMass  Species\n",
+       "187          48.4         16.3           22.0     54.00        1\n",
+       "303          49.5         19.0           20.0     38.00        2\n",
+       "196          50.5         15.9           22.2     55.50        1\n",
+       "16           38.7         19.0           19.5     34.50        0\n",
+       "85           41.3         20.3           19.4     35.50        0\n",
+       "234          47.4         14.6           21.2     47.25        1\n",
+       "145          39.0         18.7           18.5     36.50        0\n",
+       "21           37.7         18.7           18.0     36.00        0\n",
+       "30           39.5         16.7           17.8     32.50        0\n",
+       "330          42.5         17.3           18.7     33.50        2"
+      ]
+     },
+     "execution_count": 22,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "import pandas as pd\n",
+    "\n",
+    "# load the training dataset (excluding rows with null values)\n",
+    "penguins = pd.read_csv('/Users/johnnydevriese/projects/data/penguins.csv').dropna()\n",
+    "\n",
+    "# Deep Learning models work best when features are on similar scales\n",
+    "# In a real solution, we'd implement some custom normalization for each feature, but to keep things simple\n",
+    "# we'll just rescale the FlipperLength and BodyMass so they're on a similar scale to the bill measurements\n",
+    "penguins['FlipperLength'] = penguins['FlipperLength']/10\n",
+    "penguins['BodyMass'] = penguins['BodyMass']/100\n",
+    "\n",
+    "# The dataset is too small to be useful for deep learning\n",
+    "# So we'll oversample it to increase its size\n",
+    "for i in range(1,3):\n",
+    "    penguins = penguins.append(penguins)\n",
+    "\n",
+    "# Display a random sample of 10 observations\n",
+    "sample = penguins.sample(10)\n",
+    "sample"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 23,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "['CulmenLength' 'CulmenDepth' 'FlipperLength' 'BodyMass' 'Species'] SpeciesName\n",
+      "[ 42.7 18.3 19.6 40.75 0 ] Adelie\n",
+      "[ 37.9 18.6 19.3 29.25 0 ] Adelie\n",
+      "[ 39.0 17.1 19.1 30.5 0 ] Adelie\n",
+      "[ 50.2 18.8 20.2 38.0 2 ] Chinstrap\n",
+      "[ 45.2 14.8 21.2 52.0 1 ] Gentoo\n",
+      "[ 45.7 13.9 21.4 44.0 1 ] Gentoo\n",
+      "[ 38.8 20.0 19.0 39.5 0 ] Adelie\n",
+      "[ 43.8 13.9 20.8 43.0 1 ] Gentoo\n",
+      "[ 46.0 18.9 19.5 41.5 2 ] Chinstrap\n",
+      "[ 49.4 15.8 21.6 49.25 1 ] Gentoo\n"
+     ]
+    }
+   ],
+   "source": [
+    "penguin_classes = ['Adelie', 'Gentoo', 'Chinstrap']\n",
+    "print(sample.columns[0:5].values, 'SpeciesName')\n",
+    "for index, row in penguins.sample(10).iterrows():\n",
+    "    print('[',row[0], row[1], row[2],row[3], int(row[4]), ']',penguin_classes[int(row[-1])])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 24,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Training Set: 957, Test Set: 411 \n",
+      "\n",
+      "Sample of features and labels:\n",
+      "[51.1 16.5 22.5 52.5] 1 (Gentoo)\n",
+      "[50.7 19.7 20.3 40.5] 2 (Chinstrap)\n",
+      "[49.5 16.2 22.9 58. ] 1 (Gentoo)\n",
+      "[39.3 20.6 19.  36.5] 0 (Adelie)\n",
+      "[42.5 20.7 19.7 45. ] 0 (Adelie)\n",
+      "[50.  15.3 22.  55.5] 1 (Gentoo)\n",
+      "[50.2  18.7  19.8  37.75] 2 (Chinstrap)\n",
+      "[50.7 19.7 20.3 40.5] 2 (Chinstrap)\n",
+      "[49.1  14.5  21.2  46.25] 1 (Gentoo)\n",
+      "[43.2 16.6 18.7 29. ] 2 (Chinstrap)\n",
+      "[38.8  17.6  19.1  32.75] 0 (Adelie)\n",
+      "[37.8 17.1 18.6 33. ] 0 (Adelie)\n",
+      "[45.8 14.2 21.9 47. ] 1 (Gentoo)\n",
+      "[43.8 13.9 20.8 43. ] 1 (Gentoo)\n",
+      "[36.  17.1 18.7 37. ] 0 (Adelie)\n",
+      "[43.3 13.4 20.9 44. ] 1 (Gentoo)\n",
+      "[36.  18.5 18.6 31. ] 0 (Adelie)\n",
+      "[41.1  19.   18.2  34.25] 0 (Adelie)\n",
+      "[33.1 16.1 17.8 29. ] 0 (Adelie)\n",
+      "[40.9 13.7 21.4 46.5] 1 (Gentoo)\n",
+      "[45.2 17.8 19.8 39.5] 2 (Chinstrap)\n",
+      "[48.4 14.6 21.3 58.5] 1 (Gentoo)\n",
+      "[43.6 13.9 21.7 49. ] 1 (Gentoo)\n",
+      "[38.5  17.9  19.   33.25] 0 (Adelie)\n"
+     ]
+    }
+   ],
+   "source": [
+    "from sklearn.model_selection import train_test_split\n",
+    "\n",
+    "features = ['CulmenLength','CulmenDepth','FlipperLength','BodyMass']\n",
+    "label = 'Species'\n",
+    "   \n",
+    "# Split data 70%-30% into training set and test set\n",
+    "x_train, x_test, y_train, y_test = train_test_split(penguins[features].values,\n",
+    "                                                    penguins[label].values,\n",
+    "                                                    test_size=0.30,\n",
+    "                                                    random_state=0)\n",
+    "\n",
+    "print ('Training Set: %d, Test Set: %d \\n' % (len(x_train), len(x_test)))\n",
+    "print(\"Sample of features and labels:\")\n",
+    "\n",
+    "# Take a look at the first 25 training features and corresponding labels\n",
+    "for n in range(0,24):\n",
+    "    print(x_train[n], y_train[n], '(' + penguin_classes[y_train[n]] + ')')\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 25,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Libraries imported - ready to use PyTorch 1.10.0\n"
+     ]
+    }
+   ],
+   "source": [
+    "import torch\n",
+    "import torch.nn as nn\n",
+    "import torch.utils.data as torch_data\n",
+    "\n",
+    "# Set random seed for reproducability\n",
+    "torch.manual_seed(0)\n",
+    "\n",
+    "print(\"Libraries imported - ready to use PyTorch\", torch.__version__)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 26,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Ready to load data\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Create a dataset and loader for the training data and labels\n",
+    "train_x = torch.Tensor(x_train).float()\n",
+    "train_y = torch.Tensor(y_train).long()\n",
+    "train_ds = torch_data.TensorDataset(train_x,train_y)\n",
+    "train_loader = torch_data.DataLoader(train_ds, batch_size=20,\n",
+    "    shuffle=False, num_workers=1)\n",
+    "\n",
+    "# Create a dataset and loader for the test data and labels\n",
+    "test_x = torch.Tensor(x_test).float()\n",
+    "test_y = torch.Tensor(y_test).long()\n",
+    "test_ds = torch_data.TensorDataset(test_x,test_y)\n",
+    "test_loader = torch_data.DataLoader(test_ds, batch_size=20,\n",
+    "    shuffle=False, num_workers=1)\n",
+    "print('Ready to load data')\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 27,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "PenguinNet(\n",
+      "  (fully_connected1): Linear(in_features=4, out_features=10, bias=True)\n",
+      "  (fully_connected2): Linear(in_features=10, out_features=10, bias=True)\n",
+      "  (fully_connected3): Linear(in_features=10, out_features=3, bias=True)\n",
+      ")\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Number of hidden layer nodes\n",
+    "hl = 10\n",
+    "hidden_layer_nodes = 10\n",
+    "initial_input_feature_dimension = len(features)\n",
+    "output_feature_dimension = len(penguin_classes)\n",
+    "\n",
+    "# Define the neural network\n",
+    "class PenguinNet(nn.Module):\n",
+    "    def __init__(self):\n",
+    "        super(PenguinNet, self).__init__()\n",
+    "        self.fully_connected1 = nn.Linear(in_features=len(features), out_features=hl, bias=True) # bias=True is default\n",
+    "        self.fully_connected2 = nn.Linear(hl, hl)\n",
+    "        self.fully_connected3 = nn.Linear(hl, len(penguin_classes))\n",
+    "\n",
+    "    def forward(self, x):\n",
+    "        x = torch.relu(self.fully_connected1(x))\n",
+    "        x = torch.relu(self.fully_connected2(x))\n",
+    "        x = torch.relu(self.fully_connected3(x))\n",
+    "        return x\n",
+    "\n",
+    "# Create a model instance from the network\n",
+    "model = PenguinNet()\n",
+    "print(model)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 28,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch: 0\n",
+      "Training set: Average loss: 1.118814\n",
+      "Validation set: Average loss: 1.023595, Accuracy: 148/411 (36%)\n",
+      "\n",
+      "Epoch: 1\n",
+      "Training set: Average loss: 1.010274\n",
+      "Validation set: Average loss: 0.983460, Accuracy: 163/411 (40%)\n",
+      "\n",
+      "Epoch: 2\n",
+      "Training set: Average loss: 0.965314\n",
+      "Validation set: Average loss: 0.934165, Accuracy: 191/411 (46%)\n",
+      "\n",
+      "Epoch: 3\n",
+      "Training set: Average loss: 0.911513\n",
+      "Validation set: Average loss: 0.867269, Accuracy: 250/411 (61%)\n",
+      "\n",
+      "Epoch: 4\n",
+      "Training set: Average loss: 0.817720\n",
+      "Validation set: Average loss: 0.742112, Accuracy: 272/411 (66%)\n",
+      "\n",
+      "Epoch: 5\n",
+      "Training set: Average loss: 0.733329\n",
+      "Validation set: Average loss: 0.691639, Accuracy: 302/411 (73%)\n",
+      "\n",
+      "Epoch: 6\n",
+      "Training set: Average loss: 0.696301\n",
+      "Validation set: Average loss: 0.661350, Accuracy: 312/411 (76%)\n",
+      "\n",
+      "Epoch: 7\n",
+      "Training set: Average loss: 0.671731\n",
+      "Validation set: Average loss: 0.640087, Accuracy: 327/411 (80%)\n",
+      "\n",
+      "Epoch: 8\n",
+      "Training set: Average loss: 0.653092\n",
+      "Validation set: Average loss: 0.624311, Accuracy: 338/411 (82%)\n",
+      "\n",
+      "Epoch: 9\n",
+      "Training set: Average loss: 0.638097\n",
+      "Validation set: Average loss: 0.610605, Accuracy: 345/411 (84%)\n",
+      "\n",
+      "Epoch: 10\n",
+      "Training set: Average loss: 0.625696\n",
+      "Validation set: Average loss: 0.598022, Accuracy: 345/411 (84%)\n",
+      "\n",
+      "Epoch: 11\n",
+      "Training set: Average loss: 0.614685\n",
+      "Validation set: Average loss: 0.588183, Accuracy: 353/411 (86%)\n",
+      "\n",
+      "Epoch: 12\n",
+      "Training set: Average loss: 0.605506\n",
+      "Validation set: Average loss: 0.578678, Accuracy: 358/411 (87%)\n",
+      "\n",
+      "Epoch: 13\n",
+      "Training set: Average loss: 0.597361\n",
+      "Validation set: Average loss: 0.569911, Accuracy: 361/411 (88%)\n",
+      "\n",
+      "Epoch: 14\n",
+      "Training set: Average loss: 0.590228\n",
+      "Validation set: Average loss: 0.562248, Accuracy: 361/411 (88%)\n",
+      "\n",
+      "Epoch: 15\n",
+      "Training set: Average loss: 0.583250\n",
+      "Validation set: Average loss: 0.556146, Accuracy: 372/411 (91%)\n",
+      "\n",
+      "Epoch: 16\n",
+      "Training set: Average loss: 0.576846\n",
+      "Validation set: Average loss: 0.549725, Accuracy: 375/411 (91%)\n",
+      "\n",
+      "Epoch: 17\n",
+      "Training set: Average loss: 0.571098\n",
+      "Validation set: Average loss: 0.544390, Accuracy: 382/411 (93%)\n",
+      "\n",
+      "Epoch: 18\n",
+      "Training set: Average loss: 0.565975\n",
+      "Validation set: Average loss: 0.540335, Accuracy: 384/411 (93%)\n",
+      "\n",
+      "Epoch: 19\n",
+      "Training set: Average loss: 0.561476\n",
+      "Validation set: Average loss: 0.536972, Accuracy: 389/411 (95%)\n",
+      "\n",
+      "Epoch: 20\n",
+      "Training set: Average loss: 0.557517\n",
+      "Validation set: Average loss: 0.532509, Accuracy: 390/411 (95%)\n",
+      "\n",
+      "Epoch: 21\n",
+      "Training set: Average loss: 0.553931\n",
+      "Validation set: Average loss: 0.529417, Accuracy: 396/411 (96%)\n",
+      "\n",
+      "Epoch: 22\n",
+      "Training set: Average loss: 0.550773\n",
+      "Validation set: Average loss: 0.528216, Accuracy: 397/411 (97%)\n",
+      "\n",
+      "Epoch: 23\n",
+      "Training set: Average loss: 0.547976\n",
+      "Validation set: Average loss: 0.523656, Accuracy: 397/411 (97%)\n",
+      "\n",
+      "Epoch: 24\n",
+      "Training set: Average loss: 0.545466\n",
+      "Validation set: Average loss: 0.521025, Accuracy: 397/411 (97%)\n",
+      "\n",
+      "Epoch: 25\n",
+      "Training set: Average loss: 0.543647\n",
+      "Validation set: Average loss: 0.519855, Accuracy: 400/411 (97%)\n",
+      "\n",
+      "Epoch: 26\n",
+      "Training set: Average loss: 0.542047\n",
+      "Validation set: Average loss: 0.517385, Accuracy: 398/411 (97%)\n",
+      "\n",
+      "Epoch: 27\n",
+      "Training set: Average loss: 0.540234\n",
+      "Validation set: Average loss: 0.515388, Accuracy: 400/411 (97%)\n",
+      "\n",
+      "Epoch: 28\n",
+      "Training set: Average loss: 0.538977\n",
+      "Validation set: Average loss: 0.512899, Accuracy: 401/411 (98%)\n",
+      "\n",
+      "Epoch: 29\n",
+      "Training set: Average loss: 0.537303\n",
+      "Validation set: Average loss: 0.512066, Accuracy: 404/411 (98%)\n",
+      "\n",
+      "Epoch: 30\n",
+      "Training set: Average loss: 0.536062\n",
+      "Validation set: Average loss: 0.511284, Accuracy: 404/411 (98%)\n",
+      "\n",
+      "Epoch: 31\n",
+      "Training set: Average loss: 0.534580\n",
+      "Validation set: Average loss: 0.508444, Accuracy: 404/411 (98%)\n",
+      "\n",
+      "Epoch: 32\n",
+      "Training set: Average loss: 0.533200\n",
+      "Validation set: Average loss: 0.507806, Accuracy: 404/411 (98%)\n",
+      "\n",
+      "Epoch: 33\n",
+      "Training set: Average loss: 0.532376\n",
+      "Validation set: Average loss: 0.505557, Accuracy: 404/411 (98%)\n",
+      "\n",
+      "Epoch: 34\n",
+      "Training set: Average loss: 0.531220\n",
+      "Validation set: Average loss: 0.503028, Accuracy: 404/411 (98%)\n",
+      "\n",
+      "Epoch: 35\n",
+      "Training set: Average loss: 0.529759\n",
+      "Validation set: Average loss: 0.502396, Accuracy: 404/411 (98%)\n",
+      "\n",
+      "Epoch: 36\n",
+      "Training set: Average loss: 0.528576\n",
+      "Validation set: Average loss: 0.501712, Accuracy: 404/411 (98%)\n",
+      "\n",
+      "Epoch: 37\n",
+      "Training set: Average loss: 0.527694\n",
+      "Validation set: Average loss: 0.499238, Accuracy: 404/411 (98%)\n",
+      "\n",
+      "Epoch: 38\n",
+      "Training set: Average loss: 0.526515\n",
+      "Validation set: Average loss: 0.498586, Accuracy: 404/411 (98%)\n",
+      "\n",
+      "Epoch: 39\n",
+      "Training set: Average loss: 0.525752\n",
+      "Validation set: Average loss: 0.496938, Accuracy: 404/411 (98%)\n",
+      "\n",
+      "Epoch: 40\n",
+      "Training set: Average loss: 0.524745\n",
+      "Validation set: Average loss: 0.496314, Accuracy: 405/411 (99%)\n",
+      "\n",
+      "Epoch: 41\n",
+      "Training set: Average loss: 0.524034\n",
+      "Validation set: Average loss: 0.494481, Accuracy: 404/411 (98%)\n",
+      "\n",
+      "Epoch: 42\n",
+      "Training set: Average loss: 0.523150\n",
+      "Validation set: Average loss: 0.492949, Accuracy: 404/411 (98%)\n",
+      "\n",
+      "Epoch: 43\n",
+      "Training set: Average loss: 0.522167\n",
+      "Validation set: Average loss: 0.492328, Accuracy: 404/411 (98%)\n",
+      "\n",
+      "Epoch: 44\n",
+      "Training set: Average loss: 0.521537\n",
+      "Validation set: Average loss: 0.490820, Accuracy: 401/411 (98%)\n",
+      "\n",
+      "Epoch: 45\n",
+      "Training set: Average loss: 0.521010\n",
+      "Validation set: Average loss: 0.489736, Accuracy: 401/411 (98%)\n",
+      "\n",
+      "Epoch: 46\n",
+      "Training set: Average loss: 0.520252\n",
+      "Validation set: Average loss: 0.489686, Accuracy: 404/411 (98%)\n",
+      "\n",
+      "Epoch: 47\n",
+      "Training set: Average loss: 0.519929\n",
+      "Validation set: Average loss: 0.488752, Accuracy: 401/411 (98%)\n",
+      "\n",
+      "Epoch: 48\n",
+      "Training set: Average loss: 0.519249\n",
+      "Validation set: Average loss: 0.488609, Accuracy: 405/411 (99%)\n",
+      "\n",
+      "Epoch: 49\n",
+      "Training set: Average loss: 0.518899\n",
+      "Validation set: Average loss: 0.487255, Accuracy: 401/411 (98%)\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Specify the loss criteria (we'll use CrossEntropyLoss for multi-class classification)\n",
+    "loss_criteria = nn.CrossEntropyLoss()\n",
+    "\n",
+    "def train(model, data_loader, optimizer):\n",
+    "    # Set the model to training mode\n",
+    "    model.train()\n",
+    "    train_loss = 0\n",
+    "    \n",
+    "    for batch, tensor in enumerate(data_loader):\n",
+    "        data, target = tensor\n",
+    "        #feedforward\n",
+    "        optimizer.zero_grad()\n",
+    "        out = model(data)\n",
+    "        loss = loss_criteria(out, target)\n",
+    "        train_loss += loss.item()\n",
+    "\n",
+    "        # backpropagate\n",
+    "        loss.backward()\n",
+    "        optimizer.step()\n",
+    "\n",
+    "    #Return average loss\n",
+    "    avg_loss = train_loss / (batch+1)\n",
+    "    print('Training set: Average loss: {:.6f}'.format(avg_loss))\n",
+    "    return avg_loss\n",
+    "\n",
+    "def test(model, data_loader):\n",
+    "    # Switch the model to evaluation mode (so we don't backpropagate)\n",
+    "    model.eval()\n",
+    "    test_loss = 0\n",
+    "    correct = 0\n",
+    "\n",
+    "    with torch.no_grad():\n",
+    "        batch_count = 0\n",
+    "        for batch, tensor in enumerate(data_loader):\n",
+    "            batch_count += 1\n",
+    "            data, target = tensor\n",
+    "            # Get the predictions\n",
+    "            out = model(data)\n",
+    "\n",
+    "            # calculate the loss\n",
+    "            test_loss += loss_criteria(out, target).item()\n",
+    "\n",
+    "            # Calculate the accuracy\n",
+    "            _, predicted = torch.max(out.data, 1)\n",
+    "            correct += torch.sum(target==predicted).item()\n",
+    "            \n",
+    "    # Calculate the average loss and total accuracy for this epoch\n",
+    "    avg_loss = test_loss/batch_count\n",
+    "    print('Validation set: Average loss: {:.6f}, Accuracy: {}/{} ({:.0f}%)\\n'.format(\n",
+    "        avg_loss, correct, len(data_loader.dataset),\n",
+    "        100. * correct / len(data_loader.dataset)))\n",
+    "    \n",
+    "    # return average loss for the epoch\n",
+    "    return avg_loss\n",
+    "\n",
+    "# Use an \"Adam\" optimizer to adjust weights\n",
+    "# (see https://pytorch.org/docs/stable/optim.html#algorithms for details of supported algorithms)\n",
+    "learning_rate = 0.001\n",
+    "optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)\n",
+    "optimizer.zero_grad()\n",
+    "\n",
+    "# We'll track metrics for each epoch in these arrays\n",
+    "epoch_nums = []\n",
+    "training_loss = []\n",
+    "validation_loss = []\n",
+    "\n",
+    "# Train over 50 epochs\n",
+    "epochs = 50\n",
+    "# for epoch in range(1, epochs + 1):\n",
+    "for epoch in range(epochs):\n",
+    "\n",
+    "    # print the epoch number\n",
+    "    print('Epoch: {}'.format(epoch))\n",
+    "    \n",
+    "    # Feed training data into the model to optimize the weights\n",
+    "    train_loss = train(model, train_loader, optimizer)\n",
+    "    \n",
+    "    # Feed the test data into the model to check its performance\n",
+    "    test_loss = test(model, test_loader)\n",
+    "    \n",
+    "    # Log the metrics for this epoch\n",
+    "    epoch_nums.append(epoch)\n",
+    "    training_loss.append(train_loss)\n",
+    "    validation_loss.append(test_loss)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 34,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAAAyQElEQVR4nO3deXxcdb3/8ddnlmSy72mztEn3vU3TtFQKtKUFCgJlE8qigGgF9aJeRcCrInq9l/u7XC7wkMWCICoXrOxqKVgsWynQdKX7vqRpm31fZ+b7++NM1iZpkmYySebzfDzmcWbOnHPyPWjzznc5368YY1BKKRW8bIEugFJKqcDSIFBKqSCnQaCUUkFOg0AppYKcBoFSSgU5R6AL0FOJiYkmMzMz0MVQSqlBZePGjUXGmKSOvht0QZCZmUlubm6gi6GUUoOKiBzp7DttGlJKqSCnQaCUUkFOg0AppYLcoOsjUEoNLY2NjeTl5VFXVxfoogwJLpeL9PR0nE5nt8/RIFBKBVReXh5RUVFkZmYiIoEuzqBmjKG4uJi8vDxGjRrV7fO0aUgpFVB1dXUkJCRoCPQBESEhIaHHtSsNAqVUwGkI9J3e/LcMmiDYc7KS/1y1i6p6d6CLopRSA0rQBMGxkhp+++FB9pysCHRRlFIDSFlZGU8++WSPz7vssssoKyvr8pif//znrFmzppcl6z9BEwSTUqMB2HmiMsAlUUoNJJ0Fgcfj6fK8VatWERsb2+Uxv/zlL1m8ePHZFK9fBE0QpMa4iHY52H1CawRKqRb33XcfBw4cICsri9mzZ7Nw4UJuuukmpk2bBsBVV13FrFmzmDJlCitWrGg+LzMzk6KiIg4fPsykSZP45je/yZQpU7j44oupra0F4LbbbuOVV15pPv6BBx4gOzubadOmsXv3bgAKCwu56KKLyM7O5lvf+hYZGRkUFRX1638Dvw0fFZHngMuBAmPM1A6+nwg8D2QD/2aMedhfZfH9PCalRLNLg0CpAevBv+5gZ37f/hudnBrNA1dM6fT7hx56iO3bt7Nlyxbef/99vvzlL7N9+/bm4ZfPPfcc8fHx1NbWMnv2bK699loSEhLaXGPfvn289NJLPPPMM1x//fW8+uqr3HLLLaf9rMTERDZt2sSTTz7Jww8/zLPPPsuDDz7IhRdeyP3338/q1avbhE1/8WeN4PfAki6+LwHuBvwaAK1NSolm98lKvF5dp1kp1bE5c+a0GYP/+OOPM2PGDObOncuxY8fYt2/faeeMGjWKrKwsAGbNmsXhw4c7vPY111xz2jEff/wxy5YtA2DJkiXExcX13c10k99qBMaYD0Uks4vvC4ACEfmyv8rQ3qSUKGoaPBwtqSEzMaK/fqxSqpu6+su9v0REtPxueP/991mzZg3r168nPDycBQsWdDhGPzQ0tPm93W5vbhrq7Di73Y7bbY1gNCbwf5gOij4CEVkuIrkikltYWNjr60xKsTqMtXlIKdUkKiqKysqOB5GUl5cTFxdHeHg4u3fv5tNPP+3zn3/eeeexcuVKAN59911KS0v7/GecyaAIAmPMCmNMjjEmJympw3UVumX8sChsokGglGqRkJDAvHnzmDp1Kvfcc0+b75YsWYLb7Wb69On87Gc/Y+7cuX3+8x944AHeffddsrOzefvtt0lJSSEqKqrPf05XxJ/VEl/T0N866ixudcwvgKrudhbn5OSYs1mYZvEjHzAqMYJnvpbT62sopfrOrl27mDRpUqCLETD19fXY7XYcDgfr16/nrrvuYsuWLWd1zY7+m4rIRmNMh7/4gm7SuYnDo9hyrCzQxVBKKQCOHj3K9ddfj9frJSQkhGeeeabfy+DP4aMvAQuARBHJAx4AnADGmKdFZDiQC0QDXhH5PjDZGOPXdptJKdH8bdsJKuoaiXZ1f5pWpZTyh3HjxrF58+aAlsGfo4ZuPMP3J4F0f/38zkz2dRjvPlHJnFHx/f3jlVJqwBkUncV9SUcOKaVUW0EXBMOiQ4kLd7JbJ59TSikgCINARJg4PFonn1NKKZ+gCwKwmof2nKzAo1NNKKV6KDIyEoD8/Hyuu+66Do9ZsGABZxrm/uijj1JTU9P8uTvTWvtLkAZBFHWNXg4XVwe6KEqpQSo1NbV5ZtHeaB8E3ZnW2l+CNAi0w1gpZbn33nvbrEfwi1/8ggcffJBFixY1Txn95ptvnnbe4cOHmTrVela2traWZcuWMX36dG644YY2cw3ddddd5OTkMGXKFB544AHAmsguPz+fhQsXsnDhQqBlWmuARx55hKlTpzJ16lQeffTR5p/X2XTXZyvoHigDGJscid0m7DpRweXTUwNdHKVUk7fvg5Nf9O01h0+DSx/q9Otly5bx/e9/n29/+9sArFy5ktWrV/ODH/yA6OhoioqKmDt3LldeeWWn6wE/9dRThIeHs23bNrZt20Z2dnbzd7/+9a+Jj4/H4/GwaNEitm3bxt13380jjzzC2rVrSUxMbHOtjRs38vzzz/PZZ59hjOGcc85h/vz5xMXFdXu6654KyhqBy2lnTFIEu7XDWKmgN3PmTAoKCsjPz2fr1q3ExcWRkpLCT37yE6ZPn87ixYs5fvw4p06d6vQaH374YfMv5OnTpzN9+vTm71auXEl2djYzZ85kx44d7Ny5s8vyfPzxx1x99dVEREQQGRnJNddcw0cffQR0f7rrngrKGgFYzUMbDpUEuhhKqda6+Mvdn6677jpeeeUVTp48ybJly3jxxRcpLCxk48aNOJ1OMjMzO5x+urWOaguHDh3i4YcfZsOGDcTFxXHbbbed8Tpdzf/W3emueyooawRgBUF+eR1lNQ2BLopSKsCWLVvGyy+/zCuvvMJ1111HeXk5ycnJOJ1O1q5dy5EjR7o8/4ILLuDFF18EYPv27Wzbtg2AiooKIiIiiImJ4dSpU7z99tvN53Q2/fUFF1zAG2+8QU1NDdXV1bz++uucf/75fXi3pwvqGgHArhOVfGlMwhmOVkoNZVOmTKGyspK0tDRSUlK4+eabueKKK8jJySErK4uJEyd2ef5dd93F7bffzvTp08nKymLOnDkAzJgxg5kzZzJlyhRGjx7NvHnzms9Zvnw5l156KSkpKaxdu7Z5f3Z2NrfddlvzNb7xjW8wc+bMPmsG6ohfp6H2h7OdhrpJQUUdc/7jPX5++WS+ft6oM5+glPKLYJ+G2h96Og110DYNJUWFkhARolNNKKWCXtAGgYgwKSWaXTpySCkV5II2CMB6wnjPqUrcHm+gi6JUUBtsTdQDWW/+WwZ5EETT4PZyqEinmlAqUFwuF8XFxRoGfcAYQ3FxMS6Xq0fnBe2oIYCJw62RQztPVDBuWP8uFq2UsqSnp5OXl0dhYWGgizIkuFwu0tN7tuZX8ATBkU/gvV/BTX8GlxUAY5MjcdqFXScqWZoV2OIpFaycTiejRunIvUAKnqYhZxgc/QQ+X9G8K8RhY0xSpI4cUkoFteAJgtSZMO4SWP8bqG8ZKTQ5JVpnIVVKBbXgCQKA+fdCbSl8/kzzrokpUZyqqKekWqeaUEoFp+AKgvRZMHaxr1ZQBejaBEop5bcgEJHnRKRARLZ38r2IyOMisl9EtolIdkfH9bn590FNMeT+DoBpaTGE2G28u+Nkv/x4pZQaaPxZI/g9sKSL7y8Fxvley4Gn/FiWFiNmw5gLYd3j0FBNbHgIV8xI5S8b8yivbeyXIiil1EDityAwxnwIdDXh/1LgD8byKRArIin+Kk8b8++FmiLIfR6AO84bRU2Dh5c/P9ovP14ppQaSQPYRpAHHWn3O8+07jYgsF5FcEcntk4dORs6FUfNh3WPQUMPk1Gi+NDqBFz45rNNNKKWCTiCDoKPFPzt8xtwYs8IYk2OMyUlKSuqbnz7/XqgugE0vAFatIL+8jre3a1+BUiq4BDII8oARrT6nA/n99tMz50Hm+fDxo9BYx4UTkxmVGMHvPj7Ub0VQSqmBIJBB8BbwNd/ooblAuTHmRL+WYP69UHUSNr2AzSbcPi+TLcfK2HiktF+LoZRSgeTP4aMvAeuBCSKSJyJ3iMidInKn75BVwEFgP/AM8G1/laVTmefByHPh4/+FxjquzU4n2uXgOa0VKKWCiN8mnTPG3HiG7w3wHX/9/G4RgQX3wh+WwraXiZh1GzfOGckzHx0kr7SG9LjwgBZPKaX6Q3A9WdyRUfMheTLkPgfAredmIiK88MnhwJZLKaX6iQaBCMy6HU5sheObSI0N49Kpw3n582NU1bsDXTqllPI7DQKAGTeAMxw2tjxgVlnv5i+5x85wolJKDX4aBACuGJh6DXzxKtRVMHNkHNkjY3l+3WE8Xl0+Tyk1tGkQNJn1dWishi/+AsAd543maEkNa3adCnDBlFLKvzQImqRlw/BpVvOQMVwyZRipMS5e/EznH1JKDW0aBE2aOo1PfgHHN+Gw27gmO52P9xVyqqIu0KVTSim/0SBobdpXwBkBG62hpNdkp+E18OaW4wEumFJK+Y8GQWuuaJh2HWx/DerKGZ0UycyRsby68TjW829KKTX0aBC0l3M7NNbAtpUAXJudzp5TlezI16UslVJDkwZBe6kzISXLWrTGGC6fnkKI3carm/ICXTKllPILDYKO5NwOBTsgbwOx4SEsmpTMW1vyadRFa5RSQ5AGQUemXgchUc1LWV6bnU5xdQMf7OmD1dGUUmqA0SDoSGgkTP8K7HgNakuZPyGJhIgQXtuszUNKqaFHg6Azs24Hdx1s/TNOu40rs1JZs7OAspqGQJdMKaX6lAZBZ1KmW08a73wTsJqHGjxe/ratfxdRU0opf9Mg6Mr4S+HYp1BTwpTUaCYMi9LRQ0qpIUeDoCvjl4Dxwv73EBGuyU5j89EyDhZWBbpkSinVZzQIupI6EyKSYO9qAK6amYZN4LVNOuWEUmro0CDois0G4y6B/f8Aj5th0S7OG5fE65uP49V1CpRSQ4QGwZmMvwTqyuHYZwBcm53G8bJaPj1UHOCCKaVU39AgOJMxC8HmbG4eunjycCJDHby6UZuHlFJDgwbBmYRGQeY82PsOAGEhdhZNSubj/fqUsVJqaPBrEIjIEhHZIyL7ReS+Dr6PE5HXRWSbiHwuIlP9WZ5eG78EivZAyUEAZo6I5VRFPSfLdcEapdTg57cgEBE78ARwKTAZuFFEJrc77CfAFmPMdOBrwGP+Ks9ZGX+Jtd37LgBZI+MA2HKsNFAlUkqpPuPPGsEcYL8x5qAxpgF4GVja7pjJwHsAxpjdQKaIDPNjmXonfjQkjm/uJ5iUEoXTLmw5Vh7ggiml1NnzZxCkAcdafc7z7WttK3ANgIjMATKA9PYXEpHlIpIrIrmFhQFqmx9/CRz+GOorCXXYmZwSzdZjZYEpi1JK9SF/BoF0sK/94PuHgDgR2QL8C7AZcJ92kjErjDE5xpicpKSkPi9ot4xfAt5GOLAWgBkjYvnieDkefZ5AKTXI+TMI8oARrT6nA/mtDzDGVBhjbjfGZGH1ESQBh/xYpt4bcQ64YppHD81Ij6Wq3q3TTSilBj1/BsEGYJyIjBKREGAZ8FbrA0Qk1vcdwDeAD40xA3NxYLsTxi6Gfe+A18uMEbEAbNHmIaXUIOe3IDDGuIHvAu8Au4CVxpgdInKniNzpO2wSsENEdmONLvqev8rTJ8YvgepCyN/M6MQIolwODQKl1KDn8OfFjTGrgFXt9j3d6v16YJw/y9Cnxi4GscHe1djSZzEjPZateWWBLpVSSp0VfbK4J8Ljrb4C3zDSGSNi2H2ikrpGT4ALppRSvadB0FPjL4GT26Ainxnpsbi9hh35A7NbQymlukODoKfGL7G2e98hy9dhrM8TKKUGMw2CnkqaCLEjYe87JEe7SIlxaT+BUmpQ0yDoKRHImAcntgDW8wQ6ckgpNZhpEPRG4nioPAF1FcwYEcuR4hpKqxsCXSqllOoVDYLeSJpgbYv2tfQTaPOQUmqQ0iDojcSmINjDtPQYRGCrzkSqlBqkNAh6Iy7TWr6yaC+RoQ7GJUdqjUApNWhpEPSG3QEJY6BwL9DSYWyMzkSqlBp8NAh6K3EcFPmCYEQsJdUN5JXWBrhQSinVcxoEvZU4wVrD2N3Q3GGsw0iVUoORBkFvJY4H44GSg0wYHkWow6ZPGCulBiUNgt5KGm9ti/bitNuYmhajHcZKqUFJg6C3EnyzZxftAawO4y+Ol+P2eANYKKWU6jkNgt4KjYTodCjaB1hTUtc1etlzqjLABVNKqZ7RIDgbSeOh0KoRtMxEqg+WKaUGFw2Cs5E43qoReL2MjA8nNtypHcZKqUGnW0EgIt8TkWix/E5ENonIxf4u3ICXOB4aq6EyHxHRpSuVUoNSd2sEXzfGVAAXA0nA7cBDfivVYNE0+Vyr5qG9pyqpqncHsFBKKdUz3Q0C8W0vA543xmxttS94JTYNIbU6jHMy4/Aa2HSkNICFUkqpnuluEGwUkXexguAdEYkCdJxkRBK4YpuHkM4cGYdNIPdwSWDLpZRSPdDdILgDuA+YbYypAZxYzUNdEpElIrJHRPaLyH0dfB8jIn8Vka0iskNEznjNAUXEqhX4Jp+LDHUwOTWaDYe1RqCUGjy6GwRfAvYYY8pE5Bbgp0CX4yRFxA48AVwKTAZuFJHJ7Q77DrDTGDMDWAD8j4iE9KD8gZc0vnnyOYCcjHg2HyulUR8sU0oNEt0NgqeAGhGZAfwYOAL84QznzAH2G2MOGmMagJeBpe2OMUCUiAgQCZQAg6unNXECVBdArVULmDMqnrpGLzvyKwJcMKWU6p7uBoHbWJPtLwUeM8Y8BkSd4Zw04Firz3m+fa39BpgE5ANfAN8zxpz2p7SILBeRXBHJLSws7GaR+0n7DuOMOAA2HNJ+AqXU4NDdIKgUkfuBrwJ/9zX7OM9wTkejitqv3HIJsAVIBbKA34hI9GknGbPCGJNjjMlJSkrqZpH7SdPkc74hpMnRLjISwtmgHcZKqUGiu0FwA1CP9TzBSay/7P/7DOfkASNafU7H+su/tduB14xlP3AImNjNMg0MsRlgD20eOQRWP0HukVJdsUwpNSh0Kwh8v/xfBGJE5HKgzhhzpj6CDcA4ERnl6wBeBrzV7pijwCIAERkGTAAO9qD8gWezQ8LY5qYhgNmZcZRUN3CwqDqABVNKqe7p7hQT1wOfA18Brgc+E5HrujrHGOMGvgu8A+wCVhpjdojInSJyp++wXwHnisgXwHvAvcaYot7dSgAljmtuGgKYPSoe0H4CpdTg4Ojmcf+G9QxBAYCIJAFrgFe6OskYswpY1W7f063e52NNWzG4JU2AXW9BYx04XYxOjCA+IoQNh0tZNmdkoEunlFJd6m4fga0pBHyKe3Du0Jc4HowXSg4AICLkZMSRe0RrBEqpga+7v8xXi8g7InKbiNwG/J12f+kHtcS2I4cAZmfGc6S4hoKKugAVSimluqe7ncX3ACuA6cAMYIUx5l5/FmxQSRgLSJsO45xM63mCXJ2ATik1wHW3jwBjzKvAq34sy+AVEg6xI9oMIZ2aFoPLaePzQyVcNi0lgIVTSqmudRkEIlLJ6Q+BgfWwmDHGnPbwV9BKnNBmziGn3cbMEdpPoJQa+LpsGjLGRBljojt4RWkItJM0AYr2g7dlhozZmXHszK/QhWqUUgOajvzpK4njwF0L5Uebd+VkxuM1sPmo9hMopQYuDYK+kuhbtrJVh3F2hrVQja5PoJQayDQI+koHQ0ibF6rRJ4yVUgOYBkFfiUiA8IQ2HcagC9UopQY+DYK+1G7kEFgPlulCNUqpgUyDoC81TT7Xavrp2U0Plun6BEqpAUqDoC+lzYLaEsjf1LxLF6pRSg10GgR9acrV4AyHjS+02Z2TEU/uYV2oRik1MGkQ9CVXtBUG21+F+qrm3bMz4yiubmDPqcoAFk4ppTqmQdDXsm+FhirY8VrzrkWThhHmtPPk2gMBLJhSSnVMg6CvjZgDSRPbNA8lRYVyx3mjeGtrPtuPlwewcEopdToNgr4mAtlfg+O5cGpH8+7l80cTF+7kv1bvDmDhlFLqdBoE/jB9GdhDYNMfmndFu5x8Z+FYPtpXxCf7B9+yzEqpoUuDwB8iEmDSFbD1ZWsdY59b5maQGuPiodW7dQSRUmrA0CDwl+yvQV0Z7Ppr8y6X084PLhrPtrxyVn1xMnBlU0qpVjQI/CXzAojLhE1tnym4Jjud8cMiefjdPTr/kFJqQPBrEIjIEhHZIyL7ReS+Dr6/R0S2+F7bRcQjIvH+LFO/sdmsWsHhj6C4Zdio3Sb8+JKJHCqqZmXusQAWUCmlLH4LAhGxA08AlwKTgRtFZHLrY4wx/22MyTLGZAH3Ax8YY4bOXAxZN4PY23QaAyyalExORhyPrdlHbYMnQIVTSimLP2sEc4D9xpiDxpgG4GVgaRfH3wi85Mfy9L+o4TB+CWz5P/A0Nu8WEe67dCIFlfU8t+5QAAuolFL+DYI0oHXbR55v32lEJBxYArzayffLRSRXRHILCwv7vKB+NetWqC6Avavb7M7JjGfxpGSefv8ApdUNASqcUkr5Nwikg32djZm8AljXWbOQMWaFMSbHGJOTlJTUZwXsF2MWQVTqaRPRAdxzyUSqG9z89I3tOpxUKRUw/gyCPGBEq8/pQH4nxy5jqDULNbE7YOYtsH8NlBxs89WE4VH8eMlE/v7FCZ76QOchUkoFhj+DYAMwTkRGiUgI1i/7t9ofJCIxwHzgTT+WJbByboeQSPjbD9osWgPwrQtGc+WMVP77nT2s3V0QoAIqpYKZ34LAGOMGvgu8A+wCVhpjdojInSJyZ6tDrwbeNcZU+6ssARedChc9CAffh81/bPOViPBf105ncko0d7+8mYOFVR1fQyml/EQGW9t0Tk6Oyc3NDXQxes7rhT9cCSe2wnc+s8KhlbzSGq78zTriwp288Z15RLmcASqoUmooEpGNxpicjr7TJ4v7i80GVz5uDSPtoIkoPS6cJ27K5nBxDT/48xa83sEV0EqpwUuDoD/Fj4ZFP7OGkn7xl9O+/tKYBH5++WTW7Crg0TV7A1BApVQw0iDob+fcCemz4e0fQ9XpncNf+1IG1+ek8/g/9/O3bZ0NslJKqb6jQdDfbHZY+gQ0VMOqe077WkT41VVTycmI43svb+GNzccDUEilVDDRIAiEpAmw4D7Y+QbsPH3UbKjDzu+/PodzRsXzg5Vb+OP6w/1eRKVU8NAgCJRz74bh0+HvP4Sa0x+ojgx18Nxts1k0MZmfvbmDJ9/fH4BCKqWCgQZBoNidVhNRbSn8+RZorD3tEJfTzlO3zGJpVir/b/UeHnpbVzZTSvU9DYJASpkOV/8WjnwCK29tM0NpE6fdxv9en8XN54zk6Q8O8NM3tuvQUqVUn3IEugBBb9p1UFcOf/9XeOMuuHqF9cxBKzab8O9XTSXK5eTpDw5QVtvI/7t2OhGh+j+fUurs6W+SgWD2Hdb6xu/9EkKj4cv/A9J28tamNQziwp08tHo3e05W8uTN2YwfFhWYMiulhgxtGhoozvtXqwM593fwz191eti35o/hT3ecQ1lNA0t/s45XNub1YyGVUkORBsFAIQIX/RKyb4WP/gfWPd7pofPGJrLq7vOZMSKGH/1lKz9+ZasueamU6jUNgoFEBC7/X5hyNfzjZ/DZb0+bk6hJcrSLP91xDv9y4Vj+sjGPq59cxwGduVQp1QsaBAONzW51GI+/1JqG4q93g7u+w0Mddhs/vHgCv799DgWV9Vz++Mc8+9FB3B5vPxdaKTWYaRAMRI4QWPYinP9D2PQHeP4yKO98qon545NYdff5nDsmgX//+y6u/M06th4r67/yKqUGNQ2Cgcpmh0U/h+v/CIW7YcV8OLyu08OHx7h49tYcnro5m+Lqeq56ch2/eGsHlXWnP5uglFKtaRAMdJOvhG+8B64Ya2GbLvoNRIRLp6Ww5l/n87W5Gbyw/jCLH/mAt784oU8kK6U6pUEwGCRPhG/+E8ZeZPUbvLbcegitE1EuJw8uncrr355HfEQod724iZuf/Yztxzs/RykVvHSpysHE64WPHob3H7KWurzqKRh1fpenuD1eXvzsKI+u2UtZbSNXz0zjRxdPIDU2rJ8KrZQaCLpaqlKDYDDKy7VqBSUH4EvfhQt/Bk5Xl6dU1DXy5NoDPLfuEAJ84/xR3Dl/jK6NrFSQ0CAYihqq4R8/hw3PQtJEuGYFpMw442l5pTU8/M4e3tiST0JECHfOH8ON54wkUuctUmpI0yAYyvavgTe/C9WFMP8+OO/71hTXZ7Atr4z/Wr2bdfuLiXI5+OrcDG6fN4qkqFD/l1kp1e8CFgQisgR4DLADzxpjHurgmAXAo4ATKDLGzO/qmhoEHagpgVU/gu2vwrCpcMXjkD6rW6duPVbGbz88wNvbT+K027g2O53lF4xmVGKEnwutlOpPAQkCEbEDe4GLgDxgA3CjMWZnq2NigU+AJcaYoyKSbIw5fUX3VjQIurDrb9Y6yJUn4Jw74cJ/g9DuzU56qKiaZz46yCsb82j0eFk8aRhfnZvBeWMTsdnkzBdQSg1ogQqCLwG/MMZc4vt8P4Ax5j9bHfNtINUY89PuXleD4AzqKqzprDc8C9Fp1pTWE5Z0+/SCyjpe+OQwL31+jJLqBjISwrn5nJF8ZdYI4iJC/FhwpZQ/dRUE/nyOIA041upznm9fa+OBOBF5X0Q2isjX/Fie4OCKhi8/DHe8a9UGXrrBWv2s5FC3Tk+OcnHPJRNZf/+FPLYsi+SoUP5j1W7O+c/3+Nc/byH3cIk+nKbUEOPPoSIdtSe0/w3iAGYBi4AwYL2IfGqM2dvmQiLLgeUAI0eO9ENRh6ARc+BbH8Inj8OH/w27/gozboQLfgjxo894eqjDztKsNJZmpbH7ZAUvfnqU1zcf57XNxxkZH85VWalcNTON0UmR/XAzSil/CnTT0H2AyxjzC9/n3wGrjTF/6ey62jTUCxUnYN1jsPF5a13k6TfABT+ChDE9ukxVvZvV20/yxubjrDtQhDEwIz2Gq2emcfmMVBIjdcSRUgNVoPoIHFidxYuA41idxTcZY3a0OmYS8BvgEiAE+BxYZozZ3tl1NQjOQuVJa8Gb3OfAUw/TvgLn/wiSxvf4UifL6/jr1nxe33ycnScqsNuE88YmsjQrlYunDNfnEpQaYAI5fPQyrKGhduA5Y8yvReROAGPM075j7gFuB7xYQ0wf7eqaGgR9oKrAqiHkPgeNtdbEduf/sFsPpHVkz8lK3thynLe25HO8rJZQh43Fk4exdEYq8yckEeqw9/ENKKV6Sh8oUx2rLoJPn4TPn4H6Chh3sRUII+f26nLGGDYdLeXNLfn8bdsJSqobiHY5WDx5GJdOTeH8cYm4nBoKSgWCBoHqWl25FQafPgk1xZBxHpz3Axi7yFo+sxcaPV7W7S/ira35rNl5ioo6NxEhdhZMTObSqcNZMCFZm4+U6kcaBKp7GqqtFdHWPQ6V+ZA4AebeBTOWgbP3s5U2uL2sP1jM6u0n+cfOkxRVNRDisDFvTAILJyazcEIyI+LD+/BGlFLtaRConnE3wI7XYP0TcHIbhMVDztdhzjchavhZXdrjNeQeLuHt7SdZu6eAI8U1AIxJiuBCXyjkZMYT4tClMpTqSxoEqneMgSPrYP2TsGcV2Bww5WqYdStkzOt1s1HL5Q2HiqpZu6eQ9/cU8NnBEho8XsKcdmaPiufcMQmcOyaBKakx2HWaC6XOigaBOnvFB6xlMre+ZHUsx4+GrJsh6yZrkZw+UF3vZt3+ItbtL+KTA8XsK6gCIMrlYO7oBOaOTiAnI47JqdE47VpjUKonNAhU32mogV1vwaY/wpGPQWzWEpozb4GxiyGk79r6CyrrWH+gmPUHivnkQDFHS6xmJJfTxoz0WGZlxDErI47skXE6D5JSZ6BBoPyj+ABs/hNs+T+oOgmOMBizECZcBuOXQGRSn/64/LJaNh0tZeORUjYdKWVHfgVur/X/3/S4MKalxTA1LYZpvpeGg1ItNAiUf3nccPgjqx9h9yqoyAPEmu9owmUw6YoeT2fRHbUNHr44Xs6mo6V8cbyc7cfLmzufAdJiw5icGs2klGgmDY9iYko0GfHhOq22CkoaBKr/GGONNNrzNuz+u/UerAVzJl0Bk66E5Eln3dHcmfKaRrbnlzcHw64TFRwqqsZXcSDMaWfC8CjGD4tkbHIkY5KsbXpcuHZIqyFNg0AFTtlRa8GcXW/B0U8BAwljrUAYfwmkzerW0ppno67Rw75TVew6WcGuExXsPlHJvoIqiqrqm48JcdgYnRjBmKRIMhLCyUyIsLaJESRHhSJ+Ci6l+osGgRoYKk/Bbl8oHPoIjAdComDU+TB6odW/kDDWb7WF9sprGtlfWMn+gioOFFazv6CKQ0XVHCupae57AKsWkZEQTnpcOOlxYaTHhTEivul9ONEuhwaFGvA0CNTAU1MChz6Eg2vhwFooO2Ltj06HzHmQmg2pM2H4tD4didQdbo+X/LI6DhVXc6S4msNFNRwtqSavtJZjJTVUN3jaHB/qsJEYGUpSVKhvG0JSZCjxESHER4YSHx5CfEQICZEhxIWH6MNyKiA0CNTAV3LQCoSDa+HY51B1ytovdqtPITULUrIgebL1OTw+IMU0xlBW00heaS15pTUcL6ulsLLeelVZ26KqBkqq6/F28k8ryuVoCY3mAAkhPiKUmDAn0WEOol1O33sn0S4HDn1uQp0lDQI1+FScgPxNkL/Zeh3fBLUlLd9HJFuBkDzJCof0HEiaCLaBMbupx2soq2mgpLrlVdy0rbLCwgoNK0Aq69xdXi8mzElCZAgJESEkRIQS73vfEhYtARLtchIT7tQmK9WGBoEa/IyBinwo3AUFu6FgV8v7xmrrmNBoq/N5xDkwYjak5UBYbECL3V11jR5KaxqoqHVTUddIRW0j5bXWtqy2sTlIiqvqKa7yhUtNA13987XbhLhwJ7HhIadtY8KcxIY7iQ0Laa6FhIfYcTnthIc4CHPacTltGiRDiAaBGrq8Xig9BHm5cOwzq1mpYAcYr/V9zEhIHNfqNd56RSSDbXA3t3i8hqp6N5V1jacFSHltI6U1DZTWNFJa3UBpTQNlNY3Wq7aBukZvt35GeIidyFAHUS4HUS4nUS6r1hHlchAT7iTOFy4xYdY2zldLiXJZYaJBMnB0FQQ6Ibwa3Gw262G1hDEw4wZrX30lHN8IeRugcC8U7YVNn7bUHAAQcEVDaAy4Yqz3rhiITrNqFCPPgZgR/TaCqTfsNiEmzOpLIK5n59Y1epprG2U1VoDUNnqobfBQ2+ihpsFDbYOb6gYPVXVuKusbqaxzU1nnJr+sloo6N+U1jTR4Og8Uh03aBEhkqIOIUAdhIXbCnfbm9y1B4yAq1NnmnLAQu692YtfnPPxIawQqODQ1LRXthaJ9UF1oLchTX2Ft63zbkoMtgRGVagXCiLlWH0T8aAiLG9Dh0J+MMdQ0eJprG001kIraptBo2VbUuamqd1Pb4KGmwU1Ng8f3ctPo6d7voBCHjTCnFQzhIXYiXQ4iQqxwiQy1gqWpecvltBPqsDW/Dw+xnxZAESFNW3tQdMZrjUApEYhJs15jFnZ+nMdtNS0d/QyOfWptd7ze8n1oNMRlQFwmxGZYr7A4X+0ium3tIiRySIeGiBDh+ys/vYc1ktbq3Z7m2kZTM1dlXSOV9W7qWtVSahs91Dd6qfHVVKrr3VTXuzleVtv8vqbBQ53b02XfSUdCHbbmGkt4iBUWLqeNUIe9eRvqsBHqsBHS9LLbcTqEELu1P9TZUnsJc9oJC7H5AsmOwybYbYLDLjhsNhy+96EOO067BLwJTWsESp1JeR6c2AqlR6D0cNuXp77z80KiIHbk6a/oNGuBn8hkvz9VHYyMMTR4vNQ1eqlr9Fhh0tzcZQVI02crQKyaSZUvTJpCpt7tpd5thU9d07bRQ6PHur6ns/HBPWQTmgOnqSbjsFth4bTbsNsEp90KkqVZadw4Z2Svfo7WCJQ6GzHp1qs9r7eDJqZW7yvyrSk2yo5aC/zUV7S7gEBEohUKUSnWug6xI301DV9oDIFO7f4mIr6/4O1W/4mfeLyGBreXBo+XBl9oWDUYb3MNpimIvMbQ6DF4vAa31+DxeGn0GN853uZtXaOHercXt9eL22Md6/Ya3B6vdV4fhU97GgRK9ZbNBlHDrFd31JZZT1BXnIDKE9ZDc5UnoPKktT2+CWqK2p5jD4XoFKuZyRluPWXtjLDWkG793hnu2/reR6dA/Bir9qFB4hd2m1id2QyMZ1fOhgaBUv0lLNZ6pczo/JiGaqspquyo1fRUdtQKiYYaqxO7sRaqi633DTXW58Ya8DZ2fD17KMSPskIhfhREDgOHC5wua+sItbah0VaNJDpVm6uCkF+DQESWAI8BduBZY8xD7b5fALwJHPLtes0Y80t/lkmpAS0kApImWK+e8DT6QqEWGqqsMCk5YC0eVHLIer9/Tdd9GgCIFRYxaVZtImaE1UQVl9HSbBUa2evbUwOT34JAROzAE8BFQB6wQUTeMsbsbHfoR8aYy/1VDqWCgt1pvVzRwDDruYrR89se4/VaNQl3vRUY7npw11nbulIoPw4Vx33bPOvp7f1rrBpHa2HxEDvCqkW0rlW0qWl08Dk8vqXvwxXTb/9p1Jn5s0YwB9hvjDkIICIvA0uB9kGglOoPNhuERlmv7jIGqot8nd6HfU1WR6D8mNU0VVPcNlDctS1BYzydX9cV01LDiBruC40wX6j4ts5wKzwikiA8wepYH+JDcgPFn0GQBhxr9TkPOKeD474kIluBfOBHxpgd7Q8QkeXAcoCRI3s3dEop1Qsi1trTkUmQPqtn5zY1V7nrrG1NccsoqqZX8QE48klLiJgzTH1hD/UFQkTbmkjTe1esr7zDrOG5EcnW+4hE6zvtOO+QP4Ogo9huP/ZpE5BhjKkSkcuAN4Bxp51kzApgBVjPEfRxOZVS/tDUXEW09TkuA9KyOz/eGPC6W5qtGqut8Kgutobp1hRZtZOaYqu5qnVNpLYUGuugrswajeXtYDZXsVnNWuEJvle81XnfVANpDhZfrSQsDsLjrGObzguJGJI1En8GQR4wotXndKy/+psZYypavV8lIk+KSKIxpt0YOqXUkCfSKjwAkqwnuHvK620JhKoC61VdYC2GVFtiBUlNiTWdSG2Z1YHeWNfNGkmI1azV1MQWGt3y3h7Sch/WG+u9zWkFSEikb+t7OcOtcGp++Y4Xu+/p9FgrjEKj/V6T8WcQbADGicgo4DiwDLip9QEiMhw4ZYwxIjIHsAHFfiyTUmqos9msv/bD4631KnrC425pyqotbRscNcXW57oKa2LDplfZMagvt85tavQwpuW9p8HqTznjiK1OiM03dUkszL4Dzv2X3l2nC34LAmOMW0S+C7yDNXz0OWPMDhG50/f908B1wF0i4gZqgWVmsM15oZQaOuwOsEdaQ2Qjk/r22p5G6zmRpldjtVUDMfi2vpfXbQVMbalVs6kttWoutaVWf4cf6FxDSikVBLqaa0i70JVSKshpECilVJDTIFBKqSCnQaCUUkFOg0AppYKcBoFSSgU5DQKllApyGgRKKRXkBt0DZSJSCBzp5emJQLDOYxSs9673HVz0vjuXYYzp8HHpQRcEZ0NEcjt7sm6oC9Z71/sOLnrfvaNNQ0opFeQ0CJRSKsgFWxCsCHQBAihY713vO7joffdCUPURKKWUOl2w1QiUUkq1o0GglFJBLmiCQESWiMgeEdkvIvcFujz+IiLPiUiBiGxvtS9eRP4hIvt827hAltEfRGSEiKwVkV0iskNEvufbP6TvXURcIvK5iGz13feDvv1D+r6biIhdRDaLyN98n4f8fYvIYRH5QkS2iEiub99Z3XdQBIGI2IEngEuBycCNIjI5sKXym98DS9rtuw94zxgzDnjP93mocQM/NMZMAuYC3/H9bzzU770euNAYMwPIApaIyFyG/n03+R6wq9XnYLnvhcaYrFbPDpzVfQdFEABzgP3GmIPGmAbgZWBpgMvkF8aYD4GSdruXAi/43r8AXNWfZeoPxpgTxphNvveVWL8c0hji924sVb6PTt/LMMTvG0BE0oEvA8+22j3k77sTZ3XfwRIEacCxVp/zfPuCxTBjzAmwfmECyQEuj1+JSCYwE/iMILh3X/PIFqAA+IcxJijuG3gU+DHgbbUvGO7bAO+KyEYRWe7bd1b37ejjAg5U0sE+HTc7BIlIJPAq8H1jTIVIR//TDy3GGA+QJSKxwOsiMjXARfI7EbkcKDDGbBSRBQEuTn+bZ4zJF5Fk4B8isvtsLxgsNYI8YESrz+lAfoDKEginRCQFwLctCHB5/EJEnFgh8KIx5jXf7qC4dwBjTBnwPlYf0VC/73nAlSJyGKup90IR+RND/74xxuT7tgXA61hN32d138ESBBuAcSIySkRCgGXAWwEuU396C7jV9/5W4M0AlsUvxPrT/3fALmPMI62+GtL3LiJJvpoAIhIGLAZ2M8Tv2xhzvzEm3RiTifXv+Z/GmFsY4vctIhEiEtX0HrgY2M5Z3nfQPFksIpdhtSnageeMMb8ObIn8Q0ReAhZgTUt7CngAeANYCYwEjgJfMca071Ae1ETkPOAj4Ata2ox/gtVPMGTvXUSmY3UO2rH+sFtpjPmliCQwhO+7NV/T0I+MMZcP9fsWkdFYtQCwmvb/zxjz67O976AJAqWUUh0LlqYhpZRSndAgUEqpIKdBoJRSQU6DQCmlgpwGgVJKBTkNAqX6kYgsaJopU6mBQoNAKaWCnAaBUh0QkVt88/xvEZHf+iZ2qxKR/xGRTSLynogk+Y7NEpFPRWSbiLzeNBe8iIwVkTW+tQI2icgY3+UjReQVEdktIi9KMEyIpAY0DQKl2hGRScANWJN7ZQEe4GYgAthkjMkGPsB6ahvgD8C9xpjpWE82N+1/EXjCt1bAucAJ3/6ZwPex1sYYjTVvjlIBEyyzjyrVE4uAWcAG3x/rYViTeHmBP/uO+RPwmojEALHGmA98+18A/uKbDybNGPM6gDGmDsB3vc+NMXm+z1uATOBjv9+VUp3QIFDqdAK8YIy5v81OkZ+1O66r+Vm6au6pb/Xeg/47VAGmTUNKne494DrffO9N68FmYP17uc53zE3Ax8aYcqBURM737f8q8IExpgLIE5GrfNcIFZHw/rwJpbpL/xJRqh1jzE4R+SnWKlA2oBH4DlANTBGRjUA5Vj8CWNP+Pu37RX8QuN23/6vAb0Xkl75rfKUfb0OpbtPZR5XqJhGpMsZEBrocSvU1bRpSSqkgpzUCpZQKclojUEqpIKdBoJRSQU6DQCmlgpwGgVJKBTkNAqWUCnL/H8IFD/4snB5TAAAAAElFTkSuQmCC",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "%matplotlib inline\n",
+    "from matplotlib import pyplot as plt\n",
+    "\n",
+    "plt.plot(epoch_nums, training_loss)\n",
+    "plt.plot(epoch_nums, validation_loss)\n",
+    "plt.xlabel('epoch')\n",
+    "plt.ylabel('loss')\n",
+    "plt.legend(['training', 'validation'], loc='upper right')\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 35,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "fully_connected1.weight \n",
+      " [[-0.00374341  0.2682218  -0.41152257 -0.3679695 ]\n",
+      " [-0.17916061 -0.08960593  0.11843108  0.5180272 ]\n",
+      " [-0.04437202  0.13230628 -0.15110654 -0.09828269]\n",
+      " [-0.47767425 -0.33114105 -0.20611155  0.01852179]\n",
+      " [ 0.22086579  0.5711509  -0.40086356 -0.18697421]\n",
+      " [ 0.31580442  0.24776897 -0.20200174  0.39890492]\n",
+      " [-0.08059168  0.05290705  0.4527381  -0.46383518]\n",
+      " [-0.3545517  -0.15797205 -0.23337851  0.39141223]\n",
+      " [-0.32408983 -0.23016644 -0.34932023 -0.4682805 ]\n",
+      " [-0.47349784  0.8002842   0.30180416  0.15444154]]\n",
+      "fully_connected1.bias \n",
+      " [ 0.02629578 -0.20744474  0.08459234 -0.46684736 -0.35585782 -0.45410082\n",
+      "  0.31546897  0.25728968 -0.22174752  0.24439509]\n",
+      "fully_connected2.weight \n",
+      " [[ 0.20224687  0.3143725   0.12550515  0.04272011  0.21202639 -0.18619564\n",
+      "   0.05892715 -0.24517313 -0.21917307 -0.16335806]\n",
+      " [ 0.14308453  0.08098809 -0.18731831  0.09553465  0.7475572  -0.01170831\n",
+      "   0.01207405  0.03671877  0.19618031  0.71772873]\n",
+      " [-0.24369258 -0.09592994  0.12428063  0.2620103   0.44033986  0.32761905\n",
+      "   0.06293392 -0.24256472  0.02909058 -0.6438864 ]\n",
+      " [-0.29470977  0.4369507   0.2404469  -0.31544605 -0.65187347 -0.03367811\n",
+      "  -0.05203882 -0.09720274  0.12160733 -0.44794998]\n",
+      " [ 0.11592636  0.15991893  0.22637847  0.11824107 -0.31298175 -0.20513597\n",
+      "   0.15789726  0.0661869  -0.24668422 -0.1820901 ]\n",
+      " [ 0.29749104  0.33983657 -0.13788326 -0.07958971 -1.0037647   0.04011776\n",
+      "  -0.23813814 -0.21048178 -0.01742402 -0.21410409]\n",
+      " [-0.12950484  0.18764248 -0.19243696  0.2869356   0.21671084 -0.26666948\n",
+      "  -0.07870413  0.01426902  0.04613796  0.07500109]\n",
+      " [ 0.12409672  0.01894209 -0.15429662  0.1496355  -0.30334112 -0.1874303\n",
+      "  -0.07916126 -0.15403877 -0.11062703 -0.25918713]\n",
+      " [-0.06726643  0.16598707 -0.20601156 -0.01622862 -0.10633215 -0.07815906\n",
+      "   0.00878868  0.00450952  0.06399861  0.4654336 ]\n",
+      " [ 0.29954556  0.20082232  0.3002309  -0.02287012 -0.2840742  -0.14991638\n",
+      "   0.21532115 -0.00204995 -0.15717986 -0.24232906]]\n",
+      "fully_connected2.bias \n",
+      " [-0.2959424  -0.09140179 -0.24091302  0.11557585  0.17096573 -0.3224678\n",
+      "  0.19725719 -0.24745122  0.03521875 -0.1282217 ]\n",
+      "fully_connected3.weight \n",
+      " [[-0.06091028 -0.06208903 -0.28376698 -0.27304304 -0.04948315  0.0040895\n",
+      "  -0.14365433  0.11912274 -0.28462344 -0.02134135]\n",
+      " [ 0.27809682 -0.41300255  0.27310103  0.7309681  -0.2853832   0.6525562\n",
+      "  -0.03649095 -0.14116624 -0.0045454  -0.25554216]\n",
+      " [ 0.03393281 -0.19290853  0.71934235 -0.31080088  0.15194914 -0.3314264\n",
+      "  -0.07604478 -0.06650442 -1.1165304   0.17134616]]\n",
+      "fully_connected3.bias \n",
+      " [ 0.25107792  0.10447468 -0.24180876]\n"
+     ]
+    }
+   ],
+   "source": [
+    "for param_tensor in model.state_dict():\n",
+    "    print(param_tensor, \"\\n\", model.state_dict()[param_tensor].numpy())"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 36,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWIAAAElCAYAAADeAeiuAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAAAnqklEQVR4nO3dfbzlY73/8dd7xn0IDY7cRBoVTqEhpRtRkgrdGp1K5aQ6pXTzC906lXNUKqU6miLqaCRRKickNyV3Y9yLKGIyYUK5Z3j//riuXctu773W7L32fNda+/3s8X3sta7vd13fa63MZ13r872+1yXbREREc6Y13YCIiKkugTgiomEJxBERDUsgjohoWAJxRETDlmm6Af1Gy6xoLbdK083oWVs8fYOmm9DzHs1IpbYuu2T+IttrTqSO6as+yV58f9vjfP/tp9reeSLnmqgE4iWk5VZh+ae+vulm9Kxzzz+86Sb0vPsefKTpJvS8Gass+8eJ1uHFD7D802a3Pe6BSw6fMdFzTVQCcUQMJgFS063oSHLEETG4NK391kk10lGSbpN05bDyfSVdK+kqSZ9rKT9Q0vV130vb1Z8ecUQMru71iI8Gvgp85x9V60XAbsAzbD8oaa1avikwG9gMeCLwC0mb2B41J5UecUQMKMG06e23Dtg+B7hjWPG7gENsP1iPua2W7wYcZ/tB2zcA1wPbjFV/AnFEDCbRaWpihqR5Lds+HZ5hE+D5ki6QdLakrWv5usDNLcctqGWjSmoiIgaUOk1NLLI9axwnWAZYHdgW2Bo4XtKTy4n/yZhjFhOII2JwdXgxbpwWACe6TGF5oaRHgRm1fP2W49YDbhmroqQmImJwSe238fsRsEM5jTYBlgMWAScDsyUtL2kjYCZw4VgVpUccEYNJ6vhiXPuqNBfYnpJPXgB8EjgKOKoOaXsI2Kv2jq+SdDxwNbAYePdYIyYggTgiBlmXUhO29xxl1xtHOf5g4OBO608gjogBpcnOEXdNAnFEDK5p/XGLcwJxRAymoXHEfSCBOCIGV59M+pNAHBEDqnujJiZbAnFEDK6kJiIiGjTxGzaWmgTiiBhc6RFHRDQsPeKIiCblYl1ERLMyjjgiomm5xTkionnJEUdENCw94oiIhqVHHBHRoC5ODD/ZEogjYmCpT3rE/ZFAiYhYQqIE4nZbR3VJR0m6rS6LNHzfhyRZ0oyWsgMlXS/pWkkvbVd/AnFEDCZ1uHXmaGDnfzqFtD7wEuCmlrJNgdnAZvU1X5c0Zo4kgTgiBlT73nCnPWLb5wB3jLDrS8CHAbeU7QYcZ/tB2zcA1wPbjFV/csQRMbA6DLQzJM1reT7H9pwO6t4V+JPty4adZ13g/JbnC2rZqHq+RyzpVTX/8rRR9p8laVabOv5+jKRTJK02CU2NiB4zbdq0thuwyPaslq2TILwS8FHgEyPtHqHMI5T9o52dvJmG7Qn8mpJzmTDbu9i+qxt1RUQP626OeLiNgY2AyyTdCKwHzJf0L5Qe8Potx64H3DJWZT0diCWtDGwH7E0NxJJWlHScpMslfR9YseX4nSSdJ2m+pB/U1w+v88ahq5uS3ijpQkmXSvpGu4R6RPQPdTFHPJztK2yvZXtD2xtSgu9Wtv8MnAzMlrS8pI2AmcCFY9XX04EY2B34ue3fAXdI2gp4F3Cf7WcABwPPAqjB9WPAi21vBcwDPjBaxZKeDuwBbGd7C+AR4N9GOXYfSfMkzfPi+7v13iJiknVx+Npc4DzgqZIWSNp7tGNtXwUcD1wN/Bx4t+1Hxqq/1y/W7QkcVh8fV5/PBL4CYPtySZfX/dsCmwLn1g93OcoHN5odKUH8onr8isBtIx1Yc0ZzAKattNaYuZ6I6B3duqHD9p5t9m847PnBlI5iR3o2EEt6ArADsLkkA9MpCe9LGDnxLeD0dh/YsOOPsX1gN9obET1GoGm5s26iXgt8x/aTah5mfeAGYD41hSBpc+AZ9fjzge0kPaXuW0nSJmPUfwbwWklr1ePXkPSkSXovEdGAycoRd1vP9ogpaYhDhpX9ENgSWLGmJC6lJsFt3y7pLcBcScvX4z8G/G6kym1fLeljwGmSpgEPA+8G/tjl9xERDRi6WNcPejYQ295+hLKvtHnNL4Gtx6qrNZdj+/vA9yfQzIjoYQnEERFN6484nEAcEQNK6RFHRDSu3sLc8xKII2Ig5WJdREQv6I84nEAcEQMqOeKIiOYlEEdENCyBOCKiYf0y10QCcUQMpF6aS6KdBOKIGFgJxBERDUsgjohoWn/E4QTiiBhQ6p9bnPujlRERS0iA1H7rqC7pKEm3Sbqypezzkq6pCxmfJGm1ln0HSrpe0rWSXtqu/gTiiBhQXV3F+Whg52FlpwOb14WMfwccCCBpU8qq85vV13y93QrxCcQRMbC61SO2fQ5wx7Cy02wvrk/PB9arj3cDjrP9oO0bgOuBbcaqP4E4IgZWhz3iGZLmtWz7jONUbwP+rz5eF7i5Zd+CWjaqXKyLiMHUeY93ke1Z4z6N9FFgMXDsP878T0Zaef7vEogjYiAJmD59csevSdoLeAWwo+2hYLsAWL/lsPWAW8aqJ6mJiBhYXbxYN1LdOwP7A7vavq9l18nAbEnLS9oImEldbX406RFHxGBagotxbauS5gLbU/LJC4BPUkZJLA+cXgP6+bbfafsqSccDV1NSFu+2/chY9ScQR8RAKuOIuxOJbe85QvGRYxx/MHBwp/UnEEfEgMrsaxERjeuTOJxAHBEDSjAtE8NHRDSnmzniyZZAHBEDq0/icAJxRAyu9IgjIhrWJ3E4gTgiBpNysW5wbfn0DTj3gq823YyetforD2u6CT3vzp/s13QTpoj+GUfcdq4JSRtLWr4+3l7Se1tnoo+I6FXdmo94snUy6c8PgUckPYVyS99GwPcmtVUREV0wmZP+dFMngfjROgv9q4DDbL8fWGdymxURMUEd9IZ7JA53lCN+WNKewF7AK2vZspPXpIiIieunGzo66RG/FXgOcLDtG+r8mv87uc2KiJi4adPUdusFbXvEtq+WtD+wQX1+A3DIZDcsImKiBqZHLOmVwKXAz+vzLSSdPMntioiYmD7KEXeSmjiIshT0XQC2L6WMnIiI6Fmi/YiJXukxdxKIF9v+67CyMVckjYjoBd3qEUs6StJtkq5sKVtD0umSrqt/V2/Zd6Ck6yVdK+ml7ervJBBfKekNwHRJMyUdDvyms+ZHRDRn+jS13Tp0NLDzsLIDgDNszwTOqM+RtCkwG9isvubrkqaPVXkngXjfWuGDwFzgb8B+nbY+IqIJpcfbndSE7XOAO4YV7wYcUx8fA+zeUn6c7Qfr4IbrKendUXUyauI+4KN1i4joGx12eGdImtfyfI7tOR28bm3bCwFsL5S0Vi1fFzi/5bgFtWxUowZiSYfZ3k/STxghJ2x71w4aGhHRmA57vItsz+rmaUcoG/O62lg94u/Wv4eOuzkREQ2a5EERt0pap/aG1wFuq+ULgPVbjlsPuGWsikYNxLYvrg/nAffbfhSgJp2XH2/LIyKWBlGGsE2ikylTPxxS//64pfx7kr4IPBGYCVw4VkWdXKw7A1ip5fmKwC+WsMEREUuX2o+Y6HTUhKS5wHnAUyUtkLQ3JQC/RNJ1wEvqc2xfBRwPXE25Ee7dth8Zq/5OJv1ZwfY9Q09s3yNppbFeEBHRC7qVmrC95yi7dhzl+IOBgzutv5Me8b2Sthp6IulZwP2dniAiogkCpkltt17QSY94P+AHkoaSzesAe0xaiyIiuqRH4mxbnYwjvkjS04CnUr5krrH98KS3LCJignplLol2Opl9bSVgf+B9tq8ANpT0iklvWUTEBHQyz0SvxOlOcsTfBh6iTA4PZYzcZyatRRERXTJdarv1gk4C8ca2Pwc8DGD7fka+cyQioqf0yzSYnVyse0jSitRb9CRtTJkAKCKiZ5VRE023ojOdBOJPUgYlry/pWGA74C2T2aiIiAnroR5vO52Mmjhd0nxgW8qXzPtsL5r0lkVETFCfxOGOesQALwSeR0lPLAucNGktiojoAsGSTPzeqLaBWNLXgadQJoUHeIekF9t+96S2LCJiggYmNUHpDW9ue+hi3THAFZPaqoiILuiPMNxZIL4W2AD4Y32+PnD5pLUoIqILJHpmLol2OgnETwB+K2loPs2tgfMknQxZqSMielefxOGOAvEnJr0VERGTYGByxLbPBpD0BOAFwE0tq3d0laS1gS9RhsrdSbm1+nO2l3iUhqT9KIsA3tfVRkZEXxCdT/zetFFvcZb0U0mb18frAFcCbwO+W4NcV6l8df0IOMf2k20/C5hNWe9pPPbjsSuLRMRUMiCT/mxk+8r6+K3A6bZfCTybEpC7bQfgIdtHDBXY/qPtwyVNl/R5SRdJulzSOwAkbS/pLEknSLpG0rEq3ktZK+pMSWfWY/eUdIWkKyV9dugco5VHRP/rl7kmxgrErXMO7wicAmD7buDRSWjLZsD8UfbtDfzV9taUi4Vvl7RR3bclpfe7KfBkYDvbX6Gsmvoi2y+S9ETgs5RgvwWwtaTdRysffnJJ+0iaJ2ne7Ytu78Z7jYilYFoHWyckvV/SVbXDNlfSCpLWkHS6pOvq39Un0s7R3CxpX0mvAraizDdBnQBo2fGesFOSvibpMkkXATsBb5Z0KXABZSTHzHrohbYX1FWmLwU2HKG6rYGzbN9uezFwLCXfPVr5Y9ieY3uW7Vlrzlizq+8zIiaH6E6PWNK6wHuBWbY3B6ZT0qYHAGfYnklZZPmA8bZ1rEC8N6WX+hZgD9t31fJtKXMUd9tVlIAPQL1zb0dgTcpnuq/tLeq2ke3T6qGtM8E9wsgXIEf7tHvjd0lETIppar91aBlgRUnLUK493QLsBhxT9x8D7D7udo62w/Zttt9pe7eWoIftM20fOt4TjuGXwAqS3tVSNnSx7VTgXZKWBZC0iaTHtanvbmCV+vgC4IWSZkiaDuwJnD1GeUT0OanMNdFuA2YMpR7rtk9rPbb/BBwK3AQspKRJTwPWtr2wHrMQWGu8be100p9JZ9s1P/slSR8GbgfupSzT9ANKymF+HV1xO+2/feYA/ydpYc0THwicSekFn2L7xwCjlUdE/+uwx7vI9qzRdtbc727ARsBdlMWU39iN9g3pmUAMf/9WmT3K7o/UrdVZdRt6/XtaHh8OHN7y/HvA90Y454jlEdH/ujQo4sXADbZvL3XqROC5wK2S1rG9sA7xvW28J+j0omFERF8pK3So7daBm4BtJa1Uf5HvCPwWOBnYqx6zFzDuX9Oj9oglHU5dHmkktt873pNGRCwN3ehp2r5A0gmU4bWLgUsoqc+VgeMl7U0J1q8b7znGSk3MG2+lERFNk7p3i7PtT1KWjWv1IKV3PGGjBmLbx4y2LyKiH/TIjXNtdbJCx5qUkQubAisMldveYRLbFRExYX0y509HKZRjKYnpjYD/BG4ELprENkVETFgXL9ZNuk4C8RNsHwk8bPts22+j3F0XEdHT+mX2tU7GEQ9N/rNQ0sspt/aNd2rKiIilY8luYW5UJ4H4M5IeD3yQcoPEqsD7J7VVERETJGB6r3R52+hkhY6f1od/BV40uc2JiOiegekRS/o2I9zYUXPFERE9q1cmfm+nk9TET1serwC8ipInjojoWWXURNOt6EwnqYkftj6XNBf4xaS1KCKiG3poVEQ745l9bSawQbcbEhHRbb0yTridTnLEd/PYHPGfKXfaRUT0LAHT+2R+yU5SE6u0OyYioveIaX2yGlrb7wtJZ3RSFhHRS8rioX1+Z52kFShrxs2oS4UMNXlV4IlLoW0REeM3IHfWvQPYjxJ0L+YfgfhvwNcmt1kRERPX9xfrbH8Z+LKkfev6bxERfaNcrOuPQNzJNcVHJa029ETS6pL+Y/KaFBHRHd3KEUtaTdIJkq6R9FtJz5G0hqTTJV1X/64+3nZ2EojfbvuuoSe27wTePt4TRkQsDaIEuHZbh74M/Nz204BnUuZoPwA4w/ZM4Iz6fFw6acc0tdywLWk6sNx4TxgRsVSozDXRbmtbjbQq8ALgSADbD9XO6W7A0JJyxwC7j7epnQTiUykrle4oaQdgLvDz8Z4wImJpUQcbZWTYvJZtn2HVPBm4Hfi2pEskfUvS44C1bS8EqH/XGm87O7nFeX9gH+Bdtd2nAd8c7wkjIpaGoaWSOrDI9qwx9i8DbAXsa/sCSV9mAmmIkbTtEdt+1PYRtl9r+zXAVZQJ4iMieto0td86sABYYPuC+vwESmC+VdI6APXvbeNuZycHSdpC0mcl3Qh8GrhmvCeMiFg62ueHO8kR2/4zcLOkp9aiHYGrgZOBvWrZXsCPx9vSse6s2wSYDewJ/AX4PiDbWaUjInre0KiJLtkXOFbScsAfgLfW6o+XtDdwE/C68VY+Vo74GuBXwCttXw8gKWvVRUTf6NYKHbYvBUbKI+/YjfrH+sJ4DWXKyzMlfVPSjtAnUxlFRNDxqInGjXWL80nASXWYxu6UlZvXlvQ/wEm2T1s6TYx+cudP9mu6CT1vn+9f1nQTpgb1z5p1nYyauNf2sbZfAawHXEqXh25ERHSbgOlS260XLFEu2/Ydtr9he4fJalBERLf0fWoiIqLf9UiHt60E4ogYSGX4Wn9E4gTiiBhY6RFHRDRK/b9CR0REP0tqIiKiaT20SnM7CcQRMbASiCMiGqakJiIimlMmhm+6FZ1JII6IgZVRExERDUtqIiKiQUlNREQ0TukRR0Q0qo/GEXdxSaeIiN7R7fmIJU2XdImkn9bna0g6XdJ19e/q421rAnFEDKwuz0f8PuC3Lc8PAM6wPRM4gwksmJFAHBGDq0uRWNJ6wMuBb7UU7wYcUx8fQ1lSblySI46IgdXhxboZkua1PJ9je86wYw4DPgys0lK2tu2FALYXSlprvO1MII6IgdVhCniR7Vmj16FXALfZvljS9t1p2WMlEEfEwOrSoIntgF0l7QKsAKwq6X+BWyWtU3vD6wC3jfcEyRFHxEASIKnt1o7tA22vZ3tDYDbwS9tvBE4G9qqH7QX8eLxtTY84IgbT5I8jPgQ4XtLewE3A68ZbUQJxRAysbsdh22cBZ9XHfwF27Ea9CcQRMbj65M66BOKIGFCZayIionH9MtdEAnFEDKQyaqLpVnQmgTgiBlZSExERDUuPOCKiYX0Shyf3zjpJ/yLpOEm/l3S1pFMk7TM0n+cIx39L0qbjOM8W9fbDiIiik5nXeiRST1ogVrl38CTgLNsb294U+Aiw9mivsf3vtq8ex+m2AEYMxJLS64+YgsqadWq79YLJ7BG/CHjY9hFDBbYvBX4FrCzpBEnXSDq2Bm0knSVpVn18j6SDJV0m6XxJa9fy10m6spafI2k54FPAHpIulbSHpIMkzZF0GvAdSRtK+pWk+XV7bq1r+1rHSbXHfoSkzL8RMSD6pEM8qYF4c+DiUfZtCewHbAo8mTK70XCPA863/UzgHODttfwTwEtr+a62H6pl37e9he3v1+OeBexm+w2UWZFeYnsrYA/gKy3n2Qb4IPCvwMbAq4c3pKZT5kmad/ui2zt68xHRA/okEjfV+7vQ9gLbjwKXAhuOcMxDwFAu+eKWY84Fjpb0dmD6GOc42fb99fGywDclXQH8gPIF0NqWP9h+BJgLPG94Rbbn2J5le9aaM9bs5P1FRA9QB//rBZOZP70KeO0o+x5sefzIKO142LaHH2P7nZKeTVm25FJJW4xyjntbHr8fuBV4JuXL54GWfeaxhj+PiD7VIyngtiazR/xLYPnacwVA0tbACydSqaSNbV9g+xPAImB94G4eu4TJcI8HFtYe+Jt4bE96G0kb1dzwHsCvJ9K+iOgdfZKZmLxAXHuzrwJeUoevXQUcBNwywao/L+kKSVdScseXAWcCmw5drBvhNV8H9pJ0PrAJj+0tn0eZV/RK4AbKSI+I6HPdmhh+aZjUoV22bwFeP8Kub7Yc856Wx9u3PF655fEJwAn18T9dTAPuALYeox3XAc9oKTqw5fF9tkcK3hHRzyZ/YviuyRjbiBhYfRKHp/aadbbPsv2KptsREZOkC0liSetLOlPSbyVdJel9tXwNSadLuq7+XX28zZzSgTgiBlkng9c66jMvBj5o++nAtsC761QMBwBn2J4JnFGfj0sCcUQMLKn91o7thbbn18d3A78F1gV2A46phx0D7D7ediZHHBEDaQkmhp8haV7L8zm254xYp7Qh5c7gC4C1bS+EEqwlrTXetiYQR8TA6jD1sMj2rLZ1SSsDPwT2s/23bg59S2oiIgZWN1ITpR4tSwnCx9o+sRbfKmmdun8dypw245JAHBEDqxt31tXZIY8Efmv7iy27Tgb2qo/3An483nYmNRERg6l7N3RsR5ka4QpJl9ayj1DuyD1e0t7ATcDrxnuCBOKIGEhDtzhPlO1fM3rneccJn4AE4ogYYP1yZ10CcUQMrMw1ERHRsF6Z+L2dBOKIGFz9EYcTiCNicPVJHE4gjojBJMG0PkkSJxBHxODqjzicQBwRg6tP4nACcUQMrj7JTCQQR8Sg6nji98YlEEfEQFqC+Ygbl0AcEQMrgTgiomFJTURENKl702BOugTiiBhInU783gsSiCNicPVJJE4gjoiBlVucIyIa1h9hOIuHRsQg68bqoYCknSVdK+l6SQd0u5kJxBExsNTB/9rWIU0Hvga8DNgU2FPSpt1sZwJxRAykoTvr2m0d2Aa43vYfbD8EHAfs1s22Jke8hObPv3jRisvqj023Y5gZwKKmG9HD8vm012uf0ZMmWsH8+RefuuKymtHBoStImtfyfI7tOS3P1wVubnm+AHj2RNvXKoF4Cdles+k2DCdpnu1ZTbejV+XzaW8QPyPbO3epqpH6ze5S3UBSExER7SwA1m95vh5wSzdPkEAcETG2i4CZkjaStBwwGzi5mydIamIwzGl/yJSWz6e9fEajsL1Y0nuAU4HpwFG2r+rmOWR3NdURERFLKKmJiIiGJRBHRDQsgTgiomEJxANIKvcLDf2NiN6WQDxgJMn/uAK7di2bNrSvsYb1oNE+j3xOSy6f2cRk1MSAkvRu4CXAVcBNwJG2Fzfbqt7R+oUl6cXAasC1wO9t3zfsCy2GkbQV5dbf3wILbD+Qz2z8Mo54AEl6DbAH8Frgh8DFCcKP1RKEPwS8HrgBuB/4k6TP2f5rk+3rZZK2p8xGdiNwJ/B7SYfavrvBZvW1pCYGgKTtJW3WUrQ6cAiwMyW4/L963MYNNK9nSVodeAHwItt7AN8AlqX8ksjP7RFI2gI4EHid7ZcD3wKWB15V9+czG4cE4sGwJnBPDSwAf6L0WP7d9k62H5b0XuAtkqbsr6BRgsSGwEvr4wuAe6gza+Vn9mNJWhbYAtieMjUkwG+AW4FtIZ/ZeE3Zf5SDQNKWALZ/IOlJwO8kvQL4NXAKJThvDTwN2At481RNUQzLCW9OCbg3A58CdpB0l+1fSloAbFvnFHh4qgeWoc9N0jTbDwNHS1oNeJOkhbZPlXQ5sJOkxwN/m+qf2XgkEPe31wDbSPqw7UslfQY4EngDcDhlRYHPAHcAb+n2/fH9ZFhOeNdafAFlFq0LgK9L+hWwA/DKOgH4lNYShF8J7FpXqjjE9mGSHgS+LemHlF8VX09effwyaqIPDevdHQQ8C/iE7UvqaIn3AP9me76kFYBHp2pgGaEnfBSwHbAx5XPbmpIbFmW43/W2bx6luimn/sI6iPKL6mDK57WT7askfYByHeIE23Nqr/nR5lrbv5Ij7jPDhwjZPgg4FzhY0pa2vwZ8GThN0izbD0zhILxSSxBekXLh0rYftn0N5XN7IjDT9tW2z0wQ/gdJKwFPAfYGZgIrA8cA50razPYXgWOBN0vaPkF4/JKa6CPDendvoIx9vcX2IfU61Kckfdz2EZIeogwtmpLqL4F3SLqIkiN/HvA2YGgV3kNt3yjpJkrv+J++5KYySTtRersHAytQLv6+zfbvJL0cOFPSBsDPgEeA3zfW2AGQQNxHWoLwB4CXU3ojH5e0Yg3GHwe+Iuk9to9qsq1NqzcY/Ab4JWUUyXNtPyrpaMrCj6dI+hmwOyXgTPkr/i054U0o6a0DbP9F0qqUGzdmSFqLEnx/bPsB4AFJx071z26ikproA5LWqlfxqf8oNrO9I2XBxz8DJ9b83KeBnzC1e8JD82xMA+YBx1N6dFvXQ84GPk75nKZRLsxd30BTe8bQkMYahJ8EvIOyHNDQxTfXx2+lrGB8ju0Lhj7rBOGJy8W6Hlb/Q18b+D7wP8BJlOBxbP0rysD6hyS9FbjI9pVNtbdpw1I3WwJ/sP1XSc+mBJCP2J5bf3ZfZHvKfmENqV/wzwdup3xhzaTk0l8LXAx8z/ZCSasAKwGr2b62qfYOqvSIe5ztPwOHAm8CdrF9P/BjSl7zSzUIv4Vy99zfGmtow4YF4f+gBN6fSdqH0jN+C+WC5uGUIX6rj1bXFLNs3b5C+aK/2PaJwA8oC2a+TtK6tu+2fWuC8ORIjriHtfzke5TSG/mOpLdTrvavDHyjjn19NqVnfFMzLW1eSxDejXJh7hmUW5VfAaxg+yt1KNazgcNs/6GxxvaI+uV1r6Qbgc2B8ynpLmyfJGkxsAuwh6SvTtXRN0tDUhM9TtJsYD/KvfyzgVcDX7D9I0mbUn7V3Gn7T821sjfUC0nfANa1vU0t24Vyce73lBno/tJgE3tGy4W5F1Nm53sEmEX58jrF9omS1qCkKM5OT3hyJTXRYyStPaxoXeBc2wttfwk4AjiypiOus33lVA3CrXNHSFrG9m3AfwF3SToYwPYpwM8pn2PGuVY1CO9KSUk8xfbvKRcy5wO7SDoE+C5weoLw5EuPuIdIehpwNXAYcE29W2lXYEfKuNeb63E/AR6izB1xb1Pt7RWS3kG58eB24ARgLcrwqz/Y/kQ95nH5rP5B0gzKtYa31rHBz6CkJa6l3HH4JuC7tk9usJlTRnLEveVe4DzKbFavkfQsyjwImwGvlvRnyoWVe4D9p2pgab2Vto4WeSPl7q+LKFf8j6DcXfhxSR+z/Rngvqba26OWpXxWL603uKxMmWfjQ7aPlvQz24/kJpelI4G4h9i+WdKFwFaUGzZmU6Yc3LBuz6OM6fzPqXphTtLzgE0kXW57HuUi039QpmG8ADjCZdrPK4BPArdBxrq25ISfBvylDkn7KiX4Hm/755JeT5mJ7nvAYsjntrQkEPeIlp7H/sB3KD8TF1AunpxICcQLgP+2fUtT7WySpJ2B/wa+BKxai/9ISeU8YnunetxHKWmJuU20s9cM/YKoF+aOAX5df119wfaP6jE7AJ8APpjREUtfAnGPqL0VUW7SuB74IqVn/L46QuKpwG1T9SYESS8EvkqZVe6Cll2rUsZPf6NOUrML8Dpgz6Xfyt4y9OVeg/BzKKuR7E5JSewKHCTpC5Qv+I9Sbmk+tbEGT2G5WNeDatD9FXB4vW15ypO0H+X76sstZf9FmXt5MeUK/3Moy/a81/YVTbSzV9QJebYH5lKGpl0MrG57w7p/M+CVwDOBDwH32r4rOeFmZPhaD6rDhfYHptde3pTVMkRtY8qSUEPlLwM2oIxzXUxZbWNP4DVTPQhXywBXUILvo5Q5mO+X9C0Al0UCfkaZzOcJtu+q5QnCDUgg7l3nUYYRTWktgeFHwLNVlnEH+AVlWsZ5lN7wg7bvtH1HA83sKbVX+wdKkD1O0qdt30e5YeN5kr4BUL+wvmj78gabGyQQ9yyXicv3qP+Aotx+ey4wW9I2LpO7PyRpT0pe+Lxmm9cb6o0trsH4AeC9wLPqML57KdcddpH0bQDb9zTZ3iiSI46+IWldynjhHYBL+McsYbvbvrrJtjWt3t69qF6Y24lyUe7nlOk+n0yZve9M2/8t6XHALNtnN9bgeIwE4ugrKksebUUZ1vcn4Czb1zXbqmbVOao/TRlxM5cynO8Myuf0C2AOZTjkd4DTbH+qvi4X5npEAnFEH1NZTWN/yqRGm1Omrvys7Z9K2p5yU9D1wDcpwfgJti9sprUxmgTiiD5VZ987npJ2+BEl0B5C+Xe9cz3m+cDbKRfuPmf7kWZaG2NJII7oQzUdcRJlBY0jW8r/FXgfZd6S/eqFuxdSbmuesqu39LqMmojoT/dTcuQnAEhaFv4+JO0LwL8Ah9eysxOEe1sCcUR/ehywJWUiKOpER9Prvr8AlwGr1vRF9LgE4og+VO+EO5wyXeoWw3bPokydesBUH9bXLxKII/rXScBC4J119rRHJW1HSU18d6rO0tePcrEuoo/VpbVeT5mTeT5lTo5Dhqa3jP6QQBwxAGpAfhRY3vaC3KzRXxKIIyIalhxxRETDEogjIhqWQBwR0bAE4oiIhiUQR0Q0LIE4xkXSI5IulXSlpB9MZG09SUdLem19/K2xbsuVtL2k547jHDdKmjFC+dskXSHp8vpedlvSutucd8z3EwFlgcGI8bjf9hYAko4F3gl8cWinpOnjmXLR9r+3OWR74B7gN0ta93CS1qMsI7+V7b9KWpmWBUq7oYP3E5EecXTFr4Cn1N7qmZK+B1whabqkz0u6qPY43wFlZQhJX5V0taSfAWsNVSTpLEmz6uOdJc2XdJmkMyRtSAn476+98edLWlPSD+s5Lqq3+CLpCZJOk3RJXSxT/LO1gLspgR3b99i+oaUdh0n6Te0pb1PLHyfpqHquS4Z60PW9HtrSu953hPezk6Tz6nv6QQ38SDqkfhaXSzq0u//XRD9IjzgmRNIywMso66MBbANsbvsGSfsAf7W9taTlgXMlnUaZNeypwL8CawNXA0cNq3dNyqoSL6h1rWH7DklHAPfYPrQe9z3gS7Z/LWkD4FTg6cAngV/b/pSklwP7jND8y4BbgRsknQGcaPsnLfsfZ/u5kl5Q27c5pQf9S9tvk7QacKGkXwBvBjYCtrS9WNIaw97PDOBjwItt3ytpf+ADkr4KvAp4Wp07eLUOP/oYIAnEMV4rSrq0Pv4VcCTwXODCoV4lsBPwjKH8L/B4YCbwAmBuTV3cIumXI9S/LXDOUF227xilHS8GNpX+3uFdVdIq9Ryvrq/9maQ7h7/Q9iOSdga2BnYEviTpWbYPqofMrcedI2nVGiR3AnaV9KF6zArABrUdR9hePEp7twU2pXwZASxHWXn6b8ADwLfqr4OfjvI+Y4AlEMd4/T1HPKQGmHtbi4B9bZ867LhdgHb31quDY6Ck155j+/4R2tL29XU+hgspPdvTgW8DBw3tHn54bddrbF877Hzt2ivgdNt7/tOOkvbYkbK+3Hsoq1THFJIccUymU4F3qa4eIWkTlaXczwFm17zqOsCLRnjtecALJW1UXzv0U/9uYJWW406jBC/qcVvUh+cA/1bLXgasPvwEkp4oaauWoi2AP7Y836Me9zxKiuWv9T3tWwMvkrZsacc7a6qmtb1Dzge2k/SUun+l+nmsDDze9inAfrUNMcWkRxyT6VvAhsD8GrhuB3anzKO7A3AF8Dvg7OEvtH17zTGfKGkacBvwEuAnwAn1Itm+wHuBr0m6nPLf8zmUC3r/CcyVNL/Wf9MI7VsWOFTSEynpgdvra4fcKek3wKrA22rZpynL1V9e39ONwCvqe92klj9MyW9/ddj7eUtt0/K1+GOUL5YfS1qB0mt+/6ifZgyszL4WMQJJZwEfsj2v6bbE4EtqIiKiYekRR0Q0LD3iiIiGJRBHRDQsgTgiomEJxBERDUsgjoho2P8HcuuOwBgzx8YAAAAASUVORK5CYII=",
+      "text/plain": [
+       "<Figure size 432x288 with 2 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "#Pytorch doesn't have a built-in confusion matrix metric, so we'll use SciKit-Learn\n",
+    "from sklearn.metrics import confusion_matrix\n",
+    "import numpy as np\n",
+    "\n",
+    "# Set the model to evaluate mode\n",
+    "model.eval()\n",
+    "\n",
+    "# Get predictions for the test data\n",
+    "x = torch.Tensor(x_test).float()\n",
+    "_, predicted = torch.max(model(x).data, 1)\n",
+    "\n",
+    "# Plot the confusion matrix\n",
+    "cm = confusion_matrix(y_test, predicted.numpy())\n",
+    "plt.imshow(cm, interpolation=\"nearest\", cmap=plt.cm.Blues)\n",
+    "plt.colorbar()\n",
+    "tick_marks = np.arange(len(penguin_classes))\n",
+    "plt.xticks(tick_marks, penguin_classes, rotation=45)\n",
+    "plt.yticks(tick_marks, penguin_classes)\n",
+    "plt.xlabel(\"Predicted Species\")\n",
+    "plt.ylabel(\"Actual Species\")\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 37,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "model saved as penguin_classifier.pt\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Save the model weights\n",
+    "# model_file = '/User/johnnydevriese/projects/models/penguin_classifier.pt'\n",
+    "model_file = 'penguin_classifier.pt'\n",
+    "torch.save(model.state_dict(), f=model_file)\n",
+    "del model\n",
+    "print('model saved as', model_file)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 38,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "New sample: [[50.4, 15.3, 20, 50]]\n",
+      "Prediction: Gentoo\n"
+     ]
+    }
+   ],
+   "source": [
+    "# New penguin features\n",
+    "x_new = [[50.4,15.3,20,50]]\n",
+    "print ('New sample: {}'.format(x_new))\n",
+    "\n",
+    "# Create a new model class and load weights\n",
+    "model = PenguinNet()\n",
+    "model.load_state_dict(torch.load(model_file))\n",
+    "\n",
+    "# Set model to evaluation mode\n",
+    "model.eval()\n",
+    "\n",
+    "# Get a prediction for the new data sample\n",
+    "x = torch.Tensor(x_new).float()\n",
+    "_, predicted = torch.max(model(x).data, 1)\n",
+    "\n",
+    "print('Prediction:',penguin_classes[predicted.item()])"
+   ]
+  }
+ ],
+ "metadata": {
+  "interpreter": {
+   "hash": "16c7f1dc46b458d69b8d4b83cad879badbf4dbe0bbfb50262ef6f7a4b6b16937"
+  },
+  "kernelspec": {
+   "display_name": "Python 3.9.7 64-bit ('base': conda)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.12"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

code/preprocessor_config.json

@@ -0,0 +1,17 @@
+{
+  "do_normalize": true,
+  "do_resize": true,
+  "feature_extractor_type": "ViTFeatureExtractor",
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "resample": 2,
+  "size": 224
+}

code/ps2_submission.py

@@ -0,0 +1,411 @@
+import numpy as np
+import util
+
+# Character to replace with sub-problem letter in plot_path/save_path
+WILDCARD = 'X'
+
+def main_LogReg(train_path, valid_path, save_path):
+    """Problem (1b): Logistic regression with Newton's Method.
+
+    Args:
+        train_path: Path to CSV file containing dataset for training.
+        valid_path: Path to CSV file containing dataset for validation.
+        save_path: Path to save predicted probabilities using np.savetxt().
+    """
+    # Load dataset
+    x_train, y_train = util.load_dataset(train_path, add_intercept=True)
+
+    # Train a logistic regression classifier
+    clf = LogisticRegression()
+    clf.fit(x_train, y_train)
+
+    # Plot decision boundary on top of validation set set
+    x_eval, y_eval = util.load_dataset(valid_path, add_intercept=True)
+    plot_path = save_path.replace('.txt', '.png')
+    util.plot(x_eval, y_eval, clf.theta, plot_path)
+
+    # Use np.savetxt to save predictions on eval set to save_path
+    p_eval = clf.predict(x_eval)
+    yhat = p_eval > 0.5
+    print('LR Accuracy: %.2f' % np.mean( (yhat == 1) == (y_eval == 1)))
+    np.savetxt(save_path, p_eval)
+
+class LogisticRegression:
+    """Logistic regression with Newton's Method as the solver.
+
+    Example usage:
+        > clf = LogisticRegression()
+        > clf.fit(x_train, y_train)
+        > clf.predict(x_eval)
+    """
+    def __init__(self, step_size=0.01, max_iter=1000000, eps=1e-5,
+                 theta_0=None, verbose=True):
+        """
+        Args:
+            step_size: Step size for iterative solvers only.
+            max_iter: Maximum number of iterations for the solver.
+            eps: Threshold for determining convergence.
+            theta_0: Initial guess for theta. If None, use the zero vector.
+            verbose: Print loss values during training.
+        """
+        self.theta = theta_0
+        self.step_size = step_size
+        self.max_iter = max_iter
+        self.eps = eps
+        self.verbose = verbose
+
+    def gradient(self,x, y):
+        n_examples, dim = x.shape
+        logits = self.sigmoid(x)
+        # grad of logit function
+        gradient = 1 / n_examples * x.T @ (logits - y)
+        return gradient
+
+    def hessian(self, x, y):
+        n_examples, dim = x.shape
+        # sigmoid = lambda x: 1 / 1 + np.exp(- x @ self.theta)
+        logits = self.sigmoid(x)
+
+        # probs = self._sigmoid(x.dot(self.theta))
+        # diag = np.diag(logits * (1. - logits))
+        # hess = 1 / n_examples * x.T.dot(diag).dot(x)
+        # return hess
+
+        # main diag is just second derivative wrt to itself. e.g. f_xx and f_yy
+        main_diagonal = np.diag(logits * (1 - logits))
+        hessian = 1 / n_examples * x.T @ main_diagonal @ x 
+        return hessian 
+
+    def loss(self, x, y):
+        # https://developers.google.com/machine-learning/crash-course/logistic-regression/model-training
+        # also in p.16 in Supervised Learning notes 
+        n_examples, dim = x.shape
+        # sigmoid = lambda x: 1 / 1 + np.exp(- x @ self.theta)
+        logits = self.sigmoid(x)
+
+        loss = -np.mean(y * np.log(logits) + (1 + y) * np.log(1 - logits))
+        return loss
+
+
+    def sigmoid(self, x):
+        # return 1 / (1 + np.exp(-x.dot(self.theta)))
+        return 1 / (1 + np.exp(- x @ self.theta))
+
+
+    def fit(self, x, y):
+        """Run Newton's Method to minimize J(theta) for logistic regression.
+
+        Args:
+            x: Training example inputs. Shape (n_examples, dim).
+            y: Training example labels. Shape (n_examples,).
+        """
+        # *** START CODE HERE ***
+        # NOTE: look at p.18 in notes 
+        # we need to calculate theta with Newton and then maximize the 
+        # logistic regression log likelihood function l(theta) 
+        # prev_theta = theta # store for comparison 
+
+        # m = rows = number of examples 
+        # n = columns = number of features 
+
+        # breakpoint()
+        # NOTE: it looks like they prepend the '1' at the beginning of the x array! 
+        n_examples, dim = x.shape
+        if self.theta is None: 
+            self.theta = np.zeros(dim)
+        
+        # just need to init for first time. 
+        # theta_prev = np.ones(dim)
+        # # print(np.sum(np.abs(theta_prev - self.theta)) < self.eps)
+
+        # current_iteration = 0
+        # theta_difference = np.sum(np.abs(theta_prev - self.theta))
+        # while theta_difference > self.eps and current_iteration < self.max_iter:
+        for i in range(self.max_iter):
+            # current_iteration += 1
+
+            gradient = self.gradient(x, y)
+            hessian = self.hessian(x, y)
+
+            # theta_prev = self.step(gradient, hessian)
+            theta_prev = np.copy(self.theta)
+            # theta_prev = self.step()
+            # theta_prev = self.theta
+            # self.theta = self.theta - self.step_size * np.linalg.inv(hessian) @ gradient
+            self.theta -= self.step_size * np.linalg.inv(hessian).dot(gradient)
+
+            if np.sum(np.abs(theta_prev - self.theta)) < self.eps:
+                break
+
+        # *** END CODE HERE ***
+
+    def predict(self, x):
+        """Return predicted probabilities given new inputs x.
+
+        Args:
+            x: Inputs of shape (n_examples, dim).
+
+        Returns:
+            Outputs of shape (n_examples,).
+        """
+        # *** START CODE HERE ***
+        # breakpoint()
+        # sigmoid = lambda x: 1 / 1 + np.exp(- x @ self.theta)
+        prediction = self.sigmoid(x)
+        return prediction 
+        # *** END CODE HERE ***
+
+def main_GDA(train_path, valid_path, save_path):
+    """Problem (1e): Gaussian discriminant analysis (GDA)
+
+    Args:
+        train_path: Path to CSV file containing dataset for training.
+        valid_path: Path to CSV file containing dataset for validation.
+        save_path: Path to save predicted probabilities using np.savetxt().
+    """
+    # Load dataset
+    x_train, y_train = util.load_dataset(train_path, add_intercept=False)
+
+    # Train a GDA classifier
+    clf = GDA()
+    clf.fit(x_train, y_train)
+
+    # Plot decision boundary on validation set
+    x_eval, y_eval = util.load_dataset(valid_path, add_intercept=False)
+    plot_path = save_path.replace('.txt', '.png')
+    util.plot(x_eval, y_eval, clf.theta, plot_path)
+    x_eval = util.add_intercept(x_eval)
+
+    # Use np.savetxt to save outputs from validation set to save_path
+    p_eval = clf.predict(x_eval)
+    yhat = p_eval > 0.5
+    print('GDA Accuracy: %.2f' % np.mean( (yhat == 1) == (y_eval == 1)))
+    np.savetxt(save_path, p_eval)
+
+class GDA:
+    """Gaussian Discriminant Analysis.
+
+    Example usage:
+        > clf = GDA()
+        > clf.fit(x_train, y_train)
+        > clf.predict(x_eval)
+    """
+    def __init__(self, step_size=0.01, max_iter=10000, eps=1e-5,
+                 theta_0=None, verbose=True):
+        """
+        Args:
+            step_size: Step size for iterative solvers only.
+            max_iter: Maximum number of iterations for the solver.
+            eps: Threshold for determining convergence.
+            theta_0: Initial guess for theta. If None, use the zero vector.
+            verbose: Print loss values during training.
+        """
+        self.theta = theta_0
+        self.step_size = step_size
+        self.max_iter = max_iter
+        self.eps = eps
+        self.verbose = verbose
+
+    def sigmoid(self, x):
+        # return 1 / (1 + np.exp(-x.dot(self.theta)))
+        return 1 / (1 + np.exp(- x @ self.theta))
+
+
+    def fit(self, x, y):
+        """Fit a GDA model to training set given by x and y by updating
+        self.theta.
+
+        Args:
+            x: Training example inputs. Shape (n_examples, dim).
+            y: Training example labels. Shape (n_examples,).
+        """
+        # *** START CODE HERE ***
+        n_examples, dim = x.shape
+
+        # Find phi, mu_0, mu_1, and sigma
+        phi = 1 / n_examples * np.sum(y == 1)
+        mu_0 = (y == 0).dot(x) / np.sum(y == 0)
+        mu_1 = (y == 1).dot(x) / np.sum(y == 1)
+        mu_yi = np.where(np.expand_dims(y == 0, -1),
+                         np.expand_dims(mu_0, 0),
+                         np.expand_dims(mu_1, 0))
+        sigma = 1 / n_examples * (x - mu_yi).T.dot(x - mu_yi)
+
+        # Write theta in terms of the parameters
+        self.theta = np.zeros(dim + 1)
+        sigma_inv = np.linalg.inv(sigma)
+        mu_diff = mu_0.T.dot(sigma_inv).dot(mu_0) - mu_1.T.dot(sigma_inv).dot(mu_1)
+        self.theta[0] = 1 / 2 * mu_diff - np.log((1 - phi) / phi)
+        self.theta[1:] = -sigma_inv.dot(mu_0 - mu_1)
+
+        # *** END CODE HERE ***
+
+    def predict(self, x):
+        """Make a prediction given new inputs x.
+
+        Args:
+            x: Inputs of shape (n_examples, dim).
+
+        Returns:
+            Outputs of shape (n_examples,).
+        """
+        # *** START CODE HERE ***
+        prediction = self.sigmoid(x)
+        return prediction 
+        # *** END CODE HERE
+
+def main_posonly(train_path, valid_path, test_path, save_path):
+    """Problem 2: Logistic regression for incomplete, positive-only labels.
+
+    Run under the following conditions:
+        1. on t-labels,
+        2. on y-labels,
+        3. on y-labels with correction factor alpha.
+
+    Args:
+        train_path: Path to CSV file containing training set.
+        valid_path: Path to CSV file containing validation set.
+        test_path: Path to CSV file containing test set.
+        save_path: Path to save predictions.
+    
+    NOTE: You need to complete logreg implementation first (see class above)!!!
+    """
+    output_path_true = save_path.replace(WILDCARD, 'true')
+    output_path_naive = save_path.replace(WILDCARD, 'naive')
+    output_path_adjusted = save_path.replace(WILDCARD, 'adjusted')
+
+    plot_path = save_path.replace('.txt', '.png')
+    plot_path_true = plot_path.replace(WILDCARD, 'true')
+    plot_path_naive = plot_path.replace(WILDCARD, 'naive')
+    plot_path_adjusted = plot_path.replace(WILDCARD, 'adjusted')
+
+    # Problem (2a): Train and test on true labels (t)
+    full_predictions = fully_observed_predictions(train_path, test_path, output_path_true, plot_path_true)
+
+    # Problem (2b): Train on y-labels and test on true labels
+    naive_predictions, clf = naive_partial_labels_predictions(train_path, test_path, output_path_naive, plot_path_naive)
+
+    # Problem (2f): Apply correction factor using validation set and test on true labels
+    alpha = find_alpha_and_plot_correction(clf, valid_path, test_path, output_path_adjusted, plot_path_adjusted, naive_predictions)
+
+    return
+
+def fully_observed_predictions(train_path, test_path, output_path_true, plot_path_true):
+    """
+    Problem (2a): Fully Observable Binary Classification Helper Function
+
+    Args:
+        train_path: Path to CSV file containing dataset for training.
+        test_path: Path to CSV file containing dataset for testing.
+        output_path_true: Path to save observed predictions
+        plot_path_true: Path to save the plot using plot_posonly util function
+    Return:
+        full_predictions: tensor of predictions returned from applied LogReg classifier prediction
+    """
+    full_predictions = None
+    # Problem (2a): Train and test on true labels (t)
+    # Make sure to save predicted probabilities to output_path_true using np.savetxt()
+    # *** START CODE HERE ***
+    x_train, t_train = util.load_dataset(train_path, label_col='t',
+                                         add_intercept=True)
+    clf = LogisticRegression()
+    clf.fit(x_train, t_train)
+
+    x_test, t_test = util.load_dataset(test_path, label_col='t',
+                                       add_intercept=True)
+
+    full_predictions = clf.predict(x_test)
+    np.savetxt(output_path_true, full_predictions)
+    util.plot(x_test, t_test, clf.theta, plot_path_true)
+    # *** END CODE HERE ***
+    return full_predictions
+
+def naive_partial_labels_predictions(train_path, test_path, output_path_naive, plot_path_naive):
+    """
+    Problem (2b): Naive Partial Labels Binary Classification Helper Function
+
+    Args:
+        train_path: Path to CSV file containing dataset for training.
+        test_path: Path to CSV file containing dataset for testing.
+        output_path_naive: Path to save observed predictions
+        plot_path_naive: Path to save the plot using plot_posonly util function
+    Return:
+        naive_predictions: tensor of predictions returned from applied LogReg prediction
+        clf: Logistic Regression classifier (will be reused for 2f)
+    """
+    naive_predictions = None
+    clf = None
+    # Problem (2b): Train on y-labels and test on true labels
+    # Make sure to save predicted probabilities to output_path_naive using np.savetxt()
+    # *** START CODE HERE ***
+    x_train, y_train = util.load_dataset(train_path, label_col='y',
+                                         add_intercept=True)
+    clf = LogisticRegression()
+    clf.fit(x_train, y_train)
+    x_test, t_test = util.load_dataset(test_path, label_col='t',
+                                       add_intercept=True)
+    naive_predictions = clf.predict(x_test)
+    np.savetxt(output_path_naive, naive_predictions)
+    util.plot(x_test, t_test, clf.theta, plot_path_naive)
+    # *** END CODE HERE ***
+    return naive_predictions, clf
+
+def find_alpha_and_plot_correction(clf, valid_path, test_path, output_path_adjusted, plot_path_adjusted, naive_predictions):
+    """
+    Problem (2f): Alpha Correction Binary Classification Helper Function
+
+    Args:
+        clf: Logistic regression classifier from part 2b
+        valid_path: Path to CSV file containing dataset for validation.
+        test_path: Path to CSV file containing dataset for testing.
+        output_path_adjusted: Path to save observed predictions
+        plot_path_adjusted: Path to save the plot using plot_posonly util function
+        naive_predictions: tensor of predictions returned from applied LogReg prediction from 2b
+    Return:
+        alpha: corrected alpha value
+    """
+    alpha = None
+    # Problem (2f): Apply correction factor using validation set and test on true labels
+    # Plot and use np.savetxt to save outputs to output_path_adjusted
+    # *** START CODE HERE ***
+    x_valid, y_valid = util.load_dataset(valid_path, label_col='y')
+    x_valid = x_valid[y_valid == 1, :]  # Restrict to just the labeled examples
+    x_valid = util.add_intercept(x_valid)
+    y_pred = clf.predict(x_valid)
+    alpha = np.mean(y_pred)
+    print('Found alpha = {}'.format(alpha))
+    x_test, t_test = util.load_dataset(test_path, label_col='t',
+                                       add_intercept=True)
+
+    # Plot and use np.savetxt to save outputs to output_path_adjusted
+    np.savetxt(output_path_adjusted, naive_predictions / alpha)
+    util.plot(x_test, t_test, clf.theta, plot_path_adjusted, correction=alpha)
+    # *** END CODE HERE ***
+    return alpha
+
+if __name__ == '__main__':
+    '''
+    Start of Problem 1: Linear Classifiers
+    '''
+    # 1b
+    main_LogReg(train_path='ds1_train.csv',
+         valid_path='ds1_valid.csv',
+         save_path='logreg_pred_1.txt')
+    main_LogReg(train_path='ds2_train.csv',
+         valid_path='ds2_valid.csv',
+         save_path='logreg_pred_2.txt')
+    # 1e
+    main_GDA(train_path='ds1_train.csv',
+         valid_path='ds1_valid.csv',
+         save_path='gda_pred_1.txt')
+    main_GDA(train_path='ds2_train.csv',
+         valid_path='ds2_valid.csv',
+         save_path='gda_pred_2.txt')
+    
+    '''
+    Start of Problem 2: Incomplete, Positive-Only Labels
+    '''
+    main_posonly(train_path='train.csv',
+        valid_path='valid.csv',
+        test_path='test.csv',
+        save_path='posonly_X_pred.txt')

code/pytorch_m1_testing.ipynb

@@ -0,0 +1,659 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "something\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(\"something\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "tensor([[0.8833, 0.1793, 0.9218],\n",
+      "        [0.8408, 0.2123, 0.5323],\n",
+      "        [0.5581, 0.2310, 0.7946],\n",
+      "        [0.8700, 0.1769, 0.7497],\n",
+      "        [0.1971, 0.3898, 0.8916]])\n"
+     ]
+    }
+   ],
+   "source": [
+    "import torch\n",
+    "x = torch.rand(5, 3)\n",
+    "print(x)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [
+    {
+     "ename": "ImportError",
+     "evalue": "cannot import name 'batched_dot_mul_sum' from '__main__' (unknown location)",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mImportError\u001b[0m                               Traceback (most recent call last)",
+      "\u001b[0;32m/var/folders/4k/y4ljh2217c57vl68z1zkl0440000gn/T/ipykernel_49379/927675702.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     11\u001b[0m     globals={'x': x})\n\u001b[1;32m     12\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 13\u001b[0;31m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mt0\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtimeit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m100\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     14\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mt1\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtimeit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m100\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
+      "\u001b[0;32m~/miniforge3/envs/pytorch_m1/lib/python3.8/site-packages/torch/utils/benchmark/utils/timer.py\u001b[0m in \u001b[0;36mtimeit\u001b[0;34m(self, number)\u001b[0m\n\u001b[1;32m    259\u001b[0m         \u001b[0;32mwith\u001b[0m \u001b[0mcommon\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mset_torch_threads\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_task_spec\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnum_threads\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    260\u001b[0m             \u001b[0;31m# Warmup\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 261\u001b[0;31m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_timer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtimeit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnumber\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnumber\u001b[0m \u001b[0;34m//\u001b[0m \u001b[0;36m100\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    262\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    263\u001b[0m             return common.Measurement(\n",
+      "\u001b[0;32m~/miniforge3/envs/pytorch_m1/lib/python3.8/timeit.py\u001b[0m in \u001b[0;36mtimeit\u001b[0;34m(self, number)\u001b[0m\n\u001b[1;32m    175\u001b[0m         \u001b[0mgc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdisable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    176\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 177\u001b[0;31m             \u001b[0mtiming\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minner\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mit\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtimer\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    178\u001b[0m         \u001b[0;32mfinally\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    179\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mgcold\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
+      "\u001b[0;32m<timeit-src>\u001b[0m in \u001b[0;36minner\u001b[0;34m(_it, _timer)\u001b[0m\n",
+      "\u001b[0;31mImportError\u001b[0m: cannot import name 'batched_dot_mul_sum' from '__main__' (unknown location)"
+     ]
+    }
+   ],
+   "source": [
+    "import torch.utils.benchmark as benchmark\n",
+    "\n",
+    "t0 = benchmark.Timer(\n",
+    "    stmt='batched_dot_mul_sum(x, x)',\n",
+    "    setup='from __main__ import batched_dot_mul_sum',\n",
+    "    globals={'x': x})\n",
+    "\n",
+    "t1 = benchmark.Timer(\n",
+    "    stmt='batched_dot_bmm(x, x)',\n",
+    "    setup='from __main__ import batched_dot_bmm',\n",
+    "    globals={'x': x})\n",
+    "\n",
+    "print(t0.timeit(100))\n",
+    "print(t1.timeit(100))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "# packages in environment at /Users/johnnydevriese/miniconda3:\n",
+      "#\n",
+      "# Name                    Version                   Build  Channel\n",
+      "aom                       3.2.0                he49afe7_2    conda-forge\n",
+      "appnope                   0.1.2            py39h6e9494a_2    conda-forge\n",
+      "backcall                  0.2.0              pyh9f0ad1d_0    conda-forge\n",
+      "backports                 1.0                        py_2    conda-forge\n",
+      "backports.functools_lru_cache 1.6.4              pyhd8ed1ab_0    conda-forge\n",
+      "blas                      1.0                         mkl  \n",
+      "brotlipy                  0.7.0           py39h89e85a6_1003    conda-forge\n",
+      "bzip2                     1.0.8                h0d85af4_4    conda-forge\n",
+      "ca-certificates           2021.10.8            h033912b_0    conda-forge\n",
+      "certifi                   2021.10.8        py39h6e9494a_1    conda-forge\n",
+      "cffi                      1.15.0           py39he338e87_0    conda-forge\n",
+      "chardet                   4.0.0            py39h6e9494a_2    conda-forge\n",
+      "charset-normalizer        2.0.0              pyhd8ed1ab_0    conda-forge\n",
+      "colorama                  0.4.4              pyh9f0ad1d_0    conda-forge\n",
+      "conda                     4.10.3           py39h6e9494a_3    conda-forge\n",
+      "conda-package-handling    1.7.3            py39h89e85a6_1    conda-forge\n",
+      "cryptography              35.0.0           py39h209aa08_2    conda-forge\n",
+      "debugpy                   1.5.1            py39h9fcab8e_0    conda-forge\n",
+      "decorator                 5.1.0              pyhd8ed1ab_0    conda-forge\n",
+      "entrypoints               0.3             pyhd8ed1ab_1003    conda-forge\n",
+      "ffmpeg                    4.4.1                h79e7b16_0    conda-forge\n",
+      "freetype                  2.10.4               h4cff582_1    conda-forge\n",
+      "gettext                   0.19.8.1          hd1a6beb_1008    conda-forge\n",
+      "gmp                       6.2.1                h2e338ed_0    conda-forge\n",
+      "gnutls                    3.6.13               h756fd2b_1    conda-forge\n",
+      "icu                       69.1                 he49afe7_0    conda-forge\n",
+      "idna                      3.1                pyhd3deb0d_0    conda-forge\n",
+      "ipykernel                 6.5.0            py39h71a6800_1    conda-forge\n",
+      "ipython                   7.29.0           py39h71a6800_2    conda-forge\n",
+      "jbig                      2.1               h0d85af4_2003    conda-forge\n",
+      "jedi                      0.18.1           py39h6e9494a_0    conda-forge\n",
+      "jpeg                      9d                   hbcb3906_0    conda-forge\n",
+      "jupyter_client            7.0.6              pyhd8ed1ab_0    conda-forge\n",
+      "jupyter_core              4.9.1            py39h6e9494a_1    conda-forge\n",
+      "lame                      3.100             h35c211d_1001    conda-forge\n",
+      "lcms2                     2.12                 h577c468_0    conda-forge\n",
+      "lerc                      3.0                  he49afe7_0    conda-forge\n",
+      "libcxx                    12.0.1               habf9029_0    conda-forge\n",
+      "libdeflate                1.8                  h0d85af4_0    conda-forge\n",
+      "libffi                    3.4.2                h0d85af4_5    conda-forge\n",
+      "libiconv                  1.16                 haf1e3a3_0    conda-forge\n",
+      "libpng                    1.6.37               h7cec526_2    conda-forge\n",
+      "libsodium                 1.0.18               hbcb3906_1    conda-forge\n",
+      "libtiff                   4.3.0                hd146c10_2    conda-forge\n",
+      "libuv                     1.42.0               h0d85af4_0    conda-forge\n",
+      "libvpx                    1.11.0               he49afe7_3    conda-forge\n",
+      "libwebp-base              1.2.1                h0d85af4_0    conda-forge\n",
+      "libxml2                   2.9.12               h7e28ab6_1    conda-forge\n",
+      "libzlib                   1.2.11            h9173be1_1013    conda-forge\n",
+      "llvm-openmp               12.0.1               hda6cdc1_1    conda-forge\n",
+      "lz4-c                     1.9.3                he49afe7_1    conda-forge\n",
+      "matplotlib-inline         0.1.3              pyhd8ed1ab_0    conda-forge\n",
+      "mkl                       2021.4.0           h89fa619_689    conda-forge\n",
+      "mkl-service               2.4.0            py39h89e85a6_0    conda-forge\n",
+      "mkl_fft                   1.3.1            py39h7ae3660_1    conda-forge\n",
+      "mkl_random                1.2.2            py39h4d6be9b_0    conda-forge\n",
+      "ncurses                   6.2                  h2e338ed_4    conda-forge\n",
+      "nest-asyncio              1.5.1              pyhd8ed1ab_0    conda-forge\n",
+      "nettle                    3.6                  hedd7734_0    conda-forge\n",
+      "numpy                     1.21.2           py39h4b4dc7a_0  \n",
+      "numpy-base                1.21.2           py39he0bd621_0  \n",
+      "olefile                   0.46               pyh9f0ad1d_1    conda-forge\n",
+      "openh264                  2.1.1                hfd3ada9_0    conda-forge\n",
+      "openjpeg                  2.4.0                h6e7aa92_1    conda-forge\n",
+      "openssl                   1.1.1l               h0d85af4_0    conda-forge\n",
+      "parso                     0.8.2              pyhd8ed1ab_0    conda-forge\n",
+      "pexpect                   4.8.0              pyh9f0ad1d_2    conda-forge\n",
+      "pickleshare               0.7.5                   py_1003    conda-forge\n",
+      "pillow                    8.4.0            py39he9bb72f_0    conda-forge\n",
+      "pip                       21.3.1             pyhd8ed1ab_0    conda-forge\n",
+      "prompt-toolkit            3.0.22             pyha770c72_0    conda-forge\n",
+      "ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge\n",
+      "pycosat                   0.6.3           py39h89e85a6_1009    conda-forge\n",
+      "pycparser                 2.21               pyhd8ed1ab_0    conda-forge\n",
+      "pygments                  2.10.0             pyhd8ed1ab_0    conda-forge\n",
+      "pyopenssl                 21.0.0             pyhd8ed1ab_0    conda-forge\n",
+      "pysocks                   1.7.1            py39h6e9494a_4    conda-forge\n",
+      "python                    3.9.7           h1248fe1_3_cpython    conda-forge\n",
+      "python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge\n",
+      "python.app                3                py39h9ed2024_0  \n",
+      "python_abi                3.9                      2_cp39    conda-forge\n",
+      "pytorch                   1.10.0                  py3.9_0    pytorch\n",
+      "pyzmq                     22.3.0           py39h7fec2f1_1    conda-forge\n",
+      "readline                  8.1                  h05e3726_0    conda-forge\n",
+      "requests                  2.26.0             pyhd8ed1ab_0    conda-forge\n",
+      "ruamel_yaml               0.15.80         py39h89e85a6_1006    conda-forge\n",
+      "setuptools                59.1.1           py39h6e9494a_0    conda-forge\n",
+      "six                       1.16.0             pyh6c4a22f_0    conda-forge\n",
+      "sqlite                    3.36.0               h23a322b_2    conda-forge\n",
+      "svt-av1                   0.8.7                he49afe7_1    conda-forge\n",
+      "tbb                       2021.4.0             h940c156_1    conda-forge\n",
+      "tk                        8.6.11               h5dbffcc_1    conda-forge\n",
+      "torchvision               0.11.1                 py39_cpu    pytorch\n",
+      "tornado                   6.1              py39h89e85a6_2    conda-forge\n",
+      "tqdm                      4.62.3             pyhd8ed1ab_0    conda-forge\n",
+      "traitlets                 5.1.1              pyhd8ed1ab_0    conda-forge\n",
+      "typing_extensions         4.0.0              pyha770c72_0    conda-forge\n",
+      "tzdata                    2021e                he74cb21_0    conda-forge\n",
+      "urllib3                   1.26.7             pyhd8ed1ab_0    conda-forge\n",
+      "wcwidth                   0.2.5              pyh9f0ad1d_2    conda-forge\n",
+      "wheel                     0.37.0             pyhd8ed1ab_1    conda-forge\n",
+      "x264                      1!161.3030           h0d85af4_1    conda-forge\n",
+      "x265                      3.5                  h940c156_1    conda-forge\n",
+      "xz                        5.2.5                haf1e3a3_1    conda-forge\n",
+      "yaml                      0.2.5                haf1e3a3_0    conda-forge\n",
+      "zeromq                    4.3.4                he49afe7_1    conda-forge\n",
+      "zlib                      1.2.11            h9173be1_1013    conda-forge\n",
+      "zstd                      1.5.0                h582d3a0_0    conda-forge\n"
+     ]
+    }
+   ],
+   "source": [
+    "! conda list"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "usage: ipykernel_launcher.py [-h] [--batch-size N] [--test-batch-size N]\n",
+      "                             [--epochs N] [--lr LR] [--gamma M] [--no-cuda]\n",
+      "                             [--dry-run] [--seed S] [--log-interval N]\n",
+      "                             [--save-model]\n",
+      "ipykernel_launcher.py: error: unrecognized arguments: -f /Users/johnnydevriese/Library/Jupyter/runtime/kernel-59506205-59ae-4a59-8704-3ff419da213d.json\n"
+     ]
+    },
+    {
+     "ename": "SystemExit",
+     "evalue": "2",
+     "output_type": "error",
+     "traceback": [
+      "An exception has occurred, use %tb to see the full traceback.\n",
+      "\u001b[0;31mSystemExit\u001b[0m\u001b[0;31m:\u001b[0m 2\n"
+     ]
+    }
+   ],
+   "source": [
+    "from __future__ import print_function\n",
+    "import argparse\n",
+    "import torch\n",
+    "import torch.nn as nn\n",
+    "import torch.nn.functional as F\n",
+    "import torch.optim as optim\n",
+    "from torchvision import datasets, transforms\n",
+    "from torch.optim.lr_scheduler import StepLR\n",
+    "\n",
+    "\n",
+    "class Net(nn.Module):\n",
+    "    def __init__(self):\n",
+    "        super(Net, self).__init__()\n",
+    "        self.conv1 = nn.Conv2d(1, 32, 3, 1)\n",
+    "        self.conv2 = nn.Conv2d(32, 64, 3, 1)\n",
+    "        self.dropout1 = nn.Dropout(0.25)\n",
+    "        self.dropout2 = nn.Dropout(0.5)\n",
+    "        self.fc1 = nn.Linear(9216, 128)\n",
+    "        self.fc2 = nn.Linear(128, 10)\n",
+    "\n",
+    "    def forward(self, x):\n",
+    "        x = self.conv1(x)\n",
+    "        x = F.relu(x)\n",
+    "        x = self.conv2(x)\n",
+    "        x = F.relu(x)\n",
+    "        x = F.max_pool2d(x, 2)\n",
+    "        x = self.dropout1(x)\n",
+    "        x = torch.flatten(x, 1)\n",
+    "        x = self.fc1(x)\n",
+    "        x = F.relu(x)\n",
+    "        x = self.dropout2(x)\n",
+    "        x = self.fc2(x)\n",
+    "        output = F.log_softmax(x, dim=1)\n",
+    "        return output\n",
+    "\n",
+    "\n",
+    "def train(args, model, device, train_loader, optimizer, epoch):\n",
+    "    model.train()\n",
+    "    for batch_idx, (data, target) in enumerate(train_loader):\n",
+    "        data, target = data.to(device), target.to(device)\n",
+    "        optimizer.zero_grad()\n",
+    "        output = model(data)\n",
+    "        loss = F.nll_loss(output, target)\n",
+    "        loss.backward()\n",
+    "        optimizer.step()\n",
+    "        if batch_idx % args.log_interval == 0:\n",
+    "            print('Train Epoch: {} [{}/{} ({:.0f}%)]\\tLoss: {:.6f}'.format(\n",
+    "                epoch, batch_idx * len(data), len(train_loader.dataset),\n",
+    "                100. * batch_idx / len(train_loader), loss.item()))\n",
+    "            if args.dry_run:\n",
+    "                break\n",
+    "\n",
+    "\n",
+    "def test(model, device, test_loader):\n",
+    "    model.eval()\n",
+    "    test_loss = 0\n",
+    "    correct = 0\n",
+    "    with torch.no_grad():\n",
+    "        for data, target in test_loader:\n",
+    "            data, target = data.to(device), target.to(device)\n",
+    "            output = model(data)\n",
+    "            test_loss += F.nll_loss(output, target, reduction='sum').item()  # sum up batch loss\n",
+    "            pred = output.argmax(dim=1, keepdim=True)  # get the index of the max log-probability\n",
+    "            correct += pred.eq(target.view_as(pred)).sum().item()\n",
+    "\n",
+    "    test_loss /= len(test_loader.dataset)\n",
+    "\n",
+    "    print('\\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\\n'.format(\n",
+    "        test_loss, correct, len(test_loader.dataset),\n",
+    "        100. * correct / len(test_loader.dataset)))\n",
+    "\n",
+    "\n",
+    "def main():\n",
+    "    # Training settings\n",
+    "    parser = argparse.ArgumentParser(description='PyTorch MNIST Example')\n",
+    "    parser.add_argument('--batch-size', type=int, default=64, metavar='N',\n",
+    "                        help='input batch size for training (default: 64)')\n",
+    "    parser.add_argument('--test-batch-size', type=int, default=1000, metavar='N',\n",
+    "                        help='input batch size for testing (default: 1000)')\n",
+    "    parser.add_argument('--epochs', type=int, default=14, metavar='N',\n",
+    "                        help='number of epochs to train (default: 14)')\n",
+    "    parser.add_argument('--lr', type=float, default=1.0, metavar='LR',\n",
+    "                        help='learning rate (default: 1.0)')\n",
+    "    parser.add_argument('--gamma', type=float, default=0.7, metavar='M',\n",
+    "                        help='Learning rate step gamma (default: 0.7)')\n",
+    "    parser.add_argument('--no-cuda', action='store_true', default=False,\n",
+    "                        help='disables CUDA training')\n",
+    "    parser.add_argument('--dry-run', action='store_true', default=False,\n",
+    "                        help='quickly check a single pass')\n",
+    "    parser.add_argument('--seed', type=int, default=1, metavar='S',\n",
+    "                        help='random seed (default: 1)')\n",
+    "    parser.add_argument('--log-interval', type=int, default=10, metavar='N',\n",
+    "                        help='how many batches to wait before logging training status')\n",
+    "    parser.add_argument('--save-model', action='store_true', default=False,\n",
+    "                        help='For Saving the current Model')\n",
+    "    args = parser.parse_args()\n",
+    "    use_cuda = not args.no_cuda and torch.cuda.is_available()\n",
+    "\n",
+    "    torch.manual_seed(args.seed)\n",
+    "\n",
+    "    device = torch.device(\"cuda\" if use_cuda else \"cpu\")\n",
+    "\n",
+    "    train_kwargs = {'batch_size': args.batch_size}\n",
+    "    test_kwargs = {'batch_size': args.test_batch_size}\n",
+    "    if use_cuda:\n",
+    "        cuda_kwargs = {'num_workers': 1,\n",
+    "                       'pin_memory': True,\n",
+    "                       'shuffle': True}\n",
+    "        train_kwargs.update(cuda_kwargs)\n",
+    "        test_kwargs.update(cuda_kwargs)\n",
+    "\n",
+    "    transform=transforms.Compose([\n",
+    "        transforms.ToTensor(),\n",
+    "        transforms.Normalize((0.1307,), (0.3081,))\n",
+    "        ])\n",
+    "    dataset1 = datasets.MNIST('../data', train=True, download=True,\n",
+    "                       transform=transform)\n",
+    "    dataset2 = datasets.MNIST('../data', train=False,\n",
+    "                       transform=transform)\n",
+    "    train_loader = torch.utils.data.DataLoader(dataset1,**train_kwargs)\n",
+    "    test_loader = torch.utils.data.DataLoader(dataset2, **test_kwargs)\n",
+    "\n",
+    "    model = Net().to(device)\n",
+    "    optimizer = optim.Adadelta(model.parameters(), lr=args.lr)\n",
+    "\n",
+    "    scheduler = StepLR(optimizer, step_size=1, gamma=args.gamma)\n",
+    "    for epoch in range(1, args.epochs + 1):\n",
+    "        train(args, model, device, train_loader, optimizer, epoch)\n",
+    "        test(model, device, test_loader)\n",
+    "        scheduler.step()\n",
+    "\n",
+    "    if args.save_model:\n",
+    "        torch.save(model.state_dict(), \"mnist_cnn.pt\")\n",
+    "\n",
+    "\n",
+    "main()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Loss: 0.000393 after 405 batches\n",
+      "==> Learned function:\ty = -9.48 x^1 +4.39 x^2 -1.07 x^3 -3.53 x^4 +3.16\n",
+      "==> Actual function:\ty = -9.52 x^1 +4.48 x^2 -1.06 x^3 -3.55 x^4 +3.13\n"
+     ]
+    }
+   ],
+   "source": [
+    "#!/usr/bin/env python\n",
+    "from __future__ import print_function\n",
+    "from itertools import count\n",
+    "\n",
+    "import torch\n",
+    "import torch.nn.functional as F\n",
+    "\n",
+    "POLY_DEGREE = 4\n",
+    "W_target = torch.randn(POLY_DEGREE, 1) * 5\n",
+    "b_target = torch.randn(1) * 5\n",
+    "\n",
+    "\n",
+    "def make_features(x):\n",
+    "    \"\"\"Builds features i.e. a matrix with columns [x, x^2, x^3, x^4].\"\"\"\n",
+    "    x = x.unsqueeze(1)\n",
+    "    return torch.cat([x ** i for i in range(1, POLY_DEGREE+1)], 1)\n",
+    "\n",
+    "\n",
+    "def f(x):\n",
+    "    \"\"\"Approximated function.\"\"\"\n",
+    "    return x.mm(W_target) + b_target.item()\n",
+    "\n",
+    "\n",
+    "def poly_desc(W, b):\n",
+    "    \"\"\"Creates a string description of a polynomial.\"\"\"\n",
+    "    result = 'y = '\n",
+    "    for i, w in enumerate(W):\n",
+    "        result += '{:+.2f} x^{} '.format(w, i + 1)\n",
+    "    result += '{:+.2f}'.format(b[0])\n",
+    "    return result\n",
+    "\n",
+    "\n",
+    "def get_batch(batch_size=32):\n",
+    "    \"\"\"Builds a batch i.e. (x, f(x)) pair.\"\"\"\n",
+    "    random = torch.randn(batch_size)\n",
+    "    x = make_features(random)\n",
+    "    y = f(x)\n",
+    "    return x, y\n",
+    "\n",
+    "\n",
+    "# Define model\n",
+    "fc = torch.nn.Linear(W_target.size(0), 1)\n",
+    "\n",
+    "for batch_idx in count(1):\n",
+    "    # Get data\n",
+    "    batch_x, batch_y = get_batch()\n",
+    "\n",
+    "    # Reset gradients\n",
+    "    fc.zero_grad()\n",
+    "\n",
+    "    # Forward pass\n",
+    "    output = F.smooth_l1_loss(fc(batch_x), batch_y)\n",
+    "    loss = output.item()\n",
+    "\n",
+    "    # Backward pass\n",
+    "    output.backward()\n",
+    "\n",
+    "    # Apply gradients\n",
+    "    for param in fc.parameters():\n",
+    "        param.data.add_(-0.1 * param.grad)\n",
+    "\n",
+    "    # Stop criterion\n",
+    "    if loss < 1e-3:\n",
+    "        break\n",
+    "\n",
+    "print('Loss: {:.6f} after {} batches'.format(loss, batch_idx))\n",
+    "print('==> Learned function:\\t' + poly_desc(fc.weight.view(-1), fc.bias))\n",
+    "print('==> Actual function:\\t' + poly_desc(W_target.view(-1), b_target))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "tensor([[0.9464, 0.6891],\n",
+      "        [0.1501, 0.2989],\n",
+      "        [0.6019, 0.5568],\n",
+      "        [0.8334, 0.2827],\n",
+      "        [0.1098, 0.2141],\n",
+      "        [0.0985, 0.8353],\n",
+      "        [0.1616, 0.3116],\n",
+      "        [0.2264, 0.0013],\n",
+      "        [0.3426, 0.7077],\n",
+      "        [0.1323, 0.4294]])\n"
+     ]
+    }
+   ],
+   "source": [
+    "import torch\n",
+    "x = torch.rand(10, 2)\n",
+    "print(x)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "5.95 µs ± 11.5 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)\n"
+     ]
+    }
+   ],
+   "source": [
+    "%%timeit\n",
+    "# Calculate the projection matrix of x on the CPU\n",
+    "H = x.mm( (x.t().mm(x)).inverse() ).mm(x.t())"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import math\n",
+    "\n",
+    "WGS84A = 6378137.0\n",
+    "WGS84F = 1.0 / 298.257223563\n",
+    "WGS84B = WGS84A - WGS84F * WGS84A\n",
+    "\n",
+    "x = 652954.1006\n",
+    "y = 4774619.7919\n",
+    "z = -2217647.7937\n",
+    "\n",
+    "\n",
+    "\n",
+    "def ecef2GeodeticJohnny(x, y, z, a, b):\n",
+    "    e2 = (a*a - b*b) / (a*a) # first eccentricity squared\n",
+    "    d = (a*a - b*b) / b\n",
+    "    \n",
+    "    # p2 = np.square(x) + np.square(y)\n",
+    "    p2 = x * x + y * y\n",
+    "    p = p2 * p2 \n",
+    "    r = math.sqrt(p2 + z*z)\n",
+    "    tu = b*z*(1 + d/r)/(a*p)\n",
+    "    tu2 = tu*tu\n",
+    "    cu3 = (1/math.sqrt(1 + tu2))**3\n",
+    "    su3 = cu3*tu2*tu\n",
+    "    tp = (z + d*su3)/(p - e2*a*cu3)\n",
+    "    lat = math.atan(tp)\n",
+    "    \n",
+    "    lon = math.atan2(y,x)\n",
+    "    \n",
+    "    return lat, lon"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "662 ns ± 4.45 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\n"
+     ]
+    }
+   ],
+   "source": [
+    "%%timeit\n",
+    "ecef2GeodeticJohnny(x, y, z, WGS84A, WGS84B)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# running pytorch on m1 gpu \n",
+    "\n",
+    "https://pytorch.org/docs/stable/notes/mps.html"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/johnnydevriese/miniforge3/envs/pytorch-nightly/lib/python3.10/site-packages/torch/_tensor_str.py:103: UserWarning: The operator 'aten::bitwise_and.Tensor_out' is not currently supported on the MPS backend and will fall back to run on the CPU. This may have performance implications. (Triggered internally at /Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/mps/MPSFallback.mm:11.)\n",
+      "  nonzero_finite_vals = torch.masked_select(tensor_view, torch.isfinite(tensor_view) & tensor_view.ne(0))\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "tensor([1., 1., 1., 1., 1.], device='mps:0')\n"
+     ]
+    }
+   ],
+   "source": [
+    "import torch\n",
+    "\n",
+    "\n",
+    "# Check that MPS is available\n",
+    "if not torch.backends.mps.is_available():\n",
+    "    if not torch.backends.mps.is_built():\n",
+    "        print(\"MPS not available because the current PyTorch install was not \"\n",
+    "              \"built with MPS enabled.\")\n",
+    "    else:\n",
+    "        print(\"MPS not available because the current MacOS version is not 12.3+ \"\n",
+    "              \"and/or you do not have an MPS-enabled device on this machine.\")\n",
+    "\n",
+    "else:\n",
+    "    mps_device = torch.device(\"mps\")\n",
+    "\n",
+    "    # Create a Tensor directly on the mps device\n",
+    "    x = torch.ones(5, device=mps_device)\n",
+    "    # Or\n",
+    "    # x = torch.ones(5, device=\"mps\")\n",
+    "    print(x)\n",
+    "\n",
+    "    # # Any operation happens on the GPU\n",
+    "    # y = x * 2\n",
+    "\n",
+    "    # # Move your model to mps just like any other device\n",
+    "    # model = YourFavoriteNet()\n",
+    "    # model.to(mps_device)\n",
+    "\n",
+    "    # # Now every call runs on the GPU\n",
+    "    # pred = model(x)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3.10.5 ('pytorch-nightly')",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.5"
+  },
+  "orig_nbformat": 4,
+  "vscode": {
+   "interpreter": {
+    "hash": "8a8bcccfb183d1298694efece6cf41240378bc61621e95c864629a40c5876542"
+   }
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}