\lab{Data Visualization}{Data Visualization}
\label{lab:DataVis}
\objective{Correctly presenting and interpreting data with visualizations is a fundamental skills for all scientists.
In this lab we discuss and demonstrate the principles of good data visualization, including the key characteristics of good graphs, common mistakes, and how to avoid unethical visualizations.}

\section*{The Importance of Visualizations} % =================================

Using visualizations to understand data often reveals insights that may not immediately obvious from simple statistics.
The following data set, known as \emph{Anscombe's quartet}, is a famous example of the importance of graphing data.

% \begin{comment} % TODO: change the paragraph before this to account for taking the table out.
\begin{table}[H]
\small{
\begin{tabular}{rr|rr|rr|rr}

\multicolumn{2}{c|}{I}    & \multicolumn{2}{|c|}{II} &
\multicolumn{2}{|c|}{III} & \multicolumn{2}{|c}{IV} \\
\multicolumn{1}{c}{$x$}   & \multicolumn{1}{c|}{$y$} &
\multicolumn{1}{|c}{$x$}  & \multicolumn{1}{c|}{$y$} &
\multicolumn{1}{|c}{$x$}  & \multicolumn{1}{c|}{$y$} &
\multicolumn{1}{|c}{$x$}  & \multicolumn{1}{c}{$y$} \\
\hline
10.0 & 8.04  & 10.0 & 9.14 & 10.0 & 7.46  & 8.0  & 6.58  \\
8.0  & 6.95  & 8.0  & 8.14 & 8.0  & 6.77  & 8.0  & 5.76  \\
13.0 & 7.58  & 13.0 & 8.74 & 13.0 & 12.74 & 8.0  & 7.71  \\
9.0  & 8.81  & 9.0  & 8.77 & 9.0  & 7.11  & 8.0  & 8.84  \\
11.0 & 8.33  & 11.0 & 9.26 & 11.0 & 7.81  & 8.0  & 8.47  \\
14.0 & 9.96  & 14.0 & 8.10 & 14.0 & 8.84  & 8.0  & 7.04  \\
6.0  & 7.24  & 6.0  & 6.13 & 6.0  & 6.08  & 8.0  & 5.25  \\
4.0  & 4.26  & 4.0  & 3.10 & 4.0  & 5.39  & 19.0 & 12.50 \\
12.0 & 10.84 & 12.0 & 9.13 & 12.0 & 8.15  & 8.0  & 5.56  \\
7.0  & 4.82  & 7.0  & 7.26 & 7.0  & 6.42  & 8.0  & 7.91  \\
5.0  & 5.68  & 5.0  & 4.74 & 5.0  & 5.73  & 8.0  & 6.89  \\
\end{tabular}}
\end{table}

Each section of Anscombe's quartet the following statistical properties:
\begin{itemize}
\setlength\itemsep{0em}
\item The means are $9$ for $x$ and $7.5$ for $y$.
\item The sample variances are $11$ for $x$ and $3.75$ for $y$.
\item The correlation between $x$ and $y$ is $.816$.
\item The linear least squares regression line is $y=\frac{1}{2}x+3$.
\end{itemize}

Despite these similarities, each section is quite different from the others.

\begin{problem} % Describe Anscombe's quartet.
The file \texttt{anscombe.npy} contains Anscombe's quartet in the same format as displayed above.
Plot each section of the quartet separately as a scatter plot.
Also plot the regression line $y = \frac{1}{2}x + 3$ on the domain $x\in[0,20]$ over each scatter plot.

Write a few sentences describing what makes each section unique.
% How would you effectivly describe each section to another person?
\label{prob:anscombes-quartet}
\end{problem}

\begin{comment}
\section*{Principles of Good Data Visualization} % ============================

Problem \ref{prob:anscombes-quartet} demonstrate how visualizations can describe data much more accurately than simple statistics.
However, a poor or unethical visualizations can easily misrepresent data.

\begin{enumerate}
    \item \textbf{Clarity}: Use titles, label axes, and include a legend. Graphs should be self-explanatory.
    Where appropriate, use words to carefully explain the insights represented
    in the visualization.
    \item \textbf{Simplicity}: Remove unnecessary elements of a graph.
    Don't add 3D effects in for no reason.
    \item \textbf{Integrity}. Tell the truth, the whole truth, and nothing but the truth.

    Every visualization should be presented alongside all relevant information including who created it, where the data was obtained, how it was collected, whether it was cleaned or transformed, and whether there are conflicts of interest or possible biases present.

    It is almost always very easy to manipulate a visualization so that it supports a desired interpretation.

    As in any form of communication, integrity is important in data visualization.

\end{enumerate}

\end{comment}

\section*{Choosing the Right Visualization} % =================================

Most data sets are more naturally described with one type of visualization than another.
Below we review several common visualizations, their standard usages, and questions to consider during their construction.

\small{ % Get this section to fit on one page.
\begin{itemize}
\item \textbf{Line Plot}: Connects ordered $(x,y)$ points with straight lines.

Used to visualize one or two ordered arrays, such as functional outputs over an ordered domain or a sequence of values over time.
%
\begin{itemize}
    % \item What do the axes represent? How should they be labeled?
    \item Are linear or logarithmic scales more effective?
    \item Is the line an appropriate thickness and color?
\end{itemize}

% Use \li{plt.plot()} to create a line plot.

\item \textbf{Scatter Plot}: Draws $(x,y)$ points without connecting them.

Best for displaying points without a natural order or where each point is an individual instance and not inherently related to the other points in the set.
%
\begin{itemize}
    % \item What do the axes represent? How should they be labeled?
    \item Which marker is most appropriate?
    \item Are the markers an appropriate size and color?
\end{itemize}

% Use \li{plt.plot()} with a point marker (\li{'.'}, \li{','}, \li{'o'}, etc.) or \li{plt.scatter()} to draw a scatter plot.

\begin{comment} % Window size, also should go later (with ethics).
Figure \ref{fig:scatter_correlation} displays two scatter plots.
The first appears to have a weak correlation and the second appears to have a strong correlation.
However, the same data is being plotted and the only difference is the scale and window size.
Manipulating these can change your interpretation and should be done with careful consideration.
\end{comment}

\item \textbf{Histogram}: Partitions an interval into a number of bins and counts the number of values that fall into each bin.

Best for visualizing a single unordered array, such as draws from a probability distribution.
%
\begin{itemize}
    \item How many bins should be used?
    \item Over what range should the bins be?
    \item Is a linear or a logarithmic scale more appropriate for the frequency axis?
\end{itemize}

% Use \li{plt.hist()} to create a histogram.
% The arguments \li{bins} and \li{<<range>>} specify the number of bins to draw and over what domain.

\begin{comment} % Details for later.
A histogram with only a few bins or too many bins will fail to give a clear view of the distribution of the data, therefore it is important to choose the number of bins that gives the best view of the distribution of the data.

Similarly, reference lines are distracting and focus on unnecessary details instead of the shape of the distribution.
If more precise details are desired, exact figures can be presented alongside the histogram in a table.
\end{comment}

\item \textbf{Heat Map}: Assigns a color to each point on a 2-D domain, providing a 2-D picture representing a 3-D surface.

Best for visualizing functions $f:\mathbb{R}^2\rightarrow\mathbb{R}$, usually over a domain constructed with \li{np.meshgrid()}.
%
\begin{itemize}
    \item Is the domain sufficiently refined?
    \item Which color scheme is most clear and effective?
    \item Is a linear or a logarithmic scale more appropriate for the color?
\end{itemize}

\item \textbf{Contour Map}: Draws level curves of a function $f:\mathbb{R}^2\rightarrow\mathbb{R}$.

A filled contour map is discretized version of a heat map.
%
\begin{itemize}
    \item How many / which contour lines should be drawn?
    % \item Is the color scheme clear and effective?
    \item Is a linear or a logarithmic scale more appropriate for the color?
\end{itemize}

\end{itemize}}
%
% \begin{table}[H]
% \begin{tabular}{rcc}
% & \textbf{One Array} & \textbf{Two Arrays} \\
% \cline{2-3}
% \textbf{Ordered Data} &
% \multicolumn{1}{|c|}{Line Plot} & \multicolumn{1}{|c|}{Line Plot} \\
% \cline{2-3}
% \textbf{Unordered Data} &
% \multicolumn{1}{|c|}{Histogram} & \multicolumn{1}{|c|}{Scatter Plot} \\
% \cline{2-3}
% % \textbf{Categorical Data} & \multicolumn{1}{|c|}{Bar Chart} & \multicolumn{1}{|c|}{Scatter Plot} \\ \cline{2-3}
% \end{tabular}
% \end{table}

\subsection*{Scatter Plots} % -------------------------------------------------

\begin{comment}
% Example of using a marker and a line.
Line plots and scatter plots are sometimes used together.
A line connecting points implies a relationship or correlation between the points, while drawing point markers emphasizes that each point is a unique instance of data.
Use \li{plt.plot()} and specify a marker \emph{and} a line style in the third argument to do both at once.
% The keywords \li{linewidth} (or \li{ls}) and \li{markersize} (or \li{ms}) control the thickness of the line and the size of the markers, respectively.

\begin{lstlisting}
>>> x = np.linspace(0, 4, 8)
>>> data = x**2

>>> plt.subplot(131)                # Regular line plot of the data.
>>> plt.plot(x, data, 'b-', linewidth=2)

>>> plt.subplot(132)                # Scatter plot of the data.
>>> plt.plot(x, data, 'g.', markersize=15)

>>> plt.subplot(133)                # Line and scatter plot.
>>> plt.plot(x, data, 'r.-', linewidth=2, markersize=15)
\end{lstlisting}
\end{comment}

% Explain plt.scatter(), especially the 'c' and 's' keyword arguments.
While \li{plt.plot()} is an easy way to create scatter plots, \li{plt.scatter()} provides a few extra tools.
The keywords \li{s} and \li{c} correspond to marker size and marker color, respectively.
Each keyword can either be a single entry, producing uniform markers, or an array, specifying the sizes or colors of the individual markers.
This allows a scatter plot to have up to four dimensions of information.

Consider again Anscombe's quartet.
The pattern in section II can be made more pronounced by tying the size of the marker to the value of the $y$ coordinate.
Since adjusting the size of the markers may lead to overlapping markers, we specify the \emph{alpha value} of the color to make the markers transparent.
The keyword \li{alpha} accepts a value in the interval $[0,1]$; 0 makes the markers completely transparent, while a $1$ makes the markers completely opaque.

\begin{lstlisting}
>>> anscombe = np.load("anscombe.npy")
>>> x2, y2 = anscombe[:,2], anscombe[:,3]

>>> plt.subplot(121)
>>> plt.scatter(x2, y2, c='b', s=500)

>>> plt.subplot(122)
>>> plt.scatter(x2, y2, c='b', s=.5*y**4, alpha=.8)

>>> plt.show()
\end{lstlisting}

\begin{figure}[H] % Different sizing in a scatter plot.
\captionsetup[subfigure]{justification=centering}
\centering
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/scatter_1_bad.pdf}
\end{subfigure}
%
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/scatter_1_good.pdf}
\end{subfigure}
\end{figure}

Controlling the colors of the individual markers can also be illuminating. % TODO: The goal is to present 3-dimensional data, not show patterns...
As with heat maps and contour plots, a color bar can be added to indicate the values that the colors represent.

\begin{lstlisting}
>>> x3, y3 = anscombe[:,4], anscombe[:,5]

>>> plt.subplot(121)
>>> plt.scatter(x3, y3, s=500, c='b')

>>> plt.subplot(122)
>>> plt.scatter(x, y, s=500, c=y, alpha=.8)
>>> cbar = plt.colorbar()           # Draw and label the color bar.
>>> cbar.set_label("y")
\end{lstlisting}

\begin{figure}[H] % Different coloring in a scatter plot.
\captionsetup[subfigure]{justification=centering}
\centering
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/scatter_2_bad.pdf}
\end{subfigure}
%
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/scatter_2_good.pdf}
\end{subfigure}
\end{figure}

% Talk about connecting the dots, then using a regression line instead.

\begin{problem} % Scatter plots with visualizations.
The file \texttt{MLB.npy} contains measurements from over 1,000 recent Major League Baseball players, compiled by UCLA.\footnote{See \url{http://wiki.stat.ucla.edu/socr/index.php/SOCR_Data_MLB_HeightsWeights}.}
Each row in the array represents a different player; the columns are the player's height (in inches), weight (in pounds), and age (in years), in that order.

Describe the data with at least one visualization.
Your graph(s) should demonstrate whether height, weight, or age correlated with each other in the MLB.
\end{problem}

\subsection*{Histograms} % ----------------------------------------------------

Line plots should \emph{not} be used for data without a natural progression.
For example, suppose we would like to make a large number of random draws from a distribution to visualize its probability density function.
In this case, a plain line plot is completely useless, but the histogram provides an accurate picture of the statistical distribution.

\begin{comment}
\begin{lstlisting}
>>> import numpy as np
>>> from matplotlib import pyplot as plt

# Get 10,000 random samples from the standard normal distribution.
>>> data = np.random.normal(size=10000)

>>> plt.subplot(121)                # Regular line plot of the data.
>>> plt.plot(data)

>>> plt.subplot(122)                # Histogram of the data.
>>> plt.hist(data, bins=30)

>>> plt.show()
\end{lstlisting}

\begin{figure}[H] % line plot versus a histogram.
\captionsetup[subfigure]{justification=centering}
\centering
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/hist_1_bad.pdf}
\end{subfigure}
%
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/hist_1_good.pdf}
\end{subfigure}
\end{figure}
\end{comment}

Less is more.
In the case of a statistical distribution, all we really want to see is the general shape of the distribution.
The lines separating the bins, axis tick marks, and even the labels on the $y$ axis are all unnecessary (and potentially distracting) details.

Removing the lines between bins is easy: set the line width to $0$.
To ensure the gaps where the lines used to be are filled, specify the \li{histtype} as \li{"stepfilled"} (using \li{histtype="step"} will draw the outline without filling it in).
Finally, to get rid of extra markings on the axes, use \li{plt.tick_params()}.
Some of this function's keyword arguments are summarized below.

\begin{table}[H]
\begin{tabular}{r|c|l}
    \multicolumn{1}{c|}{Argument} & Options & \multicolumn{1}{|c}{Description}
    \\ \hline
    \li{axis} & \li{'x'}, \li{'y'}, \li{"both"} & Axis on which to operate. \\
    \li{which} & \li{"major"}, \li{"minor"}, \li{"both"} & Operate on major or minor ticks.\\
    \li{color} & Any Matplotlib color & Tick color. \\
    \li{labelcolor} & Any Matplotlib color & Tick label color. \\
    \li{bottom}, \li{top}, \li{left}, \li{right} & \li{"on"}, \li{"off"} & Turn ticks on or off. \\
    \li{labelbottom}, \li{labeltop}, & \li{"on"}, \li{"off"} & Turn tick labels on or off. \\
    \li{labelleft}, \li{labelright} & &
\end{tabular}
\end{table}

\begin{lstlisting}
# Get 10,000 random samples from the standard exponential distribution.
>>> data = np.random.exponential(size=N)

>>> plt.subplot(121)
>>> plt.hist(data, bins=30)

>>> plt.subplot(122)                # Histogram with no lines or tick marks.
>>> plt.hist(data, bins=30, lw=0, histtype="stepfilled")
>>> plt.tick_params(axis="both", which="both", labelleft='off',
...                              left="off", top="off", right="off")

>>> plt.show()
\end{lstlisting}

\begin{figure}[H] % Cleaning up a histogram.
\captionsetup[subfigure]{justification=centering}
\centering
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/hist_2_bad.pdf}
\end{subfigure}
%
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/hist_2_good.pdf}
\end{subfigure}
\end{figure}

A histogram plots an array of values against the frequency with which the values occur.
To plot a line plot of values against frequency (without drawing a histogram), use \li{np.histogram()}.
This function returns the number of values in each bin and the locations of the edges of the bins.
Use the edges of the bins to calculate the center of the bins, then use a line plot to connect the two.

\begin{lstlisting}
# Get samples from a beta distribution and draw its PDF.
>>> for i, N in enumerate([10000, 10000000]):
...     plt.subplot(1,2,i+1)
...     data = np.random.beta(a=5, b=2, size=N)
...     # Draw a line plot using the bin centers and frequencies.
...     frequencies, bin_edges = np.histogram(data, bins=50)
...     bin_centers = (bin_edges[:-1] + bin_edges[1:])/2.
...     plt.plot(bin_centers, frequencies, 'b-', lw=4)
...     # Plot the faded histogram for comparison.
...     plt.hist(data, bins=50, alpha=.1)
...     plt.tick_params(axis="both", which="both", labelleft='off',
...                                  left="off", top="off", right="off")
...
>>> plt.show()
\end{lstlisting}

\begin{figure}[H] % Histogram Line Plot Hybrid
\captionsetup[subfigure]{justification=centering}
\centering
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/hist_3_bad.pdf}
\end{subfigure}
%
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/hist_3_good.pdf}
\end{subfigure}
\end{figure}

\begin{problem} % Earthquake data
The file \texttt{earthquakes.npy} contains data from the 17,065 earthquakes between 2000 and 2010 that were at least a 5 on the Richter scale.\footnote{See \url{http://earthquake.usgs.gov/earthquakes/search/}.}
Each row in the array represents a different earthquake;
the columns are the earthquake's date (as a fraction of the year), magnitude, longitude, and latitude, in that order.

Because each earthquake is a distinct event, a good way to start visualizing this data might be a scatter plot of the years versus the magnitudes of each earthquake.

\begin{lstlisting}
>>> year, magnitude, longitude, latitude = np.load("earthquakes.npy").T
>>> plt.plot(year, magnitude, '.')
>>> plt.xlabel("Year")
>>> plt.ylabel("Magnitude")
>>> plt.show()
\end{lstlisting}

\begin{figure}[H] % Bad visualization: earthquake data, year vs magnitude.
    \centering
    \includegraphics[width=.7\textwidth]{figures/earthquake.pdf}
\end{figure}

Unfortunately, this plot communicates very little information because the data is so cluttered.
Describe the data with two or three better visualizations.
Your graphs should clearly answer the following questions:
\begin{enumerate}
    \item How many earthquakes happened every year?
    \item How often do stronger earthquakes happen compared to weaker ones?
    \item Where do earthquakes happen? Where do the strongest earthquakes happen?
\end{enumerate}
% (Hint: \li{plt.gca().set_aspect("equal")} fixes the aspect ratio, which may improve comparisons between longitude and latitude.)
\end{problem}

% Consider Figure \ref{fig:logexample} which shows data about large earthquakes from the United States Geological Survey.
% It appears to be somewhat random however, when transformed, the frequency of the magnitudes on a logarithmic scale show a fairly strong linear relationship between the variables.

\subsection*{Heat and Contour Maps} % -----------------------------------------

\begin{figure}[h]
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{contour_map.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{contour_map_filled.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{contour_map_rainbow_filled.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{contour_map_rainbow.png}
\end{subfigure}
\caption{Contour Maps.  A contour map (top left) and a heat map (top right).
The color choices of the bottom row are less intuitive and make the visualization harder to interpret.}
\label{fig:contour}
\end{figure}

Three-dimensional data can be displayed with a contour plot.
This draws lines in the plane where the third value is constant---like a topographical map.
Coloring the space between the contour lines produces a heat map.
Heat maps work best when only one or two colors are used.

\begin{figure}[h]
\centering
\begin{subfigure}{.5\textwidth}
  \centering
  \includegraphics[width=\textwidth]{heatmap_color.png}
\end{subfigure}%
\begin{subfigure}{.5\textwidth}
  \centering
  \includegraphics[width=\textwidth]{heatmap_hot.png}
\end{subfigure}
%\includegraphics[width=\textwidth]{heatmap.png}
\caption{Both of these pseudocolor plots depict the function
    $z = \sin(x)\sin(y)$ on the domain $[-1,4] \times [-4,1]$.
The plot on the left uses a rainbow gradient, which has no meaningful relationship to the data.
The plot on the right uses the colormap \li{afmhot}.  The lighter and darker colors are a more intuitive
representation of higher and lower values.}

\label{fig:heatmap}
\end{figure}

% TODO: In the next release of Matplotlib (version 2.0) viridis is the default.
% TODO: Reword (awkward).
A good default colormap is Matplotlib's \li{viridis}.
Other colormaps that are appropriate for the data can be selected using the argument \li{cmap='NAME\_OF\_COLORMAP'}, where \li{NAME\_OF\_COLORMAP} is one of Matplotlib's sequential or diverging colormaps.
Two sequential colormaps are \li{afmhot} and \li{cool}. Two diverging colormaps are \li{seismic} and \li{bwr}.
These colormaps along with their color scheme can be found on
\url{http://matplotlib.org/examples/color/colormaps_reference.html}.
Although three-dimensional data can be plotted as a surface in 3-space, it is often difficult to find a view where none of the important features of the data are obscured.
A heatmap avoids this problem and is easier to understand.

You can create a contour plot in Matplotlib with \li{plt.contour()}.

You can also create a pseudocolor plot in Matplotlib with the \li{plt.pcolormesh()} command.

% # This code corresponds to the figure on the top right of the contour maps.
\begin{lstlisting}
import numpy as np
from matplotlib import pyplot as plt

n=400
xran = np.linspace(-1.5,1.5,n)
yran = np.linspace(-1.5,1.5,n)
X, Y = np.meshgrid(xran,yran)
F = Y**2 - X**3 + X**2
plt.contourf(X, Y, F, [-2,-1,0.0001,1,2,3,4,5] ,cmap=plt.get_cmap('afmhot'))
\end{lstlisting}

\begin{problem} % Contour map problem. TODO: don't specify number of intervals?
Using 400 intervals, plot a filled contour map of the function \[f(x,y) = \sin(x) + \sin(y)\] over the domain $[0,12\pi]\times[0,12\pi]$.

Be sure to use an appropriate color map.
\end{problem}

\section*{Communicating Insights with Visualizations} % =======================

\begin{comment} % USE THIS AS AN EXAMPLE LATER.
\begin{figure}[H] % Histogram with and without busy information.
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{good_hist_example.png}
\end{subfigure}
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{hist_reference_grid.png}
\end{subfigure}
\caption{Histogram with and without reference lines. It is harder to see the shape of the distribution in the histogram with reference lines.}
\label{fig:histogram}
\end{figure}

Plot two histograms with data drawn randomly from a normal and exponential distribution.

\begin{lstlisting}
from numpy.random import normal
from numpy.random import exponential
normal_data = normal(size=1000)
exp_data = exponential(size=1000)
\end{lstlisting}

For each distribution, plot a histogram using 30 bins, 15 bins and 5 bins.
Does this change how we interpret the distribution of the data?
\end{comment}

\begin{comment}
\begin{figure}[h] % Line plots.
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{line_plot_bad_thin.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{line_plot_good.png}
\end{subfigure}
\caption{Line Plot.  The left plot is plotted with \li{lw=1} whereas the right plot is plotted with \li{lw=2}.  Notice that the right plot is better because it is easier to see the line.} % Irrelevant for right here.
\label{fig:lineplot_bad}
\end{figure}

\begin{figure}[h]
\centering
\begin{subfigure}{.5\textwidth}
\centering
\includegraphics[width=\textwidth]{line_plot_bad_X.png}
\end{subfigure}
\caption{Line plot. Notice that this graph appears to show that the x-values are ordered. However, in reality, they are unordered. This is a bad practice and misrepresents the data.}
\label{fig:lineplotbadX}
\end{figure}
\end{comment}

\begin{comment} % MOVE BELOW.
\begin{figure}[h] % Choose an ethical scale and window.
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{scale_scatter_zoomed_in.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{scale_scatter_zoomed_out.png}
\end{subfigure}
\caption{Scatter Plots. The data in both plots are the same but there appears to be a weaker
correlation in the first plot compared to the second.}
\label{fig:scatter_correlation}
\end{figure}
\end{comment}

\begin{comment}
\begin{figure}[h]
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{earthquake_zoomed.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{earthquake_data.png}
\end{subfigure}
\caption{USGS Earthquake data. The scatter plot on the left only plots earthquakes with
magnitude 7.5 or greater from 1980-2010 and appears to have a correlation.
However, the plot on the right shows all recorded earthquakes from 1900-2010 and shows that the correlation
is not a true correlation.}
\label{fig:scatterquake}
\end{figure}

Scatter plots may reveal insights other than correlation.
Consider Figure \ref{fig:scatterquake} which plots earthquake data from the United States Geological Survey (USGS).
The USGS data began including earthquakes with magnitude 6.5 or greater; this is clearly seen
in the scatter plot of the entire dataset.
Plots like this may also reveal errors in the dataset that are harder to detect in a table of values.
\end{comment}

\begin{comment}
\begin{figure}[h] % Bad visualization: earthquake data, year vs magnitude.
\captionsetup[subfigure]{justification=centering}
\centering
\begin{subfigure}{.5\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/earthquake_bad.pdf}
\end{subfigure}%
\begin{subfigure}{.5\textwidth}
    \centering
    \includegraphics[width=\linewidth]{freq_earthquake_log.png}
\end{subfigure}
\caption{The left plot shows earthquakes of magnitudes 7.5 or higher over time. The right plot shows the frequencies of
different magnitudes. Note the scale of the y-axis is logarithmic.}
\label{fig:logexample}
\end{figure}
\end{comment}

\begin{comment} % This needs to go somewhere else.
Understanding a data set through visualizations is an iterative process.
Start with an initial visualization---usually a broad overview---and examine the data, then adjust the visualization or create other visualizations based on what you observe.
Ask the following questions as you search for insights:
\begin{enumerate}
\item Does the data make sense?
\item Would a different visualization communicate more information?
\item Would visualizing a subset of the data provide more information?
\item Would transforming the data reveal a hidden pattern?
\end{enumerate}
\end{comment}

As in any form of communication, integrity is important in data visualization.
Every visualization should be presented alongside all relevant information including who created it, where the data was obtained, how it was collected, whether it was cleaned or transformed, and whether there are conflicts of
interest or possible biases present.
Where appropriate, use words to carefully explain the insights represented
in the visualization.
These practices will help the reader correctly interpret the visualization.

\subsection*{Things to Avoid} % -----------------------------------------------

Some popular visualizations (pie charts, radar graphs, stacked bar charts and others) interfere with our ability to interpret the information they display.
For example, it is more difficult to detect differences in a pie chart than on a bar chart with the same data.
This is because differences in area are more difficult to detect than differences in length.
As a result, charts that depend on detecting variation in area will be less effective.
Instead, choose visualizations that compare the data to the a simple standard that is easy to understand and interpret.
Figures \ref{fig:pievsbar} and \ref{fig:stacked} are examples of poorly chosen visualizations.

\begin{figure} % Bad graphs.
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{bad_pie_chart.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{bar_chart_horizontal_sorted.png}
\end{subfigure}
\caption{A pie chart on the left that has been improved by a transformation into a bar chart on the right.}
\label{fig:pievsbar}
\end{figure}

\subsection*{Keep It Simple} % ------------------------------------------------

It may be tempting to stylize your visualization by adding icons and pictures or by making certain features
look 3D.  This is called chartjunk, a term coined by Edward Tufte. Chartjunk is anything in a plot that fails to add to or
misrepresents the data.  Avoid making visualizations that are overly fancy or cluttered with chartjunk. This
includes 3D effects (as in \ref{fig:pievsbar}), bad font choices, and unnecessary icons and pictures.
Visualizations are at their best when they are as simple as possible and allow the reader to easily
interpret the information being presented.

Reference lines can distract from the visualization (see Figure \ref{fig:histogram}).
For example, we can remove axis lines, tick marks, and bar lines
with the following code:

\begin{lstlisting}
# Get current axis instance
axis = plt.gca()

# Hide top and right spines
axis.spines['right'].set_visible(False)
axis.spines['top'].set_visible(False)

# Only show bottom and left tick marks
axis.yaxis.set_ticks_position('left')
axis.xaxis.set_ticks_position('bottom')
\end{lstlisting}

\begin{problem} % Problem: convert a pie chart to a horizontal bar chart.
Regraph the pie chart in Figure \ref{fig:pievsbar} as a horizontal bar chart using the follwing data.
Spam: 21, Eggs: 20, Ham: 22, Sausage: 21, Bacon: 20, Baked Beans: 11, Lobster Thermidor: 10
\end{problem}

% TODO: WHY IS THIS PROBLEM WAY DOWN HERE??? Shouldn't this be part of the histogram section?
\begin{problem} % Problem: histogram of earthquake magnitudes.
Plot a histogram of earthquake magnitudes using the data in the file \li{earthquake.txt}.
Remove reference lines, tick marks and bar lines.
Use the numpy function \li{np.loadtxt()} to read in the data.
Is the data distributed normally or exponentially?
\end{problem}

\begin{figure} % Stacked v unstacked bar charts
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{stackedbar.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{unstackedbar.png}
\end{subfigure}
\caption{Stacked Bar Chart. The relative values are harder to compare when stacked (left) instead of adjacent (right).}
\label{fig:stacked}
\end{figure}

\subsection*{Small Multiples} % -----------------------------------------------

When we have too much information to display cleanly in one chart, we can split the visualization into many smaller visualizations and place them near each other so they can be easily compared.
This method is called \emph{small multiples} and was made famous by Edward Tufte.

As an example, consider the first six Chebyshev polynomials.
Plotting them all on the same graph makes the visualization cluttered.
Instead, each polynomial can be given its own graph and laid out in a way that makes them easy to compare, (see Figure \ref{fig:PoT}).

% add more reference material??

\begin{figure} % Chebyshev polynomials
\centering
\begin{subfigure}{.45\textwidth}
  \centering
  \includegraphics[width=\textwidth]{PoT.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
  \includegraphics[width=\textwidth]{PoT_separate.png}
\end{subfigure}
\caption{Chebyshev polynomials graphed together on the left and in small multiples on the right.
The small multiples are much easier to interpret.}
\label{fig:PoT}
\end{figure}

To make small multiples, use the Matplotlib command \li{ax = plt.subplot()} which sets up multiple plots one at a time.
% Some example code is provided below.

\begin{lstlisting}
import numpy as np
from matplotlib import pyplot as plt
from numpy.polynomial import Chebyshev as T

def Chebyshev_subplots():
    fig = plt.figure(dpi=100)
    fig.set_size_inches(10,10)
    fig.suptitle('Chebyshev Polynomials', fontsize=20)

    x = np.linspace(-1,1,500)

    for i in range(9):
        ax = plt.subplot(3,3,i+1)
        ax.plot(x, T.basis(i)(x), lw=2, label="$T_%d$"%i)
        plt.xlim(-1.1,1.1)
        plt.ylim(-1.1,1.1)
        ax.set_title('$T_%d$'%i)
        if i%3:
            # remove the inner y-ticks and labels.
            ax.set_yticklabels([])
            ax.yaxis.set_ticks_position('none')
        if i<6:
            # remove the inner x-ticks and labels.
            ax.xaxis.set_ticks_position('none')
            ax.set_xticklabels([])

    plt.show()
\end{lstlisting}

\begin{problem} % Problem: plot the Bernstein polynomials.
The $n+1$ Bernstein basis polynomials of degree $n$ are defined as

$$b_{v,n} = {{n} \choose {v}} x^v (1-x)^{n-v}, v = 0, 1,..., n.$$

Plot the Bernstein basis polynomials for $v,n$ in [0,3] as small multiples and compare that to the cluttered version of plotting them on top of each other.
\end{problem}

\begin{comment}

\subsection*{Additional Material} % ===========================================

There are many software packages that facilitate the visual exploration of data.
One Python library is Glue (see \cite{glue}).

Some packages for making nicer looking plots include \li{Seaborn} and \li{prettyplotlib}.  TODO get refs

For more about visualization of data, we highly recommend the following books and websites:

% TODO get full refs for the follwoing

\begin{itemize}
    \item \emph{How to Lie with Statistics} by Darrell Huff (1954)
    \item \emph{Envisioning Information} by Edward Tufte
    \item \emph{The Visual Display of Quantitative Information} by Edward Tufte (2nd edition)
    \item \emph{Beautiful Evidence} by Edward Tufte
    \item \emph{Now you see it} by Stephen Few
    \item \url{http://www.informationisbeautiful.net/}
\end{itemize}

\end{comment}

%\noindent\emph{At their best, graphics are instruments for reasoning about quantitative information.} \small{---Edward Tufte}
% from the introduction of his book 'The Visual Display of Quantitative Information'

\begin{comment} % Move to 'Additional Material'.
\textbf{Bar Chart}: Best for small sets of discrete, one-dimensional, categorical data.
Usually, a horizontal bar chart is preferred over a vertical bar chart because it's harder to read vertical labels than horizontal labels.
Also, horizontal labels on a vertical bar chart may not fit on the graph well (see Figure \ref{fig:barchart}).

The data in a bar chart should be sorted in a logical way.
If the chart is being used to compare the values of different data points, it is best to sort by size; if the chart is being used to look up specific values, the labels should be sorted in a way that makes it easy to find specific values (for example, alphabetically).
Use \li{plt.bar()} or \li{plt.barh()} to create a bar chart in Matplotlib.

\begin{figure}[h] % Bar chart styles
\centering
\begin{subfigure}{.45\textwidth}
  \centering
  \includegraphics[width=\textwidth]{bar_chart_horizontal_sorted.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
  \centering
  \includegraphics[width=\textwidth]{bar_chart_unsorted.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
  \centering
  \includegraphics[width=\textwidth]{bar_chart_vertical_bars_horizontal_labels.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
  \centering
  \includegraphics[width=\textwidth]{bar_chart_vertical_bars_vertical_labels.png}
\end{subfigure}

\caption{Bar Charts.
The example in the upper left is easier to use for comparison than the upper right, and easier to read than the bottom two charts.
The labels overlap in the bottom left and the labels are vertical in the bottom right, making them hard to read.}
\label{fig:barchart}
\end{figure}

\begin{lstlisting}
# This code created the top right bar chart.
labels = 'Spam', 'Eggs', 'Ham', 'Sausage', 'Bacon', 'Baked beans','Lobster thermidor'
val = [10, 11, 18, 19, 20, 21, 22]    # the bar lengths
pos = np.arange(7)+.5    # the bar centers on the y axis
plt.barh(pos,val, align='center')
plt.yticks(pos, labels[::-1])
plt.show()
\end{lstlisting}
\end{comment}


\documentclass{article}

\usepackage{minted}
\usepackage{amsmath}
\usepackage{amsfonts}

\title{ECE-210-A Homework Feedback}
\author{Jonathan Lam}
\date{Spring 2022}

\begin{document}

\maketitle{}

\section{Introduction to MATLAB}
\label{sec:hw1}

\begin{description}
\item[Submission format] Please submit in *.m (plaintext) file format, rather than *.mlx. I should have made this a requirement at the beginning. While *.mlx (live script) files may be more visually appealling, they are more difficult to grade and may not be as cross-compatible with different versions of MATLAB.
\item[Notes on style (in addition to the style notes on the assignment sheet)] Name your files consistently (e.g., \mintinline{text}|ece210_hw1_lam.m|), and use a header at the beginning of each file. It may look something like:
\begin{minted}{matlab}
% Homework 1
% Jonathan Lam
% ECE210
% 02/16/2022
clear(); clc(); close('all');

%% Q1
% ...
\end{minted}
  At the least, have your name and the commands to clear the workspace at the top of every script file\footnote{A script file is one you would run directly. This is as opposed to a file that defines a function or class, in which you do not want to call \mintinline{matlab}|clear(); clc(); close('all');| at the top of.}.
  \item[\mintinline{matlab}|linspace()| vs. the colon operator] Be wary of the use cases and what the problem specifies! One is better suited for the case of a specified number of samples, and the other is better for cases of a specified frequency or period.
\end{description}

\clearpage{}
\section{Vectorization and for loops}
\label{sec:hw2}

\begin{description}
\item[Suppress intermediate and long outputs.] I don't need to see the outputs in the command window to evaluate your HW -- there are other ways. Printing large matrices (and other I/O in general) is also very slow (it will definitely mess with timing).
\item[Keep your code DRY] (DRY = ``Don't repeat yourself,'' and is a common maxim in software engineering.) If you see a constant (e.g., 100, 1000) being repeated throughout your code, it's better (more maintainable, and better documented) to define it once as a variable and use that variable throughout. That way, you can easily change the variable by changing one value, or you can rename the variable using MATLAB's variable refactoring utilities.
\item[Use sections to separate code execution.] Sections are a double-edged sword. For one thing, they are very convenient for running a particular block of code, and they should be used to partition your code into logical segments. Being able to focus on and run a small bit of code, and inspect the workspace (environment) in the middle of your script is extremely helpful. On the other hand, it can cause problems that are addressed in the next bullet point.
\item[Make sure your code runs in a clean environment.] First of all, make sure your code compiles and runs. Submitting code that has multiple errors throughout is a big red flag, because coding is a feedback-driven problem-solving process: you are expected to run your code, and check that the results match your intuition. If your code is not in a state where it can be run, it shows that you haven't gone through this process. Additionally (related to sections), make sure that if you clear your workspace and run all sections in your code from start to finish, that it runs correctly without errors. This is important because when you were debugging, you may have accidentally changed the behavior of your program.
\item[Get familiar with standard debugging methods.] Understanding error messages is key. That should help you handle syntactical errors. For semantic errors (when your code compiles but doesn't do what you think it does), start with a minimal, working example and build up from there step by step, until something begins to fail. Learn how to build test cases and check the error from an expected or true value. Use the documentation until you get tired of it. This falls under the category of standard debugging tips that are common in software engineering.
\end{description}

\clearpage{}
\section{Logical indexing and functions}
\begin{description}
\item[Logical indexing is different than multiplying by a logical matrix] The first question involves multiplying by a matrix of logical values, which acts like a layer mask: it preserves the domain, and sets all the values corresponding to FALSE to 0. In the second question, one needs to extract the x- and y-values that meet a certain condition. In this case, we want to condense the vectors to only those that match the condition; if we multiply by the logical matrix, we end up with a lot of spurious \mintinline{matlab}|[x, y] = [0, 0]| values. In other words, make sure you understand the difference between:
\begin{minted}{matlab}
A = [ 1 2 3 ; 4 5 6 ];
B = logical([ 1 0 0 ; 0 1 0 ]);
masked = A .* B;
filtered = A(B);
\end{minted}

\item[Padding vectors] Don't pad a vector with arbitrary values to make it match the length of another vector. In the case of extending the the result of \mintinline{matlab}|diff()| to make it match the length of the original vector, do not pad with zeros -- this may create an anamolous discontinuity at the padded end. A more reasonable method is to pad with the first value at the beginning, or pad with the last value at the end. Alternatively, truncating the first or last value of the longer vector may be equally appropriate and doesn't involve introducing new datapoints.

  There may be times when padding with zeros is acceptable, e.g., when using the FFT, which expects a zero-padded vector of length $2^n,\ n\in\mathbb{N}$.

\item[The \mintinline{text}|sinc| function] As many of you discovered, the \mintinline{matlab}|sinc| function requires the Signal Processing Toolkit. With your school account, you may install this toolkit for free. (If you have trouble with installing a toolkit, please let me know.)

  You may also implement the \mintinline{matlab}|sinc| function on your own. The formula for the (normalized) \mintinline{matlab}|sinc| is well-known: \[\text{sinc }x = \frac{\sin\pi x}{\pi x}\] The problem is the discontinuity (due to a division by 0 which leaves an undefined value at $x=0$), which will cause issues with your calculus functions. Since the limit is well-defined at $x=0$, we can be more explicit: \[\text{sinc }x=
    \begin{cases}
      1, & x=0 \\
      \frac{\sin\pi x}{\pi x}, & x\ne 0
    \end{cases}
  \] We may implement this in MATLAB like so:
\begin{minted}{matlab}
x = % some vector
y = sin(pi * x) ./ (pi * x) ;
y(x == 0) = 1;  % note: logical indexing and broadcasting!
\end{minted}
\end{description}

\section{Gram Schmidt orthonormalization}
\begin{description}
\item[An open-ended problem] This assignment was the first (and only one) in this course that is really open-ended. You learned about an algorithm called Gram-Schmidt in Linear Algebra or in Signals and Systems. Now you're told to automate it. Take a specification and implement it. Word problems like this are common in computer programming.

You might wonder as you go through the process: \textit{What is the most efficient way to do this? Should this part be vectorized? Can this part even be vectorized?} What you might not realize, however, is that programming is inherently an iterative process, and you won't know until you try it the first time. And then when you run your code and get results you don't expect, you debug it and refactor until you get a polished piece of code. I will also give you feedback when you submit your code, and you are allowed to resubmit code (for credit) because that's when you expect your code to improve. Don't expect to know what you're doing when you code; treat it as an interactive process.

\item[To vectorize, or not to vectorize] Gram Schmidt is an algorithm that cannot be totally vectorized. This is because it is a sequenced algorithm; the second row depends on the first row, and the third depends on the first two, and so on and so forth. The columns are not independent and thus the algorithm is not parallelizable. Thus we need a \mintinline{matlab}{for} loop somewhere in the algorithm. However, it turns out that you can vectorize both \mintinline{matlab}{ortho_proj} and \mintinline{matlab}{is_orthonormal}, because they both don't have this restriction.

\item[Matrix multiplication as inner product] A common definition for inner product of two (complex\footnote{Did you consider the case of complex vectors?}) vectors is the following:
\[\langle x,y\rangle=\sum_ix_i^*y_i\]
Now, assume we have in MATLAB two column vectors \mintinline{matlab}{x} and \mintinline{matlab}{y}. We can write this inner product\footnote{In math notation, this would be $x^Hy$, where $^H$ indicates the Hermitian transpose (conjugate transpose).} as the expression \mintinline{matlab}{dot(conj(x), y)}. However, there's a simpler way: \mintinline{matlab}{x' * y}. The place where this really shines is if you have two matrices $A$ and $B$ (with the same interpretation as given in the problem set, where each column is a vector in the set), and want to take all the dot products $\langle A_i,B_j\rangle$ where $A_i$ indicates vector/column $i$ from matrix $A$. In this case, you can do this very simply: \[C=A^HB\] The beauty is that $C_{ij}=A_i^HB_j=\langle A_i,B_j\rangle$.

\item[Vectorizing \mintinline{matlab}{is_orthonormal}] Consider a set of vectors $M$. If the set of vectors is orthonormal, what do you expect the output of $M^HM$ to be? (Hint: see previous bullet point.)

\item[Vectorizing \mintinline{matlab}{ortho_proj}] The projection of a vector on a basis is the sum of the projections of the vector on each of the basis vectors. You can write orthogonal projection using the formula given in your first linear algebra class: \[\text{proj}_{\{\vec{a_i}\}}\vec{b}=\sum_i\text{proj}_{\vec{a_i}}\vec{b}=\sum_i\frac{\langle\vec{a_i},\vec{b}\rangle}{||\vec{a_i}||^2}\vec{a_i}\] However, we can greatly simplify this for our \mintinline{matlab}{ortho_proj} case. First of all, if we assume that $\{\vec{a_i}\}$ is an orthonormal set of vectors, then $||\vec{a_i}||=||\vec{a_i}||^2=1$. The rest is simply three operations: an inner product, a regular (element-wise) product, and a sum. Can you vectorize the composition of these operations?

\item[Additional error checking for Gram Schmidt] Consider the case when there are more vectors in the set of vectors than the dimensionality of the space (the number of elements per vector). (This will be the case when $M$ is wider than it is tall.) Then, some vectors will necessarily linearly dependent. This in turn will cause some of the vectors from Gram Schmidt orthonormalization to be zero. Do you handle this case? (Most submissions don't check if the norm is zero before normalizing, and they don't attempt to remove zero vectors from the basis during Gram Schmidt). Note that the assignment doesn't ask you to check these, but neither does it give you the assumption that the input vectors are necessarily linearly independent. (Luckily, a set of random vectors where $M$ is at least as tall as it is wide will very likely be a linearly-independent set.)

\end{description}

\section{Object-oriented programming and cell arrays}
Not much to say about this assignment, it should be pretty straightforward.

\section{Z-transform, polynomials, and basic filtering}
\begin{description}
\item[An intuitive reading of the pole-zero plot] Zeros near or on the unit circle cause increased attenuation (decreased gain), and poles near the unit circle (but hopefully not on the unit circle) caused increased gain. The rightmost part of the unit circle represents a zero frequency, whereas the leftmost part represents higher frequencies. We can use some combination of these facts to intuit what kind of filter (lowpass, highpass, bandpass, bandstop) a filter is by inspecting its pole-zero plot.
\end{description}

\end{document}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: t
%%% End:


\documentclass[11 pt]{scrartcl}
\usepackage[header, margin, koma]{tyler}

\newcommand{\hwtitle}{Discussion 2B Recap}

\tikzstyle myBG=[line width=3pt,opacity=1]

\newcommand{\drawLinewithBG}[2]
{
  \draw[white,myBG]  (#1) -- (#2);
  \draw[black,very thick] (#1) -- (#2);
}
\newcommand{\drawPolarLinewithBG}[2]
{
  \draw[white,myBG]  (#1) -- (#2);
  \draw[black,very thick] (#1) -- (#2);
}

\pagestyle{fancy}
\fancyhf{}
\fancyhead[l]{\hwtitle{}}
\fancyhead[r]{Tyler Zhu}
\cfoot{\thepage}

\begin{document}
\title{\Large \hwtitle{}}
\author{\large Tyler Zhu}
\date{\large September 11, 2020}

\maketitle

\section{Terminology}

A \textbf{graph} $G$ consists of \textbf{vertices} or \textbf{nodes}, the points, and \textbf{edges}, the lines. We frequently write $V(G)$ and $E(G)$ for the vertex and edge sets of $G$ respectively, and call $|V(G)|$ and $|E(G)|$ the \textbf{order} and \textbf{size} of the graph respectively.
\begin{figure}[!htb]
    \centering
        \begin{tikzpicture}[node distance = 2cm,
            thick,main node/.style={circle,draw}]
            \node[main node] (1) {$u$};
            \node[main node] (2) [above right of=1, yshift=-0.5cm] {$v$};
            \node[main node] (3) [below right of=1, yshift=0.5cm] {$w$};
            \node[main node] (4) [below right of=2, yshift=0.5cm] {$x$};
            \node[main node] (5) [right of=4] {$y$};

            \path
            (1) edge (2)
                edge (3)
                edge (4)
            (2) edge (4)
            (3) edge (4)
            (4) edge (5);
        \end{tikzpicture}
        \caption{A graph of order 5 and size 6.}
        \label{fig:simple}
\end{figure}

In Figure~\ref{fig:simple}, $V(G) = \{u,v,w,x,y\}$ and $E(G) = \{uv, ux, uw, vx, wx, xy\}$. If $e=uv$ is an edge of $G$, then $u$ and $v$ are said to be \textbf{joined} by the edge $e$. In this case, $u$ and $v$ are referred to as \textbf{neighbors} of each other.

%prove characterization of eulerian circuits, and maybe also do hamiltonians
For a connected graph $G$, any open trail that contains every edge of $G$ is an \textbf{Eulerian trail}. If $G$ contains a closed Eulerian trail, it is \textbf{Eulerian}

\begin{theorem}
    A nontrivial connected graph $G$ is Eulerian if and only if every vertex of $G$ has even degree.
\end{theorem}

With this, we can easily characterize graphs possessing an Eulerian trail.
\begin{corollary}
    A connected graph $G$ contains an Eulerian trail if and only if exactly two vertices of $G$ have odd degree. Furthermore, each Eulerian trail of $G$ begins at one of these odd vertices and ends at another.
    \label{cor:euler}
\end{corollary}

There is an analog to Eulerian trails. A path in a graph $G$ that contains every vertex of $G$ is called a \textbf{Hamiltonian path}.\footnote{We studied Eulerian \textit{trails} because such trails may need to have repeated vertices, unlike paths which necessarily have unique vertices.} Unfortunately, these are much less well-behaved.

\section{Banquet Arrangement}
Here's Problem 2 from the discussion worksheet.
\begin{problem}
    Suppose $n$ people are attending a banquet, and each of them has at least $m$ friends $(2\leq m\leq n)$, where friendship is mutual.
    Prove that we can put at least $m+1$ of the attendants on the same round table, so that each person sits next to his or her friends on both sides.
\end{problem}

\begin{proof}[Solution]

As an example, suppose Alice, Bob, Claire, and Dom are attending, and the friend pairings are (Alice, Claire), (Alice, Dom), (Bob, Claire), (Bob, Dom), so $n = 4$ and $m = 2$ in this case.
We actually can't choose exactly three of them so that they all sit next to their friends, but we can certainly choose to sit the four them in the order (Alice, Claire, Bob, Dom).

If you haven't realized already, it's helpful to recast this problem in terms of graphs to solve it in general.
A big tipoff is both the fact that there are people and mutual friendships which strongly hint to being vertices and edges respectively as well as the topic of this discussion. So we will set
\[ \text{ people }\mapsto \text{ vertices } \]
and
\[ \text{ friendship }\mapsto \text{ edges }.\]

If we were to represent the above problem in graphs, it'd look something like this:

\begin{figure}[!ht]
\begin{center}
\begin{tikzpicture}[>=stealth', auto, semithick, node distance=2cm]
\tikzstyle{every state}=[fill=white,draw=black,thick,text=black,scale=0.8]
\node[state]    (A)               {$A$};
\node[state]    (B)[below of=A]   {$B$};
\node[state]    (C)[right of=A]   {$C$};
\node[state]    (D)[right of=B]   {$D$};
\path
(A) edge    (C)
    edge    (D)
(B) edge    (C)
    edge    (D);
\end{tikzpicture}
\end{center}
\caption{Casting the problem in terms of graphs.}
\end{figure}

We also see that sitting people around a table so that they sit next to their friends is just asking to find a cycle of length at least $m+1$. Using the above graph, it's easy to find one such cycle, $ACBD$, by starting at $A$ and walking on the graph until you come back to $A$.

So that's the easy part of the problem. Now let's see how we would create such a cycle. Naively, we might start at an arbitrary vertex, say $v_0$, and try to walk along this graph. We know $v_0$ has at least $m$ neighbors, so let's visit one of them, $v_1$.

Now $v_1$ also has at least $m$ neighbors, but we've already been to one of them, $v_0$, so it really only has $m-1$ unvisited neighbors, so let's visit one of them, $v_2$. We can keep repeating this process of visiting unvisited neighbors until we're out of new neighbors to get a path

\[ P = v_0v_1\dots v_l.\]

One thing we should note is that in the worst case, the number of unvisited neighbors for each $v_i$ is at least $m-i$, which is if all of the $i$ vertices already in the path are neighbors of $v_i$.
So $l \geq m$, or else we would still have neighbors we could visit.

All that's left for us now is to get a cycle from this path. Since we're stuck when we reach $v_l$, all of its neighbors must be in $P$. So let's make a new path starting at $v_l$ by backpedaling from $v_l$ until we reach the last neighbor of $v_l$ in $P$, say $v_k$. Now we have a path
\[ P' = v_l v_{l-1} \dots v_k\]
where $P'$ is also at least $m+1$ since $v_l$ and all of its neighbors are in the path. But $v_k$ is neighbors with $v_l$, so this is also a cycle, and we're done.
\end{proof}

We used the principle of trying things until we get stuck, and trying to fix things until we're free. You'll find that this often works surprisingly well.

The official solution is quite slick as well, but less motivated.
It considers the \emph{longest} path in the graph, and constructs the cycle from that.
This is an example of the \textbf{Extremal Principle}, where one looks at either the largest or the smallest object satisfying some property.

\begin{proof}[Official Solution]
    Let $P=v_0v_1\dots v_l$ be a longest path in the graph.
    Such a path exists because the length of paths is bounded above by $n$.
    All neighbors of $v_0$ must be in $P$, since otherwise $P$ can be extended to be even longer by appending this edge at the beginning of path $P$.
    Let $k$ be the maximum index of neighbors of $v_0$ along $P$.
    Since $v_0$ has at least $m$ neighbors, we must have $k\geq m$.
    Then $v_0v_1\dots v_kv_0$ gives us the desired cycle.
\end{proof}

Here's a cute example of the Extremal Principle applied in another setting. Before you read on, think about it for a few minutes.

\begin{problem}
    There are $n$ students standing in a field such that the distance between each pair is distinct. Each student is holding a ball, and when the teacher blows a whistle, each student throws their ball to the nearest student. Prove that there is a pair of students that throw their balls to each other.
\end{problem}
\begin{proof}
    Consider the two students closest to each other. Since they are each other's nearest student, they must throw their balls to each other.
\end{proof}

Try this one for another extremal flavored graph problem. How would you do it without looking at the extreme cases?
\begin{exercise}
    Show that every tree $G$ contians at least two leaves, or vertices of degree 1.
\end{exercise}

\iffalse

%%%%%%%%%%%%% SECTION %%%%%%%%%%%%5
\section{Planarity Bounds}
A graph $G$ is called a \textbf{planar graph} if $G$ can be drawn in the plane so that no two of its edges cross each other. If $G$ is planar, then it divides the plane into pieces called \textbf{regions}. Recall that for any planar, connected graph $G$, if $G$ has $V$ vertices, $E$ edges, and $F$ faces, Euler's formula tells us that
    \[ V-E+F = 2. \]

You can imagine how hard it is to prove a graph to be non-planar: you can't possibly check every way of drawing the graph. Euler's Identity leads to some simple conditions on planarity.

We can derive many useful bounds using this formula that involve only two of the three quantities, which is helpful especially for showing that certain graphs are nonplanar. Usually we omit faces since those are tricky to quantify, so let's try to compare the number of faces to the number of edges.

\begin{figure}[!htb]
\begin{center}
\begin{tikzpicture}[>=stealth', auto, semithick, node distance=2cm]
\tikzstyle{every state}=[fill=white,draw=black,thick,text=black,scale=0.8]
\node[state]    (A)               {};
\node[state]    (B)[right of=A]   {};
\node[state]    (C)[right of=B]   {};
\node[state]    (D)[below of=A]   {};
\node[state]    (E)[right of=D]   {};
\path
(A) edge    node[below = 0.5cm] {$F_1$} (B)
    edge    (D)
(B) edge    node[below = 0.6cm, left] {$F_2$} (C)
    edge    (E)
(C) edge    node[below right = 0.5cm] {$F_3$} (E)
(D) edge    (E);
\end{tikzpicture}
\end{center}
\caption{An example planar graph $G_1$ with its three faces marked.}
\label{fig:graph}
\end{figure}

Let's look at the graph $G_1$ shown in Figure~\ref{fig:graph}. Notice that every edge is a part of exactly two faces. So we can count the number of edges also by looking at how many edges border each face. Let $|F_i|$ denote the number of edges that border the face $F_i$. Then,
\begin{align*}
    2E &= |F_1| + |F_2| + |F_3| \\
       &= 4 + 3 + 5 \\
       &\geq 3 + 3 + 3 \\
       &= 3F
\end{align*}
where we used the very deep fact that every face is bordered by at least 3 edges (how else do you have a face?). Hence, we've arrived at the inequality $2E \geq 3F$. Now if we solve our formula for $F$ and substitute, we get the all important bound

\[ 2 - V + E = F \leq \frac{2}{3} E \implies \boxed{E \leq 3V-6}.\]

This is useful now as a method for showing a graph is nonplanar, as otherwise we'd have to draw every possible configuration of a graph and show all of them have crossings, which is neither feasible nor convincing. We can use this to show that $K_5$, the complete graph on 5 vertices, is nonplanar. Since it has $5$ vertices and $10$ edges, $10 \not\leq 3\cdot 5 - 6 = 9$.

One extension of the above bound is to find a bound when $G$ is triangle-free, i.e. no face is bounded by 3 edges. I'll leave that as an exercise for you. Recall that the graph with six vertices, where three vertices are connected to all three other vertices, is the complete bipartite graph $K_{3,3}$. We can use this new bound to show $K_{3,3}$ is nonplanar.

\begin{exercise}
    Show that if $G$ is a connected, planar, triangle-free graph, then $E \leq 2V - 4$. Use this to show that $K_{3,3}$ is nonplanar.
\end{exercise}

As a hint, we proved that all bipartite graphs have no odd tours. So what do we know about the presence of triangles in such a graph?

\begin{figure}[!htb]
    \centering
    \begin{tikzpicture}[scale = 0.7]

  \begin{scope}[xshift=-6cm,yshift=-1cm]
    \foreach \x in {4,2,0} {
      \foreach \y in {0,2,4} {
        \drawLinewithBG{\x,0}{\y,2};
      }
    }

    \foreach \x in {0,2,4} {
      \foreach \y in {0,2} {
        \node at (\x,\y) [circle,fill=black] {};
      }
    }

    \node at (2,-1.5) {\Large$K_{3,3}$};
  \end{scope}

  \begin{scope}[xshift=3cm]
    \foreach \a in { 18, 90, 162, 234, 306 } {
      \foreach \b in { 18, 90, 162, 234, 306 } {
        \drawPolarLinewithBG{\a:2}{\b:2};
      }
    }

    \foreach \a in {18,90, 162, 234, 306 } {
      \node at (\a:2cm) [circle,fill=black] {};
    }
    \node at (0,-2.5) {\Large$K_5$};
  \end{scope}
\end{tikzpicture}
\caption{The Kuratowski graphs}
\end{figure}

We can already see how much trouble it can be to determine planarity for arbitrary graphs, let alone simple ones like $K_{3,3}$ and $K_5$. Surprisingly, the main enemies to planarity are precisely these two graphs, which leads to a simple check to see if a graph is planar, discovered in 1930. Before I can state Kuratowski's remarkable theorem, I need to state two notions.

A \textbf{subdivision} of a graph is constructed by replacing one edge by two edges with a vertex in between, and a \textbf{subgraph} is constructed by removing vertices or edges.

\begin{theorem}[Kuratowski]
    A graph $G$ is planar if and only if $G$ does not contain a subdivision of $K_5$ or $K_{3,3}$ as a subgraph.
\end{theorem}

In other words, $G$ is planar if it is not possible to subdivide the edges of $K_5$ or $K_{3,3}$, and then possibly add edges or vertices, to get $G$.

\subsection{To Take Home}
\begin{exercise}[Dis. Problem 2b]
    Consider graphs with the property $T$: For every three distinct vertices $v_1,v_2,v_3$ of graph $G$ , there are at least two edges among them.
    Prove that if $G$ is a graph on $\geq$ 7 vertices, and $G$ has property $T$, then $G$ is nonplanar.
\end{exercise}
Hint: Proof by contradiction when $v = 7$. What do we know about groups of five vertices in a planar graph?

\section{Induction on Edges and Vertices}
Problem 3 from the discussion sometimes gives people trouble, so I'll try to explain it some more in depth. Here is the problem, paraphrased.

\begin{problem}
An edge coloring of a graph is an assignment of colors to edges in a graph where any two edges incident to the same vertex have different colors.
\alphanum
    \ii Prove that any graph with maximum degree $d\geq 1$ can be edge colored with $2d-1$ colors.
    \ii Show that any tree with maximum degree $d\geq 1$ can be edge colored with $d$ colors.
\enumend
\end{problem}

The solution is to use induction on the number of edges in the first question, and induction on the number of vertices in the second. I'm not going to re-explain the solution (you can find it online) but I'll try to explain why it's fine to induct this way.

Imagine if I had the following problem:
\begin{question}
    Prove that any graph with maximum degree $4$ can be edge colored with $7$ colors.
\end{question}

Now induction, especially on the number of edges, seems like a viable approach. We're proving some statement about all graphs, so let's assume its true for a graph with $m$ edges and prove it for one with $m+1$ edges (both with max degree 4). What about this problem:

\begin{question}
    Prove that any graph with maximum degree $10$ can be edge colored with $19$ colors.
\end{question}

See what I'm getting at? Just because we have a variable $d$ in our original statement doesn't mean we need to induct over $d$. We can treat $d$ as a constant and show the statement is true for all graphs given a specific $d$, which then shows the statement is true for all $d$.

Another analogy: we don't need to induct over our variable $d$ much like how the number of vertices and edges are also variables, but we don't need to induct over both.

\fi

\end{document}


\documentclass[12pt,a4]{article}
\author{Yujie Lu \quad Jiabao Ji \quad Tong Chen}
\newcommand{\handoutdate}{2020-03-05}
\newcommand{\firstduedate}{2020-03-10}
\newcommand{\finalduedate}{2020-03-17 before class}
\usepackage{pythonhighlight}
\usepackage{import}
\usepackage[many]{tcolorbox}
\tcbuselibrary{skins, breakable, theorems}
\tcolorboxenvironment{exercise}{
  enhanced,
  borderline={0.4pt}{0.4pt}{black},
  boxrule=0.4pt,
  colback=white,
  coltitle=black,
  sharp corners,
}

\input{preamble}

\section{Bit Complexity, Recursion, and Dynamic Programming}

\subsection{Bit Complexity of Euclid's Algorithm}

We have proved that Euclid's algorithm for computing $\gcd(a,b)$ makes at most
$O(\log a)$ iterations. What is the overall running time? Each iteration computes
$u \mod v$ for some integers. This can be done by integer division. What is its running time?
There are very sophisticated algorithms, but python probably does not come with them.
Recall the ``school method'' for dividing integers. Have a look at the pdf slides on the
webpage for an illustration of the school method. It is especially simple if we are dealing
with binary numbers. If $a$ and $b$ have at most $n$ bits, then the school method
has complexity $O(n^2)$.
\begin{exercise}
  Show the following, more precise bound of the school method for integer division:
  If $a$ has $n$ bits and $b$ has $k$ bits, then the school method can be implemented
  to run in $O( k(n-k))$ operations.
\end{exercise}
\begin{proof}
First, we know $n > k$ since $a > b$. To get r, $r = a \% b, (0 \leq r < b)$,
so the bits of $r$, we write it as $b_r$. $b_r < k$.

Remind the procedure of brute-force algorithm of $a \% b$, in each iteration,
$n$ decreases(since we can drop the higher bits). In the final iteration, $n < k$,
also this $n$ is exactly $b_r$, so we don't need to do the iteration $n$ times,
but only $n - k + 1$ times. And each iteration costs $k$ subtraction operation, which is an integer operation,
in all we do no more than $k(n - k+ 1)$ times of basic operations. That is
$O\left( k\left( n-k \right)  \right) $ basic operations.
\end{proof}
\begin{exercise}
  Show that the bit complexity of Euclid's algorithm, using the school method
  to compute $a \mod b$, is $O(n^2)$. That is,
  if $a$ and $b$ have at most $n$ bits, then $\gcd(a,b)$ makes $O(n^2)$ bit operations.\\

  In order to do so, here is python code of the Euclidean algorithm:
\begin{verbatim}
  def euclid(a,b):
    while (b > 0):
        r = a % b # so a = bu+r
        if (r == 0):
            return b
        s = b % r # so b = rv + s
        a = r
        b = s
    return a
\end{verbatim}
Don't be afraid to introduce notation! I recommend to let $n$ denote the number of bits of $a$.
Take some other letters for the number of bits in $b$ and so on.
\end{exercise}
\begin{proof}
		Assume that $\gcd\left( a,b \right) $ takes $t$ iterations and $t \le  2n$. To better represent our ideas.  Let's introduce
		some notations. Conside a process of iterations as follows:
		\[
				\gcd\left( a,b \right) = \gcd\left( x_{t},x_{t-1} \right) \to
				\gcd\left( x_{t-1},x_{t-2} \right) \to \ldots \to \gcd\left( x_1,0 \right)
		.\]
		And $\forall 1\le k\le t$ let $x_{k}$ has $y_{k}$ bits. We know $y_{t}\le n$ plus $x_{k}>x_{k-1}$.
		Hence $\forall k, y_{k}\ge y_{k-1}$.

		Let the operations of all iterations be $T$. And the operations of  $k$th iteration be $T_{k}$.
		Now apply the conclusion of Exercise 1, we have
		\begin{align*}
				T &= \sum_{k=1}^{t} T_{k} \\
				  &\le \lambda \sum_{k=1}^{t} y_{k}\left( y_{k}-y_{k-1} \right) \\
				& \le \frac{\lambda}{2}\left( \sum_{k=1}^{t} \left( y_{k}-y_{k-1} \right)^2 + y_t^2 \right) \\
				  & \le  \frac{\lambda}{2} \left( \sum_{k=1}^{t} \left( y_{k}-y_{k-1} \right)  \right)^2+ \frac{\lambda}{2} y_{t}^2\\
				  & \le  \frac{\lambda}{2} n^2
		.\end{align*}
		Hence $\gcd\left( a,b \right) $ makes $O\left( n^2 \right) $ operations.
\end{proof}
\subsection{Computing the Binomial Coefficient}

Next, we will investigate the binomial coefficient ${n \choose k}$, which
you might also know by the notation $C^k_n$. The number ${n \choose k}$ is defined
as the number of subsets of $\{1,\dots,n\}$ which have size exactly $k$.
This immediately shows that ${n \choose k}$ is $0$ if $k$ is negative or larger than $n$.
You might have seen the following recurrence:
\begin{align*}
 {n \choose k} & = {n-1 \choose k-1} + {n-1 \choose k} \textnormal{ if } n,k \geq 0 \ .
\end{align*}

\begin{exercise}[A Recursive Algorithm for the Binomial Coefficient]
  Using pseudocode, write a recursive algorithm computing
  ${n \choose k}$. Implement it in python! What is
  the running time of your algorithm, in terms of $n$ and $k$? Would you say it is an efficient
  algorithm? Why or why not?
\end{exercise}
\begin{proof}
recursive code:
\begin{python}
def recursiveCompute(n, k):
    if (n <= k or k == 0):
        return 1
    else:
        return recursiveCompute(n - 1, k - 1) + recursiveCompute(n - 1, k)
\end{python}

Like recursive algorithm for computing Fib(n), we can get a recursive tree,
which has $ \binom n k$ leaves and $ \left(2 \binom n k - 1\right) $ nodes in all.
We can prove it by induction.

Firstly, prove it for a fixed n, to do that, we use an induction.
And suppose the equation is true for all $m < n$ (base $n = 1$ is trivial).\\
\textbf{Base:} $k = 0:$

Obviously, the recursive tree only has 1 node, which satisfies the equation.\\
\textbf{Induction:}

To compute $\binom n {k+1} $, we will add a node for $\binom n {k+1}$ as the new root,
and two sub recursive tree for computing $\binom {n - 1} {k}, \binom {n - 1}{k + 1}$,
whose nodes size is known by induction hypothesis.
The new tree has
\[
\binom {n-1} {k} + \binom{n - 1} {k + 1} = \binom{n}{k + 1}
.\]
leaves, and
\[
\left(2\binom{n - 1}{k} - 1 \right)+ \left(2\binom{n - 1}{k + 1} -1\right) + 1
= 2\binom n {k + 1} - 1
.\]
nodes.

Similar to the induction above, we can tell for any $n, k$, the result holds.
For the running time analysis, each node in the tree needs to do an addition,
so it's $O\left(2 \binom n k - 1\right)= O\left( \binom{n}{k}  \right) $.
If we consider the cost of addition at each node: $\log \binom{n}{k}$, we get
$O((\binom n k )log(\binom n k))$.

To sum up, the algorithm's time complexity is
\[
		\Omega\left(\binom n k\right)\text{ and } O\left(\binom n k \log\left
		(\binom n k\right)\right).
\]
It's not a good algorithm, since the time complexity is very high, and it does a lot of
redundant addition.
\end{proof}
\begin{exercise}[A Dynamic Programming Algorithm for the Binomial Coefficient]
  Using pseudocode, write a dynamic programming algorithm
  computing ${n \choose k}$. Implement it in python! What is it running time
  in terms of $n$ and $k$?
  Would you say your algorithm is efficient? Why or why not?
\end{exercise}
\begin{proof}
code:
\begin{python}
def dpCompute(n, k):
    a = [[0 for x in range(k + 1)] for y in range(n + 1)]
    for i in range (n + 1):
        a[i][0] = 1
        if i <= n - k:
            for j in range (1, min(i, k) + 1):
                a[i][j] = a[i - 1][j - 1] + a[i - 1][j]
        else:
            for j in range(i - (n - k), min(i, k) + 1):
                a[i][j] = a[i - 1][j - 1] + a[i - 1][j]
    return a[n][k]
\end{python}
The algorithm needs to do $n$ iterations, for iterations $i \le  n - k$,
we should do $\mathrm{min}  (i, k)$ additions, and for $i > n - k$, we should do $\mathrm{min} (i, k) - i - (n - k)$ additions
(since there is no node of $\binom i j $ if $ i > n - k, j < i - (n - k)$ in the recursive tree)
In all, we need to compute
$$
    \sum_{i = 1}^{n - k} \sum_{j = 1}^{min(i, k)} 1 +
    \sum_{i = n - k + 1}^{n + 1}\sum_{j = i - (n - k)}^{min(i, k) + 1} 1
    = n (n - k)
$$
additions.

This is not trivial, so we draw a picture to better visualize this
\begin{figure}[ht]
    \centering
	\incfig[0.8]{triangle}
    \caption{How much do we compute.}
    \label{fig:triangle}
\end{figure}

Consider the cost of each addition, the algorithm is $\Omega(n(n - k))$ and $O(n(n - k) log \binom n k)$.
It's much more efficcient than the recursive algorithm. I think it's efficcient.

\end{proof}

\begin{exercise}[Binomial Coefficient modulo 2]
  Suppose we are only interested in whether ${n \choose k}$ is even or odd,
  i.e., we want to compute ${n \choose k}  \mod 2$. You could do this by computing
  ${n \choose k}$ using dynamic programming and then taking
  the result modulo $2$. What is the running time? Would you say this algorithm
  is efficient? Why or why not?
\end{exercise}
\begin{proof}
  The running time is similar to Exer.4, but since we only needs to know whether the result modula 2 is 1 or 0,
  we don't need to compute addition at each node but only a bool compute. \\
  \hspace*{1em} So the dynamic algorithm is $\Theta (n(n - k))$.\\
  \hspace*{1em} To this question specifically,not efficcient,
  we don't have to compute the value of $\binom n k$ at all.\\
  Below is the python code.
  \newpage
  \begin{python}
  def hasNtwoFactor(asking):
      res = 0
      while (asking % 2 == 0):
          res += 1
          asking = asking / 2
      return res

  def isEvenOrOdd(n, k):
      numerator = 0
      dominator = 0
      for i in range(1, n + 1):
          numerator += hasNtwoFactor(i)
      for i in range(1, k + 1):
          dominator += hasNtwoFactor(i)
      for i in range(1, n - k + 1):
          dominator += hasNtwoFactor(i)
      if (numerator - dominator > 0):
          return bool(1)
      else:
          return bool(0)
  \end{python}
  it's an $\Theta(n log(n))$ algorithm, faster than dpCompute.

We also have another algorithm based on \textbf{Lucas Theorem}
\begin{python}
def C(n, k):
	if n & k == k:
        return 1;
    else:
        return 0;
\end{python}

Now, let's proof this simple algorithm why it's correct.
\begin{theorem}
	\textbf{(Lucas Theorem)} For non-negative integers \textit{m} and \textit{n} and a prime \textit{p}, the following congruence relation holds:
	$${m \choose n} \equiv \prod_{i=0}^k {m_i \choose n_i} ( mod\ p),$$
	where
	$$m=m_kp^k+m_{k-1}p^{k-1}+\cdots+m_1p+m_0$$,
	and
	$$n=n_kp^k+n_{k-1}p^{k-1}+\cdots+n_1p+n_0$$
	are the base p expansions of m and n respectively. This uses the convention that ${m\choose n}=0$ if $m<n$.
\end{theorem}
Here, we let p = 2, and we have ${0 \choose 0}={1\choose 0}={1 \choose 1}=1$ and ${0 \choose 1}=0$. Hence, we get ${n\choose k}\equiv 1 (mod\ 2)$ if and only if every bit of k is less than that of n in binary representation. That means $n\ and\ k = k$.

The time complexity of this algorithm is $\Theta(log(n))$
\end{proof}
\begin{exercise}
  Remember the ``period'' algorithm for computing $F'_n := (F_n \mod k)$ discussed in class:
  (1) find some $i,j$ between $0$ and $k^2$ for which
  $F'_{i} =  F'_{j}$ and $F'_{i+1} = F'_{j+1} $.
  Then for $d := j-i$ the sequence $F'_{n}$ will repeat every $d$ steps, as there will be a cycle.
  This cycle can either be a ``true cycle'' or a ``lasso'':
  \begin{center}
  \includegraphics[width=0.8\textwidth]{figures/cycle-and-lasso.pdf}
  \end{center}

  Show that a lasso cannot happen. That is, show
  that the smallest $i$ for which this happens is $0$, i.e, for some $j$ we have
  $F'_0 = F'_j$ and $F'_1 = F'_{j+1}$ and thus $F'_n = F'_{n \textnormal{ mod }  j}$.
\end{exercise}

\begin{proof}
		Consider a pair $g\left( i \right) =\left( F'_{i-1},F'_{i} \right) , i\ge 0$. We know that
		$\forall i, g\left( i \right) \in [0,n)\times [0,n)$ which is a finite set with $k^2$ members.

		Consider the following set
		\[
				A = \{ g\left( 1 \right) ,g\left( 2 \right) ,\ldots, g\left( k^2 \right) , g\left( k^2+1 \right) \}
		.\]
		By \textbf{Pigeonhole Principle}, we know that $\exists r< s\in \{1,2,\ldots,k^2+1\} $ such that
		$g\left( r \right) =g\left( s \right) $ which is
		\[
				F'\left( r-1 \right) = F'\left( s-1 \right) , \quad F'\left( r \right)  = F'\left( s \right)
		.\]
		Since $F'\left( r-2 \right)$ is determined by $F'\left( r \right) -F'\left( r-1 \right) $, we can imply that
		\[
				F'\left( 0 \right)  = F'\left( s-r \right) , \quad F'\left( 1 \right)  = F'\left( s-r+1 \right)
		.\]
		Hence this cycle is a ``true cycle".
\end{proof}

\newpage
{\huge\noindent \textbf{Appendix}   \\ \\ }
{\large Pseudocode for Exercise 3, 4}
\begin{algorithm}[h]
		\caption{A function $f$ using recursive algorithm.}
		\KwIn{input parameters $n,k$}
		\KwOut{$ {n\choose k }$}
		\LinesNumbered
		\If{$n=k$ or $k=0$} {
				return $1$
		}
		\Else {
				return $f\left( n-1,k-1 \right) + f\left( n-1,k \right) $
		}
\end{algorithm}

\begin{algorithm}[h]
		\caption{A function $g$ using dynamic algorithm.}
		\KwIn{input parameters $n,k$}
		\KwOut{$ {n\choose k }$}
		\LinesNumbered
		a is a 2d array to store results.

		\For{$i$ in  $\{1,2,\ldots,n+1\} $ } {
				a[i][0] = 1
				\If {$i\le n-k $ } {
						\For{$j\le   \mathrm{min}\left( i,k \right) $ }
						{
								update a[i][j]
						}
				}
				\Else {
						\For {$j \le   \mathrm{min}\left( i,k \right)  \} $} {
								update a[i][j]
						}
				}
		}
		return a[n][k]
\end{algorithm}
\end{document}