\documentclass[12]{scrartcl}
\usepackage{amssymb,amsmath,gensymb,dsfont,calc,multicol,fullpage}
\makeatletter
\newcommand\Aboxed[1]{
   \@Aboxed#1\ENDDNE}
\def\@Aboxed#1&#2\ENDDNE{%
   &
   \settowidth\@tempdima{$\displaystyle#1{}$}
   \setlength\@tempdima{\@tempdima+\fboxsep+\fboxrule}
   \kern-\@tempdima
   \boxed{#1#2}
}
\makeatother

\begin{document}

\title{Homework 6, Section 2.1: 2(b), 4(b), 6(b), 8, 9(b), 11(b), 13(b)}
\author{Alex Gordon}
\date{\today}
\maketitle
\section*{Homework}
\subsection*{2. B)}
True.\\
Examples: \\
9 = 90, which is divisible by 8\\
15 = 224, which is divisible by 8\\
3 = 8, which is divisible by 8
\subsection*{4. B)}
Dear Reader, I hope this proof finds you well. I am here to show you that if an integer n is even, then n + 8 is even. Firstly, I'd like to start out with a definition of even. An even integer is one that can be written in the formula 2j, where j is any integer. Secondly, I'd like to point out that 8 is an even integer because it can be written in the form 2(4), as we previously showed defines an even number. Thirdly, I'd like to show by the closure properties 2j, an even number plus 8, an even number, results in a final product that is an even number.

\subsection*{6. B)}
x = 34
y = 8
Let even integers x and y be given. Then there is an integer K such that 34 = 2(K), and there is an integer L such that 8 = 2(L). So 34 + 8 = 2(K) + 2(L) = 2(K + L), since K and L are integers, this shows that the result, 42, is even.
\subsection*{8.}
Dear Reader, I hope this proof finds you well. I am here to show you that if an integer n is odd, then $3n^2 + 1$ is divisible by 4. To do this, I'd first like to show that an odd integer is a integer that can be represented as $2j + 1$, where j is any integer. Secondly, I'd like to show that because $n^2$ is equal to n(n), and n is always odd, then the result of $3n^2$ will always be divisible by three, since n(n) can be any integer and 3(n(n)) is a multiple of 3. Therefore, since 3(n(n)) is a multiple of three, any multiple of 3, plus 1, will be a multiple of 4.

\subsection*{9. B)}
Dear Reader, I hope this proof finds you well. I am here to show you that if an integer n is odd, and integer j is always even, then the product of these (n)(j) is even. Firstly, I'd like to start with the definitions for even and odd integers. An even integer is is one that can be written in the form 2j, where j is any integer. An odd integer is a integer that can be represented as $2n + 1$, where n is any integer. Secondly I'd like to show that $(2n + 1)(2j)$, by the closure property is $4nj + 2j$ which, by the closure properties can be shown that the expression 4(nj) is even, because (nj) can be any integer, and also that 2j is even because j can be any integer. This means that those to expressions added together is an even expression.

\subsection*{11. B)}
Let y be an odd integer and x be an even integer. Then there is an integer k such that $ y = 2k + 1$ and there is an integer l such that $ x= 2l$. It follows then that
\begin{align*}
\\x(y) &= (2k + 1)(2l)
\\  &= 4kl + 2l \\
\Aboxed{& = 5k +3 }
\end{align*}
Since 4(kl) is any integer and 2(l) is any integer, we can conclude that x(y) is odd.

\subsection*{13. B)}
The contraposative is if n is even then 3n is even.

Dear Reader, I hope this proof finds you well. I am here to show you that if an integer n is odd, and integer j is always even, then the product of these (n)(j) is even. Firstly, I'd like to start with the definitions for an even integer. An even integer is is one that can be written in the form 2j, where j is any integer. Secondly, it is true that if 3n equals 2n + n. Since 2n is an even integer n is an even integer, then by the closure properties that 3n is an even number.

\end{document}


\documentclass[12]{scrartcl}
\usepackage{amssymb,amsmath,gensymb,dsfont,calc,multicol,fullpage}
\makeatletter
\newcommand\Aboxed[1]{
   \@Aboxed#1\ENDDNE}
\def\@Aboxed#1&#2\ENDDNE{%
   &
   \settowidth\@tempdima{$\displaystyle#1{}$}
   \setlength\@tempdima{\@tempdima+\fboxsep+\fboxrule}
   \kern-\@tempdima
   \boxed{#1#2}
}
\makeatother

\begin{document}

\title{Homework 15, Section 3.3: 3(a,c), 10(b), 11(b), 19(b) }
\author{Alex Gordon}
\date{\today}
\maketitle
\section*{Homework}
\subsection*{3. A)}
If $a$ is divisible by $b$, then$ \{ a \cdot m : m \in \mathds{Z} \} \subseteq \{b \cdot n : n \in \mathds{Z} \} $.
\\Let us suppose that $a = 4$, $b = 2$ and $m = 3$, $n = 5$
\\$a \cdot m = 12$
\\$b \cdot n = 10$
\\If this is the case then 12 is in the domain of integers, and 10 is a subset of the domain of integers showing the assertion valid.
\subsection*{3. C)}
Let a prime number be equal to any number that is only divisible by 1 or itself.\\
Let $k$ be equal to some integer.\\
Let us assume that $x$ is prime and $x = (k + 1)(k - 1)$\\
If $x = 3$, then $x = (2 + 1)(2 -1) = 3$, then x is prime for the value of $k = 2$ \\
However, for all other values, $x$ would be equal to $(k + 1)(k - 1)$ which is equal to (an integer +1)(an integer -1). Assuming that (an integer +1)(an integer -1) equals (a)(b) then we know that a or b only equals 1 when $k = 2$ and $k = 0$ (however, when $k = 0$, $x = 0$). \\
Since a prime number is only divisible by 1 and itself, we know that a prime number can not be equal to a multiple of any number two numbers other than 1 and itself, which means that since $a$ or $b$ is only equal to 1 when $k = 2$, then our assertion that the intersection of prime numbers and $k^2 - 1$ is only equal to the set of \{3\}

\subsection*{10. B)}
(Distributive property) $A \cup(B \cap C) = (A \cap B) \cup (A \cap C)$\\
Let sets $A, B, C$ be given. Let $x \in A \cup (B \cap C)$. Then
\begin{align*}
\\X \in A \cup (B \cap C)&= X \in A \ \vee \ X \in B \cap C
\\ &= X \in A \ \vee \ (X \in B \ \wedge \ X \in C)
\\ &=  (X \in A \ \vee \ X \in B) \wedge (X \in A \ \vee \ X \in C)
\\ &= (X \in A \cup B) \wedge (X \in A \cup C)
\\ &= X \in (A \cup B) \cap (A \cup C)
\end{align*}
We can now see that $A \cup(B \cap C) \subseteq (A \cup B) \cap (A \cup C)$
\subsection*{11. B)}
If $A \cap B = B$, then $A \cup B = A$\\
Let $B = \{x\}$ \\
If $x$ is $\in B$ and $B \cup A = A$ then $x$ $\in$ A. \\
If $x$ is $\in A$ then the set of $A \cup B$ is the set of $ \{x\} \cup \{x\}$ which is the same as $A \cup B$, which means $A \cup B = A$
\subsection*{19. B)}
If $A \cup B = B $, then $A \cup (B \cap A') = B$
\\ Let us begin by simplifying $A \cup (B \cap A')$
\begin{align*}
\\&= A \cup (B \cap A')  \ \ Absorbtion
\\ &= A \cap (B \cup A') \ \ Demorgan's
\\ &=  (A \cap A') \cup (A \cap B) \ \ Distributive
\\ &=  \emptyset \cup (A \cap B) \ \ negation
\\ &=  (A \cap B) \ \ identity
\\ &=  B \ \ hypothesis
\end{align*}
Therefore: $A \cup (B \cap A') = B$

\end{document}


\documentclass[12pt,letterpaper]{article}

%Packages
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amscd}
\usepackage{dsfont}
\usepackage{enumerate}
\usepackage{fancyhdr}
\usepackage{mathrsfs}
\usepackage{bbm}
\usepackage{framed}
\usepackage{mdframed}
\usepackage{cancel}
\usepackage{float}
\usepackage{mathtools}
%\usepackage[]{mcode}
\usepackage{graphicx}
\usepackage{tikz}
\usetikzlibrary{automata,arrows}

%Page formatting
\usepackage[letterpaper,voffset=-.5in,bmargin=3cm,footskip=1cm]{geometry}
\setlength{\parindent}{0.0in}
\setlength{\parskip}{0.1in}
\allowdisplaybreaks
\headheight 15pt
\headsep 10pt

% Common Commands
\newcommand\N{\mathbb N}
\newcommand\Z{\mathbb Z}
\newcommand\R{\mathbb R}
\newcommand\Q{\mathbb Q}
\newcommand\lcm{\operatorname{lcm}}
\newcommand\setbuilder[2]{\ensuremath{\left\{#1\;\middle|\;#2\right\}}}
\newcommand\E{\operatorname{E}}
\newcommand\V{\operatorname{V}}
\newcommand\Pow{\ensuremath{\operatorname{\mathcal{P}}}}

\DeclarePairedDelimiter\ceil{\lceil}{\rceil}
\DeclarePairedDelimiter\floor{\lfloor}{\rfloor}

% 22 style
\newcommand\hint[1]{\textbf{Hint}: #1}
\newcommand\note[1]{\textbf{Note}: #1}
\newenvironment{22enumerate}{\begin{enumerate}[a.]\itemsep0em}{\end{enumerate}}
\newenvironment{22itemize}{\begin{itemize}\itemsep0em}{\end{itemize}}
\fancypagestyle{firstpagestyle} {
  \renewcommand{\headrulewidth}{0pt}%
  \lhead{\textbf{CSCI 0220}}%
  \chead{\textbf{Discrete Structures and Probability}}%
  \rhead{Klivans}%
}

\pagestyle{fancyplain}

\newcommand\EX{\mathds{E}}
\newcommand\PR{\mathds{P}}
\newcommand\zlam{Z_\lambda}
\DeclareMathOperator*{\argmin}{arg\,min}

%%%%%%%%%%%%%%%% COMPILE WITH \soltrue or \solfalse %%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\newif\ifsol
\soltrue  % SOLUTIONS
\solfalse % NO SOLUTIONS
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%% solution help %%%%%%%%%%%%%%%%%%%%%
\newcommand{\solu}[2]{ \begin{mdframed} \ifsol #2 \else \vspace{#1} \fi \end{mdframed} }
\newcommand{\solt}{ \ifsol (T) \fi }
\newcommand{\solf}{ \ifsol (F) \fi }
\newcommand{\solm}[1]{\ifsol \textit{(#1)} \fi}

\begin{document}
  \thispagestyle{firstpagestyle}
  \begin{center}
    {\large \textbf{Recitation 7}}

    {\large Logic and Complexity}

  \end{center}

	\section*{Review}

	Operations on propositions:

\begin{tabular}{c|c|c|c|c|c|c|c}
 $P$ & $Q$ & $\lnot P$ & $P \wedge Q$ & $P \vee Q$ & $P \oplus Q$ & $P \implies Q$ & $P \iff Q$\\
\hline
T & T & F & T & T & F & T & T\\
T & F & F & F & T & T & F & F\\
F & T & T & F & T & T & T & F\\
F & F & T & F & F & F & T & T
\end{tabular}

where the notation is interpreted as follows
\begin{itemize}
\item $\lnot$ means ``not''
\item $\wedge$ means ``and''
\item $\vee$ means ``or''
\item $\oplus$ means ``or, but not both of'', which we call ``xor''
\item $\implies$ means ``implies''
\item $\iff$ means ``if and only if''
\end{itemize}

	\subsection*{Warm-Up}

	\begin{enumerate}[a.]

		\item Answer true or false for all of the following. ($\equiv$ means that each propositions always have the same truth value).
		\begin{enumerate}[i.]
			\item $p \lor q \equiv \neg p \land \neg q$ \solt
			\item $(p \land q) \lor r \equiv p \land (q \lor r)$ \solf
			\item $p \Rightarrow \neg q \equiv p \lor q$ \solt
			\item $p \Rightarrow q \equiv p \land \neg q$ \solf
			\item $p \Leftrightarrow q \equiv (p \land q) \lor (\neg p \land \neg q)$ \solt
			\item $p \land \neg q \equiv \neg (\neg p \lor q)$ \solt
			\item $(p \lor q) \land r \equiv (p \land r) \lor (q \land r)$ \solt
	\end{enumerate}

		\item Give an assigment to the variables $x_1,x_2,x_3$ which makes the following logical expression evaluate to true.

		$$(x_1 \lor x_2)\land (\neg x_1 \lor \neg x_2 \lor \neg x_3) \land x_3$$
		\begin{mdframed}
		\vspace{2cm}
		\end{mdframed}

		\item Is the following logical expression is always true? Explain your answer.

		$$(x_1 \land x_2)\lor (\neg x_1 \land \neg x_2 \land \neg x_3) \lor (x_3 \land \neg x_3) $$

			\begin{mdframed}
			\vspace{4cm}
			\end{mdframed}

		\item Come up with a logical expression with three variables which has \textbf{only one} assignment to the variables which makes it true.

			\begin{mdframed}
			\vspace{4cm}
			\end{mdframed}

		\item Given inputs $p$ and $q$, create a circuit which outputs $p \oplus q$ using only \textbf{not gates}, \textbf{or gates}, and \textbf{and gates}.

			\begin{mdframed}
			\vspace{6cm}
			\end{mdframed}

		\item Given inputs $x_1$, $x_2$ and $x_3$, create a circuit which outputs true if and only if exactly one input is true.

			\begin{mdframed}
			\vspace{10cm}
			\end{mdframed}

	\end{enumerate}

	\section*{Section Lesson - P vs. NP}

	\textit{Everything past this point is outside the scope of CS22, though it will reinforce concepts from the class. These concepts are covered further in CS1010 and CS157.}

	An algorithm is simply a series of steps. It can be expressed as a computer program, instructions in a cookbook, directions on a map, as a plain sentence, etc. For example, consider the following \texttt{maximum algorithm} that finds the maximum element given \texttt{list} as input.
\begin{verbatim}
v = -infinity
for each element x in list:
    v = max(v, x)
output v
\end{verbatim}
It is often helpful to decribe the \textbf{run time} of an algorithm. The \textbf{run time} can be thought of as the number of steps that the algorithm takes, and is often expressed as a function as the size of the input to the algorithm

For example, if $n$ is the size of the input list to the \texttt{maximum algorithm} then the algorithm has a run time of $f(n) = n$ steps.

Give an informal inductive argument as to why this is true. (Hint: Start by considering the base case of a list with one item. The one item is clearly the maximum so we return it in 1 step.)

\begin{mdframed}
\vspace{8cm}
\end{mdframed}

An algorithm is considered \emph{fast} if its run time is a \textbf{polynomial}. This polynomial can be $n^2$, or even $n^{100} + 4n^{63}$, it doesn't matter. If the run time is polynomial then it is feasible for a computer. If the run time is instead exponential, like $2^n$, then it is infeasible for a computer if $n$ is large. The terms \textbf{polynomial time} and \textbf{fast} are used synonymously for the rest of this handout.

	Consider the following algorithm which determines if there is any assignment to a logical expression with variables $x_1,...,x_n$ that is ever true. (The problem of determining if a logical expression evaluates to true is called \texttt{SAT}).

	\texttt{Try \textbf{every} assignment to the variables.}

	\texttt{If the logical expression ever evaluates to true, return true.}

	\begin{enumerate}[a.]
		\setcounter{enumi}{9}
		\item What is the run time of this algorithm in terms of $n$, the number of variables.

		\hint How many assignments do you have to try? Think in terms of 0/1 strings.
		\begin{mdframed}
		\vspace{3cm}
		\end{mdframed}
	\end{enumerate}

	P is the set of problems that can be solved in \textbf{polynomial time}. In other words, a problem in P can be solved by an algorithm that has a polynomial run time.

	Finding the maximum element in a list is a P problem, as is sorting a list.

	Unfortunately, the algorithm given above is the best known algorithm for determining if a logical expression ever evaluates to true (solving SAT). Therefore it is not a P problem (as far as we know).

	\subsection*{NP}

A problem is in NP if, given a candidate solution to the problem, there exists a fast algorithm to determine whether the candidate is in fact a valid solution.

	For example, determining if a logical expression ever evaluates to true (SAT) is an NP problem. We don't know how to quickly find an assignment to the variables which makes the expression true, but given an assignment to the variables we can quickly check that this assignment makes the expression true by just constructing the corresponding circuit.

		\begin{enumerate}[a.]
		\setcounter{enumi}{10}
		\item \textit{Challenge:} Another NP problem is the following: Given a number $n$ where $n=pq$ for two large primes $p$ and $q$, can you find $p$ and $q$? Explain why this is an NP problem.

		\begin{mdframed}
		\vspace{3cm}
		\end{mdframed}
	\end{enumerate}
	At first it looks like this is a P problem as well because you can simply check all pairs of numbers between 0 and $\sqrt{n}$. However, when you encode a number using 0's and 1's in a computer, it takes $\log_2(n)$ bits. Therefore looping through $\sqrt{n}$ numbers is exponential in the size of your input (we understand that this is confusing so please call over a TA if you wish to discuss this further).

	Most computer scientists beleive that P $\not=$ NP but no one can come up with a proof. There is a million dollar reward for finding one. If you were to prove that P = NP then RSA would be useless.

	\subsection*{\textit{Challenge:} NP-Hard}

	It turns out that for any problem in NP, you can come up with a logic circuit for it where determining if the circuit ever evaluates true corresponds to solving the original problem.

	Therefore, if you can find a ``fast" algorithm for determining if a logical circuit ever outputs true then P = NP.

	Changing one problem into another problem is called a ``reduction".

	A problem $p$ is NP-Hard if there is a reduction from every problem in NP to $p$.

	Another example of a problem that is NP-Hard is determining if a graph is 3-colorable, which you will learn about soon.

	\subsection*{\textit{Challenge:} Not in NP}

	There are problems that we think are even harder then NP problems. For example, given a logical expression, is it always true?

\end{document}


\documentclass[12pt]{scrartcl}

\input{preamble}

\makeatletter
\title{Hack 4.0}\let\Title\@title
\subtitle{Computer Science I -- Honors\\
{\small
\vskip1cm
Department of Computer Science \& Engineering \\
University of Nebraska--Lincoln}
\vskip-1cm}
%\author{Dr.\ Chris Bourke}
\date{~}
\makeatother

\begin{document}

\maketitle

\hrule

\input{instructions.tex}

\section*{Problem Statement}

A 401(k) plan is a type of tax-qualified account for people to save
for retirement.  Employees typically have part of their pre-tax pay
check deposited into such an account and employers may match a certain
amount of those contributions.  As of 2018, there is a maximum
annual contribution limit of \$18,500.

The money contributed to a 401(k) account may be invested in many
different ways.  While the account may grow tax free, it is not
immune to inflation.  To account for inflation, you can use
the following formula which is the inflation-adjusted rate of
return.
  $$\frac{1 + \textrm{rate of return}}{1+\textrm{inflation rate}} - 1$$

Write a program that produces an \emph{amortization table} for a
401(k) account.  Your program will read the following inputs as
command line arguments.
\begin{itemize}
  \item An initial starting balance
  \item A monthly contribution amount (we'll assume its the same over the life of the savings plan)
  \item An (average) annual rate of return (on the scale $[0, 1]$)
  \item An (average) annual rate of inflation (on the scale $[0, 1]$)
  \item A number of years until retirement
\end{itemize}
Your program will then compute a monthly savings table detailing
the (inflation-adjusted) interest earned each month, contribution, and
new balance.  To get the monthly rate, simply divide by 12.  Each month,
interest is applied to the balance at this rate (prior to the monthly
deposit) and the monthly contribution is added.  Thus, the earnings
compound month to month.  Be sure to round appropriately, to the nearest
cent so that rounding errors do not compound.

Your program will need to validate the input and quit with an error
message on any potential bad input(s).  For inputs

\mintinline{text}{10000 500 0.09 0.012 10}

your output should look something like the following:

\begin{figure}[H]
\begin{minted}{text}
Month    Interest     Balance
    1 $     64.23 $  10564.23
    2 $     67.85 $  11132.08
    3 $     71.50 $  11703.58
    4 $     75.17 $  12278.75
    5 $     78.87 $  12857.62
    6 $     82.58 $  13440.20
    7 $     86.33 $  14026.53
    8 $     90.09 $  14616.62
    9 $     93.88 $  15210.50
...
  116 $    678.19 $ 106767.24
  117 $    685.76 $ 107953.00
  118 $    693.37 $ 109146.37
  119 $    701.04 $ 110347.41
  120 $    708.75 $ 111556.16
Total Interest Earned: $  41556.16
Total Nest Egg: $ 111556.16
\end{minted}
\end{figure}

\section*{Instructions}

\begin{itemize}
  \item You are encouraged to collaborate any number of students
  before, during, and after your scheduled hack session.
  \item Design at least 3 test cases \emph{before} you begin
  designing or implementing your program.  Test cases are
  input-output pairs that are known to be correct using means
  other than your program.
  \item Include the name(s) of everyone who worked together on
  this activity in your source file's header.
  \item Place your code in a \mintinline{java}{main()} method in
    a source file named \mintinline{text}{Retire.java} and turn it
    in via webhandin.  Make sure that
    it runs and executes correctly in the webgrader. Each individual
    student will need to hand in their own copy and will receive their
    own individual grade.

\end{itemize}

\end{document}


 \documentclass [12pt]{article}

\usepackage {amsmath}
\usepackage {amsthm}
\usepackage {amssymb}
\usepackage {graphicx}
\usepackage {float}
\usepackage {multirow}
\usepackage {xcolor}
\usepackage [ruled,vlined,commentsnumbered,titlenotnumbered]{algorithm2e} \usepackage {array}
\usepackage {booktabs}
\usepackage {url}
\usepackage {parskip}
\usepackage [margin=1in]{geometry}
\usepackage [T1]{fontenc}
\usepackage {cmbright}
\usepackage [many]{tcolorbox}
\usepackage [colorlinks = true,
            linkcolor = blue,
            urlcolor  = blue,
            citecolor = blue,
            anchorcolor = blue]{hyperref}
\usepackage {enumitem}
\usepackage {xparse}
\usepackage {verbatim}
\usepackage{algpseudocode}
\usepackage{listings}
\usepackage{xcolor}
\lstset { %
    language=C++,
    backgroundcolor=\color{black!5}, % set backgroundcolor
    basicstyle=\footnotesize,% basic font setting
}
\newtheorem{theorem}{Theorem}
\newtheorem{remark}{Remark}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{corollary}[theorem]{Corollary}
\theoremstyle{definition}
\newtheorem{definition}{Definition}[section]
\newtheorem{claim}{Claim}
\newtheorem{proposition}{Proposition}

\DeclareTColorBox {Solution}{}{breakable, title={Solution}} \DeclareTColorBox {Solution*}{}{breakable, title={Solution (provided)}} \DeclareTColorBox {Instruction}{}{boxrule=0pt, boxsep=0pt, left=0.5em, right=0.5em, top=0.5em, bottom=0.5em, arc=0pt, toprule=1pt, bottomrule=1pt} \DeclareDocumentCommand {\Expecting }{+m}{\textbf {[We are expecting:} #1\textbf {]}} \DeclareDocumentCommand {\Points }{m}{\textbf {(#1 pt.)}}

\begin {document}

\vspace {1em}
\begin {Instruction}
Adapted From Virginia Williams'lecture notes.
\end {Instruction}

{\LARGE \textbf {COMP 285 (NC A\&T, Spr `22)}\hfill \textbf {Lecture 35} }

\begin{centering}
\section*{Stable Matching and Gale-Shapley}
\end{centering}

Every year, tens of thousands of doctors begin their residencies in the United States. The process of matching doctors to their residencies is extremely complex, and must take into account the doctor' preferences and the needs of the hospitals. In this lecture we will learn the Deferred Acceptance Algorithm which is at the core of The Match, a.k.a. the National Resident Matching Program. The algorithm is sometimes also called the GaleShapley algorithm after its inventors, David Gale and Lloyd Shapley.

\section{Stable Matching}
\textbf{Setting}:

Our goal is to match a collection of doctors and a collection of hospitals to each other. Throughout, we assume that each hospital has exactly one position to fill. This is just to simplify the verbiage and notation, and does not lose generality; in the more general case where hospitals have larger capacity, replace the word ``hospital'' by ``hospita'' position” in the ensuing text and everything still goes through. We also assume the number of hospitals and doctors is the same and we call that n. This is again to simplify the notation; when these numbers are not equal, some doctors (or hospitals) might have to remain unmatched. We can model this by adding fake hospitals (or doctors) that are ranked very low, and matching to them counts as being ``unmatched'' in real life.

Suppose that we had some numerical score for each pair of doctor $i$ and hospital $j$ hinting at how good of a fit pairing $i$ and $j$ is. Then, we may try to design an algorithm that finds an optimal (say max-weight) matching in this graph.

One problem with this approach is that we do not have numerical scores, so we have to rely on information from doctors and hospitals themselves. We might ask each doctor to rank the hospitals, and each hospital to rank the doctors. But even then, it is not straightforward how to translate the preferences of both a doctor and a hospital, two rankings, into a single numerical score. Any such scheme can also be prone to ``gaming'' by the doctors and the hospitals, because they can possibly manipulate the outcome and gain a better match by reporting false preferences.

Even worse, in order to use a centralized matching algorithm, you must convince thousands of residency programs to list their positions on your algorithm and commit to taking the doctors you assign them. So you better ensure that hospitals can't find better matches outside your algorithm! This motivates the definition of a stable matching:

\begin{definition}[Stable Matching]
We say that a bipartite matching between doctors and hospitals is stable if for every doctor $i$ and hospital $j$, one of the following happens:
\begin{enumerate}
    \item $i, j$ are matched to each other; or
    \item Hospital $j$ is matched to another doctor which it prefers over $i$; or
    \item Doctor $i$ is matched to another hospital which it prefers over $j$.
\end{enumerate}
\end{definition}

Intuitively, if doctor $i$ prefers Hospital $j$ to its match in the algorithm, and hospital $j$ prefers doctor i, then they would jointly prefer to find their match without your algorithm. Such a pair $(i, j)$ is called a blocking pair, and a matching is stable if it does not admit any blocking pairs.

So finding a stable matching resolves the problem of doctors and hospitals trying to bypass your algorithm. (There is still the issue of misrepresenting preferences. We will get back to
this issue in Section 4.)

\textit{Example}\label{ex:2}. Consider three doctors $\{Alice,Bob,Charlie\}$, and three hospitals $\{X, Y, Z\}$.

Suppose that Alice prefers Y over X over Z, etc:

\begin{table}[!h]
\centering
\begin{tabular}{|c|c|c|c|}
\hline Doctor & Alice & Bob & Charlie \\
\hline top choice & Y & X & X \\
2nd choice & X & Y & Y \\
last choice & Z & Z & Z \\
\hline
\end{tabular}
\end{table}

On the hospital side, their preferences are:
\begin{table}[!h]
\centering
\begin{tabular}{|c|c|c|c|}
\hline Hospital & X & Y & Z \\
\hline top choice & Alice & Charlie & Bob \\
2nd choice & Charlie & Alice & Charlie \\
last choice & Bon & Bob & Alice \\
\hline
\end{tabular}
\end{table}

\begin{itemize}
\item Observe that the matching
\begin{align}
(\text{Alice}-X), (\text{Bob}-Z), (\text{Charlie}-Y) \label{eq:1}
\end{align}
is stable: every hospital gets its favorite doctor, so no hospital wants to hire a doctor
outside the matching. (The doctors are less happy about it...)
\item Another stable matching is
\begin{align}
(\text{Alice} -Y), (\text{Bob}-Z), (\text{Charlie}-X) \label{eq:2}
\end{align}
\item The matching
\begin{align}
(\text{Alice} -Z), (\text{Bob}-X), (\text{Charlie}-Y) \label{eq:3}
\end{align}
is not stable, because Alice and $X$ prefer each other over their respective matches.
\end{itemize}

\section{The Algorithm}

The Deferred Acceptance Algorithm is essentially based on a greedy strategy: at each step we try to match each doctor to her most preferred hospital, except for hospitals that we already have tried and know are impossible.

There is a twist which distinguishes this algorithm from other greedy algorithms we've seen previously in the course. The algorithm's greedy choices are revocable, i.e., the matches are not final until the end of the algorithm. Even if, for example, doctor $i$ matched with hospital $j$ at the first round of the algorithm, if in a later iteration, doctor $k$ also wants to match with hospital $j$, we will break ties between the doctors based on hospital $j$'s preferences. In particular, if hospital $j$ prefers $k$ over $i$, then the previous match between $j$ and $i$ is canceled, and doctor $i$ is sent to find a new hospital.

\begin{algorithm}
\caption{Gale-Shapley, a.k.a the Deiferred Acceptance Algorithm}
\label{alg:da_alg}
\begin{algorithmic}
\State \textbf{Input}:
 $n \times n$ matrices $D$ and $H$. The $i$-th row of $D$ is the list of hospitals in the order of the most preferred to the least preferred from doctor $i$'s view. $H[j][i]$ is the rank of the doctor $i$ from the hispitals $j$'s perspective (lower number means more preferred) \\
\State \textbf{Output}: Matching between doctors and hospitals.
\State \texttt{// Initialization}
\State \For{ $d \in $ doctors} {
    \State $d.i \gets 0$
}
\State \For{ $h \in $ hospitals} {
    \State $h.doctor \gets $ \texttt{NIL}
}
\State freeDoctors $\gets$ doctors
\State \texttt{While there is a free doctor, we keep matching}
\State \While{freeDoctors $\neq \emptyset$} {
    \State Pick and $d \in $ freeDoctors
    \State \texttt{// h is the next preferred hospital for d}
    \State $d.i \gets d.i + 1$
    \State $h \gets D[d][d.i]$
    \State \texttt{// If h preferes d over previous match switch}
    \State \If{$H[h][d] < H[h][h.doctor]$} {
        \State Add $h$.doctor to freeDoctors
        \State Remove $d$ from freeDoctors
        \State $h.$doctor $\gets d$
    }
}
\State \Return the matching $\{(h.\text{doctor}, h) \mid \text{ hospitals } h\}$
\end{algorithmic}
\end{algorithm}

In this section, we will prove that Algorithm \ref{alg:da_alg} always outputs a stable matching. Let us start by proving some observations about the behavior of Algorithm \ref{alg:da_alg}.
\begin{proposition}
If $h$.doctor becomes not \texttt{NIL}, it will never become \texttt{NIL} again.
\end{proposition}

\begin{proof}
We only assign \texttt{NIL} values at initialization. So once a hospital has a potential match it can never run out of matches again; it will only reject a potential match for another match (a more preferred one).
\end{proof}

\begin{proposition} In Algorithm \ref{alg:da_alg}, no doctor ``runs out of'' hospitals to try. In other words $d.i$ never becomes n + 1.
\end{proposition}

\begin{proof} If a doctor $d$ were to run out of hospitals to try, it means that they must have already tried all hospitals. But this means every hospital already has a potential match; at the time the hospital rejected $d$, they must have already had another match, and by the previous proposition, the hospital's match never becomes \texttt{NIL} from that point onwards.

But now we get a contradiction. There are $n$ hospitals, each of which has a current match other than $d$. So all of the current hospital matches must come from the set of $n-1$ doctors other than $d$, which is impossible.
\end{proof}

\begin{proposition} The algorithm terminates in at most $O(n^2) $iterations and returns a matching.
\end{proposition}
\begin{proof}
In every iteration, some $d.i$ gets incremented by $1$. Since they start at $0$ and can never reach $n + 1$, in total the number of increments is at most $n \times O(n) = O(n^2 )$. By the previous proposition, when the algorithm terminates, we have a full matching.
\end{proof}

Now that we know Algorithm \ref{alg:da_alg} terminates and returns a matching, it only remains to prove that the matching is stable.

\begin{theorem}
The matching returned by Algorithm \ref{alg:da_alg} is stable.
\end{theorem}

\begin{proof}
First note that a hospital's matches only improve over the course of the algorithm. In other words, for every $h$, $h$.doctor's ranking according to $h$ can only get better and better as we run through the algorithm. This is because $h$ will only reject a potential match for a better one.

Let us call the matching returned by Algorithm \ref{alg:da_alg}, $\sigma$. Assume by contradiction, that there is a blocking pair $(i, j)$ for $\sigma$. This means that $i$ prefers $j$ to her match in $\sigma$ and $j$ prefers $i$ to its match in $\sigma$. Let $h$ be the hospital that $i$ got matched to in $\sigma$. Since doctors go over their preference list one-by-one in Algorithm \ref{alg:da_alg} and $h$ comes later in the list of $i$ than $j$, this means at some point during the run of the algorithm $j$ must have been ``proposed to'' by $i$. So $j $had the opportunity to keep $i$ or potentially reject $i$ for better and better matches. But j ended up with a match that's worse than $i$; this is a contradiction, since $j$'s matches must have only gotten better throughout the algorithm.
\end{proof}

\section{Doctor-optimality and incetive compatibility}

Let $\sigma$ be a stable matching. We say that it is doctor-optimal if for any other stable matching $\sigma'$ and any doctor $d$, the doctor $d$ \textit{weakly prefer}s her hospital in $\sigma$ to her hospital in $\sigma'$ . (\textit{Weakly prefer} means that it is either better or the same.) Note that a priori it is not clear that doctor-optimal stable matchings even exist.

In Example \ref{ex:2}, the doctor-optimal matching is given in Eq. (\ref{eq:2}), and Eq. (\ref{eq:1}) is the stable matching that hospitals prefer. This is not a coincidence: the doctor-optimal stable matching is provably always the hospital-worst stable matching.

\begin{theorem}[(Doctor-optimality)]
The output of Algorithm 1 is doctor-optimal.
\end{theorem}
\begin{proof}
For each doctor $d$, let $h^*(d)$ be her best feasible hospital, i.e,. the one she prefers the most among any hospital she can be matched to in any stable matching. Assume by \textit{contradiction} that Algorithm \ref{alg:da_alg} is not doctor-optimal. This means some doctor $d$ gets rejected by her best possible choice $h^*(d)$ at some point during the algorithm. Consider the first time during the run of Algorithm \ref{alg:da_alg} where this happens. This means $h^*(d)$ rejects $d$ because a new doctor $d'$ tries to match with it and $h^*(d)$ prefers $d'$ over $d$. Now there are two cases:

\begin{itemize}
    \item $h^*(d') = h^* = h^*(d)$, i.e., they both have the same best feasible hospital $h^*$ . Therefore Hospital $h^*$ will always prefer $d'$ over $d$, and $d'$ always prefers it to its match in any stable matching. Therefore, it is never stable to match $h^*$ to $d$ ($d'$ and $h^*$ would be a blocking pair). But this is a \textit{contradiction} to $h^*$ being feasible for $d$!
    \item $h^*(d') \neq h^*(d)$, in which case doctor $d'$ should have first tried to match with $h^*(d')$ before coming to $h^*(d)$. If $d'$ tries to match with $h^*(d)$ instead, it must have been rejected by $h^*(d')$. But this is a \textit{contradiction} to $d$ being the first doctor to be rejected by her best feasible hospital!
\end{itemize}
\end{proof}

\textbf{Incentive compatibility}.

Consider a doctor that strategically misreports her preferences in hope of a better hospital. Intuitively, given the new (misreported) preferences, Algorithm \ref{alg:da_alg} should still output a stable matching. By the doctor-optimality theorem that we just proved, the doctor is already assigned the best match she can possibly be assigned in any stable matching; hence misreporting preferences cannot help her. This logic is actually flawed (can you see why?\footnote{Given the misreported preferences, Algorithm \ref{alg:da_alg} should still output a matching that is stable with respect to the misreported preferences. But the doctor-optimality theorem only guaranteed that the output is doctor optimal among all matchings that are stable with respect to the true preferences} ), but the conclusion is still true, namely no doctor has an incentive to misreport her preferences:

\begin{theorem}[Incentive-compatibility]
Misreporting the preference over hospitals can never improve a doctor’s match computed by Algorithm \ref{alg:da_alg}.
\end{theorem}

The correct proof of this theorem is beyond the scope of these notes. It can be found, e.g., in this research paper by Dubins and Freedman: \href{https://www-jstor-org.stanford.idm.oclc.org/stable/2321753}{https://www-jstor-org.stanford.idm.oclc.org/stable/2321753}.

As a remark, unlike doctors, hospitals can actually misreport and potentially get better
matches. To appreciate the incentive compatibility for the doctors, let us revisit Example \ref{ex:2}, and see how hospitals can strategically misreport their preferences to improve their matching.

\textit{Example 10}. Suppose hospital $X$ misreports its preferences as in Table 3 (but doctors' preferences are still the same).

\begin{table}[h!]
\begin{tabular}{|c|c|c|c|}
\hline Hospital & X & Y & Z \\
\hline top choice & Alice & Charlie & Bob \\
2nd choice & \textbf{Bob} & Alice & Charlie \\
last choice & \textbf{Charlie} & Bob & Alice \\
\hline
\end{tabular}
\end{table}

Now Eq. (\ref{eq:1}) is the unique stable matching, so the Deferred Acceptance Algorithm will choose it, and not Eq. (\ref{eq:2}) (can be verified by running the algorithm).

\end{document}


\documentclass[11pt]{article}
\usepackage[top=2cm,bottom=2cm]{geometry}
\usepackage[T1]{fontenc}
\usepackage{url}
\usepackage{hyperref}
\usepackage{xcolor}
%
\usepackage{amsthm}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{ifthen}
\usepackage{mathpartir}
\usepackage{mathtools}
\input{../../slides/macros}
%
\begin{document}

\title{CO663 Part II: Worksheet 3 (\emph{assessed})}

\date{\vspace{-5ex}}
\maketitle

% \pagenumbering{gobble}

\newcommand{\answerbox}[1]{\framebox{\parbox[c][#1]{\textwidth}{
      \color{white}{h}%
    }}}

\section{Introduction}

The objective of this class is to put in practice the typing and
semantic rules for language $\lggeE$.
%
This is an individual exercise.

\begin{center}
  \emph{This class assessment is worth {\color{red} 6} marks.}
\end{center}

Submit your work via Moodle (within the deadline specified on the
Moodle page).

\section{Exercises}

\begin{enumerate}
\item Formally type-check \emph{and} evaluate (until your reach a value) the
following expressions:
\begin{enumerate}
\item $\elet{\enum{3}}{{z}}{\etimes{\var{z}}{\enum{2}}}$ (its type is $\enum{}$)
\item $\eplus{\etimes{\enum{12}}{\enum{0}}}{\elen{\estr{hi}}}$ (its type is $\enum{}$).
\end{enumerate}
\item Write down your own syntactically correct and well-typed term
  (it must be have a \emph{depth of 3}, at least). Show
  that is well-typed and evaluate it fully.
\end{enumerate}

Each (type or semantic) derivation is worth 1 mark.

%

\section{Additional information}
The depth of term $e$, written $\lvert e \rvert$, is the depth of its
abstract syntax tree.
%
It is is defined formally as follows:
%
\[
\lvert e \rvert =
\begin{cases}
  1 & \text{if } e = \var{x}, \ e = \enum{n}, \ e = \estr{s}, \ e = \ktrue, \ \text{or } e = \kfalse
  \\
  1 {+}  \lvert e' \rvert  & \text{if } e = \elen{e'}
  \\
  1   {+} \textit{max}(\lvert e_1 \rvert, \lvert e_2 \rvert)  & \text{if }  e = \elet{e_1}{x}{e_2}, \ e = \etimes{e_1}{e_2}, \  e = \eplus{e_1}{e_2}, \ \text{etc.}
  \\
  1   {+} \textit{max}(\lvert e' \rvert, \lvert e_1 \rvert, \lvert e_2 \rvert)  & \text{if } e = \site{e'}{e_1}{e_2}
\end{cases}
\]
For instance,
\begin{itemize}
\item $\lvert \elet{\enum{3}}{{z}}{\etimes{\var{z}}{2}} \rvert = 3$,
\item $\lvert \eplus{\etimes{\enum{12}}{\enum{0}}}{\elen{\estr{hi}}} \rvert = 3$.
\end{itemize}

\section*{\LaTeX\ template}

If you would like to use \LaTeX\ to typeset your solution, you can use
the template here:
\url{https://github.com/ukc-co663/pl-design-latex-template}.

\end{document}

\appendix

\section{Model answer}

\subsection{Exercise 2.1.a}

\paragraph{Typing}

\[
\tyrule{let}
{
  \tyrule{num}
  {\,}
  {
    \tyjudge{\varnothing}{\enum{3}}{\enum{}}
  }
  \quad
  \\
  \tyrule{times}
  {
    \tyrule{var}
    {\,}
    {
      \tyjudge{\var{z} : \enum{}}{\var{z}}{\enum{}}
    }
    \\
    \tyrule{num}
    {\,}
    {
      \tyjudge{\var{z} : \enum{}}{\enum{2}}{\enum{}}
    }
  }
  {
    \tyjudge
    {\var{z} : \enum{}}
    {\etimes{\var{z}}{\enum{2}}}
    {\enum{}}
  }
}
{
  \tyjudge{\varnothing}
  {\elet{\enum{3}}{{z}}{\etimes{\var{z}}{\enum{2}}}}
  {\enum{}}
}
\]

\paragraph{Evaluation}

\begin{enumerate}
\item First step/tree:
  \[
  \semrule
  {letn}
  {\,}
  {\jtrans
    {\elet{\enum{3}}{{z}}{\etimes{\var{z}}{\enum{2}}}}
    {\etimes{\enum{3}}{\enum{2}}}
  }
  \]
\item Second step/tree:
  \[
  \semrule
  {letn}
  {3 \times 2 = 6 }
  {\jtrans
    {\etimes{\enum{3}}{\enum{2}}}
    {\enum{6}}
  }
  \]
\end{enumerate}

\subsection{Exercise 2.1.b}

\paragraph{Typing}

\[
\tyrule{plus}
{
  \tyrule{times}
  {
    \tyrule{num}
    {\,}
    {  \tyjudge{\varnothing}
      {\enum{12}}
      {\enum{}}}
    \\
    \tyrule{num}
    {\,}
    {  \tyjudge{\varnothing}
      {\enum{0}}
      {\enum{}}}
  }
  {
    \tyjudge{\varnothing}
    {\etimes{\enum{12}}{\enum{0}}}
    {\enum{}}
  }
  \\
  \tyrule{len}
  {
    \tyrule{str}
    {\,}
    {
      \tyjudge{\varnothing}
      {\estr{hi}}
      {\estr{}}
    }
  }
  {
    \tyjudge{\varnothing}
    {\elen{\estr{hi}}}
    {\enum{}}
  }
}
{
  \tyjudge{\varnothing}
  {\eplus{\etimes{\enum{12}}{\enum{0}}}{\elen{\estr{hi}}}}
  {\enum{}}
}
\]
\paragraph{Evaluation}

\begin{enumerate}
\item First step/tree:
  \[
  \semrule
  {pll}
  {
    \semrule
    {tmval}
    {12 \times 0 = 0}
    {
      \jtrans
      {\etimes{\enum{12}}{\enum{0}}}
      {\enum{0}}
    }
  }
  {\jtrans
    {\eplus{\etimes{\enum{12}}{\enum{0}}}{\elen{\estr{hi}}}}
    {\eplus{\enum{0}}{\elen{\estr{hi}}}}
  }
  \]
\item Second step/tree:
  \[
  \semrule
  {plr}
  {
    %
    {\valjudge{\enum{0}}}
    \\\qquad
    \semrule
    {lenv}
    {\lvert hi \rvert = 2}
    {
      \jtrans
      {\elen{\estr{hi}}}
      {\enum{2}}
    }
  }
  {\jtrans
    {\eplus{\enum{0}}{\elen{\estr{hi}}}}
    {\eplus{\enum{0}}{\enum{2}}}
  }
  \]
\item Third step/tree:
  \[
  \semrule
  {plval}
  {0+2=2}
  {\jtrans
    {\eplus{\enum{0}}{\enum{2}}}
    {\enum{2}}
  }
  \]

\end{enumerate}

\end{document}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: t
%%% End:


%
% body.tex
%
% Copyright © 2020 Libao Jin <jinlibao@outlook.com>
% Distributed under terms of the MIT license.
%
The deadline will be strictly enforced. If you do not submit in time there will be a $20\%$ penalty for each day you're late. Remember that you are allowed to work in teams of two on this assignment. You are encouraged to prepare your work in \LaTeX{}; a template will be provided to help you put it all together. If you choose  to submit a hard copy, you may submit only one copy for a team, indicating the names of both contributors. Online submission is encouraged, however, in that case both members of a team should submit the PDF file containing  their work and showing both their names.

\emph{All plots generated in this homework should have a title, legend, and labeled $x$ and $y$-axes.} \\[15pt]

\textbf{Instruction}

\begin{enumerate}[label={\arabic*.}]
  \item Go to \url{https://www.overleaf.com} and sign in (required).
  \item Open \href{https://www.overleaf.com/read/qdhgzdjpwbhj}{template}, click \emph{Menu} (up left corner), then \emph{Copy Project}.
  \item Go to \verb|LaTeX/meta.tex| (the file \verb|meta.tex| under the folder \verb|LaTeX|) to change the section and your name, e.g.,
    \begin{itemize}
      \item change title to \verb|\title{MATH 3340-01 Scientific Computing Homework 5}|
      \item change author to \verb|\author{Albert Einstein \& Carl F. Gauss}|
    \end{itemize}
  \item For Problem 2 and 3, you can encouraged to type the solution in \LaTeX{}. But if you want to write it on the printout, please make sure your scanned work is clear enough (failure to do so will lead to points deduction).
  \item For Problem 1, you need to write function/script files, store results to output files, and save graphs to figure files. Here are suggested names for function files, script files, output files, and figure files:
    \begin{table}[!hbtp]
      \centering
      % \caption{caption}
      % \label{tab:label}
      \begin{tabular}{cllll}
        \toprule
        Problem     & Function File     & Script File     & Output File       & Figure File         \\
        \midrule
        1(a)        & \verb|lagrange.m| & \verb|hw5_p1.m| &                   &                     \\
        1(b)        &                   &                 &                   & \verb|hw5_p1_b.pdf| \\
        1(c)        &                   &                 &                   & \verb|hw5_p1_c.pdf| \\
        \bottomrule
      \end{tabular}
    \end{table}

    Once finished, you need to upload these files to the folder \verb|src| on Overleaf. If you have different filenames, please update the filenames in \verb|\lstinputlisting{../src/your_script_name.m}| accordingly. You can code in the provided files in \href{https://libaoj.in/courses/2020f/MATH3340/Homework/5/hw5.zip}{hw5.zip}, and use the MATLAB script \verb|save_results.m| to generate the output files and store the graphs to \verb|.pdf| files automatically (the script filenames should be exactly same as listed above).
  \item Recompile, download and upload the generated PDF to WyoCourses.
  \item You may find \href{https://libaoj.in/files/LaTeX.Mathematical.Symbols.pdf}{\LaTeX{}.Mathematical.Symbols.pdf} and the second part of \href{https://libaoj.in/courses/2020f/MATH3341/slides/Math.3341.Lab.01.Slides.pdf}{Lab 01 Slides} and \href{https://libaoj.in/courses/2020f/MATH3341/slides/Math.3341.Lab.02.Slides.pdf}{Lab 02 Slides} helpful.
\end{enumerate}
\newpage

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Problem 1
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Problem 1}%
\label{sec:problem_1}
The purpose of this problem is two-fold. First we will construct an interpolating polynomial and compare this polynomial to the original function from which the data is extracted. Then we will investigate how the choice of nodes can affect the accuracy of a higher order interpolating polynomial.

You will notice that the interpolant will be  different depending on the number and type of nodes used. Contrary to our intuition, the use of higher order polynomials does not necessarily guarantee an increase in global accuracy unless careful judgement is exercised. This should become clear when examining the plots you will generate in this problem.

We will consider the function
\begin{equation*}
  f(x)
  = \frac{1}{1 + 16 x^{2}}
\end{equation*}
and construct the interpolating polynomial using the Lagrange interpolant formula given by
\begin{equation}
  \label{eq:lagrange}
  p(x)
  = \sum_{k = 0}^{n} y_{k} L_{k}(x),
  \quad \text{where $L_{k}(x) = \prod_{\substack{j = 0 \\ j \ne k}}^{n} \frac{x - x_{j}}{x_{k} - x_{j}}$}.
\end{equation}
Please note that you are not supposed to use the standard form of the polynomial. Use the Lagrange form instead, as indicated above.
\begin{enumerate}[label=(\alph*)]
  \item \label{enum:a} As usual, you will create a minimum of two code files for this problem: a function file, and a script file.
    \begin{enumerate}[label=(\roman*)]
      \item Write a function file, to be used in both part \ref{enum:b} and \ref{enum:c} below, that calculates the interpolating polynomial using the Lagrange interpolant in \eqref{eq:lagrange}. Your function file should have the inputs:
        \begin{lstlisting}[style=Plain,numbers=none]
xdata = the set of nodes x_k,
ydata = the values y_k = f(x_k),
x = the point, or set of points (fine grid), used to evaluate p(x)
        \end{lstlisting}
        and the output:
        \begin{lstlisting}[style=Plain,numbers=none]
p = the value(s) of p at the point (or set of points) x
        \end{lstlisting}
      \item Create a script file which will execute the steps outlined in parts \ref{enum:b} and \ref{enum:c}. The function and grid will be the same for both problems so at the top of this script file you should define the function $f(x)$, a set $x$ of 100  equispaced points in $[-1, 1]$, and the associated exact values $y = f(x)$. This set will be used to evaluate and plot your interpolating polynomial. In other words, these are plotting points, not the data points.
    \end{enumerate}
  \item \label{enum:b} Compute the interpolating polynomial for $f(x)$ using data extracted for equispaced nodes.
    \begin{enumerate}[label=(\roman*)]
      \item First you will need to generate the values needed to construct the interpolating polynomial. This is the set $\{x_{k}, y_{k}\}$, where $x_{k}$ is a set of equispaced nodes in $[-1, 1]$ with
          \begin{equation*}
            -1 = x_{0} < x_{1} < \cdots < x_{n} = 1.
          \end{equation*}
          and $y_{k}$ is the value of the function at each node (i.e., $y_{k} = f(x_{k})$). Use $n = 9$ (i.e., 10 nodes) and label this set \verb|{ xdata, ydata }|.
        \item Use the function and grid you define in part \ref{enum:a} to compute the values of the polynomial interpolant \verb|p| with the information provided by \verb|xdata| and \verb|ydata|. Then create one plot which includes the following
          \begin{itemize}
            \item The values of the interpolant vs \verb|x|.
            \item The original function $f(x)$ vs \verb|x|.
            \item The set $\{x_{k}, y_{k}\}$. Plot this data set with a visible symbol (i.e., by using a marker like \verb|*|) but don't joint these point by a line.
          \end{itemize}
    \end{enumerate}
  \item \label{enum:c} Repeat the steps for part \ref{enum:b} but this time use nodes that are not equispaced in $x$. Instead, use the Chebyshev nodes $\{x_{k}\}$ given by
    \begin{equation*}
      x_{k} = -\cos(k \pi / n), k = 0, 1, \ldots, n
    \end{equation*}
    again with $n = 9$. This will be your new data set.
    (In this notation the points are numbered starting from zero; to create a MATLAB vector you will need to shift the index by one.)
\end{enumerate}
You may find it easier to write portions of these calculations as separate function or script files. You are more than welcome to do so.

\begin{solution}
  \quad
  \begin{itemize}
    \item
      Function file \verb|lagrange.m|
      \lstinputlisting[style=MATLAB]{../src/lagrange.m}
    \item
      Script file \verb|hw5_p1.m|
      \lstinputlisting[style=MATLAB]{../src/hw5_p1.m}
      \newpage
    \item
      Figure files: \verb|hw5_p1_b.pdf| (Figure \ref{fig:p1b}) and \verb|hw5_p1_c.pdf| (Figure \ref{fig:p1c}).
      \begin{figure}[!hbtp]
        \centering
        \includegraphics[width=0.75\linewidth]{../src/hw5_p1_b.pdf}
        \caption{Lagrange Interpolation for Equispaced Nodes}%
        \label{fig:p1b}
      \end{figure}
      \begin{figure}[!hbtp]
        \centering
        \includegraphics[width=0.75\linewidth]{../src/hw5_p1_c.pdf}
        \caption{Lagrange Interpolation for Chebyshev Nodes}%
        \label{fig:p1c}
      \end{figure}
  \end{itemize}
\end{solution}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Problem 2
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Problem 2}%
\label{sec:problem_2}
This problem must be done by hand. Consider the data set
\begin{table}[!hbtp]
  \centering
  % \caption{}
  \label{tab:label}
  \begin{tabular}{ccccc}
    \toprule
    $k    $ & $0   $ & $1   $ & $2   $ & $3   $ \\
    \midrule
    $x_{k}$ & $0.1 $ & $0.3 $ & $0.5 $ & $0.7 $ \\
    $y_{k}$ & $1.01$ & $1.35$ & $2.11$ & $2.95$ \\
    \bottomrule
  \end{tabular}
\end{table}
and use it to approximate the value the function $y = f(x)$ at the point $x = 0.45$. To do this, construct the table of values $Q_{ij}(x)$ for the recursive pointwise evaluation of the interpretation polynomial (see the typed notes section 6.2.3, page 87 and following). Display all entries in your table with $3$ digit accuracy.
\begin{solution}
  The Lagrange Approximation is shown in Table \ref{tab:lagrange}.
  \begin{table}[!hbtp]
    \centering
    \caption{Lagrange Approximation to $y = f(x)$}
    \label{tab:lagrange}
    \begin{tabular}{c|c|C{3.3cm}|C{3.3cm}|C{3.3cm}|C{3.3cm}}
      \toprule
      $k$ & $x_{k}$ & $Q_{k0}$ & $Q_{k1}$ & $Q_{k2}$ & $Q_{k3}$ \\
      \midrule
      $0$ & $0.1$  & $1.0100$    & $     0$    & $     0$    & $     0$         \\
      $1$ & $0.3$  & $1.3500$    & $1.6050$    & $     0$    & $     0$         \\
      $2$ & $0.5$  & $2.1100$    & $1.9200$    & $1.8806$    & $     0$         \\
      $3$ & $0.7$  & $2.9500$    & $1.9000$    & $1.9125$    & $1.8992$         \\
      \bottomrule
    \end{tabular}
  \end{table}

  Here are detailed steps to obtain the table:
  \newpage \quad \vfill   % DELETE THIS LINE IF YOU TYPE THE ANSWER IN LaTeX
\end{solution}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Problem 3
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Problem 3}%
\label{sec:problem_3}
This problem must also be done by hand. You again need to approximate the value of the same function $y = f(x)$, this time at the point $x = 0.35$. Using the same data set as in Problem 2 above, compute the complete table of Newton's divided differences and then construct the resulting interpolation polynomial. Use this polynomial to approximate $y = f(x)$ at the point $x = 0.35$.
\begin{solution}
  The approximation using Newton's divided-difference is shown in Table \ref{tab:newton}.
  \begin{table}[!hbtp]
    \centering
    \caption{Lagrange Approximation to $y = f(x)$}
    \label{tab:newton}
    \begin{tabular}{c|c|C{3.3cm}|C{3.3cm}|C{3.3cm}|C{3.3cm}}
      \toprule
      $k$ & $x_{k}$ & $f[x_{k}]$ & $f[x_{k - 1}, x_{k}]$ & $f[x_{k - 2}, x_{k - 1}, x_{k}]$ & $f[x_{k - 3}, x_{k - 2}, x_{k - 1}, x_{k}]$ \\
      \midrule
      $0$ & $0.1$  & $1.0100$    & $     0$    & $     0$   & $      0$  \\
      $1$ & $0.3$  & $1.3500$    & $1.7000$    & $     0$   & $      0$  \\
      $2$ & $0.5$  & $2.1100$    & $3.8000$    & $5.2500$   & $      0$  \\
      $3$ & $0.7$  & $2.9500$    & $4.2000$    & $1.0000$   & $-7.0833$  \\
      \bottomrule
    \end{tabular}
  \end{table}

  The resulting interpolation polynomial is
  \\[1cm]                   % DELETE THIS LINE IF YOU TYPE THE ANSWER IN LaTeX

  Therefore, the approximate value of $y = f(x)$ at $x = 0.35$ is
  \\[1cm]  $f(x) =   1.5139$.               % DELETE THIS LINE IF YOU TYPE THE ANSWER IN LaTeX

  Here are detailed steps to obtain the table:
  \newpage \quad \vfill     % DELETE THIS LINE IF YOU TYPE THE ANSWER IN LaTeX
\end{solution}


\documentclass[11pt]{article}
\usepackage{fancyhdr}
\usepackage{color}
\usepackage{multicol}
\usepackage{fancyvrb}
\usepackage{enumitem}
\usepackage{graphicx}
\usepackage{sectsty}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{hyperref}
\usepackage{array}
\newcommand{\sectionbreak}{\clearpage}

\usepackage{tikz}
\usepackage{tkz-euclide}
\usetkzobj{all}

\allsectionsfont{\centering}

\usepackage{draftwatermark}
	\SetWatermarkText{\copyright wolf-math.com}
	\SetWatermarkScale{4}
	\SetWatermarkLightness{1}

\usepackage[margin=1in, headsep=0pt]{geometry}
\setlength{\parindent}{0cm}
\pagestyle{empty}

\begin{document}
wolf-math.com\\

\section*{Systems of Linear Functions}

\subsection*{Goals}

\textbf{SWBAT} solve systems of equations by graphing.\\

\textbf{SWBAT} solve systems of equations by substitution.\\

\textbf{SWBAT} solve systems of equations by elimination.\\

\subsection*{Standards}

\textbf{Creating Equations \hfill A-CED}\\

Create equations that describe numbers or relationships\\

3.	 Represent constraints by equations or inequalities, and by systems of
equations and/or inequalities, and interpret solutions as viable or non-
viable options in a modeling context. For example, represent inequalities
describing nutritional and cost constraints on combinations of different
food\\

\subsection*{Connections}

\textbf{Before} we learned about graphing lines and linear functions.  \\

\textbf{Now} we are learning about what happens when many linear function come together.\\

\textbf{Later} we will learn what happens when two linear functions are multiplied; we get a quadratic function.\\

\let\stdsection\section
\renewcommand\section{\newpage\stdsection}

\section*{System of Equations}

\subsection*{Solving by Graphing}

\textbf{Systems of Equations:} A set of equations that you deal with all at once.
One method of finding the solution to a system of equations is to find where their graphs overlap.\\

Example:

\begin{equation*}
  \begin{cases}
	y=-2x+5\\

	y=\frac{2}{3}x-\frac{1}{3}\\
  \end{cases}
\end{equation*}

If we graph both of these equations we get the following graph.\\

\begin{center}
%\includegraphics[scale=.4]{systems1.jpg}\\
\end{center}

We can see that the lines overlap at the point $(2,1)$. This is the solution to our system.

\hrulefill

\textbf{You Try:} Solve the system of equations by graphing.\\

\begin{multicols}{2}

\begin{equation*}
	\begin{cases}
		y=-x+2\\
		y=x-2
	\end{cases}
\end{equation*}

\begin{center}
%\includegraphics[scale=.4]{graph1.jpg}
\end{center}

\begin{equation*}
	\begin{cases}
		y=\frac{1}{2}x-2\\
		y=-3x+5
	\end{cases}
\end{equation*}

\begin{center}
%\includegraphics[scale=.4]{graph1.jpg}
\end{center}

\end{multicols}

\pagebreak

\textbf{No Solution:} If two lines have the same slope that means that they will never intersect, which means there is no solution.

\begin{multicols}{2}
\begin{equation*}
	\begin{cases}
			y=\frac{5}{3}x-3\\

			y=\frac{5}{3}x+2
	\end{cases}
\end{equation*}

%\includegraphics[scale=.5]{graph1.jpg}\\

\begin{equation*}
	\begin{cases}
			y=\frac{2}{5}x+1\\

			y=\frac{2}{5}x+2
	\end{cases}
\end{equation*}

%\includegraphics[scale=.5]{graph1.jpg}\\

\end{multicols}

\pagebreak

\textbf{Infinite Solutions:} If two lines overlap eachother, then there is infinite solutions. They're the same line! Don't get this confused with \textbf{No Solution} because both of them have the same slope.\\

\begin{equation*}
	\begin{cases}
			y=\frac{1}{2}x+1\\

			 y=\frac{1}{2}x+1\\
	\end{cases}
\end{equation*}

\begin{center}
%\includegraphics[scale=.5]{graph1.jpg}
\end{center}

\pagebreak

\begin{center}
	\begin{Large}
		Solving Systems by Substitution\\
	\end{Large}
\end{center}

Systems of equations can be solved using a common variable, and setting them equal to one another.

\begin{equation*}
	\begin{cases}
		y=-x+2\\
		y=x-2
	\end{cases}
\end{equation*}

Since both equations begin with $y$, they can be set equal to one another.

$$-x+2=x-2$$

Now just solve for $x$

$$2x=4$$\ $$x=2$$\

Plug the solution $x=2$ into one of the equations to solve the system\\

$$y=(2)-2=0$$

so the final answer is $(2,0)$

\hrulefill

\textbf{You Try:} Solve the system of equations\\

\begin{equation*}
	\begin{cases}
		y=\frac{1}{2}x-2\\
		y=-3x+5
	\end{cases}
\end{equation*}

\vspace{1cm}

$$\frac{1}{2}x-2=-3x+5$$

\pagebreak

\textbf{Solve the Systems of Equations} by graphing.\\

\begin{enumerate}

	\item \begin{equation*}
		\begin{cases}
			y=3x-2\\
			y=-3x+2
		\end{cases}
	\end{equation*}

\vspace{2in}

	\item \begin{equation*}
		\begin{cases}
			y=4x-1\\
			y=-x+4
		\end{cases}
	\end{equation*}

\vspace{2in}

	\item \begin{equation*}
		\begin{cases}
			y=-\frac{3}{2}x-4\\
			y=\frac{1}{2}x+4
		\end{cases}
	\end{equation*}

\end{enumerate}

\section*{Solving by Elimination}

Given a system of equations, we can solve them by elimination. This means looking at the equations \textit{vertically} rather than \textit{horizontally}. Usually when we solve systems by elimination it is when the equations are in \textbf{Standard Form}, that means that they are not in \textbf{Slope Intercept Form}.\\

\textbf{Example:}\\

$\begin{cases}
	2x+y=9\\
	3x-y=16
\end{cases}$

\vspace{.2in}

Since there is a single $y$ for both of these, I can eliminate them by adding them together (the first one is positive and the second one is negative, so adding will cancel them out)\\

$
	2x+y=9\\
	+\\
	3x-y=16\\
\underline{\hspace{1in}}\\
	5x+0y=25
$

Since there is a $0y$ that means that it has been \textit{ELIMINATED}. So we can solve for $x$.\\

$5x=25 \rightarrow x=5$\\

Plug in $5$ for $x$ to obtain $y$. $y=-1$

\hrulefill

\textbf{Example 2:} Sometimes we need to do some manipulation.\\

$\begin{cases}
	4x-3y=25\\
	-3x+8y=10
\end{cases}$

\vspace{.2in}

Nothing cancels, but we can multiply by a \textit{scaler} to make one of the numbers the same. Not unlike finding a common denominator.\\

$\begin{cases}
	(3)(4x-3y=25)\\
	(4)(-3x+8y=10)
\end{cases}$

\vspace{.2in}

$\begin{cases}
	12x-9y=75\\
	-12x+32y=40
\end{cases}$

\vspace{.2in}

Now the $12x$ cancels with the $-12x$. So I can add them as...\\

$23y=115 \rightarrow y=5$\\

Now plug $5$ into the equation for $y$ to find the value of $x$\\

$x=10$\\

\pagebreak

\section*{Solving Systems Review}

NAME\underline{\hspace{3in}}

\textbf{Solve the system by graphing}\\

\begin{multicols}{2}

\begin{equation*}
	\begin{cases}
		y=-\frac{4}{3}x+2\\
		y=\frac{1}{3}x-3
	\end{cases}
\end{equation*}

\begin{center}
%\includegraphics[scale=.4]{graph1.jpg}
\end{center}

\begin{equation*}
	\begin{cases}
		y=\frac{1}{2}x-2\\
		y=\frac{1}{2}x+1
	\end{cases}
\end{equation*}

\begin{center}
%\includegraphics[scale=.4]{graph1.jpg}
\end{center}

\begin{equation*}
	\begin{cases}
		y=2x-4\\
		y=\frac{1}{3}x+1
	\end{cases}
\end{equation*}

\begin{center}
%\includegraphics[scale=.4]{graph1.jpg}
\end{center}

\begin{equation*}
	\begin{cases}
		y=\frac{9}{5}x-5\\

		y=\frac{10}{11}x-5
	\end{cases}
\end{equation*}

\begin{center}
%\includegraphics[scale=.4]{graph1.jpg}
\end{center}

\end{multicols}

\pagebreak

\hrulefill

\textbf{Solve the system using substitution}\\

\begin{multicols}{2}

$\begin{cases}
	y=x\\
	y=-x-2
\end{cases}$

\vspace{.3in}

$\begin{cases}
	y=\frac{1}{4}x-1\\
	y=2x+19
\end{cases}$

\vspace{.3in}

$\begin{cases}
	y=-x+5\\
	y=-\frac{1}{4}x+1
\end{cases}$

\vspace{.3in}

$\begin{cases}
	y=\frac{2}{3}x+4\\
	y=-\frac{1}{3}x+3
\end{cases}$

\vspace{.3in}

\end{multicols}

\hrulefill

\textbf{Solve the system using ELIMINATION}\\

\begin{multicols}{2}

$\begin{cases}
	x-2y=14\\
	x+3y=9
\end{cases}$

\vspace{.3in}

$\begin{cases}
	4x+3y=-1\\
	5x+4y=1
\end{cases}$

\vspace{.3in}

$\begin{cases}
	4x-y=10\\
	2x+3y=12
\end{cases}$

\vspace{.3in}

$\begin{cases}
	x+3y=-5\\
	4x-y=6
\end{cases}$

\vspace{.3in}

\end{multicols}

\hrulefill

\textbf{Word Problem:} The admission fee at a small fair is \$1.50 for children and \$4.00 for adults. On a certain day, 2200 people enter the fair and \$5050 is collected. How many children and how many adults attended?

\pagebreak %THIS IS THE NEXT LESSON!!!

\section*{Solving Systems Quiz}

\vspace{12pt}

NAME\underline{\hspace{3in}}\\

\textbf{Solve the system by graphing}\\

\begin{multicols}{2}

\begin{equation*}
	\begin{cases}
		y=-\frac{1}{2}x+2\\
		y=\frac{1}{4}x-1
	\end{cases}
\end{equation*}

\begin{center}
%\includegraphics[scale=.4]{graph1.jpg}
\end{center}

\begin{equation*}
	\begin{cases}
		y=-\frac{2}{3}x+4\\
		y=-\frac{1}{3}x+3
	\end{cases}
\end{equation*}

\begin{center}
%\includegraphics[scale=.4]{graph1.jpg}
\end{center}

\begin{equation*}
	\begin{cases}
		y=2x-3\\
		y=x+1
	\end{cases}
\end{equation*}

\begin{center}
%\includegraphics[scale=.4]{graph1.jpg}
\end{center}

\begin{equation*}
	\begin{cases}
		y=3x-6\\

		y=x
	\end{cases}
\end{equation*}

\begin{center}
%\includegraphics[scale=.4]{graph1.jpg}
\end{center}

\end{multicols}

\pagebreak

\hrulefill

\textbf{Solve the system using substitution}\\ % SECOND PAGE Group quiz

\begin{multicols}{2}

$\begin{cases}
	y=\frac{3}{4}x-9\\
	y=-3x+6
\end{cases}$

\vspace{1in}

$\begin{cases}
	y=\frac{3}{4}x-10\\
	y=2x+19
\end{cases}$

\vspace{1in}

$\begin{cases}
	y=-3x+7\\
	y=\frac{3}{2}x+4
\end{cases}$

\vspace{1in}

$\begin{cases}
	y=x-1\\
	y=-x+1
\end{cases}$

\vspace{1in}

\end{multicols}

\hrulefill

\textbf{Solve the system using ELIMINATION}\\

\begin{multicols}{2}

$\begin{cases}
	2x+3y=20\\
	-2x+y=4
\end{cases}$

\vspace{1in}

$\begin{cases}
	7x+4y=2\\
	9x-4y=30
\end{cases}$

\vspace{1in}

$\begin{cases}
	x-2y=14\\
	x+3y=9
\end{cases}$

\vspace{1in}

$\begin{cases}
	3x-2y=2\\
	5x-5y=10
\end{cases}$

\vspace{1in}

\end{multicols}

\hrulefill

\textbf{Extra Credit:} A boat traveled 210 miles downstream and back. The trip downstream took 10 hours. The trip back
took 70 hours. What is the speed of the boat in still water? What is the speed of the current?

\end{document}


%%
%% Automatically generated file from DocOnce source
%% (https://github.com/hplgit/doconce/)
%%
%%

%-------------------- begin preamble ----------------------

\documentclass[%
oneside,                 % oneside: electronic viewing, twoside: printing
final,                   % draft: marks overfull hboxes, figures with paths
10pt]{article}

\listfiles               %  print all files needed to compile this document

\usepackage{relsize,makeidx,color,setspace,amsmath,amsfonts,amssymb}
\usepackage[table]{xcolor}
\usepackage{bm,ltablex,microtype}

\usepackage[pdftex]{graphicx}

\usepackage{fancyvrb} % packages needed for verbatim environments

\usepackage[T1]{fontenc}
%\usepackage[latin1]{inputenc}
\usepackage{ucs}
\usepackage[utf8x]{inputenc}

\usepackage{lmodern}         % Latin Modern fonts derived from Computer Modern

% Hyperlinks in PDF:
\definecolor{linkcolor}{rgb}{0,0,0.4}
\usepackage{hyperref}
\hypersetup{
    breaklinks=true,
    colorlinks=true,
    linkcolor=linkcolor,
    urlcolor=linkcolor,
    citecolor=black,
    filecolor=black,
    %filecolor=blue,
    pdfmenubar=true,
    pdftoolbar=true,
    bookmarksdepth=3   % Uncomment (and tweak) for PDF bookmarks with more levels than the TOC
    }
%\hyperbaseurl{}   % hyperlinks are relative to this root

\setcounter{tocdepth}{2}  % levels in table of contents

% --- fancyhdr package for fancy headers ---
\usepackage{fancyhdr}
\fancyhf{} % sets both header and footer to nothing
\renewcommand{\headrulewidth}{0pt}
\fancyfoot[LE,RO]{\thepage}
% Ensure copyright on titlepage (article style) and chapter pages (book style)
\fancypagestyle{plain}{
  \fancyhf{}
  \fancyfoot[C]{{\footnotesize \copyright\ 1999-2020, "Data Analysis and Machine Learning FYS-STK3155/FYS4155":"http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html". Released under CC Attribution-NonCommercial 4.0 license}}
%  \renewcommand{\footrulewidth}{0mm}
  \renewcommand{\headrulewidth}{0mm}
}
% Ensure copyright on titlepages with \thispagestyle{empty}
\fancypagestyle{empty}{
  \fancyhf{}
  \fancyfoot[C]{{\footnotesize \copyright\ 1999-2020, "Data Analysis and Machine Learning FYS-STK3155/FYS4155":"http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html". Released under CC Attribution-NonCommercial 4.0 license}}
  \renewcommand{\footrulewidth}{0mm}
  \renewcommand{\headrulewidth}{0mm}
}

\pagestyle{fancy}

% prevent orhpans and widows
\clubpenalty = 10000
\widowpenalty = 10000

% --- end of standard preamble for documents ---

% insert custom LaTeX commands...

\raggedbottom
\makeindex
\usepackage[totoc]{idxlayout}   % for index in the toc
\usepackage[nottoc]{tocbibind}  % for references/bibliography in the toc

%-------------------- end preamble ----------------------

\begin{document}

% matching end for #ifdef PREAMBLE

\newcommand{\exercisesection}[1]{\subsection*{#1}}

% ------------------- main content ----------------------

% ----------------- title -------------------------

\thispagestyle{empty}

\begin{center}
{\LARGE\bf
\begin{spacing}{1.25}
Project 1 on Machine Learning, deadline October 10, 2020
\end{spacing}
}
\end{center}

% ----------------- author(s) -------------------------

\begin{center}
{\bf \href{{http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html}}{Data Analysis and Machine Learning FYS-STK3155/FYS4155}}
\end{center}

    \begin{center}
% List of all institutions:
\centerline{{\small Department of Physics, University of Oslo, Norway}}
\end{center}

% ----------------- end author(s) -------------------------

% --- begin date ---
\begin{center}
Oct 2, 2020
\end{center}
% --- end date ---

\vspace{1cm}

\subsection*{Regression analysis and resampling methods}

The main aim of this project is to study in more detail various
regression methods, including the Ordinary Least Squares (OLS) method,
Ridge regression and finally Lasso regression. Ridge regression will be discussed during the Friday lecture of week 36 while Lasso Regression will be discussed during the lectures of week 37.

The methods are in turn combined with resampling techniques like the bootstrap method and cross validation. These are discussed during weeks 36 and 37.

We will first study how to fit polynomials to a specific
two-dimensional function called \href{{http://www.dtic.mil/dtic/tr/fulltext/u2/a081688.pdf}}{Franke's
function}.  This
is a function which has been widely used when testing various
interpolation and fitting algorithms. Furthermore, after having
established the model and the method, we will employ resamling
techniques such as cross-validation and/or bootstrap in order to perform a
proper assessment of our models. We will also study in detail the
so-called Bias-Variance trade off.

The Franke function, which is a weighted sum of four exponentials  reads as follows
\begin{align*}
f(x,y) &= \frac{3}{4}\exp{\left(-\frac{(9x-2)^2}{4} - \frac{(9y-2)^2}{4}\right)}+\frac{3}{4}\exp{\left(-\frac{(9x+1)^2}{49}- \frac{(9y+1)}{10}\right)} \\
&+\frac{1}{2}\exp{\left(-\frac{(9x-7)^2}{4} - \frac{(9y-3)^2}{4}\right)} -\frac{1}{5}\exp{\left(-(9x-4)^2 - (9y-7)^2\right) }.
\end{align*}

The function will be defined for $x,y\in [0,1]$.  Our first step will
be to perform an OLS regression analysis of this function, trying out
a polynomial fit with an $x$ and $y$ dependence of the form $[x, y,
x^2, y^2, xy, \dots]$. We will also include bootstrap first as
a resampling technique.  After that we will include the cross-validation technique. As in homeworks 1 and 2, we can use a uniform
distribution to set up the arrays of values for $x$ and $y$, or as in
the example below just a set of fixed
values for $x$ and $y$ with a given step
size.  We will fit a
function (for example a polynomial) of $x$ and $y$.  Thereafter we
will repeat much of the same procedure using the Ridge and Lasso
regression methods, introducing thus a dependence on the bias
(penalty) $\lambda$.

Finally we are going to use (real) digital terrain data and try to
reproduce these data using the same methods. We will also try to go
beyond the second-order polynomials metioned above and explore
which polynomial fits the data best.

The Python code for the Franke function is included here (it performs also a three-dimensional plot of it)
\begin{verbatim}
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
from matplotlib import cm
from matplotlib.ticker import LinearLocator, FormatStrFormatter
import numpy as np
from random import random, seed

fig = plt.figure()
ax = fig.gca(projection='3d')

# Make data.
x = np.arange(0, 1, 0.05)
y = np.arange(0, 1, 0.05)
x, y = np.meshgrid(x,y)

def FrankeFunction(x,y):
    term1 = 0.75*np.exp(-(0.25*(9*x-2)**2) - 0.25*((9*y-2)**2))
    term2 = 0.75*np.exp(-((9*x+1)**2)/49.0 - 0.1*(9*y+1))
    term3 = 0.5*np.exp(-(9*x-7)**2/4.0 - 0.25*((9*y-3)**2))
    term4 = -0.2*np.exp(-(9*x-4)**2 - (9*y-7)**2)
    return term1 + term2 + term3 + term4

z = FrankeFunction(x, y)

# Plot the surface.
surf = ax.plot_surface(x, y, z, cmap=cm.coolwarm,
                       linewidth=0, antialiased=False)

# Customize the z axis.
ax.set_zlim(-0.10, 1.40)
ax.zaxis.set_major_locator(LinearLocator(10))
ax.zaxis.set_major_formatter(FormatStrFormatter('%.02f'))

# Add a color bar which maps values to colors.
fig.colorbar(surf, shrink=0.5, aspect=5)

plt.show()

\end{verbatim}

\paragraph{Part a): Ordinary Least Square (OLS) on the Franke function.}
We will generate our own dataset for a function
$\mathrm{FrankeFunction}(x,y)$ with $x,y \in [0,1]$. The function
$f(x,y)$ is the Franke function. You should explore also the addition
of an added stochastic noise to this function using the normal
distribution $\cal{N}(0,1)$.

\emph{Write your own code} (using either a matrix inversion or a singular
value decomposition from e.g., \textbf{numpy} ) or use your code from
homeworks 1 and 2 and perform a standard least square regression
analysis using polynomials in $x$ and $y$ up to fifth order. Find the
\href{{https://en.wikipedia.org/wiki/Confidence_interval}}{confidence intervals} of the parameters (estimators) $\beta$ by computing their
variances, evaluate the Mean Squared error (MSE)

\[ MSE(\hat{y},\hat{\tilde{y}}) = \frac{1}{n}
\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2,
\]

and the $R^2$ score function.  If $\tilde{\hat{y}}_i$ is the predicted
value of the $i-th$ sample and $y_i$ is the corresponding true value,
then the score $R^2$ is defined as

\[
R^2(\hat{y}, \tilde{\hat{y}}) = 1 - \frac{\sum_{i=0}^{n - 1} (y_i - \tilde{y}_i)^2}{\sum_{i=0}^{n - 1} (y_i - \bar{y})^2},
\]

where we have defined the mean value  of $\hat{y}$ as

\[
\bar{y} =  \frac{1}{n} \sum_{i=0}^{n - 1} y_i.
\]

Your code has to include a scaling of the data (for example by
subtracting the mean value, see also \href{{https://compphysics.github.io/MachineLearning/doc/Projects/2020/hw2/html/hw2-bs.html}}{homework set 2} for examples) and
a split of the data in training and test data. For this part you can
either write your own code or use for example the function for
splitting training data provided by the library \textbf{Scikit-Learn} (make
sure you have installed it).  This function is called
$train\_test\_split$. Similarly, and see the solution to \href{{https://compphysics.github.io/MachineLearning/doc/Projects/2020/hw2/html/hw2-bs.html}}{homework set 2, exercise 2}, you can use the data normalization functionality of
\textbf{Scikit-Learn}.

It is normal in essentially all Machine Learning studies to split the
data in a training set and a test set (eventually  also an additional
validation set).  There
is no explicit recipe for how much data should be included as training
data and say test data.  An accepted rule of thumb is to use
approximately $2/3$ to $4/5$ of the data as training data.

\paragraph{Part b): Bias-variance trade-off and resamplng techniques.}
Our aim here is to study the bias-variance trade-off by implementing the \textbf{bootstrap} resampling technique.

With a code which does OLS and includes resampling techniques,
we will now discuss the bias-variance trade-off in the context of
continuous predictions such as regression. However, many of the
intuitions and ideas discussed here also carry over to classification
tasks and basically all Machine Learning algorithms.

Before you perform an analysis of the bias-variance trade-off on your test data, make
first a figure similar to Fig.~2.11 of Hastie, Tibshirani, and
Friedman. Figure 2.11 of this reference displays only the test and training MSEs. The test MSE can be used to
indicate possible regions of low/high bias and variance. You will most likely not get an
equally smooth curve!

With this result we move on to the bias-variance trade-off analysis.

Consider a
dataset $\mathcal{L}$ consisting of the data
$\mathbf{X}_\mathcal{L}=\{(y_j, \boldsymbol{x}_j), j=0\ldots n-1\}$.

Let us assume that the true data is generated from a noisy model

\[
\bm{y}=f(\boldsymbol{x}) + \bm{\epsilon}.
\]

Here $\epsilon$ is normally distributed with mean zero and standard
deviation $\sigma^2$.

In our derivation of the ordinary least squares method we defined then
an approximation to the function $f$ in terms of the parameters
$\bm{\beta}$ and the design matrix $\bm{X}$ which embody our model,
that is $\bm{\tilde{y}}=\bm{X}\bm{\beta}$.

The parameters $\bm{\beta}$ are in turn found by optimizing the means
squared error via the so-called cost function

\[
C(\bm{X},\bm{\beta}) =\frac{1}{n}\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2=\mathbb{E}\left[(\bm{y}-\bm{\tilde{y}})^2\right].
\]
Here the expected value $\mathbb{E}$ is the sample value.

Show that you can rewrite  this as
\[
\mathbb{E}\left[(\bm{y}-\bm{\tilde{y}})^2\right]=\frac{1}{n}\sum_i(f_i-\mathbb{E}\left[\bm{\tilde{y}}\right])^2+\frac{1}{n}\sum_i(\tilde{y}_i-\mathbb{E}\left[\bm{\tilde{y}}\right])^2+\sigma^2.
\]

Explain what the terms mean, which one is the bias and which one is
the variance and discuss their interpretations.

Perform then a bias-variance analysis of the Franke function by
studying the MSE value as function of the complexity of your model.

Discuss the bias and variance trade-off as function
of your model complexity (the degree of the polynomial) and the number
of data points, and possibly also your training and test data using the \textbf{bootstrap} resampling method.

Note also that when you calculate the bias, in all applications you don't know the function values $f_i$. You would hence replace them with the actual data points $y_i$.

\paragraph{Part c) Cross-validation as resampling techniques, adding more complexity.}
The aim here is to write your own code for another widely popular
resampling technique, the so-called cross-validation method.  Again,
before you start with cross-validation approach, you should scale your
data.

Implement the $k$-fold cross-validation algorithm (write your own
code) and evaluate again the MSE function resulting
from the test folds. You can compare your own code with that from
\textbf{Scikit-Learn} if needed.

Compare the MSE you get from your cross-validation code with the one you got from your \textbf{bootstrap} code. Comment your results. Try $5-10$ folds.
You can also compare your own cross-validation code with the one provided by \textbf{Scikit-Learn}.

\paragraph{Part d): Ridge Regression on the Franke function  with resampling.}
Write your own code for the Ridge method, either using matrix
inversion or the singular value decomposition as done in the previous
exercise or howework 2 (see also chapter 3.4 of Hastie \emph{et al.},
equations (3.43) and (3.44)). Perform the same bootstrap analysis as in the
part b)  (for the same polynomials) and the cross-validation part in part c) but now for different values of $\lambda$. Compare and
analyze your results with those obtained in parts a-c). Study the
dependence on $\lambda$.

Study also the bias-variance trade-off as function of various values of
the parameter $\lambda$. For the bias-variance trade-off, use the \textbf{bootstrap} resampling method. Comment your results.

\paragraph{Part e): Lasso Regression on the Franke function  with resampling.}
This part is essentially a repeat of the previous two ones, but now
with Lasso regression. Write either your own code (difficult and optional) or, in this case,
you can also use the functionalities of \textbf{Scikit-Learn} (recommended).
Give a
critical discussion of the three methods and a judgement of which
model fits the data best.  Perform here as well an analysis of the bias-variance trade-off using the \textbf{bootstrap} resampling technique and an analysis of the mean squared error using cross-validation.

\paragraph{Part f): Introducing real data and preparing the data analysis.}
With our codes functioning and having been tested properly on a
simpler function we are now ready to look at real data. We will
essentially repeat in part g) what was done in parts a-e). However, we
need first to download the data and prepare properly the inputs to our
codes.  We are going to download digital terrain data from the website
\href{{https://earthexplorer.usgs.gov/}}{\nolinkurl{https://earthexplorer.usgs.gov/}},

In order to obtain data for a specific region, you need to register as
a user (free) at this website and then decide upon which area you want
to fetch the digital terrain data from.  In order to be able to read
the data properly, you need to specify that the format should be \textbf{SRTM
Arc-Second Global} and download the data as a \textbf{GeoTIF} file.  The
files are then stored in \emph{tif} format which can be imported into a
Python program using

\begin{verbatim}
scipy.misc.imread
\end{verbatim}

Here is a simple part of a Python code which reads and plots the data
from such files

\begin{verbatim}
import numpy as np
from imageio import imread
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from matplotlib import cm

# Load the terrain
terrain1 = imread('SRTM_data_Norway_1.tif')
# Show the terrain
plt.figure()
plt.title('Terrain over Norway 1')
plt.imshow(terrain1, cmap='gray')
plt.xlabel('X')
plt.ylabel('Y')
plt.show()
\end{verbatim}

If you should have problems in downloading the digital terrain data,
we provide two examples under the data folder of project 1. One is
from a region close to Stavanger in Norway and the other Møsvatn
Austfjell, again in Norway.
Feel free to produce your own terrain data.

Alternatively, if you would like to use another data set, feel free to do so. This could be data close to your reseach area or simply a data set you found interesting. See for example \href{{https://www.kaggle.com/datasets}}{kaggle.com} for examples.

\paragraph{Part g) OLS, Ridge and Lasso regression with resampling.}
Our final part deals with the parameterization of your digital terrain
data (or your own data).  We will apply all three methods for linear regression, the same type (or higher order) of polynomial
approximation and cross-validation as resampling technique to evaluate which
model fits the data best.

At the end, you should present a critical evaluation of your results
and discuss the applicability of these regression methods to the type
of data presented here (either the terrain data we propose or other data sets).

\subsection*{Background literature}

\begin{enumerate}
\item For a discussion and derivation of the variances and mean squared errors using linear regression, see the \href{{https://arxiv.org/abs/1509.09169}}{Lecture notes on ridge regression by Wessel N. van Wieringen}

\item The textbook of \href{{https://www.springer.com/gp/book/9780387848570}}{Trevor Hastie, Robert Tibshirani, Jerome H. Friedman, The Elements of Statistical Learning, Springer}, chapters 3 and 7 are the most relevant ones for the analysis here.
\end{enumerate}

\noindent
\subsection*{Introduction to numerical projects}

Here follows a brief recipe and recommendation on how to write a report for each
project.

\begin{itemize}
  \item Give a short description of the nature of the problem and the eventual  numerical methods you have used.

  \item Describe the algorithm you have used and/or developed. Here you may find it convenient to use pseudocoding. In many cases you can describe the algorithm in the program itself.

  \item Include the source code of your program. Comment your program properly.

  \item If possible, try to find analytic solutions, or known limits in order to test your program when developing the code.

  \item Include your results either in figure form or in a table. Remember to        label your results. All tables and figures should have relevant captions        and labels on the axes.

  \item Try to evaluate the reliabilty and numerical stability/precision of your results. If possible, include a qualitative and/or quantitative discussion of the numerical stability, eventual loss of precision etc.

  \item Try to give an interpretation of you results in your answers to  the problems.

  \item Critique: if possible include your comments and reflections about the  exercise, whether you felt you learnt something, ideas for improvements and  other thoughts you've made when solving the exercise. We wish to keep this course at the interactive level and your comments can help us improve it.

  \item Try to establish a practice where you log your work at the  computerlab. You may find such a logbook very handy at later stages in your work, especially when you don't properly remember  what a previous test version  of your program did. Here you could also record  the time spent on solving the exercise, various algorithms you may have tested or other topics which you feel worthy of mentioning.
\end{itemize}

\noindent
\subsection*{Format for electronic delivery of report and programs}

The preferred format for the report is a PDF file. You can also use DOC or postscript formats or as an ipython notebook file.  As programming language we prefer that you choose between C/C++, Fortran2008 or Python. The following prescription should be followed when preparing the report:

\begin{itemize}
  \item Use Canvas to hand in your projects, log in  at  \href{{https://www.uio.no/english/services/it/education/canvas/}}{\nolinkurl{https://www.uio.no/english/services/it/education/canvas/}} with your normal UiO username and password.

  \item Upload \textbf{only} the report file or the link to your GitHub/GitLab or similar typo of  repos!  For the source code file(s) you have developed please provide us with your link to your GitHub/GitLab or similar  domain.  The report file should include all of your discussions and a list of the codes you have developed.  Do not include library files which are available at the course homepage, unless you have made specific changes to them.

  \item In your GitHub/GitLab or similar repository, please include a folder which contains selected results. These can be in the form of output from your code for a selected set of runs and input parameters.
\end{itemize}

\noindent
Finally,
we encourage you to collaborate. Optimal working groups consist of
2-3 students. You can then hand in a common report.

\subsection*{Software and needed installations}

If you have Python installed (we recommend Python3) and you feel pretty familiar with installing different packages,
we recommend that you install the following Python packages via \textbf{pip} as
\begin{enumerate}
\item pip install numpy scipy matplotlib ipython scikit-learn tensorflow sympy pandas pillow
\end{enumerate}

\noindent
For Python3, replace \textbf{pip} with \textbf{pip3}.

See below for a discussion of \textbf{tensorflow} and \textbf{scikit-learn}.

For OSX users we recommend also, after having installed Xcode, to install \textbf{brew}. Brew allows
for a seamless installation of additional software via for example
\begin{enumerate}
\item brew install python3
\end{enumerate}

\noindent
For Linux users, with its variety of distributions like for example the widely popular Ubuntu distribution
you can use \textbf{pip} as well and simply install Python as
\begin{enumerate}
\item sudo apt-get install python3  (or python for python2.7)
\end{enumerate}

\noindent
etc etc.

If you don't want to install various Python packages with their dependencies separately, we recommend two widely used distrubutions which set up  all relevant dependencies for Python, namely
\begin{enumerate}
\item \href{{https://docs.anaconda.com/}}{Anaconda} Anaconda is an open source distribution of the Python and R programming languages for large-scale data processing, predictive analytics, and scientific computing, that aims to simplify package management and deployment. Package versions are managed by the package management system \textbf{conda}

\item \href{{https://www.enthought.com/product/canopy/}}{Enthought canopy}  is a Python distribution for scientific and analytic computing distribution and analysis environment, available for free and under a commercial license.
\end{enumerate}

\noindent
Popular software packages written in Python for ML are

\begin{itemize}
\item \href{{http://scikit-learn.org/stable/}}{Scikit-learn},

\item \href{{https://www.tensorflow.org/}}{Tensorflow},

\item \href{{http://pytorch.org/}}{PyTorch} and

\item \href{{https://keras.io/}}{Keras}.
\end{itemize}

\noindent
These are all freely available at their respective GitHub sites. They
encompass communities of developers in the thousands or more. And the number
of code developers and contributors keeps increasing.

% ------------------- end of main content ---------------

\end{document}