\newcommand{\of}{\texttt{.o}~}

\lab{Interfacing With Other Programming Languages Using Cython}{Interfacing With Other Programming Languages Using Cython}
\label{lab:cythonwrap}

\objective{Learn to interface with object files using Cython. This lab should be worked through on a machine that has already been configured to build Cython extensions using gcc or MinGW.}

There are several ways to interface with other languages from Python.
Calling code written in C or Fortran is often done for performance reasons.
There is also an easy built in interface for R, which is a statistical computing language.
Writing an interface for functions or objects that have been written in one language so that they are accessible in another language is called \emph{wrapping}.  For example, many of the linear algebra functions in NumPy wrap the corresponding function in LAPACK or BLAS.
This allows us to gain the speed of these very optimized libraries but use them in a nice, Pythonic way.

In this lab we will focus primarily on how Python can interact with C, C++, and Fortran.
Python can interface with these languages in a variety of ways.
There is a built-in library, ctypes, that allows you to interface with object files compiled from other languages.
There are also many other methods for interfacing with compiled languages like C, C++, and Fortran.
A particularly convenient way to do this is with Cython.
Since Cython already compiles as a C file, we can essentially use it to call functions from a C program.
This is done through the use of object files.

% Describe .o files and dll's
% This is mainly just to help them know what is going on.
\section*{Object Files and DLL's}
For C, C++, and Fortran, the desired functions or classes are compiled into an object file containing instructions in assembly that can be used by compilers of a variety of languages.
Object files typically use the \of extension.
An object file that can be used and modified by multiple programs is called a \emph{shared object} and generally use the \texttt{.so} extension.
When C code has been compiled with the proper headers, etc. to interface with Python, it uses the \texttt{.pyd} extension on Windows and the \texttt{.so} extension on Unix-based operating systems.
The two extensions are essentially the same, except that Python object files are distinguished from shared object files on Windows.
When you compile anything that uses functions from object files, you must tell your compiler to link to the contents of the object file.
Otherwise it will not be able to find any of the needed functions.
The part of the compiler that forms these links between the code is called the \emph{linker}.
If you do not give your compiler proper instructions for linking, it will raise a linking error.
The compiler must also be able to find the necessary header files for compiling the source code.
This can be done by including the proper directories in the system path or by passing the paths to the proper directories as arguments to the compiler at compile time.

Object files can be statically or dynamically linked.
When statically linked, the entire object file becomes part of the final program.
On the other hand, dynamic linking is just a reference to the object file.
The operating system will load any needed dynamically linked libraries at runtime.
Dynamically linked programs depend on the presence of the appropriate object files on the target system.
Since statically linked libraries are part of the program itself, there is no need to load any libraries at runtime.

These instructions will use the GNU C Compiler (gcc) to demonstrate how Python can interface with other languages.
Other compilers can be used, but some configuration may be necessary..
Python's distutils package is designed to avoid some of the difficulty that comes when interfacing with different compilers, and we will show some basic examples of how it is used here.
The best way to learn to interface with other languages is to work through simple examples.
\begin{info}
The examples in this lab are intended to provide simple templates that can be used later to wrap C and Fortran functions.
It will be good to read through and think about how each file works, but it will not be critical to understand every detail at this point.
\end{info}

\begin{warn}
When passing arrays as pointers to C, C++, and Fortran functions, be \emph{absolutely sure} that the array being passed is contiguous.
This means that the entries of the array are stored in \emph{adjacent} entries in memory.
Passing one of these functions a strided array will result in out of bounds memory accesses and could crash your computer.
When working with two dimensional arrays, C and C++ expect rows to be stored in contiguous blocks of memory.
Fortran expects columns to be stored in contiguous blocks of memory.

To avoid out of bounds memory accesses, be sure to pass in the size of the array as a separate parameter.
\end{warn}

\section*{Wrapping a C Function}
The following C function computes the solution to a tridiagonal system.
It requires pointers to four arrays.
Arrays \li{a} and \li{c} have length \li{n-1} and represent the first subdiagonal and first superdiagonal of a banded matrix with bandwidth 3.
Arrays \li{b} and \li{x} have length \li{n} and represent the main diagonal of the banded matrix and the right hand side of the system of equations.
\li{c} and \li{x} are modified in place to compute the solution of the system.
\li{c} is used to store temporary values and \li{x} is transformed into the solution of the system.

\lstinputlisting[style=fromfile, language=C, name=ctridiag.c]{./ctridiag/ctridiag.c}

This file can be compiled to a \li{.o} file so that \texttt{ctridiag} can be called by other programs.
To do this, we run the following command in a terminal:
\begin{lstlisting}[style=ShellInput]
gcc -fPIC -c ctridiag.c -o ctridiag.o
\end{lstlisting}
This calls the C-compiler gcc and tells it to compile the file \texttt{ctridiag.c} to an object file \texttt{ctridiag.o}.
The \texttt{-c} flag tells it to compile to an object file without linking as opposed to compiling to some sort of executable.
Those that are familiar with C may have noticed that we did not include a \li{main} function in our C file.
The \texttt{-c} flag is what prevents this from causing errors with the compiler.
The \texttt{-fPIC} option is required when building code for a shared object.

Once we have run this command the file has been compiled to a \of file, but the functions in the \of file cannot be used directly as python functions.
Although Python relies on C internally, it needs some special interfaces to interact with these \of files.
A general approach for accessing these external functions from Python is to define in some way what inputs each function takes.
We will show how this is done with Cython.
Since a Cython file compiles to a C file which is then compiled to a callable extension for Python, it can be used to interface with C code and other object files.
We will use the following approach:
\begin{enumerate}
\item Write a C header that specifies what the function is named and how it is called.
\item Write a Cython file that imports the header to its C file and defines a Python wrapper for C function to import.
\item Compile the Cython file to C code.
\item Compile the Cython file to a Python extension, giving it access to the object file we are accessing and giving the compiler access to the proper header files.
\end{enumerate}

An approach similar to this will work for some other types of \of files compiled from other languages as well.

The header file that defines how to interface with the C function is very simple.
This file will tell Cython how to interact with the object file once it has been loaded.

\lstinputlisting[style=fromfile, language=C, name=ctridiag.h]{./ctridiag/ctridiag.h}

The following Cython file defines a Python wrapper for the C function contained in the object file:

\lstinputlisting[style=fromfile, language=Python, name=cython_ctridiag.pyx]{./ctridiag/cython_ctridiag.pyx}

\begin{info}
This is a good example of why header files are used.
A C header file tells the C compiler how to interface with compiled C functions that it will link to later in compilation.
The compiler then allows for the use of the given function throughout the C file as if it had already been defined.

The Cython \li{cdef extern} declaration is used in the same way for the Cython compiler.
It essentially tells the Cython compiler, "When the C file generated from this .pyx file is compiled it will be linked against a C library that contains a C function that acts like this: ..."
Describing the C function this way to the Cython compiler allows it to check for undefined references and improper function use before building the C file.

It is possible to define Cython headers in much the same way as C headers.
These headers use the .pxd file extension and can be imported via the \li{cimport} statement in Cython.
.pxd files are called Cython declaration files.
When you use a \li{cimport} statement to import basic C functions or to interface with NumPy arrays and NumPy datatypes, you are really importing Cython declaration files that contain the proper signatures so that the Cython compiler can know how these functions are meant to behave.
Several different tools for automatically generating Cython declaration files from C and C++ headers are available online.
\end{info}

Now that the necessary source files are written, we can build the Python extension.
On Windows, using the compiler MinGW (a version of gcc for windows), the compilation can be performed by running the following setup file.

\lstinputlisting[style=fromfile, language=Python, name=ctridiag_setup_windows64.py]{./ctridiag/ctridiag_setup_windows64.py}

The following file works on Linux and Macintosh machines using gcc.

\lstinputlisting[style=fromfile, language=Python, name=ctridiag_setup_linux.py]{./ctridiag/ctridiag_setup_linux.py}

The distutils module in Python is designed to allow the compilation of Python extensions on different platforms with a single setup file.
The following setup file uses distutils to compile the Cython file, and may be run on Windows, Linux, or Macintosh-based computers.
This is a more standard way to build a Cython extension.

\lstinputlisting[style=fromfile, language=Python, name=ctridiag_setup_distutils.py]{./ctridiag/ctridiag_setup_distutils.py}

This setup file can be called via the command line with the following command:
\begin{lstlisting}[style=ShellInput]
python ctridiag_setup_distutils.py build_ext --inplace
\end{lstlisting}
The \texttt{--inplace} flag tells the script to compile the extension in the current directory.

Here is a simple script to test whether or not the compilation worked.

\lstinputlisting[style=fromfile, language=Python, name=ctridiag_test.py]{./ctridiag/ctridiag_test.py}

\begin{info}
It is a good idea to test whether or not these compilations work using test scripts instead of doing it manually or in an IPython notebook since you may run into trouble when trying to build an extension that replaces a file currently in use.
\end{info}

\section*{Wrapping a Fortran Function}
By making some small modifications to our original approach we can also use Cython to interface with functions that have been written in Fortran.
Here we will consider a simple Fortran implementation of the same function as before.
Here is that same algorithm implemented in Fortran.
Since this function does not return a value, it is called a subroutine in Fortran.
In this case we have used the \li{iso_c_binding} library in C to make the function accept pointers to native C types.
If we were wrapping a function or subroutine that we did not write ourselves we would have to define a Fortran function that uses the \li{iso_c_binding} library to accept pointers from C and then uses the values it receives to call the original function.
\lstinputlisting[style=fromfile, language=Fortran, name=ftridiag.f90]{./ftridiag/ftridiag.f90}

We will now write a header that tells C how to interface with the function we have just defined.
\begin{warn}
When interfacing between Fortran and C, you will have to pass pointers to \emph{all} the variables you send to the Fortran function as arguments.
Passing a variable directly will probably crash Python.
\end{warn}
Here is the file:
\lstinputlisting[style=fromfile, language=C, name=ftridiag.h]{./ftridiag/ftridiag.h}

To compile \li{ftridiag.f90} to an object file you can run the following command in your command line:
\begin{lstlisting}[style=ShellInput]
gfortran -fPIC -c ftridiag.f90 -o ftridiag.o
\end{lstlisting}
As before, if you are working on a Linux or Macintosh based machine you will probably need to include the \texttt{-fPIC} flag as well.

Aside from the small changes in the names used, the modified header file, and the use of gfortran instead of gcc, the rest of the compilation process is entirely the same.
The test script that verifies that the function works properly is the same except that it will import the function from the module \li{cython_ftridiag} instead.
The setup files specific to Windows and Linux, the setup file using distutils, and the test script are included with this lab.

\begin{warn}
When working with higher dimensional arrays Fortran differs from C in that the columns are stored in contiguous blocks of memory instead of the rows.
The default for NumPy and for C arrays is to have rows stored in contiguous blocks, but this varies, depending on what computation has already been done with the NumPy array.
This means that, whenever you need to interface with Fortran code that is meant to act on arrays with multiple dimensions, you \emph{must} be very careful about what is a row and what is a column.
When you pass a Fortran subroutine a pointer to an array it will, by default, assume that the entries of the array are arranged in Fortran order.
Many Fortran routines have options that allow you to specify whether to act on an array or its transpose, but you should still be very careful about this.
\end{warn}

\begin{info}
It may seem unusual that we have already provided a Fortran function that has been modified so that it is callable from C.
Though this is not pure Fortran, it is not necessarily an uncommon scenario.
Many larger Fortran libraries (for example, LAPACK) provide C wrappers for all their functions.
When wrapping a Fortran library, a C wrapper may already be available for use.
\end{info}

Here is a file for the next problem:
\lstinputlisting[style=fromfile, language=Fortran, name=fssor.f90]{./fssor/fssor.f90}

\begin{problem}
The Fortran subroutine \li{fssor} included with this lab uses a technique called Successive Over Relaxation to solve Laplace's equation on a rectangular domain discretized as a grid of equally spaced points.
It solves the same problem as the one involving Laplace's equation in Lab \ref{lab:NumPyArrays}.
It takes the following inputs:
\begin{itemize}
\item $U$ is expected to be a pointer to a Fortran ordered array of double precision numbers.
\item $m$ is a pointer to an integer storing the number of rows of $U$.
\item $n$ is a pointer to an integer storing the number of columns of $U$.
\item $\omega$ is a pointer to a double precision floating point value storing some parameter between $1$ and $2$.
The closer this value is to $2$, the faster the algorithm will converge, but if it is too close the algorithm may not converge at all.
For this lab just use $1.9$.
\item tol is a pointer to a floating point number storing a tolerance used to determine when the algorithm should terminate.
\item maxiters is a pointer to an integer storing the maximum allowable number of iterations.
\item info is a pointer to an integer.
This subroutine will set info to $0$ if it converged to a solution within the given tolerance.
It will set info to $1$ if it did not.
\end{itemize}

Write a corresponding C header \li{fssor.h}, Cython wrapper \li{cython_fssor.pyx}, and Cython setup file (using distutils) \li{fssor_setup_distutils.py} to wrap this subroutine for use in Python.
Call the wrapper function in your Cython file \li{cyssor}.

Fortran works with Fortran ordered arrays, but this algorithm does not need any particular orientation of axes.
Have your Cython wrapper swap the length of the axes when calling \li{fssor} if the array is C ordered.
If it is Fortran ordered, pass the lengths as they are.
Have the wrapper raise an error if the array $U$ is neither Fortran ordered nor C ordered.
Also have it raise an error if the algorithm failed to converge to within the desired tolerances (i.e. if the function sets info to 1).
Use keyword arguments to pass \li{fssor} a default tolerance of $10e^{-9}$ and a default maximum number of iterations to 10000.

Here is how you should be able to call this function:
\begin{lstlisting}
import numpy as np
from cython_fssor import cyssor
resolution = 501
U = np.zeros((resolution, resolution))
X = np.linspace(0, 1, resolution)
U[0] = np.sin(2 * np.pi * X)
U[-1] = - U[0]
cyssor(U, 1.9)
\end{lstlisting}

Generate a 3D plot of $U$ after it has been modified by the function \li{cyssor}.
Use equally spaced points between 0 and 1 as your $x$ and $y$ values.
\end{problem}

Here is some code for the next problem.
\lstinputlisting[style=fromfile, language=C, name=cssor.c]{./cssor/cssor.c}
\begin{problem}
(Optional)
The above C code is another implementation of the Successive Over Relaxation algorithm for solving Laplace's equation.
Wrap it so that it is callable from Python as well.
Produce the same plot as in the previous problem using the C version of this algorithm.
Notice that info is still passed as a pointer because the function needs to be able to modify it.
\end{problem}

\let\of\undefined


% !TEX TS-program = pdflatex
% !TEX encoding = UTF-8 Unicode

% This is a simple template for a LaTeX document using the "article" class.
% See "book", "report", "letter" for other types of document.

\documentclass[11pt]{article} % use larger type; default would be 10pt

\usepackage[utf8]{inputenc} % set input encoding (not needed with XeLaTeX)

%%% Examples of Article customizations
% These packages are optional, depending whether you want the features they provide.
% See the LaTeX Companion or other references for full information.

%%% PAGE DIMENSIONS
\usepackage{geometry} % to change the page dimensions
\geometry{a4paper} % or letterpaper (US) or a5paper or....
% \geometry{margin=2in} % for example, change the margins to 2 inches all round
% \geometry{landscape} % set up the page for landscape
%   read geometry.pdf for detailed page layout information

\usepackage{graphicx} % support the \includegraphics command and options

% \usepackage[parfill]{parskip} % Activate to begin paragraphs with an empty line rather than an indent

%%% PACKAGES
\usepackage{booktabs} % for much better looking tables
\usepackage{array} % for better arrays (eg matrices) in maths
\usepackage{paralist} % very flexible & customisable lists (eg. enumerate/itemize, etc.)
\usepackage{verbatim} % adds environment for commenting out blocks of text & for better verbatim
\usepackage{subfig} % make it possible to include more than one captioned figure/table in a single float
% These packages are all incorporated in the memoir class to one degree or another...
\usepackage{amsmath}
\usepackage{amssymb}
%%% HEADERS & FOOTERS
\usepackage{fancyhdr} % This should be set AFTER setting up the page geometry
\pagestyle{fancy} % options: empty , plain , fancy
\renewcommand{\headrulewidth}{0pt} % customise the layout...
\lhead{}\chead{}\rhead{}
\lfoot{}\cfoot{\thepage}\rfoot{}

%%% SECTION TITLE APPEARANCE
\usepackage{sectsty}
\allsectionsfont{\sffamily\mdseries\upshape} % (See the fntguide.pdf for font help)
% (This matches ConTeXt defaults)

%%% ToC (table of contents) APPEARANCE
\usepackage[nottoc,notlof,notlot]{tocbibind} % Put the bibliography in the ToC
\usepackage[titles,subfigure]{tocloft} % Alter the style of the Table of Contents
\renewcommand{\cftsecfont}{\rmfamily\mdseries\upshape}
\renewcommand{\cftsecpagefont}{\rmfamily\mdseries\upshape} % No bold!

%%% END Article customizations

%%% The "real" document content comes below...

\title{Homework based on Chapter 6\\
Computational Probability and Statistics \\
CIS 2033, Section 002}
\author{}
\date{} % Activate to display a given date or no date (if empty),
         % otherwise the current date is printed

\begin{document}
\maketitle

\section{Part 1 (Due: 9:00 AM, Friday, Feb. 20, 2015)}
\paragraph*{Question 1} Let $U$ follows a $U(0,1)$ distribution. Let $X$ follows a binomial distribution $Bin(3, 0.5)$.Describe how to simulate the outcome of $X$. (Hint: $X$ can take values from $k=\{0,1,2,3\}$, $p(k)={3 \choose k}(0.5)^k(0.5)^{3-k}$)
\paragraph*{Question 2} Somebody messed up the random number generator in your computer, instead of uniform random numbers it generates numbers with an $Par(3)$ distribution. Describe how to construct a $U(0,1)$ random variable $U$ from an $Par(3)$ distributed $X$.

\section{Part 2 (Due: 11:59 PM, Tuesday, 24, 2015)}

\paragraph*{Question 3} A random variable X has a $Exp(2)$ distribution. For details of the Exponential distribution see the notes. Describe how to construct $X$ from $U(0, 1)$ random variable.

\paragraph*{Question 3} A random variable X has a $Par(5)$ distribution. For details of the Exponential distribution see the notes. Describe how to construct $X$ from $U(0, 1)$ random variable.

\newpage
\textbf{Sample Questions}
\paragraph*{6.1} Let $U$ have a $U(0, 1)$ distribution. Describe how to simulate the outcome of a roll with a die using $U$.

\textbf{Answer}. Denote $X$ the random variable for the outcome of throwing a die. $X$ can take $k={1,2,3,4,5,6}$. The PMF of $X$ is $P(X=1) = P(X=2) = P(X=3) = P(X=4) = P(X=5) = P(X=6) = \frac{1}{6}; 0~$elsewhere. To simulate the PMF of $X$, is to define a partition on $\Omega$ of $U$ such that the area of each part corresponds to a probability of $X$.

\begin{table}[h!]
\renewcommand{\arraystretch}{1.5}
\centering
\caption{Simulation of rolling a die using $U(0, 1)$.}
\label{Ta:6.1a}
\begin{tabular}{|c|c|c|c|c|c|c|} \hline
Events of U & $0\leq U < \frac{1}{6}$ & $\frac{1}{6} \leq U < \frac{2}{6}$ &
$\frac{2}{6} \leq U < \frac{3}{6}$ & $\frac{3}{6} \leq U < \frac{4}{6}$ &
$\frac{4}{6} \leq U < \frac{5}{6}$ & $\frac{5}{6} \leq U \leq 1$ \\ \hline
Events of X & $X$=1 & $X$=2 & $X$=3 & $X$=4 & $X$=5 & $X$=6 \\ \hline
Equal probabilities & $\frac{1}{6}$ & $\frac{1}{6}$ & $\frac{1}{6}$ & $\frac{1}{6}$ & $\frac{1}{6}$ & $\frac{1}{6}$ \\ \hline
\end{tabular}
\end{table}

Notice: the second row of the table can be shuffled, which is also a valid simulation.

\paragraph*{6.6} Somebody messed up the random number generator in your computer, instead of uniform random numbers it generates numbers with an $Exp(2)$ distribution. Describe how to construct a $U(0,1)$ random variable $U$ from an $Exp(2)$ distributed $X$.

\textbf{Answer}.For the uniform distribution, $U(0, 1)$, the distribution function is $F_U(u) = u$ for $0 \leq u \leq 1$. For the exponential distribution, $Exp(2)$, the distribution function is $F_X(x) = 1 - e^{-2x}$ for $x \geq 0$. In order to construct a $U(0, 1)$ random variable $U$ from an $Exp(2)$ distributed $X$, we have to compute $F_U(u) = F_X(x)$. This means $u = 1-e^{-2x}$ for $x \geq 0$. Then, we can define the $U(0, 1)$ distributed random variable $U = F^{inv}(X) = 1-e^{-2X}$. If $U$ is a uniform distribution, then $1-U$ is also a uniform distribution, we can also construct $U = e^{-2X}$.

%\paragraph*{6.8} A random variable X has a $Par(3)$ distribution, so with distribution function $F$ with $F(x) = 0$ for $x < 1$, and $F(x) = 1-x^{-3}$ for $x \geq 1$. For details of the Pareto distribution see Section 5.4. Describe how to construct $X$ from $U(0, 1)$ random variable.
%
%For the $U(0, 1)$ random variable $U$, the distribution function is $F_U(u) = u$ for $0 \leq u \leq 1$. For the $Par(3)$ random variable $X$, the distribution function is $F_X(x) = 1 - \frac{1}{x^{3}}$, for $x \geq 1$. In order to construct a $Par(3)$ random variable $X$ from a $U(0, 1)$ distributed variable $U$, we have to compute $F_X(x) = F_U(u)$. This means $1 - \frac{1}{x^3} = u$. Then we can get $x = (1-u)^{-\frac{1}{3}}$, for $0 \leq u \leq 1$. Then, we can define the $Par(3)$ distributed random variable $X = (1 - U)^{-\frac{1}{3}}$. Since $Y = 1-U$ is also a $U(0, 1)$ distributed random variable is $U$ is a $U(0, 1)$ distributed random variable, we can also construct $X = U^{-\frac{1}{3}}$.
%
%\paragraph*{6.9} In Quick Exercise 6.1 we simulated a die by tossing three coins. Recall that we might need several attempts before succeeding.
%
%If tossing three coins, we could get eight possible outcomes, {HHH, HHT, HTH, THH, HTT, THT, TTH, TTT}. For a die, there should be six outcomes, {1, 2, 3, 4, 5, 6}. Thus, we could match
%
%\begin{table}[h!]
%\renewcommand{\arraystretch}{1.5}
%\centering
%\caption{Simulation of rolling a die using $U(0, 1)$.}
%\label{Ta:6.1a}
%\begin{tabular}{|c|c|c|c|c|c|c|} \hline
%three coins & HHH & HHT & HTH & THH & HTT & THT \\
%die & 1 & 2 & 3 & 4 & 5 & 6 \\ \hline
%\end{tabular}
%\end{table}
%If the outcome is HHT, or TTT, we toss those three coins again until we meet the other six outcomes.
%
%\subparagraph*{a.} What is the probability that we succeed on the first try ?
%
%The probability is $\frac{6}{8} = \frac{3}{4}$.
%
%\subparagraph*{b.} Let $N$ be the number of tries that we need. Determine the distribution of $N$.
%
%We know each trial is a $Ber(\frac{3}{4})$. We repeated it until succeed. We know $N \sim Geo(\frac{3}{4})$.
\end{document}


\documentclass{article}
\usepackage{ amssymb }
\usepackage{amsmath}
\usepackage{mathtools}

\begin{document}
\title{Quiz3}
\author{Malte Wegener (4672194)}

\maketitle

\section{Question 1}
MPI(Message Passage Interface) is parallel Programming Model in which there is no global shared memory. Each Processor has its own private memory. If one Procesor needs data from a different Processor, a message has to be send containing the data. There are 4 modes of messaging in MPI.
\begin{itemize}
    \item Standard Send (MPI\_SEND). In this mode, the Implementation decides whether to use synchronous or buffered mode on the fly
    \item Synchronous Send (MPI\_SSEND). In this mode, the send halts the process until a mathcing recieve is available
    \item Buffered Send (MPI\_BSEND). In this mode, the message is passed into a buffer and then send if a matching recieve is available.
    \item Ready Send (MPI\_RSEND). In this mode the message is send regardless if there is a recieve available, but only suceeds if there is a recieve available. There is no reason to ever use this.
\end{itemize}

\section{Question 2}
As the serial execution time is $T^{\star}=n^3$ and $T_p(p) = \frac{n^3}{p}+p^{1.5}n$. The efficiency is given by $E_p = \frac{T^{\star}}{p*T_p(p)}$. In order to keep the efficiency equal when doubling the number of processors, n has to be increased.

\begin{align}
    \frac{1+p^{2.5}}{n_1^2} = \frac{1+(2p)^{2.5}}{n_2^2}\\
    \frac{n_2}{n_1} = \sqrt{\frac{1+(2p)^{2.5}}{1+p^{2.5}}}
\end{align}
Thus the increase in n has to follw the the previous relation when efficiency should stay the same. In the limit as $n \to \infty$ this approaches 2.387.

When keeping execution time constant when increasing n by increasing p the following relation emerges
\begin{align}
\frac{n_1^3}{p_1}+p_1^{1.5}n_1 = \frac{n_2^3}{p_2}+p_2^{1.5}n_2
\end{align}
However when p is increased this will decrease efficiency of the program.

\section{Question 3}
The data locality ratio is the ratio between the required time for computations between 2 communications and the time required for communication. A higher ratio is better for program performance as it indicates that more time is actually spent computing rather than communicating what to compute.

\end{document}


%!TEX root = forallxcam.tex

\chapter{Alternative proof systems}
In formulating my natural deduction system, I treated certain rules of natural deduction as \emph{basic}, and others as \emph{derived}. However, I could equally well have taken various different rules as basic or derived. I shall illustrate this point by considering some alternative treatments of disjunction, negation, and the quantifiers. I shall also explain why I have made the choices that I have.

\section{Alternative negation rules}
Some systems take the following rule as their basic negation introduction rule:
\begin{proof}
	\open
	\hypo[m]{a}{\meta{A}}
	\have[n-1]{b}{\meta{B}}
	\have[n]{nb}{\enot\meta{B}}
	\close
	\have[\ ]{}{\enot\meta{A}}\by{$\enot$I*}{a-nb}
\end{proof}
and the following as their basic negation elimination rule:
\begin{proof}
	\open
	\hypo[m]{na}{\enot\meta{A}}
	\have[n][-1]{b}{\meta{B}}
	\have{nb}{\enot\meta{B}}
	\close
	\have[\ ]{a}{\meta{A}}\by{$\enot$E*}{na-nb}
\end{proof}
Using these two rules, we could have derived all of the rules governing negation and contradiction that we have taken as basic (i.e.\ $\enot$I, $\enot$E, X and TND). Indeed, we could have avoided all use of the symbol `$\ered$' altogether. Negation would have had a single introduction and elimination rule, and would have behaved much more like the other connectives.

The resulting system would have had fewer rules than ours. So why did I chose to separate out contradiction, and to use an explicit rule TND?\footnote{P.D.\ Magnus's original version of this book went the other way.}

My first reason is that adding the symbol `$\ered$' to our natural deduction system makes proofs considerably easier to work with.

My second reason is that a lot of fascinating philosophical discussion has focussed on the acceptability or otherwise of \emph{ex falso quodlibet} (i.e.\ X) and \emph{tertium non datur} (i.e.\ TND). By treating these as separate rules in the proof system, we will be  in a better position to engage with that philosophical discussion. In particular: having invoked these rules explicitly, it will be much easier for us to know what a system which lacked these rules would look like.

\section{Alternative disjunction elimination}
Some systems take DS as their basic rule for disjunction elimination. They can then treat the $\eor$E rule as a derived rule, by using the following proof scheme:
	\begin{proof}
		\have[m]{ab}{\meta{A}\eor\meta{B}}
		\open
			\hypo[i]{a}{\meta{A}} {}
			\have[j]{c1}{\meta{C}}
		\close
		\open
			\hypo[k]{b}{\meta{B}}{}
			\have[l]{c2}{\meta{C}}
		\close
		\have[n]{aic}{\meta{A} \eif \meta{C}}\ci{a-c1}
		\have{bic}{\meta{B} \eif \meta{C}}\ci{b-c2}
		\open
		\hypo{c}{\meta{C}}
		\have{c1}{\meta{C}}\by{R}{c}
		\close
		\open
		\hypo{nc}{\enot \meta{C}}
		\open
			\hypo{a1}{\meta{A}}
			\have{c2}{\meta{C}}\ce{aic, a1}
			\have{nc1}{\ered}\ri{c2, nc}
		\close
		\have{na}{\enot\meta{A}}\ni{a1-nc1}
		\have{b}{\meta{B}}\ds{ab, na}
		\have{c3}{\meta{C}}\ce{bic, b}
	\close
	\have{con}{\meta{C}}\tnd{c-c1, nc-c3}
\end{proof}
So why did I choose to take $\eor$E as basic, rather than DS?\footnote{P.D.\ Magnus's original version of this book went the other way.} My reasoning is that DS involves the use of `$\enot$' in the statement of the rule. It is in some sense `cleaner' for our disjunction elimination rule to avoid mentioning \emph{other} connectives.

\section{Alternative quantification rules}
An alternative approach to the quantifiers is to take as basic the rules for $\forall$I and $\forall$E from \S\ref{s:BasicFOL}, and also two CQ rule which allow us to move from $\forall \meta{x} \enot \meta{A}$ to $\enot \exists \meta{x} \meta{A}$ and vice versa.\footnote{Warren Goldfarb follows this line in \emph{Deductive Logic}, 2003, Hackett Publishing Co.}

Taking only these rules as basic, we could have derived the  $\exists$I and $\exists$E rules provided in \S\ref{s:BasicFOL}. To derive the $\exists$I rule is fairly simple. Suppose $\meta{A}$ contains the name $\meta{c}$, and contains no instances of the variable $\meta{x}$, and that we want to do the following:
\begin{proof}
	\have[m]{a}{\meta{A}(\ldots \meta{c} \ldots \meta{c}\ldots)}
	\have[k]{c}{\exists \meta{x} \meta{A}(\ldots \meta{x} \ldots \meta{c}\ldots)}
\end{proof}
This is not yet permitted, since in this new system, we do not have the $\exists$I rule. We can, however, offer the following:
\begin{proof}
	\hypo[m]{a}{\meta{A}(\ldots \meta{c} \ldots \meta{c}\ldots)}
	\open
		\hypo{nEna}{\enot \exists \meta{x} \meta{A}(\ldots \meta{x} \ldots \meta{c}\ldots)}
		\have{Ana}{\forall \meta{x} \enot \meta{A}(\ldots \meta{x} \ldots \meta{c}\ldots)}\cq{nEna}
		\have{nAc}{\enot\meta{A}(\ldots \meta{c} \ldots \meta{c}\ldots)}\Ae{Ana}
		\have{red}{\ered}\ri{a, nAc}
	\close
	\have{nnEa}{\enot \enot \exists \meta{x} \meta{A}(\ldots \meta{x} \ldots \meta{c}\ldots)}\ni{nEna-red}
	\have{end}{\exists\meta{x} \meta{A}(\ldots \meta{x} \ldots \meta{c}\ldots)}\dne{nnEa}
\end{proof}\noindent
To derive the $\exists$E rule is rather more subtle. This is because the $\exists$E rule has an important constraint (as, indeed, does the $\forall$I rule), and we need to make sure that we are respecting it. So, suppose we are in a situation where we \emph{want} to do the following:
\begin{proof}
	\have[m]{ExA}{\exists \meta{x} \meta{A}(\ldots \meta{x} \ldots \meta{x}\ldots)}
	\open
		\hypo[i]{Ac}{\meta{A}(\ldots \meta{c} \ldots \meta{c}\ldots)}
		\have[j]{B}{\meta{B}}
	\close
	\have[k]{end}{\meta{B}}
\end{proof}\noindent
 where $\meta{c}$ does not occur in any undischarged assumptions, or in $\meta{B}$, or in $\exists \meta{x} \meta{A}(\ldots \meta{x} \ldots \meta{x}\ldots)$. Ordinarily, we would be allowed to use the $\exists$E rule; but we are not here assuming that we have access to this rule as a basic rule. Nevertheless, we could offer the following, more complicated derivation:

\begin{proof}
	\have[m]{ExA}{\exists \meta{x} \meta{A}(\ldots \meta{x} \ldots \meta{x}\ldots)}
	\open
		\hypo[i]{Ac}{\meta{A}(\ldots \meta{c} \ldots \meta{c}\ldots)}
		\have[j]{B}{\meta{B}}
	\close
	\have[k]{condi}{\meta{A}(\ldots \meta{c} \ldots \meta{c}\ldots) \eif \meta{B}}\ci{Ac-B}
	\open
		\hypo{nB}{\enot \meta{B}}
		\have{nAc}{\enot \meta{A}(\ldots \meta{c} \ldots \meta{c}\ldots)}\mt{condi, nB}
		\have{AxnA}{\forall \meta{x} \enot \meta{A}(\ldots \meta{x} \ldots \meta{x}\ldots)}\Ai{nAc}
		\have{nEA}{\enot \exists \meta{x} \meta{A}(\ldots \meta{x} \ldots \meta{x}\ldots)}\cq{AxnA}
		\have{red2}{\ered}\ri{ExA, nEA}
	\close
	\have{nnB}{\enot\enot\meta{B}}\ni{nB-red2}
	\have{end}{\meta{B}}\dne{nnB}
\end{proof}\noindent
We are permitted to use $\forall$I on line $k+3$ because $\meta{c}$ does not occur in any  undischarged assumptions or in $\meta{B}$. The entries on lines $k+4$ and $k+1$ contradict each other, because $\meta{c}$ does not occur in $\exists \meta{x} \meta{A}(\ldots \meta{x} \ldots \meta{x} \ldots)$.

Armed with these derived rules, we could now go on to derive the two remaining CQ rules, exactly as in \S\ref{s:DerivedFOL}.

So, why did I start with all of the quantifier rules as basic, and then derive the CQ rules?

My first reason is that it seems more intuitive to treat the quantifiers as on a par with one another, giving them their own basic rules for introduction and elimination.

My second reason relates to the discussion of alternative negation rules. In the derivations of the rules of $\exists$I and $\exists$E that I have offered in this section, I invoked DNE. This is a derived rule, whose derivation essentially depends upon the use of TND. But, as I mentioned earlier, TND is a contentious rule. So, if we want to move to a system which abandons TND, but which still allows us to use existential quantifiers, we shall want to take the introduction and elimination rules for the quantifiers as basic, and take the CQ rules as derived. (Indeed, in a system without TND, we shall be \emph{unable} to derive the CQ rule which moves from $\enot \forall \meta{x} \meta{A}$ to $\exists \meta{x} \enot \meta{A}$.)


\documentclass{article}

\usepackage[utf8]{inputenc}
\usepackage[dvipsnames]{xcolor}
\usepackage{amsmath}
\usepackage{tcolorbox}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{proof}
\usepackage{float}
\usepackage{listings}
\lstset{
  basicstyle=\ttfamily,
  mathescape
}

\theoremstyle{definition}

\usepackage[
top    = 2.50cm,
bottom = 2.50cm,
left   = 2.75cm,
right  = 2.75cm]{geometry}
\usepackage{fancyhdr}
\pagestyle{fancy}
\lhead{Advanced computation}
\rhead{EPFL/Alp Ozen}
\newcommand{\floor}[1]{\lfloor #1 \rfloor}

\title{Advanced Computation 1 - CS 101}
\author{Alp Ozen}
\date{\vspace{-5ex}}
\newtheorem{theorem}{Theorem}[section]
\newtheorem{example}{Example}
\newtheorem{remark}{Remark}
\newtheorem{definition}{Definition}
\numberwithin{equation}{subsection}
\numberwithin{remark}{subsection}
\newtheorem{question}{Question \textit{?}}[subsection]
\newtheorem{answer}{Answer}[subsection]
\begin{document}

\maketitle
\section{Propositional logic and notions of sets(Week 1 - Week 4)}
\subsection{Propositions}
A proposition is a declarative sentence that is either true or false.
\begin{tcolorbox}
How much does it cost? \textbf{is not a proposition}
\\
I like red \textbf{is a proposition}
\end{tcolorbox}

To make life easier, we represent propositional statements through letters such as $p$.
\\

The conditional statement $p\implies q$ appears very often. Thus, we have the \textit{converse,contrapositive and inverse} which are:
\\
\textbf{converse}: $q \implies p$
\\
\textbf{contrapositive}: $ \neg q \implies \neg p $
\\
\textbf{inverse}: $ \neg p \implies \neg q$
\\

We note that a conditional is logically equivalent to its contrapositive.

\begin{tcolorbox}
\centering
\begin{array}{cccccc}
 p & q & p $\implies$ q & \neg q & \neg p & \neg q $\implies$ \neg p \\
 \hline
t & t & t & f & f & t\\
t & f & f & t & f & f\\
f & t & t & f & t & t\\
f & f & t & t & t & t
\end{array}
\end{tcolorbox}

\subsection{Precedence of logical operators}

\begin{figure}[h]
    \centering
    \includegraphics{figures/precedence}
\end{figure}

\subsection{Fuzzy logic}

In fuzzy logic, truth values are between 0 and 1. So if the statement "I like riding a bike" has a value of 0.8, it's negation has 1 minus this value, in this case -0.2.

\subsection{Applications of logic}
\subsubsection{Logic gates}
Here are the basic logic circuits from which more complex circuits are made:

\begin{figure}[h]
    \centering
    \includegraphics[scale= 1]{epflSemesterOne/advancedComputation/figures/logicgates.JPG}
    \caption{Logic gates}
    \label{fig:my_label}
\end{figure}

Note that the OR and AND gates accept only and only 2 inputs and output one. This input may be as compounded as possible but not exceed 'two chunks'. Thus, when given a complex logical output and reverse engineering, we identify the outer most outer operation, branch it into two or one(if it is simply a negation) and so on.

\subsubsection{More on propositions}

\begin{tcolorbox}
\begin{itemize}
    \item \textbf{Tautology} is a compound proposition that is always true regardless of the truth value of its variables
    \item \textbf{Contradiction} is a compound proposition that is always false regardless of the truth value of its variables
    \item \textbf{Contingency} compound statement that is neither tautology nor contradiction

    \begin{example}
    \\
    $p \land \neg p$ is a contradiction
    \\
    $p \lor \neg p$ is a tautology
    \end{example}
\end{itemize}
\end{tcolorbox}

Here are some examples of logical calculus:
\\
Show that $\neg(p \lor (\neg p \land q )) \equiveq \neg p \land \neg q$
\begin{align*}
    \neg(p\lor\neg p \land p \lor q)\\
    \neg(T \land p \lor q)\\
    \neg(p \lor q)\\
    \neg p \land \neg q
\end{align*}

\subsubsection{Satisfiability}

A compound proposition is \textbf{satisfiable} if a truth assignment can be made to its variables that make it true making it either a tautology or a contingency. It is \textbf{unsatisfiable} if the negation of the compound statement is a contradiction.

\subsection{Logical calculus and useful equivalences}

\begin{definition}
If $A \iff B$ is a tautology, then A is logically equivalent to B.
\end{definition}
\\

Here are some useful logical equivalences(omitting most obvious ones):

\begin{align*}
    p \implies q \equiv \neg p \lor q \equiv \neg(\neg q \lor p) \equiv \neg(q \implies p) \equiv \neg q \implies \neg p \\
    p \lor ( q \land r) \equiv (p \lor q) \land (p \lor r) \\
    \land \ \text{distributes over} \lor \text{and vice versa}\\
    p \lor \neg p \equiv T\\
    p \land \neg p \equiv F\\
    P \land T \equiv p\\
    p \lor F \equiv p
    \ \text{both $\land$ and $\lor$ are associative}
\end{align*}

For more see this figure:

\begin{figure}[h]
    \centering
    \includegraphics[scale = 0.7]{epflSemesterOne/advancedComputation/figures/logic.JPG}
    \caption{Logic, yey!}
    \label{fig:my_label}
\end{figure}

\begin{definition}
A \textbf{rule of inference} is based on the tautology $p \land (p \implies q) \implies q$. That is, whenever we are given that both $p$ and $p \implies q$ is true, we infer that q must be true. That is :
\\

\infer{q}{p & (p \implies q)}
\end{definition}

Another important fact of logic is that we may boil down all of $\lor, \oplus, \implies, \iff$ to simply propositions involving$\neg, \land$:

\begin{align*}
    p \lor q \equiv \neg \neg (p \lor q) \equiv \neg(\neg p \land \neg q)\\
    p \oplus q \equiv \neg(p \land q) \land (p \lor q)\\
    p \implies q \equiv \neg p \lor q \equiv \neg (p \land \neg q)\\
    p \iff q \equiv \neg(\neg(p\land q) \land (\neg (\neg p \land \neg q))
\end{align*}

\begin{question}
Given propositional variables and truth values of the single variables for which the compound proposition takes a value, is there a way of deducing a compound proposition?
\end{question}
\newpage
\subsection{Lec.03 notes}

The \textbf{contrapositive} is the following statement:
\begin{align*}
    p \implies q \equiv \neg p \lor q\\
    \equiv q \lor \neg p\\
    \equiv \neg q \implies \neg p\\
    \therefore p \implies q \equiv \neg q \implies \neg p
\end{align*}
\\
Some useful logical equivalences involving implication:

\begin{align*}
    (p \implies q) \land (p \implies r) \equiv (\neg p \lor q) \land (\neg p \lor r)\\
    \equiv \neg p \lor (q \land r) \equiv p \implies (q \land r)
\end{align*}
And here's a more trivial one:
\begin{align*}
    (p \implies q) \lor (p \implies r) \equiv (\neg p \lor q) \lor (\neg p \lor r)\\
    \equiv \neg p \lor \neg p \lor q \lor r \equiv \neg p \lor (q \lor r)\\
    \equiv p \implies (q \lor r)
\end{align*}
And slightly more complicated involving De Morgan:
\begin{align*}
    (p \implies r) \land (q \implies r) \equiv (\neg p \lor r) \land (\neg q \lor r)\\
    \equiv \neg r \lor (\neg p \land \neg q) \equiv \neg r \lor (\neg(p \lor q)\\
    \equiv (p \lor q) \implies r
\end{align*}

We must also add some comments on base b systems of numbers and a general algorithm for conversion. Let's take an example in base 5. Suppose we want to convert $60_{10}$ to its base 5 representation. Well the largest power of 5 less than or equal to 60 is 25 and thus we know that 60 can be \textbf{uniquely} represented as a linear combination of the powers of 5 less than or equal to it. In fact, using powers of 5 less than or equal to 60, we may represent all numbers up to $5^k - 1$ as $4\cdot5^{k-1} + \ldots + 4\cdot5^{0}$. Thus, to represent some $l$ in base $b$ the algorithm is to find the largest power of $b^k < l$, perform $\floor{\frac{l}{b^k}}$ then repeat step 1 and proceed as $l - b^k\cdot\floor{\frac{l}{b^k}}$ and repeat until $l - b^i\cdot\floor{\frac{l}{b^i}} = 0$

\subsection{Lec.04 notes}
\subsubsection{More on CMOS}
This was a lecture with a steep curve and here is a summary. First of all, we first revisit some of the concepts of the pmos and nmos resistors. A very important convention is that pmos must never be used for pull-down(connected to GND) and nmos never used for pullup(connected to VDD).

\begin{figure}[h]
    \centering
    \includegraphics[scale=0.7]{epflSemesterOne/advancedComputation/figures/cmos.JPG}
    \caption{Caption}
    \label{Nmos,Pmos,Inverter,Nand}
\end{figure}

\begin{tcolorbox}
Some principles of cmos gates:
\begin{itemize}
    \item Pmos goes to top, nmos to bottom.
    \item Never connect high voltage to low voltage to prevent a short circuit.
    \item Any circuit may be realized as a combination of the NAND and NOR ciruit.
    \item Each pmos must connect to an nmos.
\end{itemize}
\end{tcolorbox}
\\
The most challenging part of CMOS circuits is the circuit analysis itself. Consider this example to see a method of circuit analysis:

\begin{figure}[h!]
    \centering
    \includegraphics[scale = 0.6]{epflSemesterOne/advancedComputation/figures/cmosq.JPG}
    \caption{exam question}
    \label{exam question}
\end{figure}

\FloatBarrier

\begin{tcolorbox}
Now, first of all, realize that X is connected to VDD iff all of $Y_{6} \land Y_{5} \land Y_{4}$ are grounded. That is $Y_{6} \land Y_{5} \land Y_{4} = 0$. For symbolic purposes, supposing that $Y_{i} = 0 \equiv \neg Y_{i} $ we get that $X = 1 \iff \neg (Y_{4} \lor Y_{5} \lor Y_{6})$ Now given that this is a CMOS circuit, we know that the bottom part does the exact opposite of the upper part. Thus, we have that $X=0 \iff \neg(\neg (Y_{4} \lor Y_{5} \lor Y_{6})) = Y_{4} \lor Y_{5} \lor Y_{6} \equiv Y_{1} \lor Y_{2} \lor Y_{3}$. As a final step, for $\neg (A \lor B \lor C)$ to be true, we need that the output equals $\neg(A \lor B \lor C)$ and since $X=1 \iff \neg (Y_{4} \lor Y_{5} \lor Y_{6})$ we get that $Y_{1} = Y_{2} \ldots$

\end{tcolorbox}

\subsubsection{Binary addition circuit}

Given that we can construct any compound logical gate using CMOS, suppose we want to implement a binary addition calculator. Now here are all the possible cases for doing binary addition:

\begin{figure}[h]
    \centering
    \includegraphics[scale = 0.6]{epflSemesterOne/advancedComputation/figures/binary.png}
    \caption{Addition possibilities}
    \label{fig:my_label}
\end{figure}

Now suppose we want a function $f(a,b,c_{in})$ to evaluate $s$. Well notice that $s$ is only true when the parity of $a,b,c_{in}$ is odd. That is, we may describe this outcome with the function $a \oplus b \oplus c_{in}$. Similarly, devising $g(a,b,c_{in})$ to compute $c_{out}$ we notice that $c_{out}$ evaluates to 1 iff at least two variables are true. This is equivalent to $(a\land b) \lor (a \land c_{in}) \lor (b \land c_{in})$

\subsubsection{Fast Multiplication aka. Karatsuba}
We now ponder whether there is a quick way of multiplying some $v$ and $w$. Now notice that for $v$ and $w$ in base 10, $v = aX + b$ and $w = cX + d$. Now notice that $v \cdot w = (aX+b)(cX+d)$ which in turn is:

\begin{equation*}
    v\cdot w = acX^2 + (ad + bc)X + bd
\end{equation*}

And further notice that:

\begin{equation*}
    ad + bc = (ac + bd) - (a - b)(c - d)
\end{equation*}

to get:

\begin{equation*}
    v\cdot w = acX^2 + ((ac + bd) - (a - b)(c - d))X + bd
\end{equation*}

Now if for instance $v$ and $w$ were 2 digit numbers, we would normally perform 4 digit by digit multiplications but with this new method, we end up performing only 3 and a trivial subs traction.
\\
As a general result, for multiplication of two k by k digit numbers, we end up performing $3^{\log_{2}k}$ multiplications and $log_{2}k$ many additions. Finally, notice that Karatsuba is a recursive algorithm.
\subsubsection{Two's compliment}
Consider how a computer is to represent negative integers. A very smart way of doing so is \textbf{two's compliment}. That is given a binary representation, we invert all 1's with 0's and all 0's with 1's and then add 1. Note that now the 0's take the role of 1's and vice versa. The reason for adding 1 is that the most significant digit is reserved for the sign. A 1 is a negative, a 0 a positive.
\\
\begin{remark}
Note that when multiplying two numbers with non-matching number of digits, we simply pad both numbers with 0's until both have number of digits that are a power of 2.
\end{remark}

\subsection{Lec 05. notes}
\subsubsection{Main points}
In this lecture quantifiers and their properties were discussed along with common pitfalls.
\\
Consider defining a proposition on some sub-domain. That is take $D = \{0,1,2\}, S = \{2\}$ Now we want to express the proposition $\forall x \in S, P(x)$ where $P(x)$ is to mean that $x$ is even in the form $\forall x Q(x)$. A major mistake made is to try and express this as $x \in S \land P(x) \equiv Q(x)$ Now clearly $\forall x Q(x) \not \equiv \forall x \in S, P(x) $ as taking $x=1$ leads to the LHS being false. So here is the correct way to do this: Let $Q(x) \equiv x\in S \rightarrow P(x)$. It is now the case that $\forall x Q(x) \equiv \forall x \in S, P(x)$. Therefore we have that:

\begin{align}
    \forall x \in S, P(x) \equiv \forall x(x\in S \rightarrow P(x))\\
    \exists x \in T P(x) \equiv \exists x( x\in T \land P(x))
\end{align}

\begin{remark}
Note that both $\exists$ and $\forall$ have precedence over any other logical operator.
\end{remark}

We would also like to point out that whenever we have an empty domain, then both $\exists x P(x)$ and $\exists x \neg P(x)$ are both false. Similarly, $\forall x P(x)$ and $\forall x \neg P(x)$ are both true.
\\
We now ask, how do we negate the quantifiers?

\begin{align}
    \neg(\exists x P(X)) \equiv \forall x \neg P(x)\\
    \neg(\forall x P(X)) \equiv \exists x \neg P(x)
\end{align}
And we add the note that the same negation rules also hold for quantifiers over a sub-domain. Here is an example proof:

\begin{align*}
    \equiv \neg \forall x \in S \ P(x) \equiv \neg (\forall x \in S \rightarrow P(x))\\
    \equiv \exists x \neg(x \in S \rightarrow P(x))\\
    \equiv \exists x \neg(x \not \in S \lor P(x))\\
    \equiv \exists x (x \in S \land \neg P(x))\\
    \equiv \exits x \in S, \ \neg P(x)
\end{align*}

\subsection{Lec 06. notes}

First of all, we note that when solving CMOS problems, always check if the circuit is complementary. That is whenever the upper part is 1, lower part must be disconnected. Hence if the upper part evaluates $A\land B$, lower part must evaluate $\neg(A \land B)$

Here is a nice CMOS circuit problem from week 3:

\begin{figure}[H]
    \centering
    \includegraphics[scale=0.7]{epflSemesterOne/advancedComputation/figures/cmos2.JPG}
    \caption{PSET3 cmos problem}
    \label{fig:my_label}
\end{figure}

Here's how I solved it:
\\
Now notice that $X=1$ iff. $(B=0 \land \neg A = 0) \lor (\neg B = 0 \land A = 0)$ Thus we obtain:
\begin{align*}
    (\neg B \land A) \lor (B \land \neg A)\\
    \equiv \neg(B\lor \neg A) \lor \neg(\neg B \lor A)\\
    \equiv \neg(A \rightarrow B) \lor \neg(B \rightarrow A)\\
    \equiv \neg ((A \rightarrow B) \land (B \rightarrow A))\\
    \equiv \neg (A \iff B)
\end{align*}
\clearpage

And here is some more details on two's compliment:

\begin{figure}[H]
    \centering
    \includegraphics[scale = 1.2]{epflSemesterOne/advancedComputation/figures/twosc.JPG}
    \caption{More on two's compliment}
    \label{fig:my_label}
\end{figure}

And now let's say a little more about two's complement. That is, two's compliment simply represents the additive inverse of a number in $\bmod{2^k}$ That is, suppose we wanted some $b$ such that $a + b \equiv 0 \bmod{2^k}$, then we'd have that $b = 2^{k} - a$ Which is essentially the formula for two's complement for a given bit architecture.

\subsection{Lec 07. notes}

We consider how to negate the $\exists!$ expression. Realize that:
$$\exists!xP(x) \equiv \exists x(P(x)\land P(y) \rightarrow (x=y))$$
And hence the negation(using our negation laws gives):
$$\neg\exists ! xP(x) \equiv P(x) \rightarrow \exists y \not = xP(y))$$
\\
And two demonstrate the importance of nesting order on quantifiers, consider this excerpt:

\begin{figure}[H]
    \centering
    \includegraphics[scale = 0.8]{epflSemesterOne/advancedComputation/figures/nesting.JPG}
    \caption{Nesting order demo}
    \label{fig:my_label}
\end{figure}

And notice here how $\exists x \forall y S(x,y) \rightarrow \forall y \exists x S(x,y)$ This makes sense as whenever an x exists for each y, we have to have that for all y, there is an x.
\\
\clearpage
And now we come to rules of inference for quantifiers.

\begin{definition}(Inference laws for quantified statements)
\\
\textbf{Universal instantiation}
\begin{equation*}
    \infer{P(c)}{\forall x P(x)}
\end{equation*}
\textbf{Universal generalization}
\begin{equation*}
        \infer{\forall x P(x)}{P(x) \text{for arbitrary x}}
\end{equation*}
And we note that the same hold respectively for the existential quantifier.
\end{definition}

And finally we present an application of rules of inference. Suppose the following:
\\
$H(x):$ x is here
\\
$U(x):$ x likes C
\\
$L(x)$ x is a fan of D.R.(Denis Ritchie)
\\
Now we assert the following:
\\
(1) There is someone here who likes C
\\
(2) Everyone who likes C is a fan of D.R.
\\
Hence we get:
$$(1) \equiv \exists x(H(x) \land U(x))$$
$$(2) \equiv \forall x(U(x) \rightarrow L(x))$$
Now what may we infer from these?
\\
Well we know  $\exists x(H(x) \land U(x))$ hence we by instantiation we have $H(c) \land U(c)$ then $U(c),H(c)$ and since $ \forall x(U(x) \rightarrow L(x))$ to give $U(c) \rightarrow L(c), L(c)$ and finally $$H(c)\land L(c)$$
\\
And although it is not too useful to memorize their names, we include here a table of common rules of inference:

\begin{figure}[H]
    \centering
    \includegraphics[scale = 0.9]{epflSemesterOne/advancedComputation/figures/inference.JPG}
    \caption{Rules of inference}
\end{figure}

\subsection{Lec 08. notes}
\subsubsection{Proof methods}
A \textbf{proof} is a valid argument establishing the truth of a statement.
\\
We list some common proof methods and elaborate:
\\
\textbf{Direct proof:} When trying to prove a statement like $p \rightarrow q$ we consider the only case that $p \rightarrow q$ would be false and show that it can not happen. That is we take $p$ true and using our inference laws, reach that $q$ must be true as well
\\
\textbf{Proof by contraposition:} Since $p\rightarrow q \equiv \neg q \rightarrow \neg p$ we try to show that the contrapositive holds via a direct proof. And here's a mini-example: Consider the statement \textit{if $n$ is an integer then $3n+2$ is odd.} Now suppose $3n+2$ is even to give $3n+2 = 2k, \ k\in \mathbb{N}$ Then we have that $n=\frac{2k-2}{3}$ and taking $k=2$ suffices to show $n$ is not an integer.
\\
\textbf{Proof by contradiction:} Firstly, proof by contradiction is often confused with proof by contraposition. We use proof by contradiction to show that a statement $p$ is true. To do this, if we can show that $\neg p \rightarrow q$ is true where $q$ is always false, we have that $\neg p$ is false hence $p$ true. A common example is showing that $\sqrt{2}$ is irrational. We take the negation of $p$ that $\sqrt{2}$ is rational and derive the contradiction that whenever we try to write $\sqrt{2} = \frac{z}{k}$ with $\gcd{z}{k} = 1$ that $\gcd{z}{k} = 1 \land \gcd{z}{k} \not = 1$ Thus the falsity of $p$ implies a contradiction(false value) showing that $p$ itself must be true.
\\
Similarly suppose we have statement $p\equiv a \rightarrow b$. We know that $\neg p \rightarrow a \land \neg b$ and get that $a\land \neg b$ is always false showing that $p$ must be true.
\subsubsection{Graphs and planarity}
\textbf{A planar} graph is one that can be drawn without overlapping edges. Now a planar graph does not imply that for all possible drawing, overlapping edges wont exist. It simply means that a drawing could be made without overlapping edges. Now the two most important facts of graphs are:

 \begin{tcolorbox}[drop shadow, title=($K_{3,3}$ and $K_{5}$ are not planar),lower separated=true]
    Now notice that each connection we make partitions the plane into an inner and outer side following the \textbf{Jordan theorem}. We find out that when it comes to drawing the last vertex for $K_{3,3}$, the edge is surrounded by 4 curves making it impossible to not cross a curve.
    \tcblower
    \centering
        \includegraphics[scale = 1.1,valign=t]{epflSemesterOne/advancedComputation/figures/k3.png}
\end{tcolorbox}
\\
Now, \textbf{Kuratowski's theorem} states that a graph is non-planar iff it contains in one way $K_{3,3}$ or $K_{5}$
\clearpage
\subsubsection{Different surfaces}
We now consider different surfaces, those of the \textbf{Torus, Mobius strip, Klein bottle} shown respectively in the figure below.

\begin{figure}[H]
    \centering
    \includegraphics{epflSemesterOne/advancedComputation/figures/mobius.JPG}
    \caption{Rectangular abstraction}
\end{figure}

These surfaces are interesting because the laws of planarity no longer hold. For instance, on a mobius strip, $K_{3,3}$ is indeed planar as below.

 \begin{tcolorbox}[drop shadow, title=($K_{3,3}$ has become planar),lower separated=true]
    \centering
        \includegraphics[scale = 0.9,valign=t]{epflSemesterOne/advancedComputation/figures/mobiusk3.JPG}
\end{tcolorbox}

\subsection{Lec 09. notes}
\subsubsection{Sequences}
We define a \textbf{sequence} as a mapping $f: \mathbb{N} \to \mathbb{R}$. Some most essential sequences are:

\begin{align*}
    \textbf{constant sequence} \ \exists c \forall i \ a_{i} = c\\
    \textbf{arithmetic sequence} \ \exists c \forall i \ a_{i+1} - a_{i}  = c\\
    \textbf{geometric sequence} \ \exists c \forall i \ \frac{a_{i+1}}{a_{i}}= c
\end{align*}
\\
More generally, the constant and arithmetic sequence are a member of the family of consecutive polynomial sequences. That is, since a polynomial is defines as:

$$\sum_{i=0}^{j} a_{i}X^{i}$$ we may have sequences of higher degree.
\subsubsection{Summations}
Consider the summation of the first n terms of a constant sequence $a_{i} = c$. We would have $\sum_{i=1}^{n}a_{i}$ which is the same as $\sum_{i=1}^{n}c$ and taking $c=1$ we get n times 1 added which is $\sum_{i=1}^{n}c = 1$
\\
Let's now consider the summation of an arithmetic sequence by using an $nxn$ square as below:
\begin{figure}[H]
    \centering
    \includegraphics{epflSemesterOne/advancedComputation/figures/square.JPG}
    \caption{Summations, yey!}
\end{figure}

We agree that at each step there are $2i-1$ colored crosses which add up to $n^2$ many crosses. So we express it as $\sum_{i=1}^{n}(2i-1) = n^2$ and try to get a useful formula out of it.
\\
Well we have that $\sum_{i=1}^{n}(2i-1) = \sum_{i=1}^{n}(2i) - \sum_{i=1}^{n}(1)$ Which further gives $2\sum_{i=1}^{n}(i) - \sum_{i=1}^{n}(1) = n^2$ hence we obtain $$\sum_{i=1}^{n}(i) = \frac{n^2 + n}{2}$$

\textcolor{gray}{We now consider more interesting summations involving} \textbf{telescoping sequences}. A telescoping sequence is a sequence where during summation of the terms (parts of) consecutive terms cancel each other, so that only few (parts of) terms remain to be added.
To illustrate this, consider the sequence $\{b_i\}$ for $i>0$ with $$b_i=i-(i-1),$$
and let $\{a_i\}$ be a constant sequence with $a_i=1$.
Obviously, it follows that $b_i=1$ so that $b_i=a_i$ and thus $\sum_{i=1}^nb_i=\sum_{i=1}^na_i$. The latter sum was already calculated, which was a trivial exercise but in principle still required adding together $n$ terms all equal to~1. Using that $\sum_{i=1}^nb_i=\sum_{i=1}^na_i$ the same sum $\sum_{i=1}^na_i$ can be computed by computing $\sum_{i=1}^nb_i$ instead: due to the telescoping effect, this computation does not require any actual calculations at all, but just involves careful administration:
\begin{eqnarray*}
\sum_{i=1}^nb_i&=&b_n\,+\,b_{n-1}\,+\,b_{n-2}\,+\,\ldots\,+\,b_2\,+\,b_1\\
&=&\underbrace{n-(n-1)}_{b_n}\,+\,\underbrace{n-1-(n-2)}_{b_{n-1}}\,+\,\underbrace{n-2-(n-3)}_{b_{n-2}}\,+\,\ldots\,+\,\underbrace{2-(2-1)}_{b_{2}}\,+\,\underbrace{1-(1-1)}_{b_{1}}\\
&=&n\underbrace{-(n-1)+n-1}_{=0}\underbrace{-(n-2)+n-2}_{=0}\underbrace{-(n-3)+\ldots}_{=0}\ldots\underbrace{\ldots+2}_{=0}\underbrace{-1+1}_{=0}\,-\,0\\
&=&n.
\end{eqnarray*}
\clearpage
Yet a much more formal way of showing this as follows:

\begin{eqnarray*}
\sum_{i=1}^nb_i&=&\sum_{i=1}^n(i-(i-1))\\
&=&\Big(\sum_{i=1}^ni\Big)-\Big(\sum_{i=1}^n(i-1)\Big)\,\,\,\,\text{     (in 2nd sum replace $i-1$ by $j$)} \\
&=&\Big(\sum_{i=1}^{n}i\Big)-\Big(\sum_{j=0}^{n-1}j\Big)\,\,\,\,\text{     (replace $j$ by $i$)} \\
&=&\Big(\sum_{i=1}^{n}i\Big)-\Big(\sum_{i=0}^{n-1}i\Big)\,\,\,\,\text{     (identify the part that occurs in both sums)} \\
&=&\Big(n+\sum_{i=1}^{n-1}i\Big)-\Big(0+\sum_{i=1}^{n-1}i\Big)\\
&=&n-0+\Big(\sum_{i=1}^{n-1}i\Big)-\Big(\sum_{i=1}^{n-1}i\Big)\\
&=&n.
\end{eqnarray*}

We now consider a different approach to geometric sequences. We are interested in a closed form expression for $\sum_{i=0}^ng_i$, where for relevant values of $r$ it may even make sense to talk about the value of $\sum_{i=0}^\infty g_i$.
Because $$\sum_{i=0}^ng_i=\sum_{i=0}^ng_0r^i=g_0\sum_{i=0}^nr^i$$
the value~$g_0$ is just an uninteresting multiplicative factor, and we omit it (i.e., from now on we assume that $g_0=1$; note that the entire sum equals zero if $g_0=0$).
Obviously, the problem is not interesting if $r=1$: then we get (with $g_0=1$) that
$$\sum_{i=0}^ng_i=\sum_{i=0}^nr^i=\sum_{i=0}^n1^i=\sum_{i=0}^n1=n+1.$$
Also $r=-1$ is not inspiring: we get 0 if $n$ is odd and 1 if $n$ is even; but the final arguments below work for $r=-1$ as well, so $r=-1$ is not excluded below.

Assuming $r\ne1$, we want to find a closed form expression for $\sum_{i=0}^nr^i$, i.e., for $$r^0+r^1+\ldots+r^{n-2}+r^{n-1}+r^n.$$
It was noted that we are already familiar with a similar sum, namely $$r^n+r^{n-1}+r^{n-2}+\ldots+r^1+r^0$$
which is just the same, but in the reverse order.
How come we are familiar with something like $r^n+r^{n-1}+r^{n-2}+\ldots+r^1+r^0$?
Just look at the decimal number $111\ldots11$ that consists of $n+1$ ones:
that is precisely the same as our expression $r^n+r^{n-1}+r^{n-2}+\ldots+r^1+r^0$ if we assume that $r=10$ (the \emph{basis} or \emph{radix} of our decimal number system) because our decimal number ${\color{red}{1}}{\color{green}{1}}{\color{blue}{1}}\ldots{\color{pink}{1}}{\color{magenta}{1}}$ is just a convenient shorthand notation for
$${\color{red}{1}}\times10^n+{\color{green}{1}}\times10^{n-1}+{\color{blue}{1}}\times10^{n-2}+\ldots+{\color{pink}{1}}\times10^1+{\color{magenta}{1}}\times10^0:$$
that is how the value of a number in decimal notation is defined, where the present case is particularly simple because all the digits are equal to one.

Taking for instance $n=4$, we find the number $11111$, and we know that if we multiply it by $9$ (which was carefully chosen as $10-1$, i.e., as $r-1$) we get, without any effort, $99999$, so that if we add one we get 100000, for which we have the nice closed form expression $10^5$ (i.e., $r^{n+1}$). We thus found that $$(11111\times 9)+1=10^5$$
and more in general that
$$(\underbrace{111\ldots11}_{n+1 \text{ ones}}\times 9)+1=10^{n+1},$$
or even more in general that
$$(\underbrace{111\ldots11}_{n+1 \text{ ones}}\times (r-1))+1=r^{n+1},$$
where the final $111\ldots11$ should be liberally interpreted as a shorthand notation for $r^n+r^{n-1}+r^{n-2}+\ldots+r^1+r^0$
and where we use that $10$ was just a placeholder for $r$ and thus that $9$ was a placeholder for $r-1$. This final expression leads us to believe that
$$\underline{(r^n+r^{n-1}+r^{n-2}+\ldots+r^1+r^0)\times(r-1)}+1=r^{n+1}$$
and thus that
$$r^n+r^{n-1}+r^{n-2}+\ldots+r^1+r^0=\frac{r^{n+1}-1}{r-1}$$
(where you should remember that indeed $r\ne 1$). But this is precisely the closed formula for $\sum_{i=0}^nr^i$ that we were trying to find, so we are done -- except we have not proved anything yet other than by intimidation.

To actually prove this result that
$$\sum_{i=0}^nr^i=\frac{r^{n+1}-1}{r-1}$$
if $r\ne 1$, have a look at the underlined part
$$(r^n+r^{n-1}+r^{n-2}+\ldots+r^1+r^0)\times(r-1)$$
of the equation above:
apparently we need to prove that this underlined part is equal to $r^{n+1}-1$.
Pulling the factor $r-1$ inside the parentheses we get
$$(r-1)r^n+(r-1)r^{n-1}+(r-1)r^{n-2}+\ldots+(r-1)r^1+(r-1)r^0$$
and thus
$$r^{n+1}-r^n+r^n-r^{n-1}+r^{n-1}-r^{n-2}+\ldots+r^2-r^1+r^1-r^0.$$
\textcolor{gray}{This is yet another example of our earlier telescoping sum: all the intermediate terms cancel each other.}

In any case, the conclusion is that if $r\ne1$ then
$$\sum_{i=0}^nr^i=\frac{r^{n+1}-1}{r-1}$$
for any integer $n\ge0$.
It follows that if $|r|<1$ we have that
$$\sum_{i=0}^\infty r^i=\frac{1}{1-r}$$
because for $n\to\infty$ the term $r^{n+1}\to0$.
\\

We finally conclude by representing a useful identity as follows:

$$ \sum_{i=1}^{\infty}\sum_{j=1}^{i} \cdots =  \sum_{j=1}^{\infty}\sum_{i=j}^{\infty} \cdots$$
\clearpage

\subsubsection{Sum of $i^k$}
And yet another important point to mention is that all the sums of form $i^k$ may be found using the fact that the telescoping sequence given by $$f(k+1,i) = i^{k+1} - (i-1)^{k+1}$$ since using the telescoping property we have that $$\sum_{i=1}^{n} i^{k+1} - (i-1)^{k+1} = n^{k+1} $$ which means that $$\sum_{i=1}^{n} i^{k+1} = n^{k+1} + \sum_{i=1}^{n}(i-1)^{k+1} $$ where the $\sum_{i=1}^{n}(i-1)^{k+1}$ are recursively found.

\subsubsection{Hilbert's Hotel}

Some interesting questions to pose on this matter are listed below with answers.
\begin{question}
A bus carrying countably infinite number of guests arrives, can guests be accommodated?
\end{question}
\begin{answer}
Well simply assign each former guest to odd numbered room and each new guest to even numbered room.
\end{answer}

\begin{question}
Now suppose that countably infinite number of buses with countably infinite guests arrive what now?
\end{question}
\begin{answer}
Well, we move each guest in room $k$ to room $2k-1$. Now for new guests, $kth$ guest in bus 1 moves to $2^{1}(2k-1)$ hence in general $kth$ guest in bus $j$ moves to $2^{j}(2k-1)$ room.
\end{answer}
\clearpage
\section{Algorithms and more (Week 5 - )}
\subsection{Lec 10. notes}
To demonstrate how summations come in handy consider snipets of following pseudocode

\begin{verbatim}
    B <- n while(B>0){step;}
\end{verbatim}
Leads to $\infty$ steps.
\begin{lstlisting}
          B < - n while(B>0){for i=1 to B do step}){B = $\floor{\frac{B}{2}}$}
\end{lstlisting}
Now notice we first perform $n$, then approximately $\frac{n}{2}$ steps and so on. Thus in total we have $\sum_{i=0}^{\log_{2}{n} }(\frac{n}{2^j})$ many steps where this approximates to $2n$
\\
And now sum must now summation results:

\begin{figure}[H]
    \centering
    \includegraphics[scale = 0.7]{epflSemesterOne/advancedComputation/figures/sums.JPG}
    \caption{Summation formulae}
    \label{fig:my_label}
\end{figure}
And now we present \textbf{countability}:
\begin{definition} (Countabiltiy)
A set is countable iff. it is finite or there is a bijection from \mathbb{N} to the set.
\end{definition}

And some countability facts:
\begin{itemize}
    \item Union of countable sets is countable
    \item \mathbb{Z} is countable, just map each odd number to the negative integers and each even number to positive integers.
    \item Union of a finite number of countable sets is countable. That is for the finite $A_{k}$ $k$ many sets, first map all the $a_{0} \in A_{0}$ to 1, then $a_{1} \in A_{1}$ to 2 and hence $k+1 \to a_{1} \in A_{0} $
    \item Union of a countable number of countable sets is countable.
    \begin{figure}[H]
        \centering
        \includegraphics[scale = .6]{epflSemesterOne/advancedComputation/figures/walk.JPG}
        \caption{Enumeration}
        \label{fig:my_label}
    \end{figure}
    \item Cartesian product of two countable sets is countable (for which reason rational numbers are countable)
    \item Infinite union of bitsrings of finite length are countable. That is suppose we define $\bigcup_{i=0}^{j} S^{j}$ where $S = \{0,1\}$ Now $S^{0}$ has 1 element, $S^{1}$ has 1 distinct element, $S^{2}$ has 3 distinct elements and so on. Thus, we map $ 0 \to S^{0}$, then $1 \to 01 $ $2 \to 10$ $3 \to 11$. Since each $S^{k}$ has $2^{k}$ elements our mapping is defined.
    \item The set $\{0,1\}^{*}$ consisting of infinite bitstrings is \textbf{uncountable} as follows by Cantor diagonilization.
\end{itemize}
\\
A question from this week's pset asked:
\\
\begin{question}
Let $A$ be the set of real numbers with a finite number of 1 in the binary form and $B$ the set of all reals with a finite number of 1 in decimal form. Which of these sets is countable?
\end{question}
\begin{answer}
As it turns out, when we try enumerating members of A, we notice this is doable because for instance if we were to miss out enumerating one element it would have to be the case that it either contains a 0 which  we have already counted or a 1 which again we have already counted. Conversely for B, suppose we have enumerated all such numbers, but then define a new number with a digit differing in at least one decimal place of each element of B.
\end{answer}

\subsection{Lec 11. notes}
We start with the \textbf{Halting problem} which states that given a computer program $P$ with input $I$ can we write a program that always determines whether $P(I)$ will terminate or not? As it turns out the answer is no.
\begin{example*}
Now let us define a program $A(P,x)$ that determines whether the program $P$ with input $x$ terminates. Now define $Q(x)$ as a program calling $A(x,x)$. Then $Q(P)$ runs $A(P,P)$ and $Q(P)$ will stop if $A(P,P)$ returns. Now consider $Q(Q)$. We have that $A(Q,Q)$ gets called. Well, if this returns no meaning $Q(Q)$ does not determinate, we get that $Q(Q)$ should terminate and vice versa. Hence, we get that such a program $A(x)$ can not exists.
\end{example*}

\begin{definition}(Big $O$)
A function $f(x)$ is said to be $O(g(x))$ or informally $f$ is $O(g)$ if $$ \exists C,k \ \forall x>k, \ |f(x)| \leq C|g(x)|$$
\end{definition}

\begin{definition}(Big $\Omega$ )
A function $f(x)$ is said to be $\Omega(g(x))$ or informally $f$ is $O(g)$ if $$ \exists C,k \ \forall x>k, \ |f(x)| \qeq C|g(x)|$$
\end{definition}

\begin{definition}(Big $\Theta$)
if $f$ is both $O(g)$ and $\Omega(g$, then f is $\Theta(g)$ meaning that f and g are of the same order
\end{definition}

\end{document}


%%
%% Automatically generated file from DocOnce source
%% (https://github.com/hplgit/doconce/)
%%
%%
% #ifdef PTEX2TEX_EXPLANATION
%%
%% The file follows the ptex2tex extended LaTeX format, see
%% ptex2tex: http://code.google.com/p/ptex2tex/
%%
%% Run
%%      ptex2tex myfile
%% or
%%      doconce ptex2tex myfile
%%
%% to turn myfile.p.tex into an ordinary LaTeX file myfile.tex.
%% (The ptex2tex program: http://code.google.com/p/ptex2tex)
%% Many preprocess options can be added to ptex2tex or doconce ptex2tex
%%
%%      ptex2tex -DMINTED myfile
%%      doconce ptex2tex myfile envir=minted
%%
%% ptex2tex will typeset code environments according to a global or local
%% .ptex2tex.cfg configure file. doconce ptex2tex will typeset code
%% according to options on the command line (just type doconce ptex2tex to
%% see examples). If doconce ptex2tex has envir=minted, it enables the
%% minted style without needing -DMINTED.
% #endif

% #define PREAMBLE

% #ifdef PREAMBLE
%-------------------- begin preamble ----------------------

\documentclass[%
oneside,                 % oneside: electronic viewing, twoside: printing
final,                   % draft: marks overfull hboxes, figures with paths
10pt]{article}

\listfiles               %  print all files needed to compile this document

\usepackage{relsize,makeidx,color,setspace,amsmath,amsfonts,amssymb}
\usepackage[table]{xcolor}
\usepackage{bm,ltablex,microtype}

\usepackage[pdftex]{graphicx}

\usepackage[T1]{fontenc}
%\usepackage[latin1]{inputenc}
\usepackage{ucs}
\usepackage[utf8x]{inputenc}

\usepackage{lmodern}         % Latin Modern fonts derived from Computer Modern

% Hyperlinks in PDF:
\definecolor{linkcolor}{rgb}{0,0,0.4}
\usepackage{hyperref}
\hypersetup{
    breaklinks=true,
    colorlinks=true,
    linkcolor=linkcolor,
    urlcolor=linkcolor,
    citecolor=black,
    filecolor=black,
    %filecolor=blue,
    pdfmenubar=true,
    pdftoolbar=true,
    bookmarksdepth=3   % Uncomment (and tweak) for PDF bookmarks with more levels than the TOC
    }
%\hyperbaseurl{}   % hyperlinks are relative to this root

\setcounter{tocdepth}{2}  % levels in table of contents

% Tricks for having figures close to where they are defined:
% 1. define less restrictive rules for where to put figures
\setcounter{topnumber}{2}
\setcounter{bottomnumber}{2}
\setcounter{totalnumber}{4}
\renewcommand{\topfraction}{0.95}
\renewcommand{\bottomfraction}{0.95}
\renewcommand{\textfraction}{0}
\renewcommand{\floatpagefraction}{0.75}
% floatpagefraction must always be less than topfraction!
% 2. ensure all figures are flushed before next section
\usepackage[section]{placeins}
% 3. enable begin{figure}[H] (often leads to ugly pagebreaks)
%\usepackage{float}\restylefloat{figure}

% prevent orhpans and widows
\clubpenalty = 10000
\widowpenalty = 10000

% --- end of standard preamble for documents ---

% insert custom LaTeX commands...

\raggedbottom
\makeindex
\usepackage[totoc]{idxlayout}   % for index in the toc
\usepackage[nottoc]{tocbibind}  % for references/bibliography in the toc

%-------------------- end preamble ----------------------

\begin{document}

% matching end for #ifdef PREAMBLE
% #endif

\newcommand{\exercisesection}[1]{\subsection*{#1}}

% ------------------- main content ----------------------

% ----------------- title -------------------------

\thispagestyle{empty}

\begin{center}
{\LARGE\bf
\begin{spacing}{1.25}
Project 5, deadline  December 10, 2017
\end{spacing}
}
\end{center}

% ----------------- author(s) -------------------------

\begin{center}
{\bf Partial Differential equations${}^{}$} \\ [0mm]
\end{center}

\begin{center}
% List of all institutions:
\end{center}

% ----------------- end author(s) -------------------------

% --- begin date ---
\begin{center}
Fall semester 2017
\end{center}
% --- end date ---

\vspace{1cm}

\subsection{Theoretical background and description of the system}

This project is more numerically directed, with tests
of algorithms for solving partial differential equations using finite difference schemes.
Chapter 10 of the lecture notes provides the necessary theoretical background.

\textbf{For this project you can collaborate with fellow students and you can  hand in a common report.}
This project (together with projects 3 and 4) counts 1/3 of the final mark.

The project has a strong mathematical axis. However, for those interested, it should be straightforward to replace the dimensionless analysis in  parts 5a-5f) with specific boundary and initial conditions. This is done in part 5g), with a focus on the heat equation and geophysical processes. That part is optional, but gives an additional 30 points to a score of 100.

The physical problem can be that of the temperature gradient in a rod of length $L=1$ or that of channel flow
between two flat and infinite plates at $x=0$ and $x=1$, where the fluid is initially at rest
and the plate $x=1$ is given a sudden initial movement.  We are looking at a one-dimensional
problem

\begin{equation*}
 \frac{\partial^2 u(x,t)}{\partial x^2} =\frac{\partial u(x,t)}{\partial t}, t> 0, x\in [0,L]
\end{equation*}
or

\begin{equation*}
u_{xx} = u_t,
\end{equation*}
with initial conditions, i.e., the conditions at $t=0$,

\begin{equation*}
u(x,0)= 0 \hspace{0.5cm} 0 < x < L
\end{equation*}
with $L=1$ the length of the $x$-region of interest. The
boundary conditions are

\begin{equation*}
u(0,t)= 0 \hspace{0.5cm} t \ge 0,
\end{equation*}
and

\begin{equation*}
u(L,t)= 1 \hspace{0.5cm} t \ge 0.
\end{equation*}
The function $u(x,t)$  can be the temperature gradient of a  rod or represent the fluid velocity
in a direction parallel to the plates, that is normal to the $x$-axis. In the latter case,
for small $t$, only the part of the fluid
close to the moving plate is set in significant  motion, resulting in a thin boundary layer at $x=1$.
As time increases, the velocity approaches a linear variation with $x$. In this case, which can be derived
from the incompressible Navier-Stokes, the above equations constitute a model for
studying friction between moving surfaces separated by a thin fluid film.

In this project we want to study the numerical stability of three methods for partial differential equations
(PDEs).
These methods we will discuss are
the explicit forward Euler algorithm with discretized versions of time given by a forward formula and a centered difference in space resulting in
\begin{equation*}
u_t\approx \frac{u(x,t+\Delta t)-u(x,t)}{\Delta t}=\frac{u(x_i,t_j+\Delta t)-u(x_i,t_j)}{\Delta t}
\end{equation*}
and

\begin{equation*}
u_{xx}\approx \frac{u(x+\Delta x,t)-2u(x,t)+u(x-\Delta x,t)}{\Delta x^2},
\end{equation*}
or

\begin{equation*}
u_{xx}\approx \frac{u(x_i+\Delta x,t_j)-2u(x_i,t_j)+u(x_i-\Delta x,t_j)}{\Delta x^2}.
\end{equation*}
We will also discuss the implicit Backward Euler with
\begin{equation*}
u_t\approx \frac{u(x,t)-u(x,t-\Delta t)}{\Delta t}=\frac{u(x_i,t_j)-u(x_i,t_j-\Delta t)}{\Delta t}
\end{equation*}
and

\begin{equation*}
u_{xx}\approx \frac{u(x+\Delta x,t)-2u(x,t)+u(x-\Delta x,t)}{\Delta x^2},
\end{equation*}
or

\begin{equation*}
u_{xx}\approx \frac{u(x_i+\Delta x,t_j)-2u(x_i,t_j)+u(x_i-\Delta x,t_j)}{\Delta x^2},
\end{equation*}

Finally we use the implicit Crank-Nicolson scheme with  a time-centered scheme at $(x,t+\Delta t/2)$
\begin{equation*}
u_t\approx \frac{u(x,t+\Delta t)-u(x,t)}{\Delta t}=\frac{u(x_i,t_j+\Delta t)-u(x_i,t_j)}{\Delta t}.
\end{equation*}
The corresponding spatial second-order derivative reads

\begin{align*}
u_{xx}\approx &\frac{1}{2}\left(\frac{u(x_i+\Delta x,t_j)-2u(x_i,t_j)+u(x_i-\Delta x,t_j)}{\Delta x^2}\right. +\\
&\left. \frac{u(x_i+\Delta x,t_j+\Delta t)-2u(x_i,t_j+\Delta t)+u(x_i-\Delta x,t_j+\Delta t)}{\Delta x^2}\right).
\end{align*}
Note well that we are using a time-centered scheme wih $t+\Delta t/2$ as center.

\paragraph{Project 5a): Setting up the algorithms.}
Write down the algorithms for these three methods and the equations you need to implement.
For the implicit schemes show that the equations lead to a tridiagonal matrix system for the new values.

\paragraph{Project 5b): Truncation errors and analytic solutions.}
Find the truncation errors of these three schemes and investigate their stability properties.
Find also the analytic solution to the continuous problem.

\paragraph{Project 5c): Implementation.}
Implement the three algorithms in the same code and perform tests of the solution
for these three approaches
for $\Delta x=1/10$, $\Delta x=1/100$ using  $\Delta t$ as dictated by the stability limit of the explicit scheme.
Study the solutions at two time points $t_1$ and $t_2$ where $u(x,t_1)$ is smooth but still significantly curved
and $u(x,t_2)$ is almost linear, close to the stationary state.
Remember that forsolving the tridiagonal equations you can use your code from project 1.
\paragraph{Project 5d):}
Compare the solutions at $t_1$ and $t_2$ with the analytic result for the continuous problem.
Which of the schemes would you classify as the best?

\paragraph{Project 5e): Moving to two dimensions.}
Extend the code you have developed here to two
  dimensions.
It means that we deal with a $2+1$ dimensional problem. Our differential equation becomes
\[
 \frac{\partial^2 u(x,y,t)}{\partial x^2}+\frac{\partial^2 u(x,y,t)}{\partial y^2} =\frac{\partial u(x,y,t)}{\partial t}, t> 0, x,y\in [0,1],
\]
where we now have made a model with a square lattice for $x$ and $y$.
How would you extend the boundary conditions from one dimension to two dimensions? And can you
  find a closed form solution here as well?  It is left to you to decide upon what kind of boundary conditions
you deem appropriate.
\paragraph{Project 5f): Solving the two-dimensional equations numerically.}
Here you can choose between an explicit or an implicit scheme.
For the implicit scheme discussed in chapter 10 of the lecture notes, upi need to use for example an iterative method like Jacobi's.

Outline the algorithm for solving the two-dimensional diffusion equation and
implement the explicit or the implicit scheme as function of $\Delta x$ (assuming
$\Delta x = \Delta y$) and $\Delta t$. Solve the equations numerically and give a critical discussion of your results.
Compare your results with the closed-form answer. Discuss the stability
of the solution as function of different values of $\Delta x$  and $\Delta t$.

You should, if you have time, try to  parallelize the equations using MPI or OpenMP.

\paragraph{Project 5g) Temperature distribution in the lithosphere (this part is optional and gives an additional score of 30 points).}
The Scientific background is  based on geological evidence at the surface. Geologists have proposed
that there was an active subduction zone on the western coast of
Norway about 1Gy ago. When subducting, the oceanic lithosphere
releases water and other chemical components that go mostly to the
surface but may also be trapped in the mantle above the subducting
slab. This process is called the refertilization of the mantle wedge
(see Figure). These chemical components contain more radioactive
elements than the normal mantle. We can therefore expect that ancient
mantle wedges are enriched in radioactive elements and therefore
warmer than normal mantle. The question is how much warmer they are
and if we can find evidence for this enrichment in geophysical
studies.

\begin{figure}[!ht]  %
  \centerline{\includegraphics[width=0.6\linewidth]{figslides/slab.jpg}}
  \caption{
  Cartoon of the enrichment of the mantle by a subduction slab.
  }
\end{figure}
%\clearpage % flush figures

The purpose of this part is to
calculate the thermal evolution of the lithosphere up to present following the emplacement of radioactive elements in the mantle wedge 1 Gy ago.

The equation to compute is the time evolution of the temperature distribution is the heat equation, which will be used only in 2D here, namely

\begin{equation}
\vec \nabla(k \vec \nabla T) + Q = \rho c_p \frac{\partial T}{\partial t}
\label{eq:heat2D}
\end{equation}
where $T$ is the temperature, $\rho$ is the density, $k$ is the thermal conductivity, $c_p$ is the specific heat capacity and $Q$ is the heat production.

With the boundary conditions and initial conditions, you should rescale your equations in order to use the results from parts 5a-5f). The paramaters of the model are described here.

\paragraph{Parameters of the model.}
The boundary conditions are $8^{\circ}\,C$ at the surface and $1300^{\circ}C$ at the bottom at a depth of 120 km.

In this study, we assume a constant density for the lithosphere of $3.5 10^3\,Kg/m^3$,
a constant thermal conductivity of $2.5\,W/m/^{\circ}C$
and a constant specific heat capacity of $1000\,J/Kg/^{\circ}C^{-1}$.

The heat production, caused by radioactive decay, cannot be taken as uniform in the whole lithosphere as radioactive elements are much more abundant in the crust than in the mantle.
We need therefore to separate the lithosphere in 3 units: the upper crust from 0 to 20 km depth, the lower crust from 20 to 40 km depth and the mantle from 40 to 120 km depth, where the heat production is set respectively to $1.4\,\mu W/m^3$, $0.35\,\mu W/m^3$ and $0.05\,\mu W/m^3$.

\paragraph{Temperature distribution before radioactive enrichment.}
Before radioactive enrichment, the temperature is steady-state and depends only on depth. In order to test your results you should
try to obtain analytical results for  the temperature profile as function of depth neglecting the presence of radioactice sources and. Compare thereafter these results with your numerical results.  Include then the radioactive sources and compare eventual analytical results with numerical calculations.

\paragraph{Temperature distribution after radioactive enrichment.}
We assume that the lithosphere above the slab gets enriched in
radioactive elements U, Th and K 1 Gy ago and that this results in an
additional heat production of $0.5\mu W/m^3$ over the whole depth of
the mantle (not the crust) in a 150 km wide area above the
slab. Neglecting all other thermal perturbations that this slab could
produce, and assuming that the heat production will remain constant
over geological time, compute the thermal evolution of the lithosphere
from 1 Gy to present.

The radioactivity will decrease with time. Assuming that the
additional heat is produced at 40\% by U, 40\% by Th and 20\% by K,
which have halflives of 4.47 Gy, 14.0 Gy and 1.25 Gy respectively,
compute the thermal evolution of the lithosphere and compare with the
result obtained above.

\paragraph{References.}
A very good reference is the textbook by \href{{http://www.springer.com/us/book/9783540225515}}{Winther and Tveito on partial differential equations}. It is available online \href{{https://vpn2.uio.no/+CSCO+0h756767633A2F2F797661782E66636576617472652E70627A++/book/10.1007/b138016/page/1}}{from the University library}.

\subsection{Introduction to numerical projects}

Here follows a brief recipe and recommendation on how to write a report for each
project.

\begin{itemize}
  \item Give a short description of the nature of the problem and the eventual  numerical methods you have used.

  \item Describe the algorithm you have used and/or developed. Here you may find it convenient to use pseudocoding. In many cases you can describe the algorithm in the program itself.

  \item Include the source code of your program. Comment your program properly.

  \item If possible, try to find analytic solutions, or known limits in order to test your program when developing the code.

  \item Include your results either in figure form or in a table. Remember to        label your results. All tables and figures should have relevant captions        and labels on the axes.

  \item Try to evaluate the reliabilty and numerical stability/precision of your results. If possible, include a qualitative and/or quantitative discussion of the numerical stability, eventual loss of precision etc.

  \item Try to give an interpretation of you results in your answers to  the problems.

  \item Critique: if possible include your comments and reflections about the  exercise, whether you felt you learnt something, ideas for improvements and  other thoughts you've made when solving the exercise. We wish to keep this course at the interactive level and your comments can help us improve it.

  \item Try to establish a practice where you log your work at the  computerlab. You may find such a logbook very handy at later stages in your work, especially when you don't properly remember  what a previous test version  of your program did. Here you could also record  the time spent on solving the exercise, various algorithms you may have tested or other topics which you feel worthy of mentioning.
\end{itemize}

\noindent
\subsection{Format for electronic delivery of report and programs}

The preferred format for the report is a PDF file. You can also use DOC or postscript formats or as an ipython notebook file.  As programming language we prefer that you choose between C/C++, Fortran2008 or Python. The following prescription should be followed when preparing the report:

\begin{itemize}
  \item Use Devilry to hand in your projects, log in  at  \href{{http://devilry.ifi.uio.no}}{\nolinkurl{http://devilry.ifi.uio.no}} with your normal UiO username and password and choose either 'fys3150' or 'fys4150'. There you can load up the files within the deadline.

  \item Upload \textbf{only} the report file!  For the source code file(s) you have developed please provide us with your link to your github domain.  The report file should include all of your discussions and a list of the codes you have developed.  Do not include library files which are available at the course homepage, unless you have made specific changes to them.

  \item In your git repository, please include a folder which contains selected results. These can be in the form of output from your code for a selected set of runs and input parametxers.

  \item In this and all later projects, you should include tests (for example unit tests) of your code(s).

  \item Comments  from us on your projects, approval or not, corrections to be made  etc can be found under your Devilry domain and are only visible to you and the teachers of the course.
\end{itemize}

\noindent
Finally,
we encourage you to work two and two together. Optimal working groups consist of
2-3 students. For this specific report you can  hand in a common report.

% ------------------- end of main content ---------------

% #ifdef PREAMBLE
\end{document}
% #endif