% Document Type: LaTeX
% Master File: ps1-intro.tex
% this has been revised by GJS (14-jan) and LDB (16-jan, 27-jan)

\input ../6001mac

 \def\fbox#1{%
  \vtop{\vbox{\hrule%
     \hbox{\vrule\kern3pt%
 \vtop{\vbox{\kern3pt#1}\kern3pt}%
 \kern3pt\vrule}}%
 \hrule}}

\begin{document}

\psetheader{Spring Semester, 1992-93}{Problem Set 2}

\medskip

\begin{flushleft}
Issued:  February 9, 1992 \\
\smallskip
Tutorial preparation for: Week of February 15 \\
\smallskip

Written solutions are due in Recitation:
\begin{itemize}
\item Even Numbered Recitation Sections: Wednesday February 17;
\item Odd Numbered Recitation Sections: Friday, February 19.
\end{itemize}

\smallskip

Reading:
\begin{tightlist}

\item Text: Complete Chapter 1.

\end{tightlist}
\end{flushleft}

\section{1. Tutorial Exercises}

\paragraph{Exercise 2.1}
Do exercise 1.23 of the text. You should make use of the
{\tt sum} procedure defined on p. 47 of the notes.

\paragraph{Exercise 2.2}
Do exercise 1.24 of the text.

\paragraph{Exercise 2.3}
Determine the orders of growth for the {\tt my-sine}
and {\tt recurrent-sine} procedures in Problem Set 1.
Consider both the growth
of space and number of operations and their dependence on the
size of the argument and the value of the global variable {\tt epsilon},
using the number of
applications of the {\tt my-sine} and {\tt recurrent-sine}
procedures to measure the number of operations.

How would the orders of growth for the {\tt my-sine} procedure
be changed if it had been defined as

\beginlisp
(define (my-sine x)
  (if (small-enuf? x)
      x
      (* (my-sine (/ x 3))
         (- 3
            (* 4 (my-sine (/ x 3))
                 (my-sine (/ x 3)))))))
\endlisp

\paragraph{ Exercise 2.4 }
Procedure {\tt repeated} is defined as

\beginlisp
(define (repeated p n)
  (cond ((= n 0) (lambda (x) x))
        ((= n 1) p)
        (else (lambda (x) (p ((repeated p (-1+ n)) x))))))
\endlisp

Determine the values of the expressions

\beginlisp
((repeated square 2) 5)
\endlisp

and

\beginlisp
((repeated square 5) 2).
\endlisp

In tutorial you may be asked how the interpreter evaluates expressions
similar to these.

\section{2. Continued Fractions}

Continued fractions play an important role in number theory and in
approximation theory.  In this programming
assignment, you will write some short procedures that evaluate
continued fractions.  There is no predefined code for you to load.

An infinite continued fraction is an expression of the form

\begin{displaymath}
f={{N_1} \over
  {\displaystyle D_1 + {{\strut N_2} \over
                {\displaystyle D_2 + \ddots }}}}
\end{displaymath}

One way to approximate an irrational number is to expand as a
continued fraction, and truncate the expansion after a sufficient
number of terms.  Such a truncation---a so-called {\em $k$-term finite
continued fraction}---has the form
\begin{displaymath}
{{N_1} \over {\displaystyle D_1+
        {\strut {N_2} \over {\displaystyle \ddots +
        {\strut {N_K} \over {\displaystyle D_K+0}}}}}}
\end{displaymath}

For example, if the $N_i$ and the $D_i$ are all 1, it is not
hard to show that the infinite continued fraction expansion
\begin{displaymath}
{1 \over {\displaystyle 1 +
        {\strut 1 \over {\displaystyle 1 +\ddots } }}}
\end{displaymath}
converges to $1/\phi\approx .618$ where $\phi$ is the {\em golden ratio}
\begin{displaymath}
\frac{1+ \sqrt{5}}{2}
\end{displaymath}
The first few finite continued fraction approximations (also called
{\em convergents})  are:
\begin{displaymath}
1, \frac{1}{2}=.5, \frac{2}{3}\approx .667, \frac{3}{5}=.6,
\frac{5}{8}=.625, \cdots
\end{displaymath}

\paragraph{Pre-Lab Exercise 2.1}
Suppose that {\tt n} and {\tt d} are procedures
of one argument (the term index) that return the
$n_i$ and $d_i$ of the terms of the continued fraction.

Define procedures {\tt cont-frac-r} and {\tt
cont-frac-i} such that evaluating {\tt (cont-frac-r n d k)} and {\tt
(cont-frac-i n d k)} each compute the value of the $k$-term finite
continued fraction.  The process that results from applying {\tt
cont-frac-r} should be recursive, and the process that results from
applying {\tt cont-frac-i} should be iterative.  Check your procedures
by approximating $1/\phi$ using

\beginlisp
(cont-frac (lambda (i) 1)
           (lambda (i) 1)
           k)
\endlisp

for successive values of {\tt k}, where {\tt cont-frac} is {\tt
cont-frac-r} or {\tt cont-frac-i}.  How large must you make {\tt k}
in order to get an approximation that is accurate to 4 decimal places?
Turn in a listing of your procedures.

\paragraph{Lab Exercise 2.2}
One of the first formulas for $\pi$ was given around 1658 by the
English mathematician Lord Brouncker:

\begin{displaymath}
4/\pi -1  =  1/(2 + 9/(2 + 25/ (2 + 49/ (2 + 81 /(2 + \cdots
\end{displaymath}

This leads to the following procedure for approximating $\pi$:

\beginlisp
(define (estimate-pi k)
  (/ 4 (+ (brouncker k) 1)))
\endlisp

\beginlisp
(define (square x) (* x x))
\null
(define (brouncker k)
  (cont-frac (lambda (i) (square (- (* 2 i) 1)))
             (lambda (i) 2)
             k))
\endlisp

where {\tt cont-frac} is one of your procedures {\tt cont-frac-r} or
{\tt cont-frac-i}.  Use this method to generate approximations to $\pi$.
About how large must you take $k$ in order to get 2 decimal places
accuracy?  (Don't try for more places unless you are very patient.)

\subsection{Computing inverse tangents}

Continued fractions are frequently encountered when approximating
functions.  In this case, the $N_i$ and $D_i$ can themselves be
constants or functions of one or more variables.  For example, a
continued fraction representation of the inverse tangent function
was published in 1770 by the German mathematician J.H. Lambert:

\begin{displaymath}
{\tan^{-1}x} ={x \over
{\displaystyle 1+{\strut {\left( {1x} \right)^2} \over
{\displaystyle 3+{\strut {\left( {2x} \right)^2} \over
{\displaystyle 5+{\strut {\left( {3x} \right)^2} \over {7+\ddots }}}}}}}}
\end{displaymath}

\paragraph{Pre-Lab Exercise 2.3}
Define a procedure {\tt (atan-cf k x)} that computes an approximation
to the inverse tangent function based on a $k$-term continued fraction
representation of $\tan^{-1} x$ given above. Provided you use the
correct arguments, it should be possible to
define this procedure directly in terms of the {\tt cont-frac}
procedure you have already defined.

\paragraph{Lab Exercise 2.4}
Verify that your
procedure works (and check its accuracy) by using it to compute the
tangent of a number of arguments, such as $0$,
$1$, $3$, $10$, $30$, $100$.
Compare your results with those computed using
Scheme's {\tt atan} procedure\footnote{Evaluating {\tt (atan x)}
returns the same value as {\tt (atan x 1)}, an angle in the range
$(-\pi/2, +\pi/2)$.}.

How many terms are required to get
reasonable accuracy?  Turn in your procedure definition
together with some sample results.

The idea of a continued fraction can be generalized to include
arbitrary nested expressions such as (1) and (2) below:

\begin{equation}
%MathType!ZZhx46!633xZh9VgeZpaeVeH9Kppzqiq9taFeGa2-Y8WaXhsyprZBeecuFmWh
%bif9Y+se2t+8KbHa1pc8racy-lZddeFiH9enVrqiq9YaFeGu0lRu03
%Nu0l7lryp5ZtgeVeH92cFeGa2-Y8WaXhsyprZBeeFiH9RbFeGu0lRa
%1paaaaaaa8!2AB6!
\left( {N_1/\left( {D_1+N_2/\left( {D_2+\cdots +N_k/\left( {D_k+0}
\right)} \right)} \right)} \right)
\end{equation}
\begin{equation}
%MathType!ZZhx46!633xZh9VgeZpaeVeH9Kppzqiq9taFeGu0VYhsyprZBeecuFmWhbif9
%ln-aq8se2t+8KbHa1pc8rasr-kFiH9enVrqiq9YaFeGu0VKu03Nu0V
%08daXlryp5ZtgeVeH92cFeGu0VYhsyprZBeeFiH9RbFeGu0VKa1pba
%aaaap!2B00!
\left( {N_1+D_1\times \left( {N_2+D_2\times \cdots \times \left(
{N_k+D_k\times 1} \right)} \right)} \right)
\end{equation}
or in general
\begin{displaymath}
\left( T_1 {\cal O}_1 \left( T_2 {\cal O}_2 \cdots
            \left( T_k {\cal O}_k R \right) \right) \right)
\end{displaymath}
where the $T_i$ are the terms of the expression,
the ${\cal O}_i$ are the arithmetic operators, and $R$ is the
residual term (perhaps representing the effects of
terms beyond the $k$th in an approximation). For example,
in (2) the ${\cal O}_i$ are alternately the addition and the
multiplication operators, and $R=1$.

\paragraph{Pre-Lab Exercise 2.5}
Define a procedure {\tt nested-acc} such that evaluating {\tt
(nested-acc op r term k)} computes the value of a nested expression.
Argument {\tt op} is a procedure of one argument, that when applied to
the term index $i$ returns the $i$th arithmetic operation ${\cal O}_i$
(which is a procedure of two arguments).  The {\tt r} argument
represents the effect of the $R$ term.  Turn in a listing of your
procedure.

How would you use your {\tt nested-acc} procedure to
compute a $k$-term approximation to the function
\begin{displaymath}
%MathType!ZZhx46!633xZh9VgeZtbecu-esrVSZtbecu-esrVSZtbecu-esrVSsrFFWhba
%Fea8raaa!0C5C!
f(x)=\sqrt {x+\sqrt {x+\sqrt {x+\cdots }}}
\end{displaymath}

\paragraph{Lab Exercise 2.6}

Verify that your {\tt nested-acc} procedure works properly
by computing an approximation to $f(1)$, whose value is
the golden ratio. How large must you make {\tt k}
in order to get an approximation that is accurate to 4 decimal places?
Turn in a listing of your procedures.

\paragraph{Lab Exercise 2.7}
Verify that your {\tt nested-acc} procedure works
properly by using it to compute the value of the Taylor series for the
sine function discussed in Problem Set 1.

In certain continued fractions all $N_i$ and $D_i$ terms are equal
and it is convenient to regard the fraction as
being ``built-up'' from a base by a repetitive operation. For
example, given the base function $B$ and the augmentors
$N$ and $D$, one can build $N/(D+B)$. This can be computed in a
straightforward fashion by the procedure {\tt build}

\beginlisp
(define (build n d b)
  (/ n (+ d b)))
\endlisp

The value of the expression that is represented by the two-term
continued fraction

\begin{displaymath}
%MathType!ZZhx46!633xZh9VgeZtdeVeH9KdXlryprsrVSZtdeVeH9KdXlryprsrVSVeH9
%Kaaaa8!1036!
{N \over {D+{N \over {D+B}}}}
\end{displaymath}

can then be computed by evaluating {\tt (build n d (build n d b))}.
When further composition is of interest, it is possible to use the
{\tt repeated} procedure defined above.

\paragraph{ Pre-Lab Exercise 2.8 }
Write a procedure {\tt repeated-build} of four arguments {\tt k},
{\tt n}, {\tt d}, and {\tt b}, which returns a result
that is equivalent to applying the {\tt build} procedure $k$ times. In
particular {\tt (repeated-build 2 n d b)} should return the same result
as  {\tt  (build  n  d  (build  n  d  b))}.  Your  implementation  of  {\tt
repeated-build} should make use of the {\tt repeated} procedure given above
(in a non-trivial way).

\paragraph{ Lab Exercise 2.9 }
Verify that your {\tt repeated-build} procedure works properly by evaluating
the continued fraction for the reciprocal of the golden ratio:

\begin{displaymath}
{1 \over {\displaystyle 1 +
        {\strut 1 \over {\displaystyle 1 + \ddots } }}}
\end{displaymath}
converges to $1/\phi\approx .618$ where $\phi$ is the {\em golden ratio}

Now consider the problem of computing the rational functions
$r_1(x)=(1+0x)/(1+x)$, $r_2(x)=(1+x)/(2+x)$, $r_3(x)=(2+x)/(3+2x)$,
\begin{displaymath}
%MathType!ZZhx46!633xZh9VgeVeH9Nmpzq8se2Bl8raZddeFiH94bGu03Z8dbXhsyFyZB
%eeFiH9RbFeGu0l7lryVfZtgeVeH92kf9slq9taFeWhsypEq8He2h28
%gbXhsy-Asr-kcuFmWhbif9Y+se2BX8KbXlryVTWhb8He2JhasrFpZp
%eecuFmqiq9Xif9R8He2l38gbXhsy-AsrFlcuFmWhbm-aq8se2Fiaaa
%Wd!3C40!
r_k\left( x \right)={{a_k+a_{k-1}x} \over {a_{k+1}+a_kx}}={1 \over
{1+r_{k-1}\left( x \right)}}
\end{displaymath}
where $a_k$ is the $k$-th Fibonacci number ($a_0=0$).

\paragraph{ Lab Exercise 2.10 }
Making  use of your {\tt repeated-build} procedure, define a procedure
{\tt r} of one argument {\tt k} that evaluates to a procedure of
one argument {\tt x} that computes $r_k(x)$. For example,
evaluating {\tt ((r 2) 0)} should return $0.5$.

\subsection{Optional Problem}

One difficulty with using continued fractions to evaluate functions
is that there is generally no way to determine in advance how many
terms of an infinite continued fraction are required to achieve a
specified degree of accuracy. The result obtained by computing the
value of a continued fraction by working backwards from the $k$-th to
the first term cannot generally be used in the computation of the
$k+1$ term continued fraction.

Remarkably, there is a general technique for computing the value of a
continued fraction that overcomes this problem. This result is based
on the recurrence formulas
\begin{equation}
A_{-1}=1;B_{-1}=0;A_0=0;B_0=1
\end{equation}
\begin{equation}
%MathType!ZZhx46!633xZh9VgeVeH9umpzq8se2Bk8rasrFpFiH9enVrq8He2Rg8raVeH9
%umpzq8se2BQu0lTa1pb8rasr-kFiH9onVrq8He2Rg8raVeH9umpzq8
%se2BQu0lTa1pc8raaa!212C!
A_j=D_jA_{j-1}+N_jA_{j-2}
\end{equation}
\begin{equation}
%MathType!ZZhx46!633xZh9VgeVeH9Kmpzq8se2Bk8rasrFpFiH9enVrq8He2Rg8raVeH9
%Kmpzq8se2BQu0lTa1pb8rasr-kFiH9onVrq8He2Rg8raVeH9Kmpzq8
%se2BQu0lTa1pc8raaa!212F!
B_j=D_jB_{j-1}+N_jB_{j-2}.
\end{equation}
It is easy to show that the $k$-term continued fraction can be computed as
\begin{equation}
%MathType!ZZhx46!633xZh9VgeVeH9Mnpzq8se2Bl8rasrFpZpeeFiH9bnVrq8He2Vg8ra
%q8se2tY8KbXlryVTWhbaaa!1500!
f_k={{A_k} \over {B_k}}.
\end{equation}
By determining whether $f_j$ is close enough to $f_{j-1}$, it is easy
to determine whether to include more terms in the approximation.

Define a procedure that computes the value of a continued fraction
using this algorithm. The process that evolves when the procedure is
applied should be iterative. Test that your procedure works by
computing the tangent of various angles. Use the tracing feature of
SCHEME to determine how many terms are evaluated.

\vspace{10ex}

Turn in this sheet with your answers to the questions in the problem set.

\vspace{6ex}

How much time did you spend on this homework assignment? Report separately
time spent before going to lab and time spent in the 6.001 lab.

\vspace{6ex}

If you cooperated with other students working this problem set
please indicate their names in the space below. As you should know,
as long as the guidelines described in the
{\em 6.001 Policy on Cooperation}\ handout are followed,
such cooperation is allowed and encouraged.

\end{document}


 \documentclass [12pt]{article}

\usepackage {amsmath}
\usepackage {amsthm}
\usepackage {amssymb}
\usepackage {graphicx}
\usepackage {float}
\usepackage {multirow}
\usepackage {xcolor}
\usepackage {algorithmic}
\usepackage [ruled,vlined,commentsnumbered,titlenotnumbered]{algorithm2e} \usepackage {array}
\usepackage {booktabs}
\usepackage {url}
\usepackage {parskip}
\usepackage [margin=1in]{geometry}
\usepackage [T1]{fontenc}
\usepackage {cmbright}
\usepackage [many]{tcolorbox}
\usepackage [colorlinks = true,
            linkcolor = blue,
            urlcolor  = blue,
            citecolor = blue,
            anchorcolor = blue]{hyperref}
\usepackage {enumitem}
\usepackage {xparse}
\usepackage {verbatim}
\usepackage{listings}
\usepackage{xcolor}
\lstset { %
    language=C++,
    backgroundcolor=\color{black!5}, % set backgroundcolor
    basicstyle=\footnotesize,% basic font setting
}
\newtheorem{theorem}{Theorem}
\newtheorem{remark}{Remark}

\DeclareTColorBox {Solution}{}{breakable, title={Solution}} \DeclareTColorBox {Solution*}{}{breakable, title={Solution (provided)}} \DeclareTColorBox {Instruction}{}{boxrule=0pt, boxsep=0pt, left=0.5em, right=0.5em, top=0.5em, bottom=0.5em, arc=0pt, toprule=1pt, bottomrule=1pt} \DeclareDocumentCommand {\Expecting }{+m}{\textbf {[We are expecting:} #1\textbf {]}} \DeclareDocumentCommand {\Points }{m}{\textbf {(#1 pt.)}}

\begin {document}

\vspace {1em}
\begin {Instruction}
Adapted From Virginia Williams' lecture notes.
\end {Instruction}

{\LARGE \textbf {COMP 285 (NC A\&T, Spr `22)}\hfill \textbf {Lecture 10} }

\begin{centering}
\section*{QuickSort and Non-Comparison Sorts}
\end{centering}

Today we will complete our analysis of \texttt{Quicksort}, which is the last comparison-based sort we will cover in class.

Then, we will jump into learning more about non-comparison-based sorts, such as \texttt{CountingSort} and \texttt{RadixSort}.

\section{QuickSort Analysis}
\subsection{Worst-Case Analysis}
In this section we will derive a bound on the worst-case running time of \texttt{Quicksort}. If we
consider the worst random choice of pivot at each step, the running time will be $\Theta(n^2)$. This flavor of worst-case analysis (which gets an upper bound on the running time over all possible possible choices of pivots) is no different from the worst case analysis of the algorithm which picks an arbitrary pivot at every step. We are thus interested in what the running time of \texttt{Quicksort} is \textbf{on average over all possible choices of the pivots}. We should emphasize an important point: We still consider the running time for a \textbf{worst-case input}, and average only \textbf{over the random choices of the algorithm} (which is different from averaging over all possible inputs). Put differently, our analysis will guarantee that \textbf{for any input, the expected running time will be small}. We formalize the idea of averaging over the random choices of the algorithm by considering the running time of the algorithm on an input I as a random variable and bounding the expectation of that random variable.

\textbf{Proposition}. \textit{For every input array of size $n$, the expected running time of \texttt{QuickSort} is$O(n \log n)$}.

Recall that a random variable is a function that maps every element in the sample space to a real number. In the case of rolling a die, the real number (or value of the point in the sample space) would be the number on the top of the die. Here the sample space is the set of all possible choices of pivots, and an example for a random variable can be the running time of \texttt{Quicksort} on a specific input $I$.

Denote by $z_i$ the $i$-th element in the sorted array. For each $i, j$, we define a random variable $X_{i ,j} (\sigma)$ to be the number of times $z_i$ and $z_j$ are compared for a given series of pivot choices $\sigma$. What are the possible values for $X_{i ,j} (\sigma)$? It can be $0$ if $z_i$ and $z_j$ are not compared. Note that all comparisons are with the pivot, and that the pivot is not included in the elements of the arrays in the recursive calls. If $z_i$ and $z_j$ are compared, consider the first time that this happens: one of them must be the pivot at this stage and this pivot is excluded from the subarrays that recursive calls operate on. Thus, \textit{no two elements are compared twice}. Therefore, $X_{i ,j} (\sigma) \in \{0, 1\}$.

Our goal is to compute the expected number of comparisons that \texttt{Quicksort} makes. Recall the definition of expectation:

$$
\mathbb{E}[X] = \sum_{\sigma} \mathbb{P}[\sigma]X(\sigma) = \sum_{k} k\mathbb{P}[X = k]
$$

Unfortunately for us, this definition does not really give us any clue about how we can actually go about computing the expectation of the complicated random variable that we have at hand, i.e. the number of comparisons made by \texttt{Quicksort}. Can you imagine trying to figure out the probability that \texttt{Quicksort} actually performs exactly $k$ comparisons? We don’t have the foggiest idea how we might do that. If you do, please let us know!

Luckily for us, there is way around this. An important property of expectation is \textbf{linearity of expectation}. For any random variables $X_1, \cdots, X_n$:
$$
\mathbb{E}\left[\sum_{i=1}^n X_i \right] = \sum_{i=1}^n \mathbb{E}[X_i]
$$

This is a really simple to state, but amazingly useful property that makes the computation of expectations of seemingly complicated random variables much much easier. It is worth internalizing this technique that we are about to apply for analyzing the expected number of comparisons that \texttt{Quicksort} performs.

We start with computing the expected value of $X_{i ,j}$. These variables are indicator random variables, which take the value $1$ if some event happens, and $0$ otherwise. The expected value is:
\begin{align*}
\mathbb{E}[X_{i,j}] &= \mathbb{P}[X_{i,j} = 1] \cdot 1 + \mathbb{P}[X_{i,j} = 0] \cdot 0 \tag{definition of expectation} \\
&= \mathbb{P}[X_{i,j} = 1]
\end{align*}

Let $C(\sigma)$ be the total number of comparisons made by \texttt{Quicksort} for a given set of pivot choices $\sigma$:

$$
C(\sigma) = \sum_{i=1}^n \sum_{j=i+1}^n X_{i,j}(\sigma)
$$

We wish to compute $\mathbb{E}[C]$ to get the expected number of comparisons made by \texttt{Quicksort}
for an input array of size $n$.

\begin{align*}
  \mathbb{E}[C] &= \mathbb{E}\left[\sum_{i=1}^n \sum_{j=i+1}^n X_{i,j}(\sigma) \right] \tag{definition of $C$} \\
  &= \sum_{i=1}^n \mathbb{E}\left[\sum_{j=i+1}^n X_{i,j}(\sigma) \right] \tag{linearity of expectation on first sum} \\
  &= \sum_{i=1}^n \sum_{j=i+1}^n  \mathbb{E}[X_{i,j}(\sigma)] \tag{linearity of expectation on second sum} \\
  &= \sum_{i=1}^n \sum_{j=i+1}^n  \mathbb{P}[z_i, z_j \text{ are compared} ]  \tag{definition of expectation on indicator variable}\\
\end{align*}
Now we find $P[z_i, z_j \text{ are compared}]$. Apriori, this seems difficult to do, but it turns out that there is a really elegant way to analyze this. Note that each element in the array(except the pivot) is compared to the pivot at each level of the recurrence. To analyze $P[z_i, z_j \text{ are compared}]$, examine the portion of the array $[z_i, \cdots, z_j]$. After the array is split using a pivot from $[z_i, \cdots, z_j]$, $z_i$ and $z_j$ can no longer be compared. Hence, $z_i$ and $z_j$ are compared only when from the portion of the array $[z_i, \cdots, z_j]$, either $z_i$ or $z_j$ is the first one picked as the pivot. So,

\begin{align*}
\mathbb{P}[z_i, z_j \text{ are compared}] &= \mathbb{P}[z_i \text{ or } z_j \text{ is the first pivot picked from } [z_i, \cdots, z_j]] \tag{argued above} \\
&= \frac{1}{j-i+1} + \frac{1}{j - i + 1} \tag{picking from a set uniformly} \\
&= \frac{2}{j-i+1}
\end{align*}

Make sure you are able to explain the second line in the calculation above.

We return to the expected value of $C$:

\begin{align*}
\mathbb{E}[C] &= \sum_{i=1}^n \sum_{j = i + 1}^n \mathbb{P}[z_i, z_j \text{ are compared }] \\
&= \sum_{i=1}^n \sum_{j = i + 1}^n \frac{2}{j-i + 1}
\end{align*}
Note that for a fixed $i$,
\begin{align*}
  \sum_{j=i+1}^n \frac{1}{j-i+1} &= \frac{1}{2} + \frac{1}{3} + \cdots + \frac{1}{n - i + 1} \\
  &\leq \frac{1}{2} + \frac{1}{3} + \cdots + \frac{1}{n}
\end{align*}

And using $\sum_{k=2}^n \frac{1}{k} \leq \ln n$, we get that

\begin{align*}
\mathbb{E}[C] &= \mathbb{E}\left[\sum_{i=1}^n \sum_{j = i + 1}^n X_{i,j}(\sigma) \right] \\
&= \sum_{i=1}^n \sum_{j = i + 1}^n \frac{2}{j-i + 1}\\
&\leq 2n \ln n
\end{align*}

 Thus, the expected number of comparisons made by \texttt{Quicksort} is no greater than $2n \ln n = O(n \log n)$. To complete the proof, we have to show that the running time is dominated by the number of comparisons. Note that in each recursive call to \texttt{Quicksort} on an array of size $k$, the algorithm performs $k - 1$ comparisons in order to split the array, and the amount of work done is $O(k)$. In addition, \texttt{Quicksort} will be called on single-element arrays at most once for each element in the original array, so the total running time of \texttt{Quicksort} $is O(C + n)$. In conclusion, the expected running time of \texttt{Quicksort} on worst-case input is $O(n log n)$.

 \subsection{Alternative Proof}

Here we provide an alternative method for bounding the expected number of comparisons. Let $T(n)$ be the expected number of comparisons performed by \texttt{Quicksort} on an input of size $n$. In general, if the pivot is chosen to be the $i$-th order statistic of the input array (i.e. the $i$th largest element),

$$
T(n) = n - 1 + T(i - 1) + T(n - i)
$$

where we define $T(0) = 0$. Each of the $n$ possible choices of $i$ are equally likely. Thus, the
expected number of comparisons is:

\begin{align*}
T(n) &= n - 1 + \frac{1}{n} \sum_{i=1}^n \left(T(i - 1) + T(n - i)\right) \\
&= n - 1 + \frac{2}{n} \sum_{i=1}^{n-1} T(i) \tag{Each $T(i)$ shows up twice in the sum}
\end{align*}

Did you catch this minor detail: why does the second summation go from $1$ through $n - 1$, while on the previous line, the summation goes from $1$ through $n$?

Continuing the calculation, we will use two facts:

\begin{enumerate}
  \item $\sum_{i=1}^{n-1} f(i) \leq \int_1^n f(x) dx$ for an increasing function $f$.
  (How would you prove this? This gives us a way to upper-bound discrete sums by integrals of continuous functions – a nice trick to have in your toolkit that we will use below. If you understand this, what is the corresponding statement to lower-bound a discrete sum by an integral? What if $f$ was a decreasing function of $n$? It is good to get into the habit of asking yourself such questions to test your understanding.)
  \item $\int 2x \ln x dx = x^2 \ln x - \frac{x^2}{2} + C$
\end{enumerate}

Now we show that $T(n) \leq 2n \ln n$ by (strong) induction.

\textbf{Inductive Hypothesis.} $T(i) \leq 2i \ln i$.

\textbf{Base case (i = 1)} An array of size 1 requires no comparisons. Thus, $T(1) = 0$ and the inductive hypothis is true for $i = 1$.

\textbf{Inductive step}. We will show that if the inductive hypothesis is true for all $i \leq k - 1$ then
the inductive hypothesis is also true for $i = k$.

Let's bound $T(k)$

\begin{align*}
T(k) &= k - 1 + \frac{2}{k}\sum_{i=1}^{k-1}T(i) \tag{by definition} \\
&\leq k - 1 + \frac{2}{k}\sum_{i=1}^{k-1}2i \ln i \tag{using our strong inductive hypothesis} \\
&\leq k - 1 + \frac{2}{k}\int_{1}^k (2x \ln x) dx \tag{Using fact 1 above} \\
&= k - 1 + \frac{2}{k}\left[k^2 \ln k - \frac{k^2}{2} + \frac{1}{2} \right] \tag{Using fact 2 above} \\
&= 2k \ln k + k - 1 - k + \frac{1}{k} \tag{simplify} \\
&= 2k \ln k - 1 + \frac{1}{k} \\
&\leq 2k \ln k
\end{align*}

Thus the inductive hypothesis is true for $i = k$. This establishes the inductive step.

\textbf{Conclusion} By induction, we have proved that $T(i) \leq 2i \ln i$ for all $i$. Hence $T(n) \leq
2n \ln n$. This concludes the proof.

\section{Lower Bounds for Comparison-Based Sorting Algorithms}

This is not covered in this course, however, its an interesting exercise to follow. If you'd like to learn more as to why the fastest possible comparison-based sorting algorithm must be $\Omega(n \log n)$ (eg, must take at least $n \log n$ steps in the worst-case), see these \href{notes}{http://www.cs.cmu.edu/~avrim/451f11/lectures/lect0913.pdf}.

\section{Counting Sort}
Recall that our sorting lower bounds applied to the class of algorithms that can only evaluate the values being sorted via \textit{comparison queries}, namely via asking whether a given element is greater than, less than, or equal to some other element. For such algorithms, it's been proven that any correct algorithm (even a randomized one!) will require $\Omega(n \log n)$ such queries on some input. As with any lower-bound that we prove in a restricted model, it is fruitful to ask ``is it possible to have an algorithm that does not fall in this class, and hence is not subject to the lowerbound?'' For sorting, the answer is ``yes''.

We start by looking at a very simple non-comparison-based sorting algorithm called Counting Sort. Since it is not comparison-based, it is not restricted by the $\Omega(n \log n)$ lower bound for sorting. For a given input of $n$ objects, each with a corresponding key (or value) in the range $\{0, 1, \cdots, r - 1\}$, Counting Sort will sort the objects by their keys:

\begin{enumerate}
  \item Create an array $A$ of $r$ buckets where each bucket contains a linked list.
  \item For each element in the input array with key $k$, concatenate the element to the end of the linked list $A[k]$.
  \item Concatenate all the linked lists: $A[0], \cdots , A[r - 1]$.
\end{enumerate}

The algorithm correctly sorts the $n$ elements by their keys because the elements are placed into buckets by key where bucket $i$ (containing elements with key = $i$) will come before bucket $j$ (containing elements with key = $j$) in A for $i < j$. Therefore, when the algorithm concatenates the buckets, all elements with key = $i$ will come before elements with key = $j$. The worst case run time of Counting sort is $O(n+r)$ since it performs $O(1)$ passes over the $n$ input elements and $O(1)$ passes over the $r$ buckets of $A$. This is great if the size of the range $r$ is small, but we pay dearly in the running time (and space) if $r$ is large. The algorithm we discuss in the next section builds on Counting Sort and fixes this issue (that Counting Sort
behaves poorly if the range of values $r$ is very large).

An important property (which we will use in the next section) is that the algorithm described above is \textit{stable}: If two input elements $x, y$ have the same key, and $x$ appears before $y$ in the input array, $x$ will appear before $y$ in the output.

\end{document}


\documentclass[11pt, oneside]{article}

\usepackage{../../shared/preamble}
\addbibresource{../../shared/references.bib}

\usepackage{../sets/sets}
\usepackage{../topological-spaces/topological-spaces}
\usepackage{../real-numbers/real-numbers}
\usepackage{formal-specifications}

\title{The Role of Formalization and \Zed\ }
\author{Arthur Ryman, {\tt arthur.ryman@gmail.com}}
\date{\today}

\begin{document}

\maketitle

\begin{abstract}
The notion of {\it type} was introduced into set theory by Russel in an effort to establish a firm foundation for logic.
Types are now a standard feature of many computer programming languages.
\Zed\  is a formal language based on typed set theory for specifying computer programs.
Since \Zed\  is a formal language, a Z specification can itself be automatically checked for type errors.
Writing valid Z specifications requires that the author explicitly define all terms.
This encourages a style in which complex terms are gradually built up from simpler, previously defined terms.
This article proposes to use \Zed\  to formally specify mathematical structures, even if there is no intention of using
them in computer programs.
The hoped for benefits of doing so are that the exercise of formalization will lead the author to a better understanding of the subject
matter, resulting in a clearer, higher quality, result.
\end{abstract}

\section{Types}

Computer languages can be divided into two major groups, namely those that are strongly typed and those that are weakly typed.
Strongly-typed languages, such as Java, have complex type systems and require that all expressions and variable declarations
have well-defined types.
Weakly-typed languages, such as Javascript, have simpler type systems and allow greater flexibility in expressions and the use of variables.
Although no compiler can possibly tell you if your program is correct, a compiler for a strongly-typed language can at least detect type errors.
Correcting these errors at compile-time is often quicker than doing so at execution-time.

There are many similarities between mathematics and programming.
Both require the definition and use of named objects that are built up from simpler objects.
The concept of type applies to both.
Bertrand Russell discovered typed set theory in his efforts to solve some problems in the foundations of mathematics
long before computers were a reality.
Theoretical computer science now makes extensive use of types.
Mathematics can be used to specify the requirements for programs.
Specifications act as a bridge between mathematics and programs.
\Zed\  is a strongly-typed formal specification language.
However, \Zed\  can be used to formalize mathematics even if the goal is not to program a computer.

\section{Formalization}

\Zed\  can be used to precisely define mathematical objects and a Z specification can be type-checked by a program.
I have found that the exercise of expressing mathematics using \Zed\  is a great way make sure that concepts are well-defined.
This is especially helpful, although somewhat labour-intensive, when dealing with complex mathematical structures.
I propose to formalize all definitions using \Zed\  and to validate
the document using the \fuzz\ type checker.

I believe that the effort of precisely defining every needed concept pays off.
Consider how C\'{e}dric Villani \cite{villani-hbm} described the mathematical writing style of Alexander Grothendieck:
\begin{displayquote}
Let me quote a famous mathematical text by Alexander Grothendieck,
one of the most famous mathematicians of all times, who wrote long writings.
In one of them he talks about the metaphor \textit{de la noix}.
You know the parable, the metaphor, of the nut
to explain the difference between his style and
the style of his fellow mathematician Jean-Pierre Serre.
Both of them working in the same area of mathematics, but very different styles.
Grothendieck said,
\textquote{Imagine that we have a nut to open.
The style of Serre would be take a
hammer and, Bang!, smash the nut.
My style would be to take the nut and put
it in a sea of acid so that it would be
dissolved very slowly, the crust of the nut, without noticing anything.}
And yes, experts tell us that the style of Grothendieck is like everything is very
incremental from one step to another to
another by very tiny steps so that you
have the impression that nothing occurs
and we are really making no progress, and
at the end it's proven, there it is, the big theorem is is done.
\end{displayquote}
Similarly, Pierre Deligne  \cite{artin-ag1} said:
\begin{displayquote}
From Grothendieck I have also learned not to take glory in the difficulty of a proof:
difficulty means we have not understood. The ideal is to be able to paint a landscape in which the proof is obvious.
I admire how often he succeeded in reaching this ideal.
\end{displayquote}
The message is clear: a slow, methodical build-up of concepts is a good thing in mathematics.

\section{Given Sets and Signatures}

Although the integers are built-in to \Zed, the real numbers are not.
In principle, one could first build up the rational numbers from the integers,
and then the real numbers from the rational numbers, but that would take a lot of time and not really
accomplish much.
I'll assume that the real numbers are sufficiently well-understood and do not need a complete formalization.
Instead, I'll define real numbers as a given set and then declare the types and signatures of the usual constants and operations of real arithmetic.

\section{Names and Symbols}

Integers and reals are distinct types in the sense of typed set theory.
But distinct types are disjoint sets.
This implies that the $0$ element of the integers is not the same object as the $0$ element of the reals.
In fact, they aren't even comparable within a Z specification since doing so is a type error and the
specification would therefore not be valid.
The name \texttt{0} is built-in to \Zed\  as the name of the $0$ element of the integers.
A Z specification must therefore introduce a new name to refer to the $0$ element of the reals.
However, working mathematicians regard the integers as a subset of the reals and therefore have no need to deal with duplicate names.
As Henri Poincar\'{e} \cite{poincare-fm} said:
\begin{displayquote}
I think I have already said somewhere that mathematics is the art of giving the same name to different things.
It is enough that these things, though differing in matter, should be similar in form, to permit of their being, so to speak, run in the same mould.\end{displayquote}

How then can we make a Z specification valid while still honouring normal mathematical practice?
The way out of this difficulty is that if a mathematician would normally regard two distinct objects as the same
then we give then distinct \Zed\  names, but display them using the same typographic symbol.
To paraphrase Poincar\'{e}:
\begin{quote}
Formalizing mathematics is the art of giving different names, but the same symbol, to different things.
It is enough that these things, though differing in type, should be similar in typography,
to permit of their being, so to speak, run in the same mould.
\end{quote}

For example, Mike Spivey \cite{spivey-fm} describes three typographic symbols defined in the \fuzz\ package
that are each used to display two distinct objects:
\begin{displayquote}
A few symbols have two names, reflecting two different uses for the
symbol in Z:
\begin{itemize}
\item The symbol $\semi$ is called
\cmd{semi} when it is used as an operation on schemas, and
\cmd{comp} when it is used for composition of relations.
\item The symbol $\hide$ is called
\cmd{hide} as the hiding operator of the schema calculus, and
\cmd{setminus} as the set difference operator.
\item The symbol $\project$ is called
\cmd{project} as the schema projection operator, and
\cmd{filter} as the filtering operator on sequences.
\end{itemize}
Although the printed appearance of
each of these pairs of symbols is the same, the type checker
recognizes each member of the pair only in the appropriate
\hbox{context}.
\end{displayquote}
This approach keeps both the Z type checker and the mathematician reader happy.
For example, in my formalization of the reals:
\begin{itemize}
\item The symbol $0$ is called
\texttt{0} when it is used as an integer, and
\cmd{zeroR} as a real number.
\item The symbol $+$ is called
\texttt{+} as the addition operator on the integers, and
\cmd{addR} as the addition operator on the reals.
\end{itemize}
The mathematician reader will always be able to infer the actual type of an object from its context.

\section{Theorems, Examples, and Remarks}

In mathematical writing it is good practice to follow a definition with some examples.
When an example is given, there is a proof obligation associated with it,
namely that whatever the author is asserting about the example is in fact true.
Similarly, when the author makes a remark or states a theorem, there is also a proof obligation.
In each of these cases, the author makes a statement and asserts it to be true, but may not offer any proof.

\Zed\ allows the author to add constraints to a specification.
However, if the constraint is a logical consequence of the preceding definitions then it adds nothing to the specification.
Therefore, we can freely add examples, remarks, and theorems to a specification without changing its meaning.
These statements should be placed in a \cmd{zed} box that is enclosed in an appropriate \LaTeX\ theorem-like environment.

For example, the following statement is true and therefore does not change the meaning of
the specification but will be type-checked by \fuzz, .

\begin{example}

\begin{zed}
	1 + 1 = 2
\end{zed}

\end{example}

The benefit of placing theorem-like statements in \cmd{zed} boxes is that the \fuzz\ type checker will check their type-correctness.

\subsection{Generic Theorem-Like Statements}

It will often be the case that a theorem-like statement will involve generic constructs.
Although \Zed\ provides a mechanism for introducing formal generic parameters into the definition of
generic constants and schemas, it does not provide such a mechanism for constraints.
We can work around this limitation by declaring a set of global given sets which are understood to be used in theorem-like
statements where they are taken to represent arbitrary sets.
The article on topological spaces introduces the global given sets $\setX$, $\setY$, and $\setZ$ for this purpose.

For example, we can remark that every element $x$ of an arbitrary set $\setX$ is a member of that set.

\begin{remark}

\begin{zed}
	\forall x: \setX @ x \in \setX
\end{zed}

\end{remark}

\section{Simulating Standard Mathematical Notation}

The expression $(a,b)$ usually denotes an ordered pair in mathematical writing.
Sometimes an author asks us to interpret $(a,b)$ as an open interval of the real number line.
In this case we recognize that the symbols used in the expression $(a,b)$ do not have their usual meanings
and we interpret them accordingly.
The notation $(a,b)$ for denoting an open interval of the real number line is
arguably very compact, convenient, and easy to understand.
In contrast, the \Zed\  function application $\intervalR(a,b)$, defined in the article on real numbers,
may strike the reader as somewhat verbose and clumsy, albeit precise.

\subsection{Prefix, Infix, and Postfix Operators in \fuzz}

There is a way to achieve some of the compactness of standard mathematical notation in Z while
preserving its precision and type-correctness.
The approach is to take advantage of the \fuzz\ type checker's ability to define infix and postfix
operator symbols.
In the grammar used by \fuzz, postfix operators have higher precedence than prefix operators,
and prefix operators have higher precedence than infix operators.
Infix operators are assigned numerical priorities of 1 through 6 with higher priorities taking precedence
over lower ones.
We can use these relative precedences to define operators that allow us to construct expressions
that resemble usual mathematical writing.

\subsection{\zcmd{lowerBound}, \zcmd{upperBound}, and \zcmd{intersect}}

First, we use \LaTeX\ bold styling to provide a visual cue that helps the reader
distinguish the interval $\lowerBound a \intersect b \upperBound$ from the
usual ordered pair $(a,b)$.
Next, we interpret the symbols as prefix, infix, and postfix operators.
The symbol $\lowerBound$ is a prefix operator that maps $a$ to $\lowerBound a$,
the open interval bounded below by $a$.
Similarly, $\upperBound$ is a postfix operator that maps $b$ to $b \upperBound$,
the open interval bounded above by $b$.
Finally, we interpret the symbol $\intersect$ as an infix operator that forms the intersection of the two intervals.

\begin{axdef}
	\lowerBound: \R \fun \power \R \\
	\_ \upperBound: \R \fun \power \R \\
	\_ \intersect \_ : \power \R \cross \power \R \fun \power \R
\where
	\forall a: \R @ \lowerBound a = \{~ x: \R | a \ltR x ~\}
\also
	\forall b: \R @ b \upperBound = \{~ x: \R | x \ltR b ~\}
\also
	\forall a, b: \R @ \lowerBound a \intersect b \upperBound = \{~ x: \R | a \ltR x \ltR b ~\}
\end{axdef}

This design is a step in the right direction, but it doesn't prevent the writer from abusing the notation.
For example, the following paragraph looks odd but makes perfect sense to \fuzz.

\begin{zed}
	\forall a, b: \R @ \lowerBound a \intersect b \upperBound = b \upperBound \intersect \lowerBound a
\end{zed}

When we are asked to interpret $(a,b)$ as an interval, we are, in a sense, being asked to parse a sentence
in a new mathematical mini-language that defines intervals.
If we want to enforce the usual syntax for intervals,
we can introduce new types to represent fragments of the expression so that the operators can only be applied to
fragments in some prescribed order.

To illustrate this point, let's expand the example to include closed intervals $[a,b]$ as well as semi-closed intervals
$(a,b]$ and $[a,b)$.
Regard $(a$ and $[a$ as open and closed right half lines, and $b)$ and $b]$ as open and closed left half lines.
Only allow a right half line to be combined with a left half line.

\subsection{Right Half Lines: \zcmd{openLowerBound}, and \zcmd{closedLowerBound}}

Let $RightHalfLine$ denote the type of syntactic fragments that define open and closed right half-lines.

\begin{zed}
	RightHalfLine ::= \openLowerBound \ldata \R \rdata | \closedLowerBound \ldata \R \rdata
\end{zed}

\begin{example}

\begin{zed}
	\openLowerBound \zeroR \in RightHalfLine
\also
	\closedLowerBound \oneR \in RightHalfLine
\end{zed}

\end{example}

Let $rightSet(R)$ denote the set of real numbers that the right half-line syntactic fragment $R$ represents.

\begin{axdef}
	rightSet: RightHalfLine \fun \power \R
\where
	\forall a : \R @ rightSet(\openLowerBound a) = \{~ x: \R | a \ltR x ~\}
\also
	\forall a : \R @ rightSet(\closedLowerBound a) = \{~ x: \R | a \leR x ~\}
\end{axdef}

For example

\begin{zed}
	\zeroR \notin rightSet(\openLowerBound \zeroR)
\also
	\oneR \in rightSet(\closedLowerBound \oneR)
\end{zed}

\subsection{Left Half Lines: \zcmd{openUpperBound}, and \zcmd{closedUpperBound}}

Let $LeftHalfLine$ denote the type of syntactic fragments that define open and closed left half-lines.

\begin{zed}
	LeftHalfLine ::= (\_ \openUpperBound)  \ldata \R \rdata | (\_ \closedUpperBound)  \ldata \R \rdata
\end{zed}

For example

\begin{zed}
	\zeroR \openUpperBound \in LeftHalfLine
\also
	\oneR \closedUpperBound \in LeftHalfLine
\end{zed}

Let $leftSet(L)$ denote the set of real numbers that the left half-line syntactic fragment $L$ represents.

\begin{axdef}
	leftSet: LeftHalfLine \fun \power \R
\where
	\forall b : \R @ leftSet(b \openUpperBound) = \{~ x: \R | x \ltR b ~\}
\also
	\forall b : \R @ leftSet(b \closedUpperBound) = \{~ x: \R | x \leR b ~\}
\end{axdef}

For example

\begin{zed}
	\zeroR \notin leftSet(\zeroR \openUpperBound)
\also
	\oneR \in leftSet(\oneR \closedUpperBound)
\end{zed}

\subsection{\zcmd{intersectRightLeft}}

Let $R \intersectRightLeft L$ denote the set of real numbers in the intersection of the sets represented by
the half-line syntactic fragments $R$ and $L$.

\begin{axdef}
	\_ \intersectRightLeft \_ : RightHalfLine \cross LeftHalfLine \fun \power \R
\where
	\forall R: RightHalfLine; L: LeftHalfLine @ \\
	\t1	R \intersectRightLeft L = rightSet~R \cap leftSet~L
\end{axdef}

For example

\begin{zed}
	\zeroR \in \closedLowerBound \zeroR \intersectRightLeft \oneR \openUpperBound
\end{zed}

\subsection{Improvements}

After reading the above, my reaction is that it would be more readable to first define the relevant objects using meaningful names
and then define the symbols as abbreviations for them.

TODO: Fix up the above discussion.

TODO: Use the example environment for examples.

TODO: Improve the Summary. It should summarize the whole article, not just how to simulate mini-languages.

\subsection{Summary}

A lot of standard mathematical notation can be written directly using the built-in capabilities of \Zed\  and \fuzz.
However, mathematics contains many specialized notations including those for intervals, probability, and quantum mechanics (Dirac bra-ket notation).
One can view these specialized notations as mathematical mini-languages.
As demonstrated above for the case of intervals, it is possible to simulate these notations in Z by introducing new syntactic types along with infix and postfix operators that construct, combine, and reduce fragments of this syntax.
The general pattern is for the opening and closing symbols to correspond to prefix and postfix operators that
construct fragments, and for internal symbols to correspond to infix operators that combine and then finally
reduce fragments.
Reduction occurs when the final infix operator is applied and maps the completed mini-language sentence to a
some usual mathematical type.

\printbibliography

\end{document}


%%
%% Automatically generated file from DocOnce source
%% (https://github.com/hplgit/doconce/)
%%
%%
% #ifdef PTEX2TEX_EXPLANATION
%%
%% The file follows the ptex2tex extended LaTeX format, see
%% ptex2tex: http://code.google.com/p/ptex2tex/
%%
%% Run
%%      ptex2tex myfile
%% or
%%      doconce ptex2tex myfile
%%
%% to turn myfile.p.tex into an ordinary LaTeX file myfile.tex.
%% (The ptex2tex program: http://code.google.com/p/ptex2tex)
%% Many preprocess options can be added to ptex2tex or doconce ptex2tex
%%
%%      ptex2tex -DMINTED myfile
%%      doconce ptex2tex myfile envir=minted
%%
%% ptex2tex will typeset code environments according to a global or local
%% .ptex2tex.cfg configure file. doconce ptex2tex will typeset code
%% according to options on the command line (just type doconce ptex2tex to
%% see examples). If doconce ptex2tex has envir=minted, it enables the
%% minted style without needing -DMINTED.
% #endif

% #define PREAMBLE

% #ifdef PREAMBLE
%-------------------- begin preamble ----------------------

\documentclass[%
oneside,                 % oneside: electronic viewing, twoside: printing
final,                   % draft: marks overfull hboxes, figures with paths
10pt]{article}

\listfiles               %  print all files needed to compile this document

\usepackage{relsize,makeidx,color,setspace,amsmath,amsfonts,amssymb}
\usepackage[table]{xcolor}
\usepackage{bm,ltablex,microtype}

\usepackage[pdftex]{graphicx}

\usepackage{ptex2tex}
% #ifdef MINTED
\usepackage{minted}
\usemintedstyle{default}
% #endif

\usepackage[T1]{fontenc}
%\usepackage[latin1]{inputenc}
\usepackage{ucs}
\usepackage[utf8x]{inputenc}

\usepackage{lmodern}         % Latin Modern fonts derived from Computer Modern

% Hyperlinks in PDF:
\definecolor{linkcolor}{rgb}{0,0,0.4}
\usepackage{hyperref}
\hypersetup{
    breaklinks=true,
    colorlinks=true,
    linkcolor=linkcolor,
    urlcolor=linkcolor,
    citecolor=black,
    filecolor=black,
    %filecolor=blue,
    pdfmenubar=true,
    pdftoolbar=true,
    bookmarksdepth=3   % Uncomment (and tweak) for PDF bookmarks with more levels than the TOC
    }
%\hyperbaseurl{}   % hyperlinks are relative to this root

\setcounter{tocdepth}{2}  % levels in table of contents

% prevent orhpans and widows
\clubpenalty = 10000
\widowpenalty = 10000

% --- end of standard preamble for documents ---

% insert custom LaTeX commands...

\raggedbottom
\makeindex
\usepackage[totoc]{idxlayout}   % for index in the toc
\usepackage[nottoc]{tocbibind}  % for references/bibliography in the toc

%-------------------- end preamble ----------------------

\begin{document}

% matching end for #ifdef PREAMBLE
% #endif

\newcommand{\exercisesection}[1]{\subsection*{#1}}

% ------------------- main content ----------------------

% ----------------- title -------------------------

\thispagestyle{empty}

\begin{center}
{\LARGE\bf
\begin{spacing}{1.25}
Fourth and fifth days: Homework set 3
\end{spacing}
}
\end{center}

% ----------------- author(s) -------------------------

\begin{center}
{\bf Data Analysis and Machine Learning${}^{}$} \\ [0mm]
\end{center}

\begin{center}
% List of all institutions:
\end{center}

% ----------------- end author(s) -------------------------

% --- begin date ---
\begin{center}
Jul 3, 2020
\end{center}
% --- end date ---

\vspace{1cm}

\subsection{Day four and five exercises}

The exercises here are somewhat longer and we expect to use at least two days on them.

\paragraph{Exercise 1, Bias-Variance tradeoff and Bootstrap.}
This exercise is a continuation of exercise 2 from the second homework set.
In that exercise we computed the MSE-score for the training
data and the test data as functions of the complexity of a polynomial,
that is the degree of a given polynomial.

One of the aims of that exercise was
to reproduce Figure 2.11 of \href{{https://github.com/CompPhysics/MLErasmus/blob/master/doc/Textbooks/elementsstat.pdf}}{Hastie et al}.

Our data is defined by $x\in [-3,3]$ with a total of for example $100$ data points.
\bpycod
np.random.seed()
n = 100
maxdegree = 14
# Make data set.
x = np.linspace(-3, 3, n).reshape(-1, 1)
y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)
\epycod
where $y$ is the function we want to fit with a given polynomial.

\paragraph{Part (1a) Proving the bias-variance tradeoff.}
Consider a
dataset $\mathcal{L}$ consisting of the data
$\mathbf{X}_\mathcal{L}=\{(y_j, \boldsymbol{x}_j), j=0\ldots n-1\}$.

Let us assume that the true data is generated from a noisy model

\[
\bm{y}=f(\boldsymbol{x}) + \bm{\epsilon}.
\]

Here $\epsilon$ is normally distributed with mean zero and standard
deviation $\sigma^2$.

In our derivation of the ordinary least squares method we defined then
an approximation to the function $f$ in terms of the parameters
$\bm{\beta}$ and the design matrix $\bm{X}$ which embody our model,
that is $\bm{\tilde{y}}=\bm{X}\bm{\beta}$.

The parameters $\bm{\beta}$ are in turn found by optimizing the means
squared error via the so-called cost function

\[
C(\bm{X},\bm{\beta}) =\frac{1}{n}\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2=\mathbb{E}\left[(\bm{y}-\bm{\tilde{y}})^2\right].
\]

Show that you can rewrite  this as
\[
\mathbb{E}\left[(\bm{y}-\bm{\tilde{y}})^2\right]=\frac{1}{n}\sum_i(f_i-\mathbb{E}\left[\bm{\tilde{y}}\right])^2+\frac{1}{n}\sum_i(\tilde{y}_i-\mathbb{E}\left[\bm{\tilde{y}}\right])^2+\sigma^2.
\]

Explain what the terms mean, which one is the bias and which one is
the variance and discuss their interpretations.

\paragraph{Part (1b) Adding Bootstrap and Bias-Variance Tradeoff.}
Add now bootstrapping as discussed in the \href{{https://compphysics.github.io/MachineLearningECT/doc/pub/Day1/html/Day1.html}}{Regression lectures} (scroll down to the bias-variance code).
Add also the expressions for the bias and the variance as discussed above.

Discuss the bias and variance tradeoff as function
of your model complexity (the degree of the polynomial) and the number
of data points, and possibly also your training and test data.

Try to make a figure similar to Fig.~2.11 of Hastie et al.~You should include an analysis of the bias and variance for the test results. Figure 2.11 displays only the test and training MSEs while indicating regions of low/high bias and variance. You will most likely not get an
equally smooth curve! Note also that when you calculate the bias, in all applications you don't know the function values $f_i$. You would hence replace them with the actual data points $y_i$.

\paragraph{Exercise 2, Linear Regression for  a two-dimensional function.}
This is a longer  exercise and the aim is to study in more detail various
regression methods, including the Ordinary Least Squares (OLS) method,
Ridge regression and finally Lasso regression.
The methods are in turn combined with resampling techniques.

We will study how to fit polynomials to a specific
two-dimensional function called \href{{http://www.dtic.mil/dtic/tr/fulltext/u2/a081688.pdf}}{Franke's
function}.  This
is a function which has been widely used when testing various
interpolation and fitting algorithms. Furthermore, after having
established the model and the method, we will employ resamling
like the bootstrap from the previous exercise in order to perform a
proper assessment of our models. We will also study in detail the
so-called Bias-Variance trade off.

The Franke function, which is a weighted sum of four exponentials  reads as follows
\begin{align*}
f(x,y) &= \frac{3}{4}\exp{\left(-\frac{(9x-2)^2}{4} - \frac{(9y-2)^2}{4}\right)}+\frac{3}{4}\exp{\left(-\frac{(9x+1)^2}{49}- \frac{(9y+1)}{10}\right)} \\
&+\frac{1}{2}\exp{\left(-\frac{(9x-7)^2}{4} - \frac{(9y-3)^2}{4}\right)} -\frac{1}{5}\exp{\left(-(9x-4)^2 - (9y-7)^2\right) }.
\end{align*}

The function will be defined for $x,y\in [0,1]$.  Our first step will
be to perform an OLS regression analysis of this function, trying out
a polynomial fit with an $x$ and $y$ dependence of the form $[x, y,
x^2, y^2, xy, \dots]$. We will also include cross-validation (or bootstrap) as
resampling technique.  As in homeworks 1 and 2, we can use a uniform
distribution to set up the arrays of values for $x$ and $y$, or as in
the example below just a set of fixed
values for $x$ and $y$ with a given step
size.  We will fit a
function (for example a polynomial) of $x$ and $y$.  Thereafter we
will repeat much of the same procedure using the Ridge and Lasso
regression methods, introducing thus a dependence on the bias
(penalty) $\lambda$.

Finally we are going to use (real) digital terrain data and try to
reproduce these data using the same methods. We will also try to go
beyond the second-order polynomials metioned above and explore
which polynomial fits the data best.

The Python fucntion for the Franke function is included here (it performs also a three-dimensional plot of it)
\bpycod
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
from matplotlib import cm
from matplotlib.ticker import LinearLocator, FormatStrFormatter
import numpy as np
from random import random, seed

fig = plt.figure()
ax = fig.gca(projection='3d')

# Make data.
x = np.arange(0, 1, 0.05)
y = np.arange(0, 1, 0.05)
x, y = np.meshgrid(x,y)

def FrankeFunction(x,y):
    term1 = 0.75*np.exp(-(0.25*(9*x-2)**2) - 0.25*((9*y-2)**2))
    term2 = 0.75*np.exp(-((9*x+1)**2)/49.0 - 0.1*(9*y+1))
    term3 = 0.5*np.exp(-(9*x-7)**2/4.0 - 0.25*((9*y-3)**2))
    term4 = -0.2*np.exp(-(9*x-4)**2 - (9*y-7)**2)
    return term1 + term2 + term3 + term4

z = FrankeFunction(x, y)

# Plot the surface.
surf = ax.plot_surface(x, y, z, cmap=cm.coolwarm,
                       linewidth=0, antialiased=False)

# Customize the z axis.
ax.set_zlim(-0.10, 1.40)
ax.zaxis.set_major_locator(LinearLocator(10))
ax.zaxis.set_major_formatter(FormatStrFormatter('%.02f'))

# Add a color bar which maps values to colors.
fig.colorbar(surf, shrink=0.5, aspect=5)

plt.show()

\epycod

\paragraph{(2a) Ordinary Least Square on the Franke function  with resampling.}
We will generate our own dataset for a function
$\mathrm{FrankeFunction}(x,y)$ with $x,y \in [0,1]$. The function
$f(x,y)$ is the Franke function. You should explore also the addition
an added stochastic noise to this function using the normal
distribution $\cal{N}(0,1)$.

Write your own code (using either a matrix inversion or a singular
value decomposition from e.g., \textbf{numpy} ) or use your code from
homeworks 1 and 2 and perform a standard least square regression
analysis using polynomials in $x$ and $y$ up to fifth order. You can use \textbf{scikit-learn} as well.

Evaluate the Mean Squared error (MSE)

\[ MSE(\hat{y},\hat{\tilde{y}}) = \frac{1}{n}
\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2,
\]

and the $R^2$ score function.  If $\tilde{\hat{y}}_i$ is the predicted
value of the $i-th$ sample and $y_i$ is the corresponding true value,
then the score $R^2$ is defined as

\[
R^2(\hat{y}, \tilde{\hat{y}}) = 1 - \frac{\sum_{i=0}^{n - 1} (y_i - \tilde{y}_i)^2}{\sum_{i=0}^{n - 1} (y_i - \bar{y})^2},
\]

where we have defined the mean value  of $\hat{y}$ as

\[
\bar{y} =  \frac{1}{n} \sum_{i=0}^{n - 1} y_i.
\]

To set up the design matrix, the following code can be used
\bpycod
def FrankeFunction(x,y):
	term1 = 0.75*np.exp(-(0.25*(9*x-2)**2) - 0.25*((9*y-2)**2))
	term2 = 0.75*np.exp(-((9*x+1)**2)/49.0 - 0.1*(9*y+1))
	term3 = 0.5*np.exp(-(9*x-7)**2/4.0 - 0.25*((9*y-3)**2))
	term4 = -0.2*np.exp(-(9*x-4)**2 - (9*y-7)**2)
	return term1 + term2 + term3 + term4

def create_X(x, y, n ):
	if len(x.shape) > 1:
		x = np.ravel(x)
		y = np.ravel(y)

	N = len(x)
	l = int((n+1)*(n+2)/2)		# Number of elements in beta
	X = np.ones((N,l))

	for i in range(1,n+1):
		q = int((i)*(i+1)/2)
		for k in range(i+1):
			X[:,q+k] = (x**(i-k))*(y**k)

	return X

# Making meshgrid of datapoints and compute Franke's function
n = 5
N = 1000
x = np.sort(np.random.uniform(0, 1, N))
y = np.sort(np.random.uniform(0, 1, N))
z = FrankeFunction(x, y)
X = create_X(x, y, n=n)
\epycod

\paragraph{Part (2b) Resampling techniques, adding more complexity.}
Perform a resampling of the data where you split the data in training
data and test data. Here you can write your own function or use the
function for splitting training data provided by \textbf{Scikit-Learn}.
This function is called $train\_test\_split$.   You should also renormalize your data.

It is normal in essentially all Machine Learning studies to split the
data in a training set and a test set (sometimes also an additional
validation set).  There
is no explicit recipe for how much data should be included as training
data and say test data.  An accepted rule of thumb is to use
approximately $2/3$ to $4/5$ of the data as training data.

Use then the _bootstrap code you developed in the previous exercise to resample your data
and evaluate again the MSE function resulting
from the test data.

\paragraph{Part (2c): Bias-variance tradeoff.}
With a code which does OLS and includes bootstrap
we will now discuss the bias-variance tradeoff in the context of
continuous predictions such as regression. However, many of the
intuitions and ideas discussed here also carry over to classification
tasks and basically all Machine Learning algorithms.

Use the code from exercise 1 above and implement the bootstrap
resampling and perform a bias-variance tradeoff analysis like you did
in exercise 1.

\paragraph{Part (2d): Ridge Regression on the Franke function  with resampling.}
Write your own code for the Ridge method, either using matrix
inversion or the singular value decomposition ir use \textbf{scikit-learn}
Perform the same analysis as in the
previous three steps (for the same polynomials and include resampling
techniques) but now for different values of $\lambda$. Compare and
analyze your results with those obtained in parts 2a-2c). Study the
dependence on $\lambda$.

Study also the bias-variance tradeoff as function of various values of
the parameter $\lambda$. Comment your results.

\paragraph{Part (2e): Lasso Regression on the Franke function  with resampling.}
This part is essentially a repeat of the previous ones, but now
with Lasso regression. Write either your own code or
use the functionalities of \textbf{Scikit-Learn} (recommended).
Give a
critical discussion of the three methods and a judgement of which
model fits the data best.

% ------------------- end of main content ---------------

% #ifdef PREAMBLE
\end{document}
% #endif


\documentclass[12]{scrartcl}
\usepackage{amssymb,amsmath,gensymb,dsfont,calc,multicol,fullpage}
\makeatletter
\newcommand\Aboxed[1]{
   \@Aboxed#1\ENDDNE}
\def\@Aboxed#1&#2\ENDDNE{%
   &
   \settowidth\@tempdima{$\displaystyle#1{}$}
   \setlength\@tempdima{\@tempdima+\fboxsep+\fboxrule}
   \kern-\@tempdima
   \boxed{#1#2}
}
\makeatother

\begin{document}

\title{Homework 28, Section 4.5: 3, 9, 14, 20, 22, 25}
\author{Alex Gordon}
\date{\today}
\maketitle
\section*{Homework}
\subsection*{3.}
This subspace is $H = $Span $\{v_1, v_2, v_3\}, $ where $v_1 =  \begin{bmatrix} 0 \\ 1 \\ 0 \\ 1  \end{bmatrix}, v_2 =  \begin{bmatrix} 0 \\ -1 \\ 1 \\ 2  \end{bmatrix},$ and $v_3 =  \begin{bmatrix} 2 \\ 0 \\ -3 \\ 0  \end{bmatrix}$. Theorem 4 in 4.3 shows that the columns of this set are linearly independent. $v_1 \neq 0, v_2$ is not a multiple of $v_1$ and since its first entry is not zero, $v_3$ is not a linear combination of $v_1$ or $v_2$ Thus, this set is a basis for H and the dimension of the subspace is 3.
\subsection*{9.}
This subspace is $H = \begin{bmatrix} a \\ b \\ a  \end{bmatrix}: a, b $ in $R$ = Span $\{v_1, v_2 \}$ and where $v_1 = \begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix}$ and $v_2 = \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}$. Since $v_1$ and $v_2$ are not multiples of each other, they are linearly independent and are a basis for H. This means the dimension of H is 2.
\subsection*{14.}
The matrix is in echelon form. There are two pivot columns so the dimension of Col $A$ is 3. Since there are three columns without pivots, the equation $Ax = 0 $ has three free variables thus the dimension of Nul $A$ is 2.
\subsection*{20. A)}
False. the set $R^2$ is not even a subset of $R^3$.
\subsection*{20. B)}
False. The number of free variables is qual to the dimension of Null A.
\subsection*{20. C)}
False. A basis could still have only finitely many elements, which would make the vector space finite.
\subsection*{20. D)}
False. The set S must also have n elements.
\subsection*{20. E)}
True. The subspaces can only be subsets of themselves.

\subsection*{22.}
The matrix whose columns are the coordinate vectors of polynomials with the standard basis of $P_3$ is:
$\begin{bmatrix} 1 & 1 & 2 & 6 \\ 0 & -1 & -4 & -18 \\ 0 & 0 & 1 & 9 \\ 0 & 0 & 0 & -1  \end{bmatrix}$. This matrix has 4 pivots so it's columns are linearly independent. Since their coordinate vectors for a linearly independent set, the polynomials themselves are linearly independent in $p_3$. The Basis Theorem then states they form a basis for $P_3$.
\subsection*{25.}
Suppose that S Spans V and that S contains fewer than n vectors. This means that by the spanning set theorem, some subset of $S'$ of S is a basis for V. Since S contains fewer than n vectors, and $S'$  is a subset of S, $S'$ also contains fewer than n vectors. Thus there is a basis $S'$ for V with fewer than n vectors. However, this is inpossible because of theorem 10, because dimV = n. Thus S cannot span V.

\end{document}


\documentclass[oneside]{amsart}
%\usepackage{amsmath}
\usepackage{txfonts}
\usepackage{courier}
\usepackage{superdate}
\usepackage[margin=1.25in]{geometry}

\title{A Simple Graph type for Julia}
\author{Ed Scheinerman}
\thanks{Document version \texttt{\superdate}. Note: This documentation
might not quite be in sync with the functionality found in the
\texttt{SimpleGraphs} module. I will likely want to redo this at some
point.}
\begin{document}
\maketitle

\section{Fundamentals}

This is documentation for a \verb|SimpleGraph| data type for Julia. The
goal is to make working with graphs as painless as possible. The
\verb|SimpleGraph| data type is for simple graphs (undirected edges, no
loops, no multiple edges). Vertices in a \verb|SimpleGraph| are of a
given Julia data type, which might be \verb|Any|.

The key constructor is
\verb|SimpleGraph| which creates a graph whose vertices may be any Julia
type. Alternatively, \verb|G=SimpleGraph{T}()| sets up \verb|G| to be a
graph in which all vertices must be of type \verb|T|. Two special
cases are built into this module: \verb|IntGraph()| is a synonym for
\verb|SimpleGraph{Int}()| and \verb|StringGraph()| is a synonym for
\verb|SimpleGraph{String}|.

Vertices and edges are added to a graph with \verb|add!| and deleted
with \verb|delete!|. Membership is checked with \verb|has|.
\begin{verbatim}
julia> using SimpleGraphs

julia> G = IntGraph()
SimpleGraph{Int64} (0 vertices, 0 edges)

julia> add!(G,5)
true

julia> add!(G,1,2)
true

julia> G
SimpleGraph{Int64} (3 vertices, 1 edges)
\end{verbatim}
Notice that adding an edge automatically adds its end points to the
graph.

The number of vertices and edges can be queried with \verb|NV| and
\verb|NE|. The vertex and edge sets are returned as arrays by
\verb|vlist(G)| and \verb|elist(G)|.
\begin{verbatim}
julia> vlist(G)
3-element Array{Int64,1}:
 1
 2
 5

julia> elist(G)
1-element Array{(Int64,Int64),1}:
 (1,2)
\end{verbatim}

Use \verb|deg(G,v)| for the degree of a vertex and \verb|deg(G)| for
the graph's degree sequence.
\begin{verbatim}
julia> deg(G,1)
1

julia> deg(G)
3-element Array{Int64,1}:
 1
 1
 0
\end{verbatim}

The neighbors of a vertex can be sought with \verb|neighbors(G,v)| or,
alternatively, with \verb|G[v]|. Edges can be queried with
\verb|G[v,w]|.
\begin{verbatim}
julia> G[1]
1-element Array{Int64,1}:
 2

julia> G[1,2]
true

julia> G[1,5]
false
\end{verbatim}

The \verb|Complete| method can be used to create complete graphs
and complete bipartite graphs with \verb|Int| vertices.
\begin{verbatim}
julia> Complete(5)
SimpleGraph{Int64} (5 vertices, 10 edges)

julia> Complete(3,3)
SimpleGraph{Int64} (6 vertices, 9 edges)
\end{verbatim}
We also provide \verb|Cycle(n)| and \verb|Path(n)| to create
the graphs $C_n$ and $P_n$. An instance of a Erd\H{o}s-R\'enyi random
graph $G_{n,p}$ is returned by \verb|RandomGraph(n,p)|.

The adjacency, Laplacian, and vertex-edge incidence matrices can be
found with \verb|adjaceny|, \verb|Laplacian|, and \verb|incidence|. It
is important to note that the indexing of the rows/columns of these
matrices might not correspond to the natural order of the underlying
vertices (or edges).

The incidence matrix method takes an optional argument as to whether
the matrix is signed (a $1$ and a $-1$ in each column) or unsigned
(only positive $1$s). Also, it is a sparse matrix that can be
converted to dense storage with \verb|full|.

Finally, the \verb|vertex_type| function can be used to query the
data type of the vertices in the graph.
\begin{verbatim}
julia> G = StringGraph()
SimpleGraph{String} (0 vertices, 0 edges)

julia> vertex_type(G)
String (constructor with 1 method)
\end{verbatim}

\section{Look but don't touch}
\label{sect:fastN}

The \verb|SimpleGraph| objects have various internal fields. It is not
safe to change these directly, but there is no problem examining their
values.

\begin{itemize}
\item \verb|:V|

  This holds the vertex set of the graph. If \verb|G| is a
  \verb|SimpleGraph{T}| then \verb|G.V| is of type \verb|Set{T}|.

\item \verb|:E|

  This holds the edge set of the graph. If \verb|G| is a
  \verb|SimpleGraph{T}| then \verb|G.E| is a \verb|Set{(T,T)}|.

\item \verb|:Nflag|

  This boolean value indicates whether or not fast neighborhood lookup
  has been activated for this graph. See the discussion below and the
  description of the function \verb|fastN!|.

\item \verb|:N|

  This is a \verb|Dict| that keeps track of the neighborhood of each
  vertex. It is only active if the \verb|Nflag| is set to
  \verb|true|.

\item \verb|:cache|

  This is a dictionary that holds previously computed properties
  of the graph. See Section~\ref{sect:cache} for details and user
  functions for manipulating the cache.

\item \verb|cache_flag|

  Used to determine if caching should be enabled.

\end{itemize}

This design is redundant. One can test if two vertices are adjacent by
looking for the edge in \verb|E|.  One can determine the neighbors of
a given vertex by iterating over the edge set \verb|E|.  However, this
approach is slow. By providing the extra \verb|N| data structure,
these operations are very fast.

By calling \verb|fastN!(G,false)| the redudant neighborhood structure
\verb|N| is deleted. All functions will still work, but perhaps more
slowly. Calling \verb|fastN!(G,true)| rebuilds the \verb|N|
structure.

We recommend using the default structure unless the graph is so large
that it consumes too much memory.

\section{List of all functions}

\subsection*{Creators}

These are functions that create new graphs.

\begin{itemize}
\item \verb|SimpleGraph|

  Use \verb|G=SimpleGraph()| to create a graph whose vertices can be
  of \verb|Any| type. To create a graph with vertices of a particular
  type \verb|T| use \verb|G=SimpleGraph{T}()|.

\item \verb|IntGraph|

  This a synonym for \verb|SimpleGraph{Int}|. Use \verb|G=IntGraph()|
  to create a vertex whose vertices are integers (type \verb|Int|).

  Use \verb|IntGraph(n)| to create a \verb|SimpleGraph{Int}| graph
  with vertex set $\{1,2,\ldots,n\}$ and no edges.

\item \verb|StringGraph|

  This is a synonym for \verb|SimpleGraph{String}|. Use
  \verb|G=StringGraph()| to create a graph whose vertices are
  character strings.

  One can also call \verb|G=StringGraph(filename)| to read in a graph
  from a file. The file must have the following format:
  \begin{itemize}
  \item Each line should contain one or two tokens (words) that do not
    contain any whitespace.
  \item If a line contains one token, that token is added to the graph
    as a vertex.
  \item If a line contains two tokens, an edge is added with those
    tokens as end points. If those two tokens are the same, no edge is
    created (since we do not allow loops) but a vertex is added (if
    not already in the graph).
  \item If there are three or more tokens on a line, only the first
    two are read and the rest are ignored.
  \item If the line is blank, it is ignored.
  \item If the line begins with a \verb|#|, the entire line is
    ignored. (This does put a mild limitation on the names of
    vertices.)
  \end{itemize}

  \item \verb|Complete|

    The \verb|Complete| function can be used to create a complete
    graph $K_n$, a complete bipartite graph $K_{n,m}$, or a complete
    multipartite graph $K(n_1,n_2,\ldots,n_p)$.

    \begin{itemize}
    \item Use \verb|Complete(n)| to create a complete graph $K_n$.

    \item Use \verb|Complete(n,m)| to create a complete bipartite
      graph $K_{n,m}$.

    \item Use \verb|Complete([n1,n2,...,np])| to create a complete
      multipartite graph $K(n_1,n_2,\ldots,n_p)$.
    \end{itemize}

    Note the last version requires that the part sizes be in an
    array. In this way we distinguish \verb|Complete(n)| and
    \verb|Complete([n])|. The first makes $K_n$ and the second an
    edgeless graph with $n$ vertices, i.e., $\overline{K_n}$.
    However, \verb|Complete(n,m)| and \verb|Complete([n,m])| build
    exactly the same graphs.
    \small
\begin{verbatim}
julia> G = Complete(4,5)
SimpleGraph{Int64} (9 vertices, 20 edges)

julia> H = Complete([4,5])
SimpleGraph{Int64} (9 vertices, 20 edges)

julia> G == H
true
\end{verbatim}
\normalsize

  \item \verb|Cube|

    Create the cube graph. The $2^n$ vertices of \verb|Cube(n)| are
    \verb|String| objects. For example:
\small
\begin{verbatim}
julia> G = Cube(3)
SimpleGraph{String} (8 vertices, 12 edges)

julia> vlist(G)
8-element Array{String,1}:
 "000"
 "001"
 "010"
 "011"
 "100"
 "101"
 "110"
 "111"

julia> elist(G)
12-element Array{(String,String),1}:
 ("000","001")
 ("000","010")
 ("000","100")
 ("001","011")
 ("001","101")
 ("010","011")
 ("010","110")
 ("011","111")
 ("100","101")
 ("100","110")
 ("101","111")
 ("110","111")
\end{verbatim}
\normalsize

\item \verb|Path|

  Use \verb|Path(n)| to create a path graph with $n$ vertices.

  Also, given a list of vertices \verb|verts|, then \verb|Path(verts)|
  creates a path graph with edges \verb|(verts[k],verts[k+1])| when
  \verb|k=1:n-1|. It's the user's responsibility that there be no
  repeated entries in \verb|verts|.

\item \verb|Grid|

  Use \verb|Grid(n,m)| to create an $n\times m$ grid graph.

\item \verb|Cycle|

  Use \verb|Cycle(n)| to create a cycle graph with $n$ vertices. It is
  required that $n\ge3$.

\item \verb|Wheel|

  Use \verb|Wheel(n)| to create the wheel graph with $n$
  vertices. That is, a graph composed of an $(n-1)$-cycle with one
  additional vertex adjacent to every vertex on the cycle. This
  requires $n\ge4$.

\item \verb|BuckyBall|

  Create the Buckyball graph with $30$ vertices and $90$ edges.

\item \verb|RandomGraph|

  Use \verb|RandomGraph(n,p)| to create an Erd\H{o}s-R\'enyi random
  graph with $n$ vertices with edge probability $p$. If the argument
  \verb|p| is omitted, it is assumed $p=\frac12$.

\item \verb|RandomTree|

  Use \verb|RandomTree(n)| to create a random tree on vertex set
  \verb|1:n|. All $n^{n-2}$ trees are equally likely.

  This works by creating an $n-2$-long sequence of random values, each
  in the range \verb|1:n|. It then converts that Pr\"ufer code to a
  tree using \verb|code_to_tree|. This latter function is exposed for
  use by the user. It takes as input an array of integers and returns
  a tree assuming, that is, that the input is valid. No checks are
  done on the input so user beware.

\item \verb|RandomSBM|

  Use \verb|RandomSBM(bmap,pmat)| to create a random stochastic block
  model graph. Here \verb|bmap| is an $n$-long list of integers between
  $1$ and $b$ that assigns the $n$ vertices to blocks. The
  $i,j$-entry of the matrix \verb|pmat|
  gives the probability that a vertex in block $i$
  is adjacent to a vertex in block $j$.

  This can also be invoked by \verb|RandomSBM(n,pvec,pmat)| where
  \verb|n| is the number of vertices, \verb|pvec| is a probability vector
  whose $j$-th entry is the probability a vertex is placed in the $j$-th block.

\item \verb|Kneser|

  Use \verb|Kneser(n,k)| to create the Kneser graph with those
  parameters (with $0 \le k \le n$). This is a graph with $\binom nk$
  vertices that are the $k$-element subsets of $\{1,2,\ldots,n\}$. Two
  vertices $u$ and $v$ are adjacent iff $u \cap v = \varnothing$.

  Part of this implementation is a function \verb|subsets(A,k)| where
  \verb|A| is a \verb|Set| and \verb|k| is an \verb|Int|. This creates
  the set of all $k$-element subsets of $A$.

\item \verb|Petersen|

  Use \verb|Peteren()| to create Petersen's graph. This is created by
  calculating \verb|Kneser(5,2)|.

  To remap the vertex names to $\{1,2,\ldots,10\}$ use
  \verb|relabel(Petersen())|.

  \item \verb|Frucht|

    Use \verb|Frucht| to create the Frucht graph, 12-vertex, 3-regular graph with
    no non-nontrivial automorphisms.

  \item \verb|Hoffman|

    Use \verb|Hoffman()| to return a graph that is cospectral, but not
    isomorphic, to the 4-cube.

  \item \verb|HoffmanSingleton|

    Use \verb|HoffmanSingleton()| to create the Hoffman-Singleton graph:
    A $7$-regular, diameter-$2$, girth-$5$ graph.

  \end{itemize}

\subsection*{Graph operations}
These are operations that create new graphs from old.

\begin{itemize}

\item \verb|line_graph|

  Use \verb|line_graph(G)| to create the line graph of $G$. Note that
  if \verb|G| has vertex type \verb|T|, then this creates a graph with
  vertex type \verb|(T,T)|.

\item \verb|complement| and \verb|complement!|

  Use \verb|complement(G)| to create the graph $\overline{G}$. The
  original graph is not changed and the vertex type of the new graph
  is that same the vertex type of \verb|G|.

  We can use \verb|G'| in lieu of \verb|complement(G)|.

  Use \verb|complement!(G)| to complement a graph in place (i.e.,
  replace \verb|G| with its own complement).

\item \verb|copy|

  Use \verb|copy(G)| to create an independent copy of \verb|G|.

\item \verb|induce|

  Use \verb|induce(G,A)| to create the induced subgraph of \verb|G| on
  vertex set \verb|A|.

\item \verb|spanning_forest|

  Given a graph, this creates a maximal, acyclic, spanning
  subgraph. If the original graph is connected, this produces a
  spanning tree.

\item \verb|cartesian|

  Use \verb|cartesian(G,H)| to compute the Cartesian product $G\times
  H$ of $G$ and $H$. For example, to create a grid graph, do this:
  \verb|cartesian(Path(n), Path(m))|.

  Note that \verb|G*H| is equivalent to \verb|cartesian(G,H)|.

\item \verb|relabel|

  Create a new graph, isomorphic to the old graph, in which the
  vertices have been renamed. Use \verb|relabel(G,label)| where
  \verb|G| is a simple graph and \verb|labels| is a dictionary mapping
  vertices in \verb|G| to new names. Trouble ensues if two vertices
  are mapped to the same label (we don't check).

  Note that if the vertex type of \verb|G| is \verb|S|, then
  \verb|label| must be of type \verb|Dict{S,T}|. The new graph
  produced with have type \verb|SimpleGraph{T}|.

  Calling this with one argument, \verb|relabel(G)|, will produce a
  relabeled version of \verb|G| using consecutive integers starting
  with $1$.

\item \verb|disjoint_union|

  The disjoint union of two graphs is formed by taking
  nonoverlapping copies of the two graphs and merging them into a
  single graph (with no additional edges). In Julia, we do this by
  appending the intger 1 or 2 to the vertex names.  Thus, if a vertex
  of the first graph is \verb|"alpha"|, then in the disjoint union its
  name will be \verb|("alpha",1)|.

  Use \verb|disjoint_union(G,H)| to form the disjoint union. If the
  vertex types of the two graphs are both \verb|T|, then the vertex
  type of the result is type \verb|(T,Int)|. But if the two graphs
  have different vertex types, then the result has vertex type
  \verb|Any|.

  We append a 1 or a 2 to vertex names to ensure that we have two
  independent copies of the graphs. If the user knows that the two
  graphs have no vertices in common, then \verb|union| might be a
  preferrable choice.

\item \verb|join|

  The join of two graphs is formed by taking nonoverlapping copies of
  the two graphs and then adding all possible edges between the two
  copies. To ensure the two copies of the given graphs are on distinct
  vertex sets, we append a 1 or a 2 to the vertex names (see the
  description for \verb|disjoint_union|).

  Use \verb|join(G,H)| to form the join. If the two graphs have the
  same vertex type \verb|T|, then the result has vertex type
  \verb|(T,Int)|. Otherwise, the resulting graph has vertex type
  \verb|Any|.

\item \verb|union|

  Given graphs $G$ and $H$, the union has vertex set $V(G)\cup V(H)$
  and edge set $E(G) \cup E(H)$.

\item \verb|trim|

  Trimming a graph means to repeatedly remove vertices of a given
  degree $d$ or less until either all vertices have been removed or the
  remaining vertices induce a subgraph all of whose vertices have
  degree greater than $d$.

  Use \verb|trim(G,d)| to trim the graph, with \verb|trim(G)|
  equivalent to \verb|trim(G,0)|. The latter simply removes all
  isolated vertices.

\end{itemize}

\subsection*{Manipulators and inspectors}

These are functions that are used to modify a graph and to inspect its
structure.

\begin{itemize}

\item \verb|isequal|

  Test two graphs to see if they are the same; that is, the graphs
  must have equal vertex and edge sets. They need not be the same
  object. While this can be invoked as \verb|isequal(G,H)| it is more
  convenient to use \verb|G==H|.

  Note: If the vertex type of either graph cannot be sorted by
  \verb|<| then equality testing is slower (unless fast neighhborhood
  lookup is engaged).

\item \verb|add!|

  Use this to add vertices or edges to a graph. The syntax
  \verb|add!(G,v)| adds a vertex and calling \verb|add!(G,v,w)| adds an edge.

  These return \verb|true| if the operation succeeded in adding a
  \emph{new} vertex or edge.

\item \verb|delete!|

  Use this to delete vertices or edges from a graph. The syntax
  \verb|delete!(G,v)| to delete a vertex (and all edges incident
  thereon) and \verb|delete!(G,v,w)| to delete an edge.

  Returns \verb|true| if successful. If the vertex or edge slated for
  removal was not in the graph, returns \verb|false|.

\item \verb|contract!|

  Mutates a graph by contractin an edge. Calling
  \verb|contract!(G,u,v)| adds all vertices in \verb|v|'s neighborhood
  to \verb|u|'s neighborhood and then deletes vertex
  \verb|v|. Typically \verb|(u,v)| is an edge of the graph but this is
  not necessary.

  This returns \verb|true| is the operation is successful, but
  \verb|false| if either \verb|u| or \verb|v| is not a vertex of the
  graph or if \verb|u==v|.

\item \verb|has|

  Test for the presence of a vertex of edge. Use \verb|has(G,v)| to
  test if \verb|v| is a vertex of the graph and \verb|has(G,v,w)| to
  test if the edge is present. Returns \verb|true| if so and
  \verb|false| if not.

  Note that \verb|G[v,w]| is a synonym for \verb|has(G,v,w)|.

\item \verb|vlist|

  Use \verb|vlist(G)| to get the vertex set of the graph as a
  one-dimensional array. If possible, the vertices are sorted in
  ascending order.

\item \verb|elist|

  Use \verb|elist(G)| to get the edge set of the graph as a
  one-dimensional array of 2-tuples. If possible, the edges are sorted
  in ascending lexicographic order.

\item \verb|neighbors|

  Use \verb|neighbors(G,v)| to get the set of neighbors of vertex
  \verb|v| as a one-dimensional array.

  Note that \verb|G[v]| is a synonym.

\item \verb|deg|

  Use \verb|deg(G,v)| to get the degree of vertex \verb|v| and
  \verb|deg(G)| to get the degree sequence of $G$ as a one-dimensional
  array (in decreasing order).

\item \verb|fastN!|

  This is explained in \S\ref{sect:fastN}.

  Use this to switch on \verb|fastN!(G,true)| or to switch off
  \verb|fastN!(G,false)| rapid neighborhood lookup. If off,
  neighborhood lookup can be slow. If on, the data structure
  supporting the graph is roughly tripled in size.

  The difference is especially striking when looking for paths between
  vertices with \verb|find_path|.

\item \verb|NV| and \verb|NE|

  Use \verb|NV(G)| to get the number of vertices and \verb|NE(G)| to
  get the number of edges.

\item \verb|is_connected|

  Use \verb|is_connected(G)| to determine if the graph is connected.

\item \verb|num_components|

  Use \verb|num_components(G)| to determine the number of connected
  components in the graph.

\item \verb|components|

  The function \verb|components(G)| determines the vertex sets of the
  connected components of the graph. The return value is a set of
  sets. That is, if the graph has vertex type \verb|T|, then this
  function produces a \verb|Partition| (requires \verb|SimplePartitions|).

\item \verb|find_path|

  The function \verb|find_path(G,u,v)| finds a shortest path from
  \verb|u| to \verb|v| if one exists. An empty array is returned if
  there is no such path. An error is raised if either vertex is absent
  from the graph.

\item \verb|dist| and \verb|dist_matrix|

  These are used to find distances between vertices in a graph. The
  distance between vertices $u$ and $v$ is defined to be the number of
  edges in a shortest $(u,v)$-path. If there is no such path, one
  typically says that $d(u,v)$ is undefined of $\infty$. However since
  these functions report distances as \verb|Int| values, we signal the
  absence of a $(u,v)$ path by the value $-1$.

  Use \verb|dist(G,v,w)| to find the distance between the specified
  vertices in the graph.

  Use \verb|dist(G,v)| to find the distances from vertex \verb|v| to
  all vertices in the grpah. This is returned as a \verb|Dict|.

  Use \verb|dist(G)| to find all distances between all vertices in the
  graph. For example:
  {\small
\begin{verbatim}
julia> G = Cycle(10)
SimpleGraph{Int64} (10 vertices, 10 edges)

julia> d = dist(G);

julia> d[(3,9)]
4
\end{verbatim}
  }

  The function \verb|dist_matrix| creates an $n\times n$-matrix whose
  $i,j$-entry is the distance between the $i^{\text{th}}$ and
  $j^{\text{th}}$ vertex of the graph where the order is produced by
  \verb|vlist|.

\item \verb|wiener_index|

  The function \verb|wiener_index| computes the sum of the distances
  between distinct vertices in the graph:
  \[
  \frac12 \sum_{v\in V} \sum_{w \in V} d(v,w).
  \]

\item \verb|diam|

  Compute the diameter of a graph. Note that \verb|diam(G)| returns
  $-1$ if the graph is not connected.

\item \verb|eccentricty|

  The \emph{eccentricity} of a vertex $v$ of a graph $G$ is the
  maximum distance from $v$ to another vertex of $G$.
  \verb|eccentricity(G,v)| returns this
  value, but if the graph is not connected it returns $-1$.

\item \verb|radius|

  The \emph{radius} of a graph is the minimum eccentricty of a vertex.
  \verb|radius(G)| returns $-1$ if the graph is not connected.

\item \verb|is_ayclic|

  Determine if the graph is acyclic.

\item \verb|is_cut_edge|

  Determine if a given edge is a cut edge. This can be called either
  as \verb|is_cut_edge(G,u,v)| where \verb|u| and \verb|v| are
  vertices or as \verb|is_cut_edge(G,e)| where \verb|e| is an edge
  (i.e., a 2-tuple of vertices.

  If \verb|(u,v)| is not an edge of the graph, an error is raised.

\item \verb|euler|

  This is used to find Eulerian trails and tours in a graph. Typical
  call is \verb|euler(G,u,v)| to find an Eulerian trail starting at
  \verb|u| and ending at \verb|v|.  The first element of that arrary
  is \verb|u| and the last is \verb|v|. If a trail is found, the
  length of the array is \verb|NE(G)+1|. Otherwise, an empty array is
  returned.

  The graph may have isolated vertices, and these are ignored.

  The call \verb|euler(G,u)| is shorthand for \verb|euler(G,u,u)|. A
  simple call to \verb|euler(G)| will attempt to find an Euler tour
  from some vertex in the graph.

  If the graph is edgeless, then \verb|euler(G,u)| and
  \verb|euler(G,u,u)| return the 1-element array \verb|[u]|. Calling
  \verb|euler(G)| will pick \verb|u| for you.  An empty array is
  returned if the graph has no vertices. (This is mildly unfortunate
  as an empty array indicates failure to find a trail for nonempty
  graphs.)

\item \verb|bipartition| and \verb|two_color|

  Used to find a bipartition or two-coloring of a graph if the graph
  is bipartite; otherwise, return an error. The function
  \verb|bipartition| returns a \verb|Partition| with the two
  color classes.
  The function
  \verb|two_color| returns a map (\verb|Dict|) from the vertex set to
  the set $\{1,2\}$.
  {\small
\begin{verbatim}
julia> G = Cycle(6)
SimpleGraph{Int64} (6 vertices, 6 edges)

julia> two_color(G)
[5=>1,4=>2,6=>2,2=>2,3=>1,1=>1]

julia> bipartition(G)
{{2,4,6},{1,3,5}}
\end{verbatim}
  }

\item \verb|girth| and \verb|girth_cycle|

  Determine the girth of a simple graph and find a shortest cycle
  in that graph. If the graph is acyclic, \verb|girth| returns
  \verb|0| and \verb|girth_cycle| returns an empty array.

\item \verb|greedy_color| and \verb|random_greedy_color|

  This is a simple graph coloring function. Given an ordering of the
  vertices of the graph, \verb|greedy_color| creates a proper, greedy
  coloring. If the ordering is not provided, then a degree-decreasing
  ordering is given. Use \verb|greedy_color(G,seq)| where \verb|seq|
  is a permutation of the vertex set (if you wish to specify the
  order) or simply \verb|greedy_color(G)| in which case a
  degree-decreasing ordering is used.

  The second function performs multiple greedy colorings on random
  orderings of the vertex set. Use \verb|random_greedy_color(G,nreps)|
  where \verb|nreps| is the number of random orders generated.

  In all cases a \verb|Dict| is returned that maps the vertex set to a
  range of the form \verb|[1:k]|.

\end{itemize}

\subsection*{Graph matrices}

These functions return standard matrices associated with graphs.

\begin{itemize}
\item \verb|adjacency|

  Use \verb|adjacency(G)| to return the adjacency matrix of the graph.

\item \verb|laplace|

  Use \verb|laplace(G)| to return the Laplacian matrix of the graph.

\item \verb|incidence|

  Use \verb|incidence(G)| to return the signed incidence matrix of the
  graph. This is equivalent to \verb|incidence(G,true)|. Calling
  \verb|incidence(G,false)| returns the unsigned incidence matrix.

  Assignment of $+1$ and $-1$ in each column tries to put the $+1$ on
  the vertex that sorts lower than the vertex that gets a $-1$. If the
  vertices are not comparable by \verb|<| (less than), the assignment
  is unpredictable.

  Note that \verb|incidence| returns a sparse matrix. Use
  \verb|full(incidence(G))| if a full-storage matrix is desired.
\end{itemize}

\subsection*{Converting}

\textbf{Note}: The feature is currently turned off because of
errors generated by the \verb|Graphs| module.

This is explained in \S\ref{sect:convert}.

\begin{itemize}
\item \verb|convert_simple|

  Use \verb|convert_simple(G)| to create a Julia
  \verb|Graphs.simple_graph| version of a graph, together with
  dictionaries to translate between one vertex set and the other.
\end{itemize}

\section{Errors and gotchas}

\subsection*{Errors raised}
The functions in the \verb|SimpleGraphs| module generally do not
raise errors. Function such as \verb|add!| and \verb|delete!| return
\verb|false| if the graph is not changed by the requested
modification.

However, there are some instances in which an error might be raised.
\begin{itemize}
\item An error is raised if one attempts to add a vertex of a type
  that is incompatible with the vertex type of the graph. Here's an
  example:
\begin{verbatim}
julia> G = StringGraph()
SimpleGraph{String} (0 vertices, 0 edges)

julia> add!(G,4)
ERROR: no method convert(Type{String},Int64)
\end{verbatim}

\item An error is raised if one attempts to find the neighborhood or
  the degree of a vertex that is not in the graph. Here's an example:
\begin{verbatim}
julia> G = Complete(5)
SimpleGraph{Int64} (5 vertices, 10 edges)

julia> G[6]
ERROR: Graph does not contain requested vertex
\end{verbatim}

\item An error is raised if one attempts to create a cycle with fewer
  than three vertices.
\end{itemize}

Of course, code can be wrapped in a \verb|try|-\verb|catch| block to
handle these possibilities gracefully.

\subsection*{Please specify the vertex type}
Although \verb|G=SimpleGraph()| allows \verb|Any| type vertex, we
recommend specifying the type of vertex desired. Here's why.

Because we do not allow loops, the code for \verb|add!(G,v,w)| first
checks if \verb|v==w| and if so, does not add the edge and returns the
value \verb|false|. So far so good.

Internally, the vertices of the graph are held in a Julia \verb|Set|
container. Now \verb|Set| either does or does not contain a given
object; the object cannot be in the \verb|Set| twice. Where this gets
us into a bit of trouble is that the integer value \verb|1| and the
floating point value \verb|1.0| are different objects and therefore
may cohabit the same \verb|Set|. Here's an illustration:
\begin{verbatim}
julia> A = Set()
Set{Any}()

julia> push!(A,1)
Set{Any}(1)

julia> push!(A,1.0)
Set{Any}(1.0,1)
\end{verbatim}
The implication of this is that the vertex set of a \verb|SimpleGraph|
might contain both the integer \verb|1| and the floating point number
\verb|1.0|. However, we cannot add an edge between these two vertices
because the test \verb|1==1.0| returns \verb|true|.

Here's how this plays out:
\begin{verbatim}
julia> G = SimpleGraph()
SimpleGraph{Any} (0 vertices, 0 edges)

julia> add!(G,1)
true

julia> has(G,1.0)
false

julia> add!(G,1.0)
true

julia> add!(G,1,1.0)
false
\end{verbatim}

In principle, we could fix this problem by using a more liberal filter
in \verb|add!(G,v,w)| that allows the addition of an edge with
\verb|v==w| provided \verb|typeof(v)| and \verb|typeof(w)| are
different. But that would not entirely solve the problem.

Consider this example:
\begin{verbatim}
julia> G = SimpleGraph()
SimpleGraph{Any} (0 vertices, 0 edges)

julia> add!(G,1)
true

julia> add!(G,BigInt(1))
true

julia> vlist(G)
2-element Array{Any,1}:
 1
 1
\end{verbatim}
It appears that the number 1 is in the vertex set twice!

The preferred solution is to avoid using \verb|G=SimpleGraph()|
and, instead, to use \verb|G=SimpleGraph{T}()| where \verb|T| is the
data type of the vertices. The ready-to-use \verb|IntGraph| and
\verb|StringGraph| are handy for these popular vertex types.
\begin{verbatim}
julia> G = IntGraph()
SimpleGraph{Int64} (0 vertices, 0 edges)

julia> add!(G,1)
true

julia> add!(G,BigInt(1))
ERROR: 1 is not a valid key for type Int64

julia> add!(G,1.0)
ERROR: 1.0 is not a valid key for type Int64
\end{verbatim}

\subsection*{Unsortable vertex types}

Edges in a \verb|SimpleGraph| are held as a tuple. If the end points
of the edge can be compared using the \verb|<| operator, then the
smaller end point comes first in the pair. Otherwise, the order is
arbitrary. In the latter case, graph equalty checking is slower.

In general, it's best to specify vertex types for graphs, preferring
\verb|SimpleGraph{T}()| for some type \verb|T| that supports \verb|<|
comparison.

\section{Convert to Graphs.jl}
\label{sect:convert}

\textbf{Note}: The feature is currently turned off because of
errors generated by the \verb|Graphs| module.

The \verb|Graphs| module defined in \verb|Graphs.jl| is another tool
for dealing with graphs. We provide the function \verb|convert_simple|
to convert a graph from a \verb|SimpleGraph| to a
\verb|simple_graph| type graph from the \verb|Graphs| module
distributed with Julia.

A \verb|simple_graph|'s vertex set is always of the form \verb|1:n|,
so the output of \verb|convert_simple| provides two dictionaries for
mapping from the vertex set of the \verb|SimpleGraph| to the vertex set
of the \verb|simple_graph|, and back again.

\section{Results Caching}
\label{sect:cache}

Some graph-theoretic computations can be expensive. To avoid computing
the same item twice for a given graph, the results of these
calculations are saved in a data structure attached to the graph. Note
that any change to the graph will automaticaly clear the cache.

Some values held in the cache may be mutable Julia objects. Such
objects are deep copied when fetched from the cache.

The following functions are available to the user to manipulate the
cache.

\begin{itemize}
\item \verb|cache_clear(G)|: Removes all items held in the cache.

\item \verb|cache_clear(G,item)|: Remove a specific item from the
  cache. The variable \verb|item| is a \verb|Symbol| whose name
  corresponds to the function it caches (for example,
  \verb|:components| or \verb|:girth|).

\item \verb|cache_on(G)|: Enable the caching system for this graph.

\item \verb|cache_off(G)|: Disable the caching system for this graph.
  Note thie does not wipe the graph's cache, so to save space, also
  use \verb|cache_clear(G)|.

\item \verb|cache_check(G,item)|: See if there is a value associated
  with the symbol \verb|item| held in the graph's cache.

\item \verb|cache_recall(G,item)|: Copy a value associated with the
  symbol \verb|item| from the cache. \textbf{Warning}: If there is
  no such symbol held in the cache an error will be thrown. Be sure
  to first check using \verb|cache_check|.

\end{itemize}

\end{document}