% Generated by GrindEQ Word-to-LaTeX 2008
% ========== UNREGISTERED! ========== Please register! ==========
% LaTeX/AMS-LaTeX

\documentclass[a4paper]{article}
\usepackage{anysize}
\marginsize{1cm}{1cm}{1cm}{1cm}

\usepackage{amssymb}
\usepackage{amsmath}
\usepackage[dvips]{graphicx}
\usepackage{listings}
\lstset{language=haskell}
\lstset{commentstyle=\textit}
\lstset{mathescape=true}
%\lstset{labelstep=1}
%\lstset{backgroundcolor=,framerulecolor=}
\lstset{backgroundcolor=,rulecolor=}
\linespread{2.0}

\begin{document}

%%% remove comment delimiter ('%') and select language if required
%\selectlanguage{spanish}

\title{Customizable Objects}

\textit{Giuseppe Maggiore}
\textit{Michele Bugliesi}
\textit{Universitá Ca'  Foscari -- Venice}

\section{Introduction}

 In this paper we present a technique that aims towards supporting many different programming paradigms and techniques with a clear and intuitive syntax (and no particular knowledge requirements) for the developer. We also wish to make it possible to easily switch between one paradigm and the other without modifying the client code: at the ``flipping of a switch'' the actual execution of a developer code could radically change to support different features \textit{without any modification whatsoever}. The kind of techniques we wish to support as a ``runtime backend'' for client programs is quite a large set; among these various techniques we discuss some of our favorites: transactional systems, concurrent/distributed systems and reactive programs. We absolutely wish to avoid creating a new language or to extend the syntax of an existing one; instead we try to push the expressivity o the highly flexible Haskell programming language so that it supports our desired constructs. As a starting point we build a framework for working with mutable objects; this system must (of course) be type-safe and also as generic as possible: for this reason we will use meta-programming (type predicates and type functions) a lot. Our object system heavily uses heterogeneous lists as a basis for a customized type-safe heap. We define references to values inside our heap to offer support for mutable operations such as assignment and in-place modification; our heap requires some notion of subtyping to allow references that expect a smaller heap to work wherever a larger heap is active. Whenever a reference represents a value based on our heterogeneous lists then we give a selection operator that returns a reference to a field inside the references heterogeneous list; thanks to this selection operator we now have built a framework for handling mutable records. We extend our mutable records with support for methods and method selection and invocation; since our methods are not invoked directly but through our customized selection operator we can control not only the automated passing of the \textit{this} value to each method as its first parameter, but also the propagation of effects into \textit{this }after the method invocation. Some work is dedicated to overloading the same selection operator so as to not having to distinguish between selecting a field and a method. We add support for inheritance so that the fields and methods of the inherited object can be accessed from the inheriting object in a fully transparent way, and we also add an explicit casting operator to convert a reference to an object to a reference to the \textit{base} (inherited) object.

 We pay attention to working on two distinct levels. On one hand we define all our constructs as very generic type predicates and type functions, thereby giving the general shape of our operators. In the most decoupled way possible we instance these functions and predicates with heterogeneous lists, a concrete representation for references, record labels, methods, etc. At the interface level we also define a way to write our computations as statements in the state monad. This allows a developer to write code that uses very general objects, without caring about the plumbing involved whenever the various assignment, selection, method invocation etc. operations occur; by instancing our generic operators multiple times we get the bonus of executing client code with a completely different runtime behavior than simple stateful objects. We will show a few examples of how we can do so and what kind of constructs we may implement. As a first example we will show how we could add three simple operators (\textit{beginT}, \textit{commitT} and \textit{discardT}) to enable transactional code. Then we will show how we could support reactive programming by modifying the behavior of the binding and selection operators, storing and passing on appropriate callbacks that signal the need for propagating changes in the operators to the results of computations. Finally we will discuss how we could make our objects distributed or concurrent, either with shared memory or outright message-passing: by storing a process ID or machine ID inside each reference we could track to which process the current reference belongs to so that whenever we try and access the value represented by such reference we can check whether or not this reference is local and if it is not we will use some synchronization/message-passing scheme to access it.

 Our last step will compare our approach to various, more specific approaches for supporting mutable objects, reactive computations, concurrency, etc. on top of pure functional languages.

 Note on syntax: we will use an ambiguous set of operators with some conventions. The ambiguity will always be clearly resolved by the context, since some operators will both be present at the type level and at the value level, but there is never any confusion as to whether we are manipulating types or values. Also, we will use a Haskell-like syntax:

\begin{enumerate}
\item  $x\ =\ v$ for binding

\item  $let\ x\ =\ v_1\ in\ \ v_2$ for binding

\item  $T=t$ for defining types

\item  $F\ :{\left(*\ \to \ \right)}^n*$ for giving the kind of a type function

\item  $F\ T_1\dots \ \ T_n=T_{n+1}$ for specifying the returned value of a type function with certain input parameters

\item  $P\ v_1\dots \ v_n$ for defining a type predicate

\item  $P_1\ v_{11}\dots \ v_{1m}\Rightarrow P_1\ v_{21}\dots v_{2m}$ for defining a type predicate that requires another type predicate

\item  $P\ T_1\dots \ T_n$ for instancing a type predicate
\end{enumerate}

 Rather than use the full Haskell syntax, we will try and adhere to a more standard syntax. For example, the fact that $x$ has type $T$ we will write

\begin{lstlisting}
x:T
\end{lstlisting}

Rather than the peculiar

\begin{lstlisting}
x::T
\end{lstlisting}

Used in Haskell programs. Also, when defining a type predicate and the operators it enables we will just say:

\begin{lstlisting}
P $v_1 \dots v_n$
$f_1$ : $T_1$
   $\vdots$
$f_m$ : $T_m$
\end{lstlisting}

Rather than the Haskell

\begin{lstlisting}
class P $v_1 \dots v_n$  where
  $f_1$ : $T_1$
    $\vdots$
  $f_m$ : $T_m$
\end{lstlisting}

\section{State Monad}

 We build a system for performing stateful computation. This kind of computations can be expressed very cleanly with the state monad, which essentially allows us to express the denotational semantics of our statements and which automatically concatenates the semantics of those statements that must be executed sequentially. An ulterior incentive towards using monads is that as language constructs monads have seen an impressive support in many languages, starting from Haskell and ranging to F\# and C\#; this is probably related to the expressive power of monads and to the way they allow extending a language cleanly and seamlessly, especially if supported by proper syntactic sugar. A statement that evaluates to a value $\alpha $ with a state $s$ in the state monad has type

\begin{lstlisting}
ST s $\alpha$ = s -> ($\alpha \times$s)
\end{lstlisting}

A monad is a triplet of:

\begin{enumerate}
\item  a type constructor $M$ that denotes the type of our monads

\item  a $unit$ operator with type $\alpha \to M\ \alpha $ that generates a monad encapsulating a value

\item  a $bind$ operator with type $M\ \alpha \to \left(\alpha \to M\ \beta \right)\to M\ \beta $ that ``concatenates'' two monads into one
\end{enumerate}

For the state monad we have:

\begin{lstlisting}[frame=tb,mathescape]{somecode}
M = ST s
unit x = ST($\lambda$s.(x,s))
bind (ST m) k=ST($\lambda$s.
		let (x,s')=m s in
		let (ST m')=k x in
		m' s')
\end{lstlisting}

We also add an evaluation operation for values of the state monad:

\begin{lstlisting}[frame=tb]
runST : ST s $\alpha$ -> s -> $\alpha$
runST m s = fst (m s)
\end{lstlisting}

We will use Haskell style syntactic sugar, so that using monads may feel more intuitive than using the explicit syntax; the kind of sugarization we use is based on two (purely cosmetic) simplifications, one for the $unit$ operator and the other for the $bind\ $operator:

\begin{lstlisting}[frame=tb]
return $\equiv$ unit
bind m ($\lambda$v. $\dots$) $\equiv$ do v $\leftarrow$ m
		      $\dots$
\end{lstlisting}

\section{Heap}

 We start by giving a definition of our notion of heap. The heap will be the basis for all our computations, in that it will provide the main, shared storage to which all statements and references in our programs refer to. We do not define our heap as a concrete type, but rather we give an ``interface'' (a Haskell type class) that specifies the operations that must be available on some generic type for it to be used in whichever context a heap is expected. Such an interface may also contain functions from types to types that specify some type transformations that our heap will need to offer. We define this predicate on a type variable $h$ as:

\begin{lstlisting}
Heap h
\end{lstlisting}

We start with type-level functions. A heap $h$ must support a type function that, given $h$ and a type $\alpha $ (some new item), returns a new heap; we will expect the returned heap to contain not only all the elements that the original heap $h$ contained, but also a new slot for storing a value of type $\alpha $:

\begin{lstlisting}
New h : *$\to$*
\end{lstlisting}

Thanks to this definition we can now proceed to giving the two fundamental operators on a heap: $new$ and $delete$, which respectively allow us to add an element to our heap and remove the last added element to our heap. The allocation and deallocation operators have the following types:

\begin{lstlisting}
new : h$\to\alpha\to$New h $\alpha$
del : New h $\alpha$$\to$h
\end{lstlisting}

\section{References}

 Our notion of heap is too poor for our purposes. We immediately extend our heap predicate so that instead of having ``just a heap'' (which would be relatively pointless), whenever we have a heap $h$ we also get a type $ref$ for expressing references to within our heap $h$. This way we strongly couple the idea of some state (the heap itself) and the idea of doing mutable computations on it (the references we will define shortly). References are represented as pointing to some value $\alpha $ inside some heap $h$; because of this a reference will be represented with a functor $ref$ that takes in two type parameters.

Our heap predicate now takes two type variables as parameters: the heap $h$ and the functor $ref$; the signature now becomes:

\begin{lstlisting}
Heap h ref
\end{lstlisting}

Followed by the various functions and operators that both $h$ and $ref$ must support. We give a small ``calculus of references'' that allows us to manipulate references by creating references from other references (selection), invoking methods from references (method invocation), and generating stateful statements from references. Statements that evaluate to a value of type $\alpha $ in a context with a heap $h$ will have the form (known from the state monad):

\begin{lstlisting}
ST h $\alpha$
\end{lstlisting}

Some of the operators on references we will not introduce right away; these operators will be presented after some more background is covered. Two simple and basic operators on references are the evaluation and assignment operators:

\begin{lstlisting}
eval : ref h $\alpha\to$ST h $\alpha$
\end{lstlisting}

Which evaluates a reference to a statement that will return the same value contained inside the reference when passed a concrete heap, and:

\begin{lstlisting}
(:=) : ref h $\alpha\to\alpha\to$ST h Unit
\end{lstlisting}

Which assigns a reference of type $\alpha $ a new value of type $\alpha $ and returns a statement that when evaluated will return a value of type $Unit$ ($()$) and the new heap where the assignment has been made.

 Another kind of assignment operator is the in-place modification operator:

\begin{lstlisting}
(*=) : ref h $\alpha\to(\alpha\to\alpha)\to$ST h Unit
\end{lstlisting}

which takes a reference of type $\alpha $, a function of type $(\alpha \to \alpha )$ and returns a statement which, when passed a concrete value of the heap $h$, will evaluate the initial value of the reference, apply the function and return the result of assigning it to the original reference. This operator is quite handy for implementing common operators such as the in-place increment by one of an integer $(++)$ or the in-place increment by some integer value $(+=)$ that are widely used in stateful programming languages such as C, Java or C\#. The alternative to having such an operator is to evaluating and then assigning a reference; indeed, in light of this consideration we may realize that we already could define the body of this operator in terms of the state monad:

\begin{lstlisting}[frame=tb,mathescape]{somecode}
r *= f =
	do v $\leftarrow$ eval r
	   r := (f v)
\end{lstlisting}

It is interesting to realize that this definition is indeed extremely abstract, and implies absolutely nothing in terms of what is happening exactly inside the various operators: whether this operation simply writes a mutable state, synchronizes access to some shared memory location or even sends and receives some network messages with some other instance of the same distributed process is indeed unknown by just looking at this piece of code.

 A very important operation on some reference $ref$ and some heap $h$ is the ``introduction-and-elimination'' operator that:

\begin{enumerate}
\item  Allocates an initial value of type $\alpha $ on our heap $h$

\item  Builds a reference to this value

\item  Performs some arbitrary computation on this newly create reference, returning a result of type $\beta $

\item  Cleans up the heap deleting the new value of type $\alpha $ before returning the $\beta \ $result
\end{enumerate}

 This operator is called $\left(\gg +\right)$, in order to recall the (similar) concept of binding in a monad. In fact, this operator will bind a new value of type $\alpha $ to a corresponding reference. The type of this operator is:

\begin{lstlisting}
(>>+) : $\alpha\to$(ref (New h $\alpha$) $\alpha\to$ST (New h $\alpha$) $\beta$)$\to$ST h $\beta$
\end{lstlisting}

We use the same representation that the state monad uses for values where our heap $h$ is the mutable state of the computation. It is also interesting to notice that this operator works in a way that is very similar to a ``scoping'' operator. To realize this, notice that the value of type $\alpha $ is accessed through a reference that is exclusively available within the confines of the function that is passed as the second parameter to $(\gg +)$. As an example, let us consider the following pseudo-code:

\begin{lstlisting}[frame=tb,mathescape,language=C++]{somecode}
A a = new A(...);
{
  B b = new B(...);
  {
    // do something with a and b
  }
}
\end{lstlisting}

Would be written, with our $(>>+)$ operator:

\begin{lstlisting}[frame=tb,mathescape]{somecode}
A(...) $>>+$ ($\lambda$a.
B(...) $>>+$ ($\lambda$b.
  -- do something with a and b
))
\end{lstlisting}

\section{First Examples:}

 Thanks to the system we have put together, we may already start writing some simple examples. These examples will only use the common operators that are required to exist for every heap $h$ with references $ref$.

 For the first example, let us say that we have a reference $i$ to an integer within some heap $h$ such that:

\begin{lstlisting}
i : ref h Int
\end{lstlisting}

We can write:

\begin{lstlisting}[frame=tb,mathescape]{somecode}
$ex_1$ =
	do v $\leftarrow$ eval i
	   i := v+2
	   v' $\leftarrow$ eval i
	   return v
\end{lstlisting}

The type of the code above is:

\begin{lstlisting}
Heap h ref $\Rightarrow$ $ex_1$ : ST h Int
\end{lstlisting}

%Why $ex_1$ has this type is maybe more obvious if we consider the de-sugarized version of the same code:
%\[ex^'_1=\underbrace{bind\underbrace{\left(eval\ i\right)}_{ST\ h\ Int}\underbrace{\left(\lambda \underbrace{v}_{Int}.%\underbrace{bind\underbrace{\left(i?v+1\right)}_{ST\ h\ Int}\underbrace{\left(\lambda \underbrace{{\rm \_}}_{Unit}.%\underbrace{bind\underbrace{\left(eval\ i\right)}_{ST\ h\ Int}\ \left(\lambda \underbrace{v^'}_{Unit}.\underbrace{return\ %v^'}_{ST\ h\ Int}\right)}_{ST\ h\ Int}\right)}_{Unit\to \ ST\ h\ Int}}_{ST\ h\ Int}\right)}_{Int\to ST\ h\ Int}}_{ST\ h\ Int}\]

The example above makes a quite limited use of our operators, but most importantly it assumes that a reference has already been created outside the example and is given. Let us show now how we can generate our local references in a self-contained example. The following example is more interesting since it shows the most reasonable pattern of introduction for references: we expect that a reference will be allocated with the $\gg +$ operator, used in its scope and then simply discarded, exactly as we would do in a ``typical'' imperative language. The example is:

\begin{lstlisting}[frame=tb,mathescape]{somecode}
$ex_2$ =
	do 10 $>>=$ ($\lambda$i.
	do "hello " $>>=$ ($\lambda$s.
	do s *= ($\lambda$x.x++"world")
	   v$\leftarrow$eval s
	   x$\leftarrow$eval i
	   return v ++ show x))
\end{lstlisting}

In this example we start by allocating an integer on the heap, then we allocate a string on the heap, and finally we manipulate the two references to these values on the heap before returning the concatenation of their values. Sadly, the example above does not compile without some modifications. We will see very shortly how we can write the example above so that at least it compiles correctly.

 The last example we see is another term which does not type correctly. This time however this behavior is quite desired:

\begin{lstlisting}[frame=tb,mathescape]{somecode}
$ex_{wrong}$ =
	do $i'$ $\leftarrow$ 10 $>>=$ ($\lambda$i.return i)
	   $i'$:=20
\end{lstlisting}

In this example we try to access a location on the heap that is not available, but luckily the last line cannot be typed since ${{\rm i}}^{{\rm '}}{\rm :}{\rm ref}{\rm \ }\left({\rm New}{\rm \ }{{\rm h}}_0{\rm \ }{\rm Int}\right){\rm Int}$ and ${{\rm i}}^{{\rm '}}?{\rm 20\ :}{\rm ST}{\rm \ }\left({\rm New}{\rm \ }{{\rm h}}_0{\rm \ }{\rm Int}\right){\rm \ }{\rm ()}$ but the expected value for the first type parameter is ${{\rm h}}_0\ne {\rm (New\ }{{\rm h}}_0{\rm \ Int)}$.

\end{document}

% == UNREGISTERED! == GrindEQ Word-to-LaTeX 2008 ==


\documentclass{article}
\usepackage[latin9]{inputenc}
\usepackage{geometry}
\geometry{verbose,tmargin=2cm,bmargin=2cm,lmargin=2cm,rmargin=2cm}
\usepackage{float}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{graphicx}
\usepackage{esint}
\usepackage[unicode=true,pdfusetitle,
 bookmarks=true,bookmarksnumbered=false,bookmarksopen=false,
 breaklinks=false,pdfborder={0 0 1},backref=section,colorlinks=false]
 {hyperref}
\usepackage{breakurl}

\makeatletter

\providecommand{\tabularnewline}{\\}

\@ifundefined{date}{}{\date{}}
\usepackage{subfigure}
\usepackage{cancel}

\makeatother

\begin{document}

\title{CSE397: Group Assignment \#3}
\author{Ben Crestel, Nick Malaya, Prabhakar Marepalli, Ajay Vadakkepatt}

\maketitle

\section*{Problem Statement}

\section{Selection of calibration}

Select what you consider to be the best calibration of the ball-drop
model from group problem 2. You may either use one of the calibrations
from problem 2, or revise your calibration based on any feedback you
received (your choice). Be sure to document what calibration you are
using (the priors, error models and likelihood formulation) and your
justification for selecting this particular formulation.

\subsubsection*{Solution}

For this project, we choose the parameters inferred from the data
of both the ball drops together (inferring same alpha and sigma, but
different $C_{d}'s$). This selection is one of the realistic cases
as $\alpha$ and $\sigma$ only depend on the measurement error (motion
of the iPad while recording) whereas it is quite reasonable to infer
different $C_{d}'s$ for different balls as $C_{d}$ is a function
of the surface roughness and diameter of the ball. Also, in contrast
with our solution procedure for Project 2, the parameter $H$ is not
inferred here. This change is from the feedback received for that
report. We outline the calibration procedure below:

\subsubsection*{Prior for $\alpha$}

The coefficient $\alpha$ defined above is said to be ``of order
$0.5$~m/s with uncertainty of the same order''. Since we could
not exclude negative values of $\alpha$ we can choose $\alpha$ to
be Gaussian with mean $0.5$~m/s and $95\%$~confidence interval
given by the interval of uncertainty. Here the uncertainty is said
to be ``of the same order'' as the mean. In order to be conservative,
we'll pick our $67\%$~confidence interval to be $[-1,1]$~m/s.
\[
\alpha\sim\mathcal{N}(0.5,1).
\]

\subsubsection*{Prior for $\sigma$}

For $\sigma$, we know it has to be non-negative. We can therefore
assume that $\sigma$ is uniformly distributed over the interval $[0,0.2]$~m.
\[
\sigma\sim\mathcal{U}(0,0.2).
\]

\subsubsection*{Prior on $C_{d}$, the coefficient of drag}

The coefficient of drag is a non-dimensionalization of the force that
results from a scaling analysis.
\[
\frac{F_{D}}{\frac{1}{2}\rho V^{2}A}=C_{D}(Re).
\]
 Thus, the coefficient depends only on the Reynolds number of the
flow. Let's estimate the Reynolds number to give us a sense of the
range we believe may be possible in this particular experiment.

The Reynolds number is defined as,
\[
Re=\frac{VL}{\nu}
\]
 Let's estimate the characteristic velocity (V), and length (L) as
well as the viscosity, $\nu$. An upper bound for the velocity would
be from the calculating the velocity in the absence of drag. Here,
$V=gt_{\text{final}}$, and with $t_{\text{final}}\approx3$ seconds,
our V is $\approx29.4$ m/s. The characteristic length is the ball
diameter, which for the basketball is 23.33 cm. Finally, the kinematic
viscosity of air is $15.68*10^{-6}\text{m}^{2}/s$. Thus, our $\text{Re}\approx437,000$,
which means the flow is certainly turbulence. Furthermore, we expect
roughness effects to become significant around $Re\approx10^{5}$.
Given that we do not know, with confidence, if the balls are rough
or smooth, our prior must be sufficiently broad to capture the possibility
of either condition.

Looking at a NASA chart for the measured drag coefficient on a sphere,
at low reynolds numbers (when we release the ball) we expect that
our drag coefficient might be as large as 2. Should the sphere be
smooth, the drag coefficient could be as small as 0.1 at the end.
We also know that our prior should not be capable of negative values,
that would be aphysical. For these reasons, we decided to define $C_{d}$
as
\begin{align*}
C_{d}\sim\mathcal{U}(0,2).
\end{align*}

\subsubsection*{Likelihood}

We will infer (at most) for the following parameters: $\alpha,\sigma,C_{d}$.
The likelihood gives us the probability that the results from our
simulation~$h_{ode}$ match the data~$h_{d}$ we have.
\[
\pi\left(h_{ode}=h_{d}\,|\,\alpha,\sigma,C_{d}\right).
\]
 In the hypothetical case we'd know all parameters exactly ($\alpha,\sigma,C_{d}$),
the data would be connected to the results of our simulation through
the following relation
\begin{align*}
h_{d}=\left(h^{\text{ode}}+\varepsilon\right),
\end{align*}
We already detailed the distribution of the random variable $\varepsilon$.
It is Gaussian with mean and covariance given by the time measurements
of each experiment and random variables $\alpha$ and $\sigma$ (which
are assumed to be sampled when evaluating the likelihood). Calling
$\mu$ and $R$ the mean and covariance matrix corresponding to the
right format of $h$ (ie, stacking information from all 4 experiments
'on top of each other'), we get the following distribution for the
likelihood,
\begin{align*}
\pi(h_{ode}=h_{d}\,|\,\alpha,\sigma,C_{d})=\frac{1}{\sqrt{(2\pi)^{n}|R|}}\exp\left\{ -\frac{1}{2}\left(h_{d}-h^{\text{ode}}-\mu\right)^{T}\cdotp R^{-1}\cdotp\left(h_{d}-h^{\text{ode}}-\mu\right)\right\} ,
\end{align*}
 where $n=t_{f}^{1}+t_{f}^{2}+t_{f}^{3}+t_{f}^{4}$ is the number
of data (ie, the dimension of $h_{d}$) and $h_{d}$ is defined as
\begin{align*}
h_{d}=\begin{bmatrix}h_{1}(t_{1}^{1})\\
h_{1}(t_{2}^{1})\\
\vdots\\
h_{1}(t_{f}^{1})\\
h_{2}(t_{1}^{2})\\
\vdots\\
h_{2}(t_{f}^{2})\\
\vdots\\
h_{4}(t_{f}^{4})
\end{bmatrix}.
\end{align*}
 Note that the times used in $h_{d}$ (and hence $h_{ode},\mu,R$)
can be a subset of the times used in the actual experiments. For instance,
the last measurements look highly suspicious and should be probably
excluded from our inference. $\mu$ and $R$ are defined as follows
\begin{align*}
\mu=\alpha\begin{bmatrix}t_{1}^{1}\\
t_{2}^{1}\\
\vdots\\
t_{f}^{1}\\
t_{1}^{2}\\
\vdots\\
t_{f}^{4}
\end{bmatrix},\quad R=\begin{bmatrix}R_{1} & 0 & 0 & 0\\
0 & R_{2} & 0 & 0\\
0 & 0 & R_{3} & 0\\
0 & 0 & 0 & R_{4}
\end{bmatrix}
\end{align*}
 where
\begin{align*}
R_{i}=\sigma^{2}\begin{bmatrix}1 & e^{-|t_{1}^{i}-t_{2}^{i}|/\tau} & e^{-|t_{1}^{i}-t_{3}^{i}|/\tau} & \ldots & e^{-|t_{1}^{i}-t_{f-1}^{i}|/\tau} & e^{-|t_{1}^{1}-t_{f}^{i}|/\tau}\\
\vdots & 1 & e^{-|t_{2}^{i}-t_{3}^{i}|/\tau} & \ldots & e^{-|t_{2}^{i}-t_{f-1}^{i}|/\tau} & e^{-|t_{2}^{i}-t_{f}^{i}|/\tau}\\
\\
\vdots &  & \ddots &  &  & \vdots\\
\\
 & \text{symm} &  &  & 1 & e^{-|t_{f-1}^{i}-t_{f}^{i}|/\tau}\\
\\
 &  & \ldots &  &  & 1
\end{bmatrix}
\end{align*}
 Because of the very specific form of our covariance matrix (block-diagonal),
we can simplify the expression for the likelihood as follows
\begin{align*}
 & \pi(h_{ode}=h_{d}\,|\,\alpha,\sigma,C_{d})=\ldots\\
 & \frac{1}{\sqrt{(2\pi)^{n}\prod_{i=1}^{4}|R_{i}|}}\exp\left\{ -\frac{1}{2}\sum_{i=1}^{4}\left(h_{d}-h^{\text{ode}}-\mu\right)_{i}^{T}\cdotp R_{i}^{-1}\cdotp\left(h_{d}-h^{\text{ode}}-\mu\right)_{i}\right\} ,
\end{align*}
 where
\begin{align*}
\left(h_{d}-h^{\text{ode}}-\mu\right)_{i}
\end{align*}
 corresponds to the data, computed results and mean for the $i^{\text{th}}$
experiment. This formulation clarifies the fact that we work experiment
per experiment as each experiment is independent from the others.

\begin{figure}[H]
\begin{centering}
\includegraphics[width=8cm]{figs/a_post}
\par\end{centering}

\caption{Prior and posterior probability function of parameter $\alpha$. }

\label{fig:Uc}
\end{figure}

\begin{figure}[H]
\begin{centering}
\includegraphics[width=8cm]{figs/s_post}
\par\end{centering}

\caption{Prior and posterior probability distribution function of parameter
$\sigma$.}

\label{fig:Uc}
\end{figure}

\begin{figure}[H]
\begin{centering}
\includegraphics[width=8cm]{figs/C1_post}
\par\end{centering}

\caption{Prior and posterior probability function of drag coefficient of bowling
ball $C_{d}1$. }

%Cd1

\label{fig:Uc}
\end{figure}

\begin{figure}[H]
\begin{centering}
\includegraphics[width=8cm]{figs/C2_post}
\par\end{centering}

\caption{Prior and posterior probability function of drag coefficient of basket
ball $C_{d}2$. }

%Cd1

\label{fig:Uc}
\end{figure}

\section{Independent Samples}

Generate samples of the joint posterior distribution of the calibration
parameters for use in performing the validation calculations. Instead
of using every sample in the MCMC chain, you may want to take every
$n^{th}$ sample in the chain, so the samples will be closer to independent.
Again be sure to document what you do. You will need enough nearly
independent samples to accurately perform the Monte Carlo integrals
in (3).

\subsubsection*{Solution}

We created samples of the joint posterior distribution for $C_{d},\sigma$
and $\alpha$ using MCMC. The correlation between the samples were
measured using the program `Autoregressive process modeling tools'
(https://github.com/RhysU/ar). Table \ref{table:creatingindesamples}
shows the number of steps for each of the parameters required so that
the samples are independent (correlation fall below a specified value).
$C_{d}$ has by far the longest correlation between steps, more than
30 chain steps before decorrelation.

An example of the arsel output is:
\begin{verbatim}
[nick@magus data]$ arsel -s < cd.dat | grep '#'
# absrho    true
# criterion CIC
# eff_N     32.751365129344251
# eff_var   0.12904055664629624
# gain      5.0428322668791719
# maxorder  512
# minorder  0
# mu        0.67116199032442436
# mu_sigma  0.062769462607289453
# N         1000
# AR(p)     71
# sigma2eps 0.024807597117980005
# sigma2x   0.12510055121028832
# submean   true
# T0        30.533078424386947
# window_T0 1
\end{verbatim}
Where $T_0$ presents the estimated envelope window of time of
correlation between samples.

We generated a set of samples 300,000 long, and downsampled to 10,000
uncorrelated samples for the integrals. We then confirmed that these
samples were uncorrelated using arsel on the downsampled chains. Later,
we found that these 10k uncorrelated samples were complete overkill with
respect to the integration. Only a few hundred samples were needed to
compute the posterior distributions shown below, which are generally
quite smooth (and no further smoothing function or KDE was utilized afterwards).

\begin{table}
\begin{centering}
\begin{tabular}{|c|c|}
\hline
Parameter & Chain steps to decorrelation\tabularnewline
\hline
\hline
$C_{d}$ & 30.533078\tabularnewline
\hline
$\sigma$ & 12.089690\tabularnewline
\hline
$\alpha$ & 11.870957\tabularnewline
\hline
\end{tabular}
\par\end{centering}

\caption{Creating independent samples.}

\label{table:creatingindesamples}
\end{table}

\section{Posterior of the observations}

Recall that the posterior distribution of an observation $y$, including
the uncertainty in the observation measurement is given by:

\begin{equation}
p\left(y|D,X\right)=\int_{\tilde{y}}\int_{\theta}p\left(y|\tilde{y},\theta,X\right)p\left(\tilde{y}|\theta,X\right)p\left(\theta|D,X\right)d\theta d\tilde{y}
\end{equation}

where $\tilde{y}$ is the unknown true value of the observable, and
$\theta$ is the vector of parameters. The conditional probabilities
in the integrand are:
\begin{itemize}
\item $p\left(y|\tilde{y},\theta,X\right)$ -- the probabilistic representation
of measurement error. It is conditioned on because of the calibration
parameters in the error model.
\item $p\left(\tilde{y}|\theta,X\right)$ -- this is the physical model,
which in our case is deterministic. Therefore, $p\left(\tilde{y}|\theta,X\right)=\delta\left(\tilde{y}-Y\left(\theta\right)\right)$
, where $Y\left(\theta\right)$ is the model predicted value of $\tilde{y}$
for given parameters , and $\delta\left(\right)$ is the Dirac delta
function. You made use of this fact when formulating the likelihood.
\item $p\left(\theta|D,X\right)$ is the posterior distribution of the parameters
obtained from the Bayesian calibration.
\end{itemize}
Because the model is deterministic, the $\tilde{y}$ integral can
be done trivially, further the samples of the parameters obtained
from the MCMC algorithm are samples from the posterior distribution
$p\left(\theta|D,X\right)$, so the $\theta$ integral can be performed
using a Monte Carlo method to obtain the approximation:

\begin{equation}
p\left(y|D,X\right)=\int_{\theta}p\left(y|Y\left(\theta\right),\theta,X\right)p\left(\theta|D,X\right)d\theta\approx\frac{1}{N}\sum_{j=1}^{N}p\left(y|Y\left(\theta_{j}\right),\theta_{j},X\right)
\end{equation}

where $\theta_{j}$ is the $j^{th}$ sample of $\theta$. Use this
Monte Carlo approximation, the samples from (2) and the measurement
error model developed for problem 2 to compute the posterior distribution
$p\left(y|D,X\right)$ for each observation of both balls considered
in problem 2. \textbf{Hint}: You can define a grid in $y$, with grid
points $y_{i}$ and evaluate the sum above for each $y_{i}$ . Be
sure to choose the grid spacing in $y$ to be fine enough to accurately
represent the distribution.

\subsubsection*{Solution}

In the general case, the posterior distribution of the observations~$y$
can be marginalized over the parameters~$\theta$ and the true value
of the observed quantity~$\tilde{y}$,
\begin{align*}
p(y|D,X)=\int_{\tilde{y}}\int_{\theta}p(y,\tilde{y},\theta|D,X)\, d\theta d\tilde{y}.
\end{align*}
 We can then apply Bayes' theorem twice to get
\begin{align*}
p(y|D,X) & =\int_{\tilde{y}}\int_{\theta}p(y|\tilde{y},\theta,D,X)p(\tilde{y},\theta|D,X)\, d\theta d\tilde{y}\\
 & =\int_{\tilde{y}}\int_{\theta}p(y|\tilde{y},\theta,D,X)p(\tilde{y}|\theta,D,X)p(\theta|D,X)\, d\theta d\tilde{y}.
\end{align*}
 We can now simplify this expression a bit more by noticing that the
first two conditional probability density functions~(pdf) do not
depend on the observed quantities~$D$. We end up with the expression
found in the problem statement
\begin{equation}
p(y|D,X)=\int_{\tilde{y}}\int_{\theta}p(y|\tilde{y},\theta,X)p(\tilde{y}|\theta,X)p(\theta|D,X)\, d\theta d\tilde{y}.\label{eq:postobsfull}
\end{equation}
 In this exercise, we did not assume any model inadequacy. Therefore,
the second pdf inside the integral is simply given by the deterministic
physical model, i.e.
\[
p(\tilde{y}|\theta,X)=\delta(\tilde{y}-Y(\theta)),
\]
 where $Y$ is the deterministic physical model. Then expression~\ref{eq:postobsfull}
simplifies more to
\begin{equation}
p(y|D,X)=\int_{\theta}p(y|Y(\theta),\theta,X)p(\theta|D,X)\, d\theta,\label{eq:postobs}
\end{equation}
 where $p(y|Y(\theta),\theta,X)$ is the observational model and $p(\theta|D,X)$
is the posterior distribution coming from the calibration.

Applying a Monte-Carlo approximation we evaluate integral~(\ref{eq:postobs})
thanks to the samples drawn in question~2. This gives
\begin{equation}
p(y|D,X)\approx\frac{1}{N}\sum_{i=1}^{N}p(y|Y(\theta_{i}),\theta_{i},X).\label{eq:postobsmc}
\end{equation}
 The variable~$y$ represents the observation and could be as such
understood as a vector of all the observations made. However multi-dimensional
model validation is cumbersome and instead we'll treat each observation
separately. For each ball, we had the results from two experiments.
Let's number these experiments and call the observations $y^{1}(t),y^{2}(t),y^{3}(t),y^{4}(t)$.
In each experiment, the ball position was only recorded at a discrete
number of times, we in fact we are dealing with the data
\[
y^{1}(t_{1}^{1}),y^{1}(t_{2}^{1}),\ldots,y^{1}(t_{N_{1}}^{1}),y^{2}(t_{1}^{2}),\ldots,y^{2}(t_{N_{2}}^{2}),\ldots,y^{4}(t_{N_{4}}^{4}).
\]
 And therefore for any $i=1,\ldots,4$ and $j=1,\ldots,N_{i}$, let
$y^{i,j}:=y^{i}(t_{j}^{i})$. We then want to evaluate
\[
p(y^{i,j}|D,X)\approx\frac{1}{N}\sum_{s=1}^{N}p(y^{i,j}|Y(\theta_{s}),\theta_{s},X).
\]
 We are going to evaluate that sum on a grid. For storage efficiency,
each observation will have its own grid centered around a reasonable
value (to be defined later). The grid is then defined as
\[
y_{k}^{i,j}=y_{c}^{i,j}+k\Delta y,\quad\forall k\in\{-m^{i,j},m^{i,j}\}.
\]
 For a given $i=1,\ldots,4$ and $j=1,\ldots,N_{i}$, we are going
to store the values of
\[
\frac{1}{N}\sum_{s=1}^{N}p(y^{i}(t_{j}^{i})=y_{k}^{i,j}|Y(\theta_{s}),\theta_{s},X),\quad\forall k\in\{-m^{i,j},m^{i,j}\}.
\]
 We can now write down explicitly the formula for that quantity. In
the group project~\#2, we defined the measurement error as a multivariate
Gaussian with mean and covariance depending on the calibration parameters~$\alpha$
and $\sigma$. But here we only care about the marginalized distribution
of a single observation. Fortunately, marginalized distributions of
Gaussian are also Gaussian with mean and covariance being given by
the restriction of the multivariate mean and covariance to the variables
of interest. In our case, we end up with a Gaussian of mean $\alpha t_{j}^{i}$
and covariance $\sigma^{2}$. Let's call $y_{c}^{i,j}$ the computed
solution of the ball drop without noise. We can therefore write
\[
y^{i,j}=y_{c}^{i,j}+\varepsilon,
\]
 with $\varepsilon\sim\mathcal{N}(\alpha t_{j}^{i},\sigma^{2})$.
Then we store at each grid point
\[
\frac{1}{N}\frac{1}{\sqrt{2\pi}\sigma}\sum_{s=1}^{N}e^{-(y_{k}^{i,j}-y_{c}^{i,j}-\alpha t_{j}^{i})^{2}/2\sigma^{2}},\quad\forall k\in\{-m^{i,j},m^{i,j}\}.
\]

\section{Credibility}

For each observation, compute the the credibility $1-\beta$, where
the datum is the maximum or minimum value of a $\beta-$credibility
set. You may use either an HPD or quantile credibility set. Plot these
credibility values as a function of time for the four ball drops considered.
\textbf{Hint}: you can approximate the required integrals using the
grid of values obtained in 3) and a quadrature rule, such as trapezoidal
rule.

\subsubsection*{Solution}

The credibility of a given distribution is defined as alpha = 1 \textendash{}
beta, where beta = integral(P(y)dy). In this project, we made the
decision to
use the HPD method to compute the credibility. HPD was selected because
we did not know if the distributions would be skewed (in reality, they
did not appear to have been strongly skewed) and HPD would provide a
more robust estimate than the quartile analysis in this case.
In the figures below, we
plot the credibility as a function of time for the 4 sets of measurements
(2 basket ball and 2 bowling ball). As one can clearly observe from
the figures, the data seems to be credible (wrt the model) only for
the first fractions of a second and then becomes incredible later on. In other
words, this means the predictions of the model are quickly inconsistent with the
observations. This figure depicts the credibility interval for the first
data set with the basketball, but all data sets and balls were
qualititively similar.

\begin{figure}[H]
\begin{centering}
\includegraphics[width=8cm]{figs/credibility}
\par\end{centering}
\caption{Credibility as a function of time.}

\label{fig:Uc}
\end{figure}

For all the datasets, after 1.5 seconds, the HPD credibility is assessed
as exceeding low, at less than $1e-5$. Around 2.5 seconds, every data
set is completely implausible, with credibilities assessed at less than
$1e-10$. In summary, the credibility decreases as a function of time,
and after a few seconds, the predictions of our model are completely
inconsistent with the observations.

\section{Validation test}

Choose two or more data points for each drop (including those with
the highest and lowest credibility), and plot the predicted posterior
distribution of the observation $p\left(y|D,X\right)$ along with
the value of the actual observation.

\subsubsection*{Solution}

The plots below show the best and the worst credibility intervals
along with credibility at few other time steps. The cases where $\beta$
falls within distribution give credence that the model is plausible \textendash{}
i.e. the model prediction is not inconsistent with the data. For the
values far outside the distribution, the model prediction is
inconsistent with the data, which certainly implies that we cannot trust
the model predictions to match those of observations. This is evidence
for an invalidation of the model, but further investigations would be
necessary to actually determine what is the ultimate cause of the
inconsistency (such as overly optimistic observation error bounds,
inaccurate scenario measurements, etc.).

\subsection{Ball drop - 1 (Basket Ball) }

\begin{figure}[H]
\begin{centering}
\subfigure[]{\includegraphics[width=6cm]{figs/Basket_Ball_1_5}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Basket_Ball_1_20}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Basket_Ball_1_35}}
\par\end{centering}

\caption{Predicted posterior distribution at time (a) $t=5$ (b) $t=20$ and
(c) $t=35$ secs vs the actual observation}

\label{balldrop1t5}

\end{figure}

\begin{figure}[H]
\begin{centering}
\subfigure[]{\includegraphics[width=6cm]{figs/Basket_Ball_1_maxbeta22}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Basket_Ball_1_minbeta1}}
\par\end{centering}

\caption{Predicted posterior distribution at (a) maximum $\beta$ (b) minimum
$\beta$}

\label{balldrop1t5-3}
\end{figure}

\subsection{Ball drop - 2 (Basket Ball)}

\begin{figure}[H]
\begin{centering}
\subfigure[]{\includegraphics[width=6cm]{figs/Basket_Ball_2_5}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Basket_Ball_2_20}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Basket_Ball_2_35}}
\par\end{centering}

\caption{Predicted posterior distribution at time (a) $t=5$ (b) $t=20$ and
(c) $t=35$ secs vs the actual observation}

\label{balldrop1t5-1}
\end{figure}

\begin{figure}[H]
\begin{centering}
\subfigure[]{\includegraphics[width=6cm]{figs/Basket_Ball_2_maxbeta40}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Basket_Ball_2_minbeta10}}
\par\end{centering}

\caption{Predicted posterior distribution at (a) maximum $\beta$ (b) minimum
$\beta$}

\label{balldrop1t5-3-1}
\end{figure}

\subsection{Ball drop - 1 (Bowling Ball)}

\begin{figure}[H]
\begin{centering}
\subfigure[]{\includegraphics[width=6cm]{figs/Bowling_Ball_1_5}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Bowling_Ball_1_20}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Bowling_Ball_1_35}}
\par\end{centering}

\caption{Predicted posterior distribution at time (a) $t=5$ (b) $t=20$ and
(c) $t=35$ secs vs the actual observation}

\label{balldrop1t5-2}
\end{figure}

\begin{figure}[H]
\begin{centering}
\subfigure[]{\includegraphics[width=6cm]{figs/Bowling_Ball_1_maxbeta24}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Bowling_Ball_1_minbeta0}}
\par\end{centering}

\caption{Predicted posterior distribution at (a) maximum $\beta$ (b) minimum
$\beta$}

\label{balldrop1t5-3-2}
\end{figure}

\subsection{Ball drop - 2 (Bowling Ball)}

\begin{figure}[H]
\begin{centering}
\subfigure[]{\includegraphics[width=6cm]{figs/Bowling_Ball_2_5}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Bowling_Ball_2_20}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Bowling_Ball_2_35}}
\par\end{centering}

\caption{Predicted posterior distribution at time (a) $t=5$ (b) $t=20$ and
(c) $t=35$ secs vs the actual observation}

\label{balldrop1t5-4}
\end{figure}

\begin{figure}[H]
\begin{centering}
\subfigure[]{\includegraphics[width=6cm]{figs/Bowling_Ball_2_maxbeta20}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Bowling_Ball_2_minbeta0}}
\par\end{centering}
\caption{Predicted posterior distribution at (a) maximum $\beta$ (b) minimum
$\beta$}

\label{balldrop1t5-3-3}
\end{figure}

\section{Consistency of the model}

Given the results of the above validation calculations, is the model
consistent with the observations for all drops? If not, what might
be wrong?

\subsubsection*{Solution}

From the above results, the model is clearly not consistent with the
observations for all ball drops. Rather, the model is accurate for
a finite time horizon, after which the predictions are no longer accurate.
Possible reasons for the model failure:
\begin{enumerate}
\item The assumptions made in the model may not be accurate. For
      instance, the ball may not be starting at rest, but with a
      non-zero downward velocity.
\item The model may be inadequate, without rich enough physics to
      capture all the significant dynamics.
\item The measurement data may not be reliable
\item The scenario parameters may have been inaccurate: for instance,
      the height of the bridge, or the size of the basketball.
\end{enumerate}

In addition, we note that the model appears to be systematically
overestimating the balls drop rate. In other words, it appears as if our
predictions are consistently projecting the ball to fall farther than it
actually does. This hints that our drag model may be at fault, because
it is not reducing the rate at which the ball falls as much as will in
reality.

\end{document}


\lab{Data Visualization}{Data Visualization}
\objective{Use data visualizations to explore data and communicate it to others.}
\label{lab:DataVis}

\noindent\emph{The purpose of visualization is insight, not pictures.} \small{---Card, Mackinlay, \& Shneiderman (after Hamming)}

\medskip

\noindent\emph{The ability to take data---to be able to understand it, to process it,
to extract value from it, to visualize it, to communicate it---that's
going to be a hugely important skill in the next decades,... because
now we really do have essentially free and ubiquitous data. So the
complimentary scarce factor is the ability to understand that data and
extract value from it.} \small{---Hal Varian (Google's Chief Economist)}
% in The McKinsey Quarterly, Jan 2009

\section*{What is data visualization?}
Data visualizations (or graphs) are used to understand and explore data as well as communicate results to others.  Often data is more easily interpreted or understood in graphical format than as a list of numbers or data points.

\section*{Types of visualizations}

Here are some of the most useful types of data visualizations.  We explain each of these in more detail below.

\begin{enumerate}

\item A \emph{line plot} plots $(x,y)$-pairs as points and connects them with a line.
You should use a line plot for plotting continuous functions or when there is a natural order to your two-dimensional data---for example when the data is a time series (a sequence of values over time).

\item A \emph{bar chart} is generally the preferred method for displaying a small (discrete) collection of one-dimensional data points. Each discrete data point is represented as a bar whose length is determined by the value of the data.

\item A \emph{histogram} is a special type of  bar graph where the length of each bar corresponds to the number of data points in a certain range (sometimes called a \emph{bin}).
(see Figure \ref{fig:healthcare}).
Histograms are useful for revealing statistical distributions.

\item A \emph{scatter plot} plots $(x,y)$ tuples as discrete points.  This should be used instead of line plot when the data is two-dimensional but has no natural order.

A scatter plot can reveal correlation (or lack thereof) between $x$ and $y$.

\item Three dimensional data can be displayed on a two-dimensional page with a \emph{contour plot}.  This draws lines in the plane where the third value is constant---like a geological map.  Filling in the contours with successive colors gives a  \emph{heat map}.

\item A \emph{pseudocolor plot} or \emph{heat map} uses color to display a third dimension on a two-dimensional page (see Figure \ref{fig:heatmap}).

Three-dimensional data can also be plotted as a surface in 3-space, but it is often difficult to find a view where none of the important features of the data are obscured.
A contour plot or a heatmap can help avoid this problem and is often easier for the viewer to decode.

%
%
%\item A \emph{sparkline} is a word-sized graphic that is often embedded inline with the text.
%Created by Edward Tufte, an innovator and recognized leader in the field of data visualizations, sparklines can be used to quickly communicate large-scale trends in a data set. They are also ideal for binary data sets.
%Two sources for creating sparkplots in Python are \cite{sparkplots} and \cite{sparklineshtml}.
%%TODO: Add an example of a sparkline
%
%
%\item \emph{Small multiples} are several minature graphics of the same type in a single visualization.
%Like sparklines, they were pioneered by Edward Tufte. Any of the plots discussed so far may be assembled into a small multiple.
%Small multiples are useful for viewing trends over time, when the data associated to each time is more complex than a single number.
%More generally, they can be used to compare many sets of similar data (\cite{tufte1990} p. 67).
%Figure \ref{fig:log_plots} could be called a small multiple, though frequently they contain more subplots.
%Recall that this kind of plot can be created in matplotlib with the command \li{plt.subplot()}.
\end{enumerate}

\subsection*{Bar Charts}

Bar charts are best for relatively small sets of discrete, one-dimensional data.  They are usually best presented with the bars running horizontally because when the bars are vertical the labels tend to run together (and the other possible solution to that problem---making the text vertical---is harder to read) as seen in Figure \ref{fig:barchart}.

\begin{figure}[h]
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{bar_chart_horizontal_sorted.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{bar_chart_unsorted.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{bar_chart_vertical_bars_horizontal_labels.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{bar_chart_vertical_bars_vertical_labels.png}
\end{subfigure}

\caption{Bar Chart Examples}
\label{fig:barchart}
\end{figure}

The data in a bar chart should generally be sorted in some natural way, based on the intended use.  If the chart will be used to compare the value of different data points, we usually sort by size; whereas if the chart will be used for looking up specific values, the data could be sorted alphabetically or in some other logical order that facilitates finding specific values.

As a general rule, bars should always start at zero because people are conditioned to expect that.  Failing to do this can be confusing or misleading.

\begin{problem}

The following table provides average heights of men and women in 17 countries. Order the data and plot a horizontal bar chart.

\begin{tabular}{ | l || l | l | }
\hline
Country & Average Male Height & Average Female Height \\
\hline
Austria & 179.2 cm (5 ft 7.5 in) & 167.6 cm (5 ft 6 in) \\
Bolivia & 160.0 cm (5 ft 3 in) & 142.2 cm (4 ft 8 in) \\
England & 177.8 cm (5 ft 10 in) & 164.5 cm (5 ft 5 in) \\
Finland & 178.9 cm (5 ft 10.5 in) & 165.3 cm (5 ft 5 in) \\
Germany & 178 cm (5 ft 10 in) & 165 cm (5 ft 5 in) \\
Hungary & 176 cm (5 ft 9.5 in) & 164 cm (5 ft 4.5 in) \\
Japan & 172.5 cm (5 ft 7.5 in) & 158 cm (5 ft 2 in) \\
North Korea & 165.6 cm (5 ft 5 in) & 154.9 cm (5 ft 1 in) \\
South Korea & 170.8 cm (5 ft 7 in) & 157.4 cm (5 ft 2 in) \\
Montenegro & 183.2 cm (6 ft 0 in) & 168.4 cm (5 ft 6.5 in) \\
Norway & 182.4 cm (6 ft 0 in) & 168 cm (5 ft 6 in) \\
Peru & 164 cm (5ft 4.5 in) & 151 cm (4 ft 11.5 in) \\
Sri Lanka & 163.6 cm (5 ft 4.5 in) & 151.4 cm (4 ft 11.5 in) \\
Switzerland & 175.4 cm (5 ft 9 in) & 164 cm (5 ft 4.5 in) \\
Turkey & 174 cm (5 ft 8.5 in) & 158.9 cm (5 ft 2.5 in) \\
U.S. & 176.1 cm (5 ft 9.5 in) & 162.1 cm (5 ft 4 in) \\
Vietnam & 165.7 cm (5 ft 5 in) & 155.2 cm (5 ft 1 in) \\
\hline
\end{tabular}
\label{prob1}
\end{problem}

\subsection*{Histograms}

A histogram depicts a distribution of numerical data. They can be used to look for trends over time and to investigate statistical distributions.
Histograms can give a clear view of the way data is distributed, but sometimes changing the size and position of the bins (a small interval of that data) can change the picture substantially.

You can create a histogram in matplotlib with the command \li{plt.hist()}.

When plotting histograms, it can sometimes be useful to include reference lines.  A nice way to do this without cluttering the figure is to make the reference lines white.  You can do this in matplotlib with the command \li{plt.grid(True, color='w',linestyle='-')}.

%TODO example of a basic histogram.  Be sure to show the data so the reader can see how this is different from a standard bar.  Also give an example of the same data with bins a different size that gives a different picture.

\begin{lstlisting}
from numpy.random import randn
from matplotlib import pyplot as plt

data = randn(75) + randn(75) + randn(75) + randn(75)
plt.hist(data)
plt.grid(True, color = 'w', linestyle = '-')

plt.title("histogram)
plt.show()

\end{lstlisting}

\begin{figure}
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{histogram.png}
\end{subfigure}
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{histogram_referenceLines.png}
\end{subfigure}
\caption{Histogram and Histogram with Reference Lines}
\label{fig:histogram}
\end{figure}

\begin{problem}
Using 1000 random numbers from a Gaussian distribution,
\begin{lstlisting}
from numpy.random import normal
gaussian_numbers = normal(size=1000)
\end{lstlisting}
plot a histogram with white reference lines. Vary the number of bins and compare.

\end{problem}

\subsection*{Line Plot}

Line plots should not be used for data that has no natural ordering or progression because it gives the misimpression that there is an order or progression when there is none (see Figure \ref{fig:lineplot_bad}.

Recall that you can create a line plot in matplotlib with the command \li{plt.plot()}.

When plotting line graphs in matplotlib, the default line width is usually a little too thin.  It will help your reader (or you) if you set the line width to 2 or more with the argument \li{lw=2}.

%\li{plt.plot(x,y,lw=2)}

\begin{figure}
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{line_plot_bad_thin.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{line_plot_bad_thick.png}
\end{subfigure}
\caption{Line Plot with previous bar chart data}
\label{fig:lineplot_bad}
\end{figure}

\begin{problem}
Using 1000 subintervals, plot the equation

$$y = x^2np.sin(x)$$,
$$x \in [0,100]$$

It should look like Figure \ref{fig:lineplot}.
\end{problem}

\begin{figure}
\centering
\begin{subfigure}{.5\textwidth}
\centering
\includegraphics[width=\textwidth]{line_plot.png}
\end{subfigure}
\caption{solution to Problem 3}
\label{fig:lineplot}
\end{figure}

\subsection*{Scatter Plots}

Scatter plots should be used instead of line plot when the data is two-dimensional but has no natural order.

You can create a scatter plot in matplotlib with the command \li{plt.scatter()}.

\begin{figure}[h]
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{scale_scatter_zoomed_in.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{scale_scatter_zoomed_out.png}
\end{subfigure}
\caption{scatter plots with and without a strong correlation}
\label{fig:scatter_correlation}
\end{figure}

Figure \ref{fig:scatter_correlation} displays two scatter plots, the first appearing to have no strong correlation and the second a strong correlation. However, the same data is being plotted and the only difference is the scale and window size. Manipulating these can change your interpretation and should be done only after careful consideration.

\begin{problem}
Plot the average heights of men against women using the data from Problem \ref{prob1}. Zoom in and zoom out to see what difference that makes.
\end{problem}

\subsection*{Contour Plots and Heatmaps}

Three dimensional data can be displayed on a two-dimensional page with a \emph{contour plot}.  This draws lines in the plane where the third value is constant---like a geological map.

You can create a contour plot in matplotlib with
\li{plt.contour()}.

Filling in the contours with successive colors gives a  \emph{heat map}.

Note that the default color map for matplotlib is the so-called \emph{rainbow color map}.  This is a very poor choice for a heat map or colored contour map because people do not naturally interpret one color to be greater than another.  To solve this problem in matplotlib, use the argument \li{cmap='NAME\_OF\_COLORMAP'}
where \li{NAME\_OF\_COLORMAP} is one of matplotlib's sequential or diverging colormaps.

\begin{lstlisting}
import numpy as np
from matplotlib import pyplot as plt

n=400
xrange = np.linspace(-1.5,1.5,n)
yrange = np.linspace(-1.5,1.5,n)
X, Y = np.meshgrid(xrange,yrange)
F = Y**2 - X**3 + X**2
plt.contourf(X, Y, F, [-2,-1,0.0001,1,2,3,4,5] ,cmap=plt.get_cmap('afmhot'))
\end{lstlisting}

\begin{figure}[h]
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{contour_map.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{contour_map_filled.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{contour_map_rainbow_filled.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{contour_map_rainbow.png}
\end{subfigure}
\caption{Contour Maps}
\label{fig:contour}
\end{figure}

TODO add in the list (and pix) of sequential and diverging colormaps from \url{http://matplotlib.org/examples/color/colormaps_reference.html}

\begin{problem}
Using 400 intervals, plot a filled contour map of

$$z = sin(x) + sin(y)$$,$$ x\in[0,12\pi], y\in[0,12\pi]$$.

Be sure to use an appropriate color map.
\end{problem}

\begin{figure}
\centering
\begin{subfigure}{.5\textwidth}
  \centering
  \includegraphics[width=\textwidth]{heatmap_color.png}
\end{subfigure}%
\begin{subfigure}{.5\textwidth}
  \centering
  \includegraphics[width=\textwidth]{heatmap_hot.png}
\end{subfigure}
%\includegraphics[width=\textwidth]{heatmap.png}
\caption{Both of these pseudocolor plots depict the function $z = \sin(x)\sin(y)$ on the domain $[-1,4] \times [-4,1]$.
The plot at left uses a rainbow gradient which is pretty to look at but whose choice of colors has no meaningful relationship to the data.
The plot at right uses the colormap called \li{afmhot}, which has the important benefit that the lighter colors are naturally associated with higher values, while the darker colors are naturally associated with lower values.}
\label{fig:heatmap}
\end{figure}

You can also create a pseudocolor plot in matplotlib with the \li{plt.pcolormesh()} command.

One situation where heatmaps are very useful is when data is naturally associated to a geographic location.  When a heatmap is plotted on a map it is often called a \emph{cloropleth}.
\begin{comment}
TODO example of a cloropleth---maybe percentage income donated to charity by county? (makes Utah look good).

As in the case of a histogram, if the data is relatively coarse then the way the data is divided up can have an effect on the overall picture of the heatmap.

TODO make an example where things are binned in two different ways  and give a different-looking result.

\end{comment}

Finally, three-dimensional data can also be plotted as a surface in 3-space, but it is often difficult to find a view where none of the important features of the data are obscured.
A heatmap can help avoid this problem and is often easier for the viewer to decode.

TODO give example of how the surface can obscure your view and how the contour or heatmap gives a clearer picture.

\section*{Exploring data with visualizations}
When you visualize data you may notice trends that are not apparent from the numbers.
The following problem is a famous example of this phenomenon.

\begin{problem}\label{prob:anscombe}
The data sets I-IV in Table \ref{table:anscombe} are known as Anscombe's quartet.
Each dat set has identical statistical properties.
In each case,
\begin{itemize}
\item The mean of $x$ is 9 and the mean of $y$ is $7.5$.
\item The variance of $x$ is 11 and the variance of $y$ is 4.127.
\item The correlation between $x$ and $y$ is .816.
\item The linear regression line is $y=3+5x$.
\end{itemize}
Plot each data set. What do you notice?

\begin{table}[H]
\begin{tabular}{l l  |  l l  |  l l  |  l l }
I & & II & & III & & IV\\
x & y & x & y & x & y & x & y \\
\hline
10.0 & 8.04 & 10.0 & 9.14 & 10.0 & 7.46 & 8.0 & 6.58 \\
8.0 & 6.95 & 8.0 & 8.14 & 8.0 & 6.77 & 8.0 & 5.76 \\
13.0 & 7.58 & 13.0 & 8.74 & 13.0 & 12.74 & 8.0 & 7.71 \\
9.0 & 8.81 & 9.0 & 8.77 & 9.0 & 7.11 & 8.0 & 8.84 \\
11.0 & 8.33 & 11.0 & 9.26 & 11.0 & 7.81 & 8.0 & 8.47 \\
14.0 & 9.96 & 14.0 & 8.10 & 14.0 & 8.84 & 8.0 & 7.04 \\
6.0 & 7.24 & 6.0 & 6.13 & 6.0 & 6.08 & 8.0 & 5.25 \\
4.0 & 4.26 & 4.0 & 3.10 & 4.0 & 5.39 & 19.0 & 12.50 \\
12.0 & 10.84 & 12.0 & 9.13 & 12.0 & 8.15 & 8.0 & 5.56 \\
7.0 & 4.82 & 7.0 & 7.26 & 7.0 & 6.42 & 8.0 & 7.91 \\
5.0 & 5.68 & 5.0 & 4.74 & 5.0 & 5.73 & 8.0 & 6.89 \\
\end{tabular}
\caption{These four sets of data are known as Anscombe's quartet.}
\label{table:anscombe}
\end{table}
\end{problem}

As Problem \ref{prob:anscombe} demonstrates, a picture can quickly reveal properties of the data that are difficult to see in a list of numbers.

\subsection*{The Iterative Process of Exploratory Visualization}

Understanding a data set through visualization is often an iterative process. We start with an initial visualization---usually a broad overview.  Examining the result leads to observations that provoke questions.  We then filter or transform the data and adjust the visualization based on those questions and observations, and then repeat the process until we converge on a useful result.

Some of the questions that you should be asking in this iterative process include:
\begin{enumerate}
\item Are the data correct and reliable?
\item Are there any clear patterns, trends, or irregularities?
\item Would a different type of visualization give more information?
\item Is something unusual or interesting going on outside my view?
\item Would filtering the data to look at a particular subset give a more informative picture?
\item Would a transformation of the data give a more informative picture?
\end{enumerate}

\subsubsection*{Data integrity}

Visualization is a powerful way to identify problems with your data set.  Ask yourself, whether the results look like what you expect.  Are they the right magnitude? Are there missing values?

\subsubsection*{Trends, patterns, and irregularities}

Some patterns you may look for:
\begin{enumerate}
\item Are there trends in time (up, down, flat, cyclical)?
\item Does the data look random, cyclical, or patterned?
\item Are there clear correlations between the $x$- and $y$- values?
\item Is the trend linear or curved?
\item Do lines intersect, or are they parallel?
\item Are the data symmetric or skewed?
\item What is the variance (wide vs narrow)?
\item Are there clusters and gaps?
\item Are the data dense or sparse?
\end{enumerate}

\subsubsection*{Using different types of visualization}

Data can be visualized using different types of visualization and is relatively easy to transform.

\begin{problem}
This data

\begin{tabular}{ | l || l | l | }
\hline
Number of students playing each instrument \\
\hline
cello & 14 \\
clarinet & 41 \\
flute & 30 \\
guitar & 15 \\
piano & 53 \\
saxophone & 18 \\
violin & 22 \\
\hline
\end{tabular}

 is visualized in Figure \ref{fig:pie} as a pie chart.

Choose another type of visualization and plot the data again. What are the relative merits of your plot compared to the pie chart?

\end{problem}

\begin{figure}
\centering
\begin{subfigure}{.5\textwidth}
  \centering
  \includegraphics[width=\textwidth]{pie_instruments.png}
\end{subfigure}%
\caption{Pie Chart for Instrument Problem}
\label{fig:pie}
\end{figure}
\subsubsection*{Changing the view}

By choosing to plot only a portion of your data, you may be able to focus your attention on the most relevant data.  On the other hand, you may also miss some interesting behavior in the data you didn't graph. Figures \ref{fig:scatter_correlation} and \ref{fig:line_scale} are two such examples.

Another issue to consider is scale. If a number changes from 50,000 to 55,000 over some period, that  increase can appear small or large, depending on where we start the $y$-axis and how long we stretch out the $x$-axis (see Figure \ref{fig:dog_plots}).

You can modify the scale of an existing plot using \li{plt.yscale()} or \li{plt.xscale()}.

\begin{figure}
\centering
\begin{subfigure}{.5\textwidth}
  \centering
  \includegraphics[width=\textwidth]{scale_line_zoomed_in.png}
\end{subfigure}%
\begin{subfigure}{.5\textwidth}
  \centering
  \includegraphics[width=\textwidth]{scale_line_zoomed_out.png}
\end{subfigure}
\caption{Scale Issues}
\label{fig:line_scale}
\end{figure}

\begin{problem}
Return to your plot of

$$y = x^2sin(x)$$ $$x\in[0,100]$$

from problem 3 and plot again a few times adjusting the scale.
\end{problem}

\subsubsection*{Filtering the data}

Sometimes important information about a subset of the data can be obscured by considering the whole data set.
\begin{comment}
TODO example where scatterplot of all people looks random, but separating by something like education level shows clear trends for one level.
\end{comment}

\subsubsection*{Transforming the data}

Often a transformation of the data can reveal useful information that was previously hidden.

For example, data that grows very rapidly or that is tightly clustered may be better visualized by plotting the logarithm of the data, instead of the original data, A log-lin plot (also called a log plot) is constructed by by taking the logarithm of only the $y$-values of your data set.

This is easily done in matplotlib with the command \li{plt.semilogy(x,y)}.  The syntax is the same as \li{plt.plot}.

On the left side of Figure \ref{fig:exp}, $x^{14}$ and $x^{13}$ look almost idential. Graphic them on a log-lin plot (as on the right side of Figure \ref{fig:exp}), their differences become apparent.

\begin{figure}[h]
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{exponential.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{exponential_loglin}
\end{subfigure}
\caption{Plotting exponential graphs on a log-lin plot}
\label{fig:exp}
\end{figure}

Similarly, if the data grows very slowly, you may find that a lin-log plot (given by by taking the logarithm of only the $x$-values) gives a more useful view.
This is easily done in matplotlib with the command \li{plt.semilogx(x,y)}

Transformations can also be useful for histograms.
As an example, let us look at the cost of health care claims.
Our data set is bogus but created to closely resemble real-life data.
An initial plotting of the data produces the histogram on the left in Figure \ref{fig:healthcare}.
From this picture, it is hard to see the nature of the data---the plot just looks like a single bar.
We may think that all health care claims are cheap.
However, when we take the logarithm of the claims prices, we get the histogram on the right in Figure \ref{fig:healthcare}.
Graphed on this scale, the data has a bell-shaped distribution similar to a normal distribution.
Moreover, we see that there are some very expensive claims being submitted, though they are few.
\begin{figure}
\centering
\begin{subfigure}{.5\textwidth}
  \centering
  \includegraphics[width=\textwidth]{healthcare_linscale.pdf}
\end{subfigure}%
\begin{subfigure}{.5\textwidth}
  \centering
  \includegraphics[width=\textwidth]{healthcare_logscale.pdf}
\end{subfigure}
\caption{The same data (meant to resemble the prices of health care claims) is plotted in both histograms above.
The plot at left is a lin-lin scale, whereas the plot at right is a log-lin scale.
In this case, changing scales revealed important information about the data.}
\label{fig:healthcare}
\end{figure}

As a general rule, log-lin plots are useful when the range of the $y$-values is orders of magnitude larger than the range of the $x$-values.
Our health care data was an example of this.
Similarly, lin-log plots are useful when the range of the $x$-values is much bigger than the range of the $y$-values.
In any case, you can try applying a log scale to one or both of your axes as a way to explore your data.

Let us analyze the mathematics behind log plots.
Suppose we have some data that roughly follows the line $y=cx^a$ where $c$ and $a$ are constants.
Taking the logarithm of both sides yields
\[
\log(y) = a\log(x) + \log(c).
\]
If we set new variables $X = \log(x)$ and $Y = \log(y)$, then this equation becomes $Y = aX + \log(c)$, which is the equation of a straight line.
So  on a log-log plot, polynomial-shaped data looks linear.
Similarly, taking the log of $y = a^x$ yields
\[
\log(y) = \log(a)x,
\]
so exponential data looks linear on a log-lin plot.
Finally, data of the form $y = a\log(x)+b$ will look linear on a lin-log plot.
See Figure \ref{fig:log_plots}.

\begin{figure}
\centering
\includegraphics[width=\textwidth]{log_plots.pdf}
\caption{Here are some functions plotted on various logarithmic and linear scales.
The dark blue line is $y=x$, the green line is $y=\log(x)$, the red line is $y=e^x$, and the light blue line is $y=x^3$.
We have used different ranges on the $y$-axes to highlight appropriate parts of the graphs.}
\label{fig:log_plots}
\end{figure}

Be warned that many different functions can look linear on log plots.
Thus, if you graph your data on a log-log plot and you see a line, you CANNOT conclude that your data must follow a polynomial.
In general, you need more information.

\begin{problem}
Graph your data from Problem 1 in the Complexity and Sparse Matrices lab on a (a) log-log scale, (b) log-lin scale, and (c) lin-log scale.
Which plot makes the data easiest to understand?

In this problem you will analyze the complexity of the following functions.

\begin{enumerate}
\renewcommand{\theenumi}{\alph{enumi}}
\item (Optional)
\begin{lstlisting}
# Single ``for'' loop
def func1(n):
    n = 500*n
    sum(xrange(n))
\end{lstlisting}

\item
\begin{lstlisting}
# Double ``for'' loop
def func2(n):
    n = 3*n
    t = 0
    for i in xrange(n):
        for j in xrange(i):
            t += j
\end{lstlisting}

\item
\begin{lstlisting}
# Square a matrix
def func3(n):
    n = int(1.2*n)
    A = np.random.rand(n, n)
    np.power(A, 2)
\end{lstlisting}

\item
\begin{lstlisting}
# Invert a matrix
from scipy import linalg as la
def func4(n):
    A = np.random.rand(n, n)
    la.inv(A)
\end{lstlisting}

\item (Optional)
\begin{lstlisting}
# Find the determinant of a matrix.
from scipy import linalg as la
def func5(n):
    n = int(1.25*n)
    A = np.random.rand(n, n)
    la.det(A)
\end{lstlisting}
\end{enumerate}

Do this as follows:
\begin{enumerate}
\item Time how long each function takes to run on an input of size $n$ for $n=100, 200, 400$, and $800$.
\item Graph your data as follows. Note that we rescaled the inputs to the functions so that your lines would begin at roughly the same place on the y-axis.
\begin{enumerate}
\item For each function, plot a line using $n=100, 200, 400$, and $800$ for $x$-values and your runtimes for $y$-values. Make sure your $y$-values are all in the same units! When you call the function \li{plt.plot()}, specify a value for the keyword parameter \li{label} so you can reference each line later. For example, the call \li{plt.plot( x, y, label="Function 1")} attaches the label "Function 1" to this line.
\item After plotting all your lines, call \li{plt.legend()} to draw a legend on your graph. Use the keyword argument \li{loc} to specify the location of the legend. Some possible values are \li{'upper left'} and \li{'right'}. See the documentation for more options. Your final graph should look something like the figure below.\footnote{From this graph, you can see that inverting a matrix is an incredibly time consuming operation. In Lab \ref{lab:ChangeBasis} you will learn fast algorithms for solving linear systems without inverting matrices. In fact, inverting matrices is so costly in terms of temporal and spatial complexity that there is hardly ever a good reason to do it.}
\end{enumerate}
\item For each function, discuss how its spatial and temporal complexity could have contributed to its runtime.
\end{enumerate}

\end{problem}

\begin{problem}
Given the following data,

\begin{tabular}{| l | l | l | l | l | l | l |}
\hline
male & female & age & marital status & college & pet owner & employed \\
\hline
Y & N & 21 & S &  N & N & N \\
N & Y & 25 & M & Y & Y & Y\\
Y & N & 26 & M & Y & Y & Y\\
Y & N & 20 & S & Y & N & Y\\
Y & N & 27 & M & Y & N & Y\\
N & Y & 22 & S & N & N & N\\
Y & N & 28 & M & Y & Y & Y\\
N & Y & 29 & M &  N & N & N\\
N & Y & 23 & S &  N & N & N\\
N & Y & 28 & M & N & N & Y\\
Y & N &  24 & S & Y & Y & Y\\
\end{tabular}

explore it. Plot it in different ways, zoom in and out, and do a transformation.
\end{problem}

\section*{Communicating data with visualizations}

As the saying has it, ``a picture is worth a thousand words.''
Certainly, a data visualization is far more valuable than a thousand data points to a colleague who wants to understand the results of your analysis.
Data visualizations are critical for communication in both business and research.

\subsection*{Choosing the right visualization}
At this point, you have already used visualization to explore your data and draw conclusions, and you are ready to tell your results to someone else.
The first step in creating a graphic to do this is to choose the right type of visualization.
As we saw in the section ``Types of Visualizations,'' there are many possibilities, each with different strengths.
You should carefully consider which one best suits your data set and communication needs.  Your goal should always be communicating as clearly, effectively, and accurately as possible.

\subsubsection*{Visualizations to avoid}
There are also several popular types of visualizations that you should generally \textbf{not} use.  These may look pretty, but they tend to obscure information or are easily confused, because of natural difficulties humans have interpreting them.

For example, people have much more difficulty judging changes and variation in area than they do in length.
As a result, charts that depend on distinguishing variation in area will be less effective.  One of the most common charts that has this problem is the pie chart. Figure \ref{fig:pievsbar} indicates the difficult in judging the difference in area between the lobster and beans.

\begin{figure}
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{pie_chart_sorted.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{bar_chart_horizontal_sorted.png}
\end{subfigure}
\caption{A pie chart on the left that has been transformed into a bar chart on the right}
\label{fig:pievsbar}
\end{figure}

As bad as the pie chart may be, there is a way to make things worse---make the chart three-dimensional.  It seems to be very popular for people to convert a visualization that is naturally 2 or 1 dimensional into one that is artificially three dimensional, but this inevitably makes things more confusing.  For example, we already have difficulty distinguishing the variation in area in a pie chart, and when we add a third dimension, it becomes even harder (see Figure \ref{fig:pie3d}.

\begin{figure}
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{pie_chart3d.png}
\end{subfigure}
\caption{a 3d pie chart}
\label{fig:pie3d}
\end{figure}

Other visualization types that tend to be confusing and ineffective include the stacked bar and the radar graph.  The radar graph is just generally confusing.  While problem with stacking bars is that the second bar is not based at zero, so it becomes difficult to compare the different parts, as in Figure \ref{fig:stacked}

\begin{figure}
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{stackedbar.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{stackedbarinv.png}
\end{subfigure}
\caption{Stacked Bar Chart comparison}
\label{fig:stacked}
\end{figure}

In these examples differences in design give completely different impressions of the data, so the reality of the data is obscured rather than illuminated by the design.  As the famous visualization guru, Edward Tufte, says, ``Show data variation, not design variation.''  Pie charts and stacked bar charts show design variation more than data variation.

\subsection*{Simplify}

\noindent\emph{A sentence should contain no unnecessary words, a paragraph no unnecessary sentences for the same reason that a drawing should contain no unnecessary lines and a machine no unnecessary parts.} \small{---Strunk and White}

Once you have chosen the visualization best suited to your data, you should design the details so that all elements contribute to the communication of data.

Edward Tufte offers two principles for simplifying graphics: (1) erase ink that does not communicate data, and (2) erase ink that communicates data redundantly.  (\cite{tufte2001} pp.96-100). According to these principles, decorative backgrounds, fancy lettering, and cute graphics in the corner of you plots should all be deleted.

Other opportunities for simplification are harder to notice and implement. As an example, let us examine the plot on the right of Figure \ref{fig:healthcare}.
This plot was created with the following code.

\begin{lstlisting}
import numpy as np
import scipy as sp
from matplotlib import pyplot as plt

m = 2.07
s = 0.63
num_samples = 10000
samples = []

for i in xrange(num_samples):
    samples.append(sp.random.lognormal(m, s))

sp_samples = sp.array(samples)

# Plot the histogram
plt.hist(sp_samples, 100)
\end{lstlisting}

For purposes of this example, the only important part of the above code is the line \li{plt.hist(sp_samples, 100)} that plots the histogram.
What ink in this plot can we erase because it communicates no data?

First, let's get rid of the vertical black lines that separate the bars of the histograph and the black outline. These are meaningless for our application and only distract.
We can do this by modifying our call to \li{plt.hist()} as follows.

\begin{lstlisting}
plt.hist(sp_samples, 100, edgecolor='none')
\end{lstlisting}

Next, let us turn off the top and right lines that box in the graph.
To do this, we need to access the ``axis'' object associated with the figure.
We can access the ``axis'' object with the command \li{plt.gca()} (get current axis).

\begin{lstlisting}
# Get current axis instance
axis = plt.gca()

# Hide top and right spines
axis.spines['right'].set_visible(False)
axis.spines['top'].set_visible(False)
\end{lstlisting}

These commands only turn off the sides of the box, not the tick marks.
To turn off the tick marks we run the following commands.

\begin{lstlisting}
# Only show bottom and left tick marks
axis.yaxis.set_ticks_position('left')
axis.xaxis.set_ticks_position('bottom')
\end{lstlisting}

Finally, we do not need so many tick marks on the $x$- and $y$-axes.
We adjust those along with specifying the range on each axis.

\begin{lstlisting}
# Fix x and y ranges
plt.xlim(0,70)
plt.ylim(0, 800)

# Use fewer axis ticks
plt.xticks(np.arange(0, 71, 35))
plt.yticks(np.arange(0, 801, 200))
\end{lstlisting}

The final graph is shown in Figure \ref{fig:simplify}.
Note how much cleaner this looks than the original.

\begin{figure}
\centering
\includegraphics[width=\textwidth]{simplify.pdf}
\caption{A simplified version of the histogram in Figure \ref{fig:healthcare}.}
\label{fig:simplify}
\end{figure}

%TODO: Rewrite this problem to use a specific image with a specific context
\begin{problem}
Choose a graph you previously created that you think could use simplification.
Apply at least 3 of the following to simplify your plot:
\begin{enumerate}
\item Turn off some of the spines.
\item Fix the x or y range to better fit your data.
\item Change the number of x or y ticks.
\item Choose a better color scheme.
\end{enumerate}
Simplify your image in any other ways you can think of.
\end{problem}

\subsection*{Small Multiples}

Often we have too much information to display cleanly in one chart.  One indicator of this is if the reader must often look back and forth between a legend and the chart to decode the images.  A very useful way to deal with this problem is the method of \emph{small multiples}, an idea made famous by Edward Tufte.

The idea is to separate the data into many small graphics, placed near each other in a way that allows us to easily compare them.

For example, one might wish to compare the first six Chebyshev polynomials.  It would be natural to plot them together on the same graph, as in Figure \ref{fig:PoT}.  But this graph is too cluttered and one must constantly refer back to the legend to identify which colored line is which.

\begin{figure}
\centering
\begin{subfigure}{.45\textwidth}
  \centering
  \includegraphics[width=\textwidth]{PoT.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
  \includegraphics[width=\textwidth]{PoT_separate.png}
\end{subfigure}
\caption{Plotting on top variations and plotting separately}
\label{fig:PoT}
\end{figure}

The small multiple approach is to plot each polynomial on its own set of axes, adjacent to each other and aligned in such a way that you can easily compare the different plots as in Figure\ref{fig:PoT}

Note how even with 9 plots instead of 6, it is much easier to identify the different polynomials and to compare them.

The main tool you need to make small multiples is the command \li{ax = plt.subplot()}.  This allows you to set up multiple plots, and access them one at a time.

\begin{lstlisting}
import numpy as np
from matplotlib import pyplot as plt
from numpy.polynomial import Chebyshev as T

def Chebyshev_subplots():
    fig = plt.figure(dpi=100)
    fig.set_size_inches(10,10)
    fig.suptitle('Chebyshev Polynomials', fontsize=20)

    x = np.linspace(-1,1,500)

    for i in xrange(9):
        ax = plt.subplot(3,3,i+1)
        ax.plot(x, T.basis(i)(x), lw=2, label="$T_%d$"%i)
        plt.xlim(-1.1,1.1)
        plt.ylim(-1.1,1.1)
        ax.set_title('$T_%d$'%i)
        if i%3:
            ax.set_yticklabels([])
            ax.yaxis.set_ticks_position('none')
        if i<6:
            ax.xaxis.set_ticks_position('none')
            ax.set_xticklabels([])

    plt.show()
\end{lstlisting}

\begin{problem}

The $n+1$ Bernstein basis polynomials of degree $n$ are defined as

$$b_{v,n} = {{n} \choose {k}} x^v (1-x)^{n-v}, v = 0, 1,..., n.$$

Plot the first few Bernstein basis polynomials up to $v,n$ [0,3] as small multiples and compare that to the cluttered version of plotting them on top of each other.

\end{problem}

\subsection*{Tell the truth}

Just as graphics can tell your reader about your data, they can also give your viewer false impressions.
You should never use data visualizations to mislead your reader.
There are many ways pictures can be used to lie, and some of them can be unintentional.

\subsubsection*{2- and 3-d effects}
For example, if we see differences in flat shapes by area, not width or height.
Suppose you are creating a pictogram to demonstrate the recent increase in housing prices, which have doubled in the last $n$ years.
So, you draw two houses, one whose sides are 1cm and one whose sides are 2cm.
The area of the second house is now $2^2=4$ times as big as the area of the first. And your reader will likely interpret the picture as a four-fold increase rather than just double.
Hence, you should make two-dimensional pictures proportional in \emph{area} to the numbers they represent.

Even worse, volume is cubed rather than squared, so if a picture is intended to look like a three-dimensional object, doubling the side lengths will give the impression of an eight-fold increase.

%TODO add figures with 2- and 3-d houses doubling in side length.

Three-dimensional special effects also tend to be misleading.  As mentioned above, a 3-D pie chart, for example, can be used to distort the sizes of its slices.
Similarly, a 3-D histogram can use perspective to distort the relative differences between bars.

\subsubsection*{Base, Scale, and Labels}

As mentioned above, bar charts should be based at 0 and all charts should be scaled in the way the most clearly conveys the most important and relevant information.  When there is a possibility of confusion, axes and key data values should be labeled for clarity.

\begin{figure}
\centering
\includegraphics[width=\textwidth]{dog_plots.pdf}
\caption{The graph of some data can look very different depending on the scale.
Here are two graphs of the ``same'' line over the same range (1 to 4).
The only difference is the $y$-range. Notice easy it is to manipulate the y-scale to imply either a large or small increase in $y$.
These figures are especially misleading when we leave the range markers off of the $y$-axes.}
\label{fig:dog_plots}
\end{figure}

As in any form of communication, integrity is important in data visualization.
You will win more respect for yourself and those you represent if you avoid misleading diagrams.

\subsection*{Additional Resources}

There are many software packages that facilitate the visual exploration of data.
One Python library is Glue (see \cite{glue}).

Some packages for making nicer looking plots include \li{Seaborn} and \li{prettyplotlib}.  TODO get refs

For more about visualization of data, we highly recommend the following books:

TODO get full refs for the follwoing

\begin{itemize}
\item \emph{How to lie with statistics}
\item \emph{Envisioning Information} by Edward Tufte
\item \emph{The visual display of quantitative information} by Edward Tufte (2nd edition)
\item \emph{Now you see it} by Stephen Few
\end{itemize}

\printbibliography