# PHY321: Conservative forces, examples and theory
**[Morten Hjorth-Jensen](http://mhjgit.github.io/info/doc/web/)**, Department of Physics and Astronomy and Facility for Rare Ion Beams (FRIB), Michigan State University, USA and Department of Physics, University of Oslo, Norway

Date: **Feb 18, 2022**

Copyright 1999-2022, [Morten Hjorth-Jensen](http://mhjgit.github.io/info/doc/web/). Released under CC Attribution-NonCommercial 4.0 license

## Aims and Overarching Motivation

### Monday February 14

Discussion of conditions for conservative forces and summing up our
discussion on conservative forces. Discussion of potential surfaces
and their interpretations.

**Reading suggestion**: Taylor sections 4.6, 4.9, 4.10 and 5.1 and 5.2 on harmonic oscillations.

### Wednesday February 16

The Earth-Sun problem and energy-conserving algorithms and how to encode in more efficient ways various algorithms for solving the equations of motion (Euler, Euler-Cromer and Velocity Verlet).

* [Links to Julie's material on code reusability](https://github.com/mhjensen/Physics321/tree/master/doc/pub/week7/ipynb)

**Reading suggestions**: Taylor section 4.8 and these notes

### Friday February 18

Working on the Earth-Sun problem and hw 5. Hints on various exercises.

**Reading suggestions:** Taylor chapters 3 and 4 and these notes.

## The curl of a force and link between Line Integrals and conservative forces

The concept of line integrals plays an important role in our discussion of energy conservation,
our definition of potentials and conservative forces.

Let us remind ourselves of some the basic elements (most of you may
have seen this in a calculus course under the general topic of vector
fields).

We define a path integration $C$, that is we integrate
from a point $\boldsymbol{r}_1$ to a point $\boldsymbol{r}_2$.
Let us assume that the path $C$ is represented by an arc length $s$. In three dimension we have the following representation of $C$

$$
\boldsymbol{r}(s)=x(s)\boldsymbol{e}_1+y(s)\boldsymbol{e}_2+z(s)\boldsymbol{e}_3,
$$

then our integral of a function $f(x,y,z)$ along the path $C$ is defined as

$$
\int_Cf(x,y,z)ds=\int_a^bf\left(x(s),y(s),z(s)\right)ds,
$$

where the initial and final points are $a$ and $b$, respectively.

## Exactness and Independence of Path

With the definition of a line integral, we can in turn set up the
theorem of independence of integration path.

Let us define
$f(x,y,z)$, $g(x,y,z)$ and $h(x,y,z)$ to be functions which are
defined and continuous in a domain $D$ in space. Then a line integral
like the above is said to be independent of path in $D$, if for every
pair of endpoints $a$ and $b$ in $D$ the value of the integral is the
same for all paths $C$ in $D$ starting from a point $a$ and ending in
a point $b$. The integral depends thus only on the integration limits
and not on the path.

## Differential Forms

An expression of the form

$$
fdx+gdy+hdz,
$$

where $f$, $g$ and $h$ are functions defined in $D$, is a called a first-order differential form
in three variables.
The form is said to be exact if it is the differential

$$
du= \frac{\partial u}{\partial x}dx+\frac{\partial u}{\partial y}dy+\frac{\partial u}{\partial z}dz,
$$

of a differentiable function $u(x,y,z)$ everywhere in $D$, that is

$$
du=fdx+gdy+hdz.
$$

It is said to be exact if and only if we can then set

$$
f=\frac{\partial u}{\partial x},
$$

and

$$
g=\frac{\partial u}{\partial y},
$$

and

$$
h=\frac{\partial u}{\partial z},
$$

everywhere in the domain $D$.

## In Vector Language

In vector language the above means that the differential form

$$
fdx+gdy+hdz,
$$

is exact in $D$ if and only if the vector function (it could be a force, or velocity, acceleration or other vectors we encounter in this course)

$$
\boldsymbol{F}=f\boldsymbol{e}_1+g\boldsymbol{e}_2+h\boldsymbol{e}_3,
$$

is the gradient of a function $u(x,y,z)$

$$
\boldsymbol{v}=\boldsymbol{\nabla}u=\frac{\partial u}{\partial x}\boldsymbol{e}_1+\frac{\partial u}{\partial y}\boldsymbol{e}_2+\frac{\partial u}{\partial z}\boldsymbol{e}_3.
$$

## Path Independence Theorem

If this is the case, we can state the path independence theorem which
states that with functions $f(x,y,z)$, $g(x,y,z)$ and $h(x,y,z)$ that fulfill the above
exactness conditions,  the line integral

$$
\int_C\left(fdx+gdy+hdz\right),
$$

is independent of path in $D$ if and only if the differential form under the integral sign is exact in $D$.

This is the path independence theorem.

We will not give a proof of the theorem. You can find this in any vector analysis chapter in a mathematics textbook.

We note however that the path integral from a point $p$ to a final point $q$ is given by

$$
\int_p^q\left(fdx+gdy+hdz\right)=\int_p^q\left(\frac{\partial u}{\partial x}dx+\frac{\partial u}{\partial y}dy+\frac{\partial u}{\partial z}dz\right)=\int_p^qdu.
$$

Assume now that we have a dependence on a variable $s$ for $x$, $y$ and $z$. We have then

$$
\int_p^qdu=\int_{s_1}^{s_2}\frac{du}{ds}ds = u(x(s),y(s),z(s))\vert_{s=s_1}^{s=s_2}=u(q)-u(p).
$$

This last equation

$$
\int_p^q\left(fdx+gdy+hdz\right)=u(q)-u(p),
$$

is the analogue of the usual formula

$$
\int_a^bf(x)dx=F(x)\vert_a^b=F(b)-F(a),
$$

with $F'(x)=f(x)$.

## Work-Energy Theorem again

We remember that a the work done by a force
$\boldsymbol{F}=f\boldsymbol{e}_1+g\boldsymbol{e}_2+h\boldsymbol{e}_3$ on a displacemnt $d\boldsymbol{r}$

$$
W=\int_C\boldsymbol{F}d\boldsymbol{r}=\int_C(fdx+gdy+hdz).
$$

From the path independence theorem, we know that this has to result in
the difference between the two endpoints only. This is exact if and
only if the force is the force $\boldsymbol{F}$ is the gradient of a scalar
function $u$. We call this scalar function, which depends only the
positions $x,y,z$ for the potential energy $V(x,y,z)=V(\boldsymbol{r})$.

We have thus

$$
\boldsymbol{F}(\boldsymbol{r})\propto \boldsymbol{\nabla}V(\boldsymbol{r}),
$$

and we define  this as

$$
\boldsymbol{F}(\boldsymbol{r})= -\boldsymbol{\nabla}V(\boldsymbol{r}).
$$

Such a force is called **a conservative force**. The above expression can be used to demonstrate
energy conservation.

## Additional Theorem

Finally we can define the criterion for exactness and independence of
path. This theorem states that if $f(x,y,z)$, $g(x,y,z)$ and
$h(x,y,z)$ are continuous functions with continuous first partial derivatives in the domain $D$,
then the line integral

$$
\int_C\left(fdx+gdy+hdz\right),
$$

is independent of path in $D$ when

$$
\frac{\partial h}{\partial y}=\frac{\partial g}{\partial z},
$$

and

$$
\frac{\partial f}{\partial z}=\frac{\partial h}{\partial x},
$$

and

$$
\frac{\partial g}{\partial x}=\frac{\partial f}{\partial y}.
$$

This leads to the **curl** of $\boldsymbol{F}$ being zero

$$
\boldsymbol{\nabla}\times\boldsymbol{F}=\boldsymbol{\nabla}\times\left(-\boldsymbol{\nabla}V(\boldsymbol{r})\right)=0!
$$

## Summarizing

A conservative force $\boldsymbol{F}$ is a defined as the partial derivative of a scalar potential which depends only on the position,

$$
\boldsymbol{F}(\boldsymbol{r})= -\boldsymbol{\nabla}V(\boldsymbol{r}).
$$

This leads to conservation of energy and a path independent line integral as long as the curl of the force is zero, that is

$$
\boldsymbol{\nabla}\times\boldsymbol{F}=\boldsymbol{\nabla}\times\left(-\boldsymbol{\nabla}V(\boldsymbol{r})\right)=0.
$$

## Graphing the potential energy and what we can learn from that

This is taken from homework 4, exercises 5.

A particle is under the influence of a force $F=-kx+kx^3/\alpha^2$, where $k$ and $\alpha$ are constants and $k$ is positive.

Determine $V(x)$ and discuss the motion. It can be convenient here to
make a sketch/plot of the potential as function of $x$.

We assume that the potential is zero at say $x=0$. Integrating the force from zero to $x$ gives

$$
V(x) = -\int_0^x F(x')dx'=\frac{kx^2}{2}-\frac{kx^4}{4\alpha^2}.
$$

## Making the plot

The following code plots the potential. We have chosen values of $\alpha=k=1.0$. Feel free to experiment with other values. We plot $V(x)$ for a domain of $x\in [-2,2]$.

```
%matplotlib inline

import numpy as np
import matplotlib.pyplot as plt
import math

x0= -2.0
xn = 2.1
Deltax = 0.1
alpha = 1.0
k = 1.0
#set up arrays
x = np.arange(x0,xn,Deltax)
n = np.size(x)
V = np.zeros(n)
V = 0.5*k*x*x-0.25*k*(x**4)/(alpha*alpha)
plt.plot(x, V)
plt.xlabel("x")
plt.ylabel("V")
plt.show()
```

## Interpreting the results

From the plot here (with the chosen parameters)
1. we see that with a given initial velocity we can overcome the potential energy barrier

and leave the potential well for good.
1. If the initial velocity is smaller (see next exercise) than a certain value, it will remain trapped in the potential well and oscillate back and forth around $x=0$. This is where the potential has its minimum value.

2. If the kinetic energy at $x=0$ equals the maximum potential energy, the object will oscillate back and forth between the minimum potential energy at $x=0$ and the turning points where the kinetic energy turns zero. These are the so-called non-equilibrium points.

## Final interpretations

What happens when the energy of the particle is $E=(1/4)k\alpha^2$? Hint: what is the maximum value of the potential energy?

From the figure we see that
the potential has a minimum at at $x=0$ then rises until $x=\alpha$ before falling off again. The maximum
potential, $V(x\pm \alpha) = k\alpha^2/4$. If the energy is higher, the particle cannot be contained in the
well. The turning points are thus defined by $x=\pm \alpha$. And from the previous plot you can easily see that this is the case ($\alpha=1$ in the abovementioned Python code).

## The Earth-Sun system

We will now venture into a study of a system which is energy
conserving. The aim is to see if we (since it is not possible to solve
the general equations analytically) we can develop stable numerical
algorithms whose results we can trust!

We solve the equations of motion numerically. We will also compute
quantities like the energy numerically.

We start with a simpler case first, the Earth-Sun system  in two dimensions only.  The gravitational force $F_G$ on the earth from the sun is

$$
\boldsymbol{F}_G=-\frac{GM_{\odot}M_E}{r^3}\boldsymbol{r},
$$

where $G$ is the gravitational constant,

$$
M_E=6\times 10^{24}\mathrm{Kg},
$$

the mass of Earth,

$$
M_{\odot}=2\times 10^{30}\mathrm{Kg},
$$

the mass of the Sun and

$$
r=1.5\times 10^{11}\mathrm{m},
$$

is the distance between Earth and the Sun. The latter defines what we call an astronomical unit **AU**.

## The Earth-Sun system, Newton's Laws

From Newton's second law we have then for the $x$ direction

$$
\frac{d^2x}{dt^2}=-\frac{F_{x}}{M_E},
$$

and

$$
\frac{d^2y}{dt^2}=-\frac{F_{y}}{M_E},
$$

for the $y$ direction.

Here we will use  that  $x=r\cos{(\theta)}$, $y=r\sin{(\theta)}$ and

$$
r = \sqrt{x^2+y^2}.
$$

We can rewrite

$$
F_{x}=-\frac{GM_{\odot}M_E}{r^2}\cos{(\theta)}=-\frac{GM_{\odot}M_E}{r^3}x,
$$

and

$$
F_{y}=-\frac{GM_{\odot}M_E}{r^2}\sin{(\theta)}=-\frac{GM_{\odot}M_E}{r^3}y,
$$

for the $y$ direction.

## The Earth-Sun system, rewriting the Equations

We can rewrite these two equations

$$
F_{x}=-\frac{GM_{\odot}M_E}{r^2}\cos{(\theta)}=-\frac{GM_{\odot}M_E}{r^3}x,
$$

and

$$
F_{y}=-\frac{GM_{\odot}M_E}{r^2}\sin{(\theta)}=-\frac{GM_{\odot}M_E}{r^3}y,
$$

as four first-order coupled differential equations

$$
\frac{dv_x}{dt}=-\frac{GM_{\odot}}{r^3}x,
$$

$$
\frac{dx}{dt}=v_x,
$$

$$
\frac{dv_y}{dt}=-\frac{GM_{\odot}}{r^3}y,
$$

$$
\frac{dy}{dt}=v_y.
$$

## Building a code for the solar system, final coupled equations

The four coupled differential equations

$$
\frac{dv_x}{dt}=-\frac{GM_{\odot}}{r^3}x,
$$

$$
\frac{dx}{dt}=v_x,
$$

$$
\frac{dv_y}{dt}=-\frac{GM_{\odot}}{r^3}y,
$$

$$
\frac{dy}{dt}=v_y,
$$

can be turned into dimensionless equations or we can introduce astronomical units with $1$ AU = $1.5\times 10^{11}$.

Using the equations from circular motion (with $r =1\mathrm{AU}$)

$$
\frac{M_E v^2}{r} = F = \frac{GM_{\odot}M_E}{r^2},
$$

we have

$$
GM_{\odot}=v^2r,
$$

and using that the velocity of Earth (assuming circular motion) is
$v = 2\pi r/\mathrm{yr}=2\pi\mathrm{AU}/\mathrm{yr}$, we have

$$
GM_{\odot}= v^2r = 4\pi^2 \frac{(\mathrm{AU})^3}{\mathrm{yr}^2}.
$$

## Building a code for the solar system, discretized equations

The four coupled differential equations can then be discretized using Euler's method as (with step length $h$)

$$
v_{x,i+1}=v_{x,i}-h\frac{4\pi^2}{r_i^3}x_i,
$$

$$
x_{i+1}=x_i+hv_{x,i},
$$

$$
v_{y,i+1}=v_{y,i}-h\frac{4\pi^2}{r_i^3}y_i,
$$

$$
y_{i+1}=y_i+hv_{y,i},
$$

## Code Example with Euler's Method

The code here implements Euler's method for the Earth-Sun system using a more compact way of representing the vectors. Alternatively, you could have spelled out all the variables $v_x$, $v_y$, $x$ and $y$ as one-dimensional arrays.

```
# Common imports
import numpy as np
import pandas as pd
from math import *
import matplotlib.pyplot as plt
import os

# Where to save the figures and data files
PROJECT_ROOT_DIR = "Results"
FIGURE_ID = "Results/FigureFiles"
DATA_ID = "DataFiles/"

if not os.path.exists(PROJECT_ROOT_DIR):
    os.mkdir(PROJECT_ROOT_DIR)

if not os.path.exists(FIGURE_ID):
    os.makedirs(FIGURE_ID)

if not os.path.exists(DATA_ID):
    os.makedirs(DATA_ID)

def image_path(fig_id):
    return os.path.join(FIGURE_ID, fig_id)

def data_path(dat_id):
    return os.path.join(DATA_ID, dat_id)

def save_fig(fig_id):
    plt.savefig(image_path(fig_id) + ".png", format='png')

DeltaT = 0.001
#set up arrays
tfinal = 10 # in years
n = ceil(tfinal/DeltaT)
# set up arrays for t, a, v, and x
t = np.zeros(n)
v = np.zeros((n,2))
r = np.zeros((n,2))
# Initial conditions as compact 2-dimensional arrays
r0 = np.array([1.0,0.0])
v0 = np.array([0.0,2*pi])
r[0] = r0
v[0] = v0
Fourpi2 = 4*pi*pi
# Start integrating using Euler's method
for i in range(n-1):
    # Set up the acceleration
    # Here you could have defined your own function for this
    rabs = sqrt(sum(r[i]*r[i]))
    a =  -Fourpi2*r[i]/(rabs**3)
    # update velocity, time and position using Euler's forward method
    v[i+1] = v[i] + DeltaT*a
    r[i+1] = r[i] + DeltaT*v[i]
    t[i+1] = t[i] + DeltaT
# Plot position as function of time
fig, ax = plt.subplots()
#ax.set_xlim(0, tfinal)
ax.set_ylabel('y[AU]')
ax.set_xlabel('x[AU]')
ax.plot(r[:,0], r[:,1])
fig.tight_layout()
save_fig("EarthSunEuler")
plt.show()
```

## Problems with Euler's Method

We notice here that Euler's method doesn't give a stable orbit. It
means that we cannot trust Euler's method. In a deeper way, as we will
see in homework 5, Euler's method does not conserve energy. It is an
example of an integrator which is not
[symplectic](https://en.wikipedia.org/wiki/Symplectic_integrator).

Here we present thus two methods, which with simple changes allow us to avoid these pitfalls. The simplest possible extension is the so-called Euler-Cromer method.
The changes we need to make to our code are indeed marginal here.
We need simply to replace

```
    r[i+1] = r[i] + DeltaT*v[i]
```

in the above code with the velocity at the new time $t_{i+1}$

```
    r[i+1] = r[i] + DeltaT*v[i+1]
```

By this simple caveat we get stable orbits.
Below we derive the Euler-Cromer method as well as one of the most utlized algorithms for sovling the above type of problems, the so-called Velocity-Verlet method.

## Deriving the Euler-Cromer Method

Let us repeat Euler's method.
We have a differential equation

$$
\begin{equation}
y'(t_i)=f(t_i,y_i)
\label{_auto1} \tag{1}
\end{equation}
$$

and if we truncate at the first derivative, we have from the Taylor expansion

$$
\begin{equation}
y_{i+1}=y(t_i) + (\Delta t) f(t_i,y_i) + O(\Delta t^2), \label{eq:euler} \tag{2}
\end{equation}
$$

which when complemented with $t_{i+1}=t_i+\Delta t$ forms
the algorithm for the well-known Euler method.
Note that at every step we make an approximation error
of the order of $O(\Delta t^2)$, however the total error is the sum over all
steps $N=(b-a)/(\Delta t)$ for $t\in [a,b]$, yielding thus a global error which goes like
$NO(\Delta t^2)\approx O(\Delta t)$.

To make Euler's method more precise we can obviously
decrease $\Delta t$ (increase $N$), but this can lead to loss of numerical precision.
Euler's method is not recommended for precision calculation,
although it is handy to use in order to get a first
view on how a solution may look like.

Euler's method is asymmetric in time, since it uses information about the derivative at the beginning
of the time interval. This means that we evaluate the position at $y_1$ using the velocity
at $v_0$. A simple variation is to determine $x_{n+1}$ using the velocity at
$v_{n+1}$, that is (in a slightly more generalized form)

$$
\begin{equation}
y_{n+1}=y_{n}+ v_{n+1}+O(\Delta t^2)
\label{_auto2} \tag{3}
\end{equation}
$$

and

$$
\begin{equation}
v_{n+1}=v_{n}+(\Delta t) a_{n}+O(\Delta t^2).
\label{_auto3} \tag{4}
\end{equation}
$$

The acceleration $a_n$ is a function of $a_n(y_n, v_n, t_n)$ and needs to be evaluated
as well. This is the Euler-Cromer method.

**Exercise**: go back to the above code with Euler's method and add the Euler-Cromer method.

## Deriving the Velocity-Verlet Method

Let us stay with $x$ (position) and $v$ (velocity) as the quantities we are interested in.

We have the Taylor expansion for the position given by

$$
x_{i+1} = x_i+(\Delta t)v_i+\frac{(\Delta t)^2}{2}a_i+O((\Delta t)^3).
$$

The corresponding expansion for the velocity is

$$
v_{i+1} = v_i+(\Delta t)a_i+\frac{(\Delta t)^2}{2}v^{(2)}_i+O((\Delta t)^3).
$$

Via Newton's second law we have normally an analytical expression for the derivative of the velocity, namely

$$
a_i= \frac{d^2 x}{dt^2}\vert_{i}=\frac{d v}{dt}\vert_{i}= \frac{F(x_i,v_i,t_i)}{m}.
$$

If we add to this the corresponding expansion for the derivative of the velocity

$$
v^{(1)}_{i+1} = a_{i+1}= a_i+(\Delta t)v^{(2)}_i+O((\Delta t)^2)=a_i+(\Delta t)v^{(2)}_i+O((\Delta t)^2),
$$

and retain only terms up to the second derivative of the velocity since our error goes as $O(h^3)$, we have

$$
(\Delta t)v^{(2)}_i\approx a_{i+1}-a_i.
$$

We can then rewrite the Taylor expansion for the velocity as

$$
v_{i+1} = v_i+\frac{(\Delta t)}{2}\left( a_{i+1}+a_{i}\right)+O((\Delta t)^3).
$$

## The velocity Verlet method

Our final equations for the position and the velocity become then

$$
x_{i+1} = x_i+(\Delta t)v_i+\frac{(\Delta t)^2}{2}a_{i}+O((\Delta t)^3),
$$

and

$$
v_{i+1} = v_i+\frac{(\Delta t)}{2}\left(a_{i+1}+a_{i}\right)+O((\Delta t)^3).
$$

Note well that the term $a_{i+1}$ depends on the position at $x_{i+1}$. This means that you need to calculate
the position at the updated time $t_{i+1}$ before the computing the next velocity.  Note also that the derivative of the velocity at the time
$t_i$ used in the updating of the position can be reused in the calculation of the velocity update as well.

## Adding the Velocity-Verlet Method

We can now easily add the Verlet method to our original code as

```
DeltaT = 0.01
#set up arrays
tfinal = 10 # in years
n = ceil(tfinal/DeltaT)
# set up arrays for t, a, v, and x
t = np.zeros(n)
v = np.zeros((n,2))
r = np.zeros((n,2))
# Initial conditions as compact 2-dimensional arrays
r0 = np.array([1.0,0.0])
v0 = np.array([0.0,2*pi])
r[0] = r0
v[0] = v0
Fourpi2 = 4*pi*pi
# Start integrating using the Velocity-Verlet  method
for i in range(n-1):
    # Set up forces, air resistance FD, note now that we need the norm of the vecto
    # Here you could have defined your own function for this
    rabs = sqrt(sum(r[i]*r[i]))
    a =  -Fourpi2*r[i]/(rabs**3)
    # update velocity, time and position using the Velocity-Verlet method
    r[i+1] = r[i] + DeltaT*v[i]+0.5*(DeltaT**2)*a
    rabs = sqrt(sum(r[i+1]*r[i+1]))
    anew = -4*(pi**2)*r[i+1]/(rabs**3)
    v[i+1] = v[i] + 0.5*DeltaT*(a+anew)
    t[i+1] = t[i] + DeltaT
# Plot position as function of time
fig, ax = plt.subplots()
ax.set_ylabel('y[AU]')
ax.set_xlabel('x[AU]')
ax.plot(r[:,0], r[:,1])
fig.tight_layout()
save_fig("EarthSunVV")
plt.show()
```

You can easily generalize the calculation of the forces by defining a function
which takes in as input the various variables. We leave this as a challenge to you.

## Hints for exercises 2 and 3

**Taylor exercise 3.11:**

This exercise is discussed in Taylor's chapter 3.2.

Consider the rocket of mass $M$ moving with velocity $v$. After a
brief instant, the velocity of the rocket is $v+\Delta v$ and the mass
is $M-\Delta M$. Momentum conservation gives

$$
\begin{eqnarray*}
Mv&=&(M-\Delta M)(v+\Delta v)+\Delta M(v-v_e)\\
0&=&-\Delta Mv+M\Delta v+\Delta M(v-v_e),\\
0&=&M\Delta v-\Delta Mv_e.
\end{eqnarray*}
$$

## Exercise 2

In the second step we ignored the term $\Delta M\Delta v$ since we
can assume it is small. The last equation gives

$$
\begin{eqnarray}
\Delta v&=&\frac{v_e}{M}\Delta M,\\
\nonumber
\frac{dv}{dt}dt&=&\frac{v_e}{M}dM.
\end{eqnarray}
$$

Here we let $\Delta v\rightarrow dv$ and $\Delta M\rightarrow dM$.
We have also assumed that $M(t) = M_0-kt$.
Integrating the expression with lower limits $v_0=0$ and $M_0$, one finds

$$
\begin{eqnarray*}
v&=&v_e\int_{M_0}^M \frac{dM'}{M'}\\
v&=&v_e\ln(M/M_0)\\
&=&v_e\ln[(M_0-k t)/M_0].
\end{eqnarray*}
$$

We have ignored gravity here. If we add gravity as the external force, we get when integrating an additional terms $-gt$, that is

$$
v(t)=v_e\ln[(M_0-k t)/M_0]-gt.
$$

## Exercise 3, more rockets

This is a continuation of the previous exercise and most of the relevant background material can be found in Taylor chapter 3.2.

Taking the velocity from the previous exercise and integrating over time we find the height

$$
y(t) = y(t_0=0)+\int_0^tv(t')dt'.
$$

You need to insert $v(t)$ from the previous exercise

To do the integral over time we recall that $M(t)=M_0-\Delta M t$. We assumed that $\Delta M=k$ is a constant.
We use that $M_0-M=kt$ and assume that mass decreases by a constant $k$ times time $t$.

## Some more manipulations which have to be done

We will need  to compute an integral which goes like

$$
\int_0^t \ln{M(t')}dt' = \int_0^t \ln{(M_0-kt')}dt'.
$$

and defining the variable $u=M_0-kt'$, with $du=-kdt'$ and the new limits $M_0$ when $t=0$ and $M_0-kt$ when time is equal to $t$, we have

$$
\int_0^t \ln{M(t')}dt' = \int_0^t \ln{(M_0-kt')}dt'=-\frac{1}{k}\int_{M_0}^{M_0-kt} \ln{(u)}du=-\frac{1}{k}\left[u\ln{(u)}-u\right]_{M_0}^{M_0-kt}.
$$


```python
from IPython.core.display import HTML, Image
css_file = 'style.css'
HTML(open(css_file, 'r').read())
```

<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Philosopher:400,700,400italic,700italic' rel='stylesheet' type='text/css'>

<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Philosopher', sans-serif;
    font-weight: 400;
    font-size: 2.2em;
    line-height: 100%;
    color: rgb(0, 80, 120);
    margin-bottom: 0.1em;
    margin-top: 0.1em;
    display: block;
}
.text_cell_render h2 {
    font-family: 'Philosopher', serif;
    font-weight: 400;
    font-size: 1.9em;
    line-height: 100%;
    color: rgb(245,179,64);
    margin-bottom: 0.1em;
    margin-top: 0.1em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Philosopher', serif;
    margin-top:12px;
    margin-bottom: 3px;
    font-style: italic;
    color: rgb(94,127,192);
}

.text_cell_render h4 {
    font-family: 'Philosopher', serif;
}

.text_cell_render h5 {
    font-family: 'Alegreya Sans', sans-serif;
    font-weight: 300;
    font-size: 16pt;
    color: grey;
    font-style: italic;
    margin-bottom: .1em;
    margin-top: 0.1em;
    display: block;
}

.text_cell_render h6 {
    font-family: 'PT Mono', sans-serif;
    font-weight: 300;
    font-size: 10pt;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

.CodeMirror{
        font-family: "PT Mono";
        font-size: 100%;
}

</style>

```python
from sympy import init_printing, Matrix, symbols
init_printing()
```

# Solving $A\underline{x}=\underline{b}$

## Finding the complete solution to the set for in a linear system

In this section, we focus on the row view of matrices in order to solve systems of linear equations.  Our aim to to find possible solutions for $\underline{x}$.  Consider the example in (1), taken from the previous lecture.

$$\begin{align}&{x_1}+2{x}_{2}+2{x}_{3}+2{x}_{4}={b}_{1}\\&2{x}_{1}+4{x}_{2}+6{x}_{3}+8{x}_{4}={b}_{2}\\&3{x_1}+6{x}_{2}+8{x}_{3}+10{x}_{4}={b}_{3}\end{align}\tag{1}$$

The matrix of coefficients and vectors are shown in (2).

$$\begin{bmatrix}1&2&2&2\\2&4&6&8\\3&6&8&10\end{bmatrix}\begin{bmatrix}{x}_{1}\\{x}_{2}\\{x}_{3}\\{x}_{4}\end{bmatrix}=\begin{bmatrix}{b}_{1}\\{b}_{2}\\{b}_{3}\end{bmatrix}\tag{2}$$

Multiplying the matrix of coefficients and the column vector of unknowns is shown in (3), indication that we are adding scalar multiples of vectors.

$${x}_{1}\begin{bmatrix}1\\2\\3\end{bmatrix}+{x}_{2}\begin{bmatrix}2\\4\\6\end{bmatrix}+{x}_{3}\begin{bmatrix}2\\6\\8\end{bmatrix}+{x}_{4}\begin{bmatrix}2\\8\\10\end{bmatrix}=\begin{bmatrix}{b}_{1}\\{b}_{2}\\{b}_{3}\end{bmatrix}\tag{4}$$

Since the third row is the addition of ($1\times$) row one and ($1\times$) row two, solutions on the right-hand side must be of the form shown in (5).

$${b}_{3}={b}_{1}+{b}_{2}\tag{5}$$

The question now is: _What values can $\underline{x}$ possibly take?_

From what we have seen up until now, $A\underline{x}=\underline{b}$ is solvable (exactly) when $\underline{b}$ is in the column space of $A$, that is to say, it must be a linear combination of the columns.

We need to find two solutions, one called the _particular_ and one the _nullspace_ solution.

### The particular solution: $\underline{x}_\text{particular}$

A particular solution is one that solves a specific case for $\underline{b}$.  Consider the reduced row-echelon form of the matrix of coefficients, $A$.

```python
A.rref()
```

Note that columns $2$ and $4$ have zero values as their respective pivots.

To find $\underline{x}_\text{particular}$ or $\underline{x}_\text{p}$, we set all free variables to $0$ (in this example case that would be ${x_2}={x_4}=0$, since these are _columns without pivots_).  Then we solve $A\underline{x}=\underline{b}$ for the pivot variables.

Let's see that in action and calculate $\underline{x}_\text{p}$ for (6).

$$\underline{b} =\begin{bmatrix}1\\5\\6\end{bmatrix}\tag{6}$$

The equation (6) above is a proper solution as $1+5=6$ in accordance with (5).

Below, we create an augmented matrix (that is one with $\underline{b}$ included).

```python
A_augm = Matrix([[1, 2, 2, 2, 1], [2, 4, 6, 8, 5], [3, 6, 8, 10, 6]])
A_augm
```

We can reduce this to reduced row-echelon form using the `.rref()` method.

```python
A_augm.rref() # The video example is not solved to reduced row echelon form
```

So, we are left with a new matrix (called `A1` below) that omits columns $2$ and $4$.

```python
A1 = Matrix([[1, 0, -2] ,[0, 1, (3 / 2)] ,[0, 0, 0]]) # The video example is not solved to reduced row echelon form
A1
```

From the last row in the reduced row-echelon form, we note that we can set $x_4$ to any value.  We have already set is to $0$, though.  From the second last row, we can read that $1{x_3}+{x_4}=\frac{3}{2}$, or then, ${x_3}=\frac{3}{2}$.  Knowing that ${x_2}=0$, we read from the first row that $1{x_1}+2{x_2}+0{x_3}-2{x_4}={x_1}=-2$ as shown in (7).

$$\begin{align}&x_4=0\\&{x_3}=\frac{3}{2}\\&{x_1}=-2\end{align}\tag{7}$$

Thus, for $\underline{x}_\text{p}$ we have (8).

$$\underline{x}_\text{p}=\begin{bmatrix}-2\\0\\{\frac{3}{2}}\\0\end{bmatrix}\tag{8}$$

### The nullspace solution: $\underline{x}_\text{nullspace}$

Here we have $\underline{b}$ as in (9).  (We will simplify the notation to $\underline{x}_\text{n}$.)

$$\underline{b}=\begin{bmatrix}0\\0\\0\end{bmatrix}\tag{9}$$

Reduction to reduce row-echelon form and following the same principles as above leads us to two nullspace solutions that we can easily calculate using the `.nullspace()` method for the augmented matrix.

```python
A_augmented_null = Matrix([[1, 2, 2, 2, 0], [2, 4, 6, 8, 0], [3, 6, 8, 10, 0]])
A_augmented_null.rref()
```

```python
A.nullspace()
```

$$\begin{bmatrix}\left[\begin{matrix}-2\\1\\0\\0\end{matrix}\right], & \left[\begin{matrix}2\\0\\-2\\1\end{matrix}\right]\end{bmatrix}$$

Let's have a look at how we got here.  We begin with the last row of the reduced row-echelon form.  Since we are free to set a value for $x_4$ (the whole row consisting of zeros), we set it to $t$.  Solving back up through rows $2$ and $1$, we have (10) below.

$$\begin{align}&{x_4}=t\\&{x_3}+2t=0\\&{x_3}=-2t\\&{x_1}+2{x_2}-2t=0\\&{x_1}=2t-2{x_2}\end{align}\tag{10}$$

The simplest value to give $t$ is $0$ and with ${x_2}=1$, we have (11).

$$\begin{align}&{x_4}=0\\&{x_3}=0\\&{x_2}=1\\&{x_1}=-2\end{align}\tag{11}$$

If we set $t=1$ and ${x_2}=0$, we have (12), the other nullspace solution.

$$\begin{align}&{x_4}=1\\&{x_3}=-2\\&{x_2}=0\\&{x_1}=2\end{align}\tag{12}$$

Why did we alternate by giving the unknowns the values $0$ and $1$?  Well, these are the simplest values to give (other than $0$ and $0$, which will be trivial).

### The full set of solutions

We get the full set of solutions as $\underline{x}=\underline{x}_\text{p}+\underline{x}_\text{n}$.  This is shown to be so in (13).

$$\begin{align}&A{\underline{x}}_\text{p}=b\\&A{\underline{x}}_\text{n}=0\\{\therefore}\quad&{A}\left({\underline{x}}_\text{p}+{\underline{x}}_\text{n}\right)=\underline{b}\end{align}\tag{13}$$

Since we can have constant multiples of the nullspace vector, we have a final solution in (14).

$$ \overline { x } ={ \overline { x }  }_{ P }+{ \overline { x }  }_{ N }=\begin{bmatrix} -2 \\ 0 \\ \frac { 3 }{ 2 }  \\ 0 \end{bmatrix}+{ c }_{ 1 }\begin{bmatrix} -2 \\ 1 \\ 0 \\ 0 \end{bmatrix}+{ c }_{ 2 }\begin{bmatrix} -2 \\ 0 \\ -2 \\ 1 \end{bmatrix} \tag{14}$$

## Rank

We discussed rank in the previous lecture.  For any ${m}\times{n}$ matrix we have a rank (number of pivots).  We cannot have more pivots than rows, therefor, for a matrix $A$, $m\le\text{rank}\left(A\right)$ and $n\le\text{rank}\left(A\right)$.

### The case of full column rank for a matrix $A$, i.e. $\text{rank}\left(A\right)=n$

This implies that there are no free variables and the nullspace only has the zero vector (it is a subspace and MUST contain the zero vector.  Thus $\underline{x}=\underline{x}_\text{p}$ ONLY (if it exists).

Consider the example with $2$ columns below.  One column is not a linear combination of the other and the rank will be $2$.  With only 2 unknowns and a rank of $2$, the nullspace will contain only the zero vector, $\underline{0}$.

```python
A = Matrix([[1, 3], [2, 1], [6, 1], [5, 1]])
A
```

```python
A.nullspace()
```

```python
A.rref() # Just to show the reduced row echelon form
```

### The case of full row rank, i.e. $\text{rank}\left(A\right)=m$

Here, every row has a pivot.  For which $\underline{b}$ will the the set be solvable?  For ALL $\underline{b}$.  How many free variables?  We are left with $n-\text{rank}\left(A\right)$ or $n-m$ free variables.

```python
A2 = A.transpose() # Creating a new matrix, which is the transpose of A above (just as an example)
A2
```

```python
A2.nullspace() # Showing the two nullspace solutions
```

```python
A2.rref()
```

### The case of full (row and column) rank, i.e. $\text{rank}\left(A\right)=m=n$

Let's do an example.

```python
A = Matrix([[1, 2], [3, 1]])
A
```

```python
A.nullspace() # Only the zero vector
```

```python
A.rref()
```

This is by definition an invertible matrix (which we will learn about later).

Let's look at an non-invertible matrix.  For this $2\times2$ matrix we will have one free variable (no pivot).

```python
A = Matrix([[2, 4], [3, 6]])
A
```

```python
A.nullspace()
```

```python
A.rref()
```

```python

```

Mathematics/Linear Algebra/0.0 MIT-18.06 - Jupyter/Lecture_8_Solving_nonhomogeneous_systems.ipynb


# Homework #5 (Due 10/14/2021, 11:59pm)
## Hierarchical Models and the Theory of Variational Inference

**AM 207: Advanced Scientific Computing**

**Instructor: Weiwei Pan**

**Fall 2021**

**Name: Jiahui Tang**

**Students collaborators: Yujie Cai**

### Instructions:

**Submission Format:** Use this notebook as a template to complete your homework. Please intersperse text blocks (using Markdown cells) amongst `python` code and results -- format your submission for maximum readability. Your assignments will be graded for correctness as well as clarity of exposition and presentation -- a “right” answer by itself without an explanation or is presented with a difficult to follow format will receive no credit.

**Code Check:** Before submitting, you must do a "Restart and Run All" under "Kernel" in the Jupyter or colab menu. Portions of your submission that contains syntactic or run-time errors will not be graded.

**Libraries and packages:** Unless a problems specifically asks you to implement from scratch, you are welcomed to use any `python` library package in the standard Anaconda distribution.

```python
### Import basic libraries
import numpy as np
import pandas as pd
import sklearn as sk
from sklearn.linear_model import LinearRegression
import matplotlib.pyplot as plt
%matplotlib inline

```

```python
# data_df is the dataframe of your data
# estimates_df is a numpy array of cancer rate estimates, one for each county
def scatter_plot_cancer_rates(data_df, estimates=None):
    ax = data_df.plot(kind='scatter', x="pop",y="pct_mortality", alpha=0.1, color="grey")
    bot_kcancer_counties = data_df.sort_values(by='pct_mortality',ascending=True)[:300]
    top_kcancer_counties = data_df.sort_values(by='pct_mortality',ascending=False)[:300]
    top_kcancer_counties.plot(kind='scatter',x="pop",y="pct_mortality",
                              alpha=0.1, color="red", ax=ax, logx=True, label = "highest cancer rate")
    bot_kcancer_counties.plot(kind='scatter',x="pop",y="pct_mortality",
                              alpha=0.1, color="blue", ax=ax, logx=True, label = "lowest cancer rate")
    if estimates is not None:
        ax.plot(data_df['pop'], 5 * estimates, '.', alpha=0.2, color="green", label='mean estimates')
    ax.set_ylim([-0.0001, 0.0003])

```

```python
# data_df is the dataframe of your data
# estimates_df is a numpy array of cancer rate estimates, one for each county
def scatter_plot_cancer_rates_orange(data_df, estimates=None):
    ax = data_df.plot(kind='scatter', x="pop",y="pct_mortality", alpha=0.1, color="grey")
    bot_kcancer_counties = data_df.sort_values(by='pct_mortality',ascending=True)[:300]
    top_kcancer_counties = data_df.sort_values(by='pct_mortality',ascending=False)[:300]
    top_kcancer_counties.plot(kind='scatter',x="pop",y="pct_mortality",
                              alpha=0.1, color="red", ax=ax, logx=True, label = "highest cancer rate")
    bot_kcancer_counties.plot(kind='scatter',x="pop",y="pct_mortality",
                              alpha=0.1, color="blue", ax=ax, logx=True, label = "lowest cancer rate")
    if estimates is not None:
        ax.plot(data_df['pop'], 5 * estimates, '.', alpha=0.2, color="orange", label='mean estimates')
    ax.set_ylim([-0.0001, 0.0003])

```

# Problem Description: Understanding EM and Variational Inference

In this problem, we will draw concrete connections between EM and variational inference by applying both methods to a certain class of latent variable models. You'll need to refer to relevant lecture notes on the derivations of EM and the derivation of the variational inference objective. This is an essay question that requires you to engage with complex derivations at a productive but still high level. No implementation is required.

#### Non-Bayesian Latent Variable Model
Recall the class of latent variable models we studied in lecture:

#### Bayesian Latent Variable Model
A Bayesian version of the same class of models involve adding priors for the model parameters:

1. **(Comparing ELBOs)** For the above type of Bayesian latent variable model, write down the ELBO for variational inference with a mean field variational family. Compare the variational inference ELBO for the Bayesian model to the expectation maximization ELBO for the non-Bayesian model. What are the differences and similarities between these two ELBOs?

  In both EM and variational inference we optimize the ELBO. Compare the update steps in EM to the update steps in Coordinate Ascent Variational Inference, draw a concrete analogy between them.

  ***Hint:*** To make both ELBO's comparable, make sure that both are in terms of $z, y, \theta, \phi$.

**Answer:**

ELBO for **EM** for non-Bayesian model:

$$ELBO^{\text{EM}}(\theta, \phi, q(z))  = \mathbb{E}_{z\sim q(z)}\left[\log\left(\frac{p(y, z|\theta, \phi)}{q(z)}\right)\right] = \mathbb{E}_{z\sim q(z)} \left[\log \left(\frac{P(y|z,\phi)p(z|\theta)}{q(z)}\right)\right]
$$

The ELBO for **Variational Inference** with a mean field variational family, where  $\psi=(\theta,\phi,Z)$:

$$ELBO^{\text{VI}}(\psi) = \mathbb{E}_{\psi \sim q(\psi|\lambda)}\left[\log\left(\frac{p(\psi, Y_1, \ldots, Y_N |a,b))}{q(\psi | \lambda)} \right)\right] $$

Write out in terms of $z, y, \theta, \phi$

$$
ELBO = \mathbb{E}_{z, \phi, \theta \sim q(z, \phi, \theta |\lambda_1, \lambda_2, \lambda_3)} \left[\log\left(\frac{p(y, z, \theta, \phi|a, b)}{q(z, \phi, \theta |\lambda_1, \lambda_2, \lambda_3)}\right)\right]
$$

**The differences and similarities between these two ELBOs:**

**Similarities:**
 1. They all incorporate an auxilary function $q(z)$ in order to take the gradients.
 2. Both take an expectation log ratio between joint distribution over distributions that are not that of the observed variable (auxilary distribution $q$ or variational distribution $q$).

**Differences:**
1. For VI, we introduce an additional parameter $\lambda$ for the variation family $Q$, and it is maximized using expetctation wrt the variation family.
2. In EM, $\phi$ and $\theta$ are known constants, and produce point estimation; In VI, they are random variables with distribution and we derive posterior over them;  Thus in EM, we optimize the ELBO wrt $\theta$ and $\phi$ as point estimates; In variational inference, we optimize over the distribution of $\theta$ and $\phi$
3. The difference between the ELBO and the KL divergence is the log normalizer (i.e. the evidence), which is the quantity that the ELBO bounds.
4. In VI, we take the expectation over the distribution of a hyperparameter. In EM, we take the expectation over the distribution of the latent variable.

**Special Case**:

EM is a special case of ELBO if we use mean-field VI where the variational distributions $q$ are assumed to be point estimations. That is, Let $\theta^{*}$ and $\phi^{*}$ be the unknown location of this point mass:
    $$q_{\theta}(\theta)=\delta\left(\theta-\theta^{*}\right)$$
     $$q_{\phi}(\phi)=\delta\left(\phi-\phi^{*}\right)$$

VI will minimize a KL divergence, and so when we minimumize over the variational family, it is the E-step of EM; and the further when minimizing over $\theta^{*}$ is the M-step of EM.

2. **(Comparing ELBOs and KL-divergences)** Recall that the original objective of variational inference is to minimize a KL-divergence, we rewrote the objective to be that of maximizing the ELBO. Why is directly minimizing the KL-divergence in the original objective difficult (be specific about wherein the difficulty lies)?

  In the derivation of the E-step of EM, we reframed an maximization of the ELBO problem as a minimization of a KL-divergence problem. In this case, why was the KL-divergence easier to minimize and the ELBO harder to maximize (use the instantiation of the E-step for Gaussian Mixture Models in Lecture 7 to help support your answer)?

  In the notes for Lecture 8, we introduce a way to maximize the variational inference ELBO -- through coordinate ascent. In the derivation of the updates for coordinate ascent, there is a place where we reframed an maximization of the ELBO problem as an equivalent minimization of a KL-divergence problem. Write down the exact form of this equivalence (the two expressions are separated in the derivation by a bunch of lines, you'll need to identify both parts that you need). In this case, why was the KL-divergence easier to minimize and the ELBO harder to maximization (use the instantiation of the update for Gaussian Mixture Models in Lecture 8 to help support your answer)?

  Based on this analysis, can you draw some general conclusions about when we'd prefer to minimize the KL-divergence versus when we'd prefer to maximize the ELBO?

**Answer:**

Recall that the original objective of variational inference is to minimize a KL-divergence, we rewrote the objective to be that of maximizing the ELBO. Why is directly minimizing the KL-divergence in the original objective difficult (be specific about wherein the difficulty lies)?

In VI, we are finding a tractable distribution $q$ that best approximate the complex distribution $p$ by minimizing KL divergence.

$$
\begin{aligned}
\lambda^* &= \underset{\lambda}{\text{argmin}}\; D_{\text{KL}}(q(\psi|\lambda) \| p(\psi|Y_1, \ldots, Y_N, a, b))) \\
&= \underset{\lambda}{\text{argmin}}\; \mathbb{E}_{\psi \sim q(\psi|\lambda)}\left[\log\left(\frac{q(\psi | \lambda)}{p(\psi|Y_1, \ldots, Y_N))}\right) \right]
\end{aligned}$$

Write out in terms of $z, y, \theta, \phi$

$$
\underset{\lambda}{\min} D_{\text{KL}}[q(z, \phi, \theta |\lambda_1, \lambda_2, \lambda_3) \| p(\theta, \phi, z|y, a, b)] = \underset{\lambda}{\min} \mathbb{E}_{z, \phi, \theta \sim q(z, \phi, \theta |\lambda_1, \lambda_2, \lambda_3)} \left[\log \left( \frac{q(z, \phi, \theta |\lambda_1, \lambda_2, \lambda_3)}{p(\theta, \phi, z|y, a, b)}\right) \right]
$$

By Bayes' rule, the denominator in the log ratio

$$
p(\theta, \phi, z|y, a, b) = \frac{p(y, z, \theta, \phi| a, b)}{p(y | a, b)} = \frac{p(y, z, \theta, \phi| a, b)}{\int p(y, z, \theta, \phi| a, b) \mathrm{d}(z, \theta,\phi)} \propto p(y, z, \theta, \phi| a, b)
$$

Therefore
\begin{aligned}
D_{\text{KL}}[q(z, \phi, \theta |\lambda_1, \lambda_2, \lambda_3) \| p(\theta, \phi, z|y, a, b)] &= \mathbb{E}_{z, \phi, \theta \sim q(z, \phi, \theta |\lambda_1, \lambda_2, \lambda_3)} \left[\log \left( \frac{q(z, \phi, \theta |\lambda_1, \lambda_2, \lambda_3) p(y | a, b)}{ p(y, z, \theta, \phi| a, b)}\right) \right]\\
    & =  \mathbb{E}_{z, \phi, \theta \sim q(z, \phi, \theta |\lambda_1, \lambda_2, \lambda_3)} \left[\log q(z, \phi, \theta |\lambda_1, \lambda_2, \lambda_3) + \log p(y | a, b) - \log p(y, z, \theta, \phi| a, b) \right]\\
    \end{aligned}

But that $\log p(y| a, b) = \int p(y, z, \theta, \phi| a, b)\mathrm{d}(z, \theta,\phi) $ involve integration that's intractable to integrate over $(z, \theta,\phi)$.

But with ELBO,

\begin{aligned}
ELBO &= \mathbb{E}_{z, \phi, \theta \sim q(z, \phi, \theta |\lambda_1, \lambda_2, \lambda_3)} \left[\log\left(\frac{p(y, z, \theta, \phi|a, b)}{q(z, \phi, \theta |\lambda_1, \lambda_2, \lambda_3)}\right)\right]\\
&= \mathbb{E}_{z, \phi, \theta \sim q(z, \phi, \theta |\lambda_1, \lambda_2, \lambda_3)} \left[\log p(y, z, \theta, \phi|a, b) - \log q(z, \phi, \theta |\lambda_1, \lambda_2, \lambda_3) \right]
\end{aligned}

The term no longer involves $\log p(y| a, b)$, making it easier to compute.
Thus, directly minimizing the KL-divergence in the original objective is difficult, we used maximizing ELBO.The issue is that the gradient taken is with respect to the parameter $\psi$ of the distribution of which we are taking the expectation - i.e. we cannot push the gradient into the expectation.

In the derivation of the E-step of EM, we reframed an maximization of the ELBO problem as a minimization of a KL-divergence problem. In this case, why was the KL-divergence easier to minimize and the ELBO harder to maximize (use the instantiation of the E-step for Gaussian Mixture Models in Lecture 7 to help support your answer)?

**Why KL-divergence is easier to minimize and the ELBO is harder to maximize in EM:**

1. Theoretically, when KL is minimized, we know the approximated distribution is the same as the true distribution.
2. Since we already know what one of the distributions are, we can just equate the other distribution to it and be done. Hence, KL divergence is easier to minimize in this situation
3. Practically, in ELBO, we need to conduct two sets of iterations, one for E-step and another for M step. Also, we only converge to the lower bound, which is not guaranteed to be the true optimized value.
4. Besides, for the E-step of EM, it is hard to maximize the ELBO with respect to the auxiliary $q$ function due to complex integration and multivariate gradients.

To be specific, in E-step,
$$
\underset{q}{\max} ELBO(\theta^*, \phi^*, q(z)) = \underset{q}{\max} \mathbb{E}_{z\sim q(z)}\left[\log\left(\frac{p(y, z|\theta^*, \phi^*)}{q(z)}\right)\right]
$$
(after M step, we get $\theta^*, \phi^*$ as optimized values)

To get that, we will take gradient of ELBO, that is, $\nabla \mathbb{E}_{z\sim q(z)}\left[\log\left(\frac{p(y, z|\theta^*, \phi^*)}{q(z)}\right)\right]$. But taking gradient with respect to the auxiliary $q$ function due to complex integration and multivariate gradients will be very hard.

Therefore, we think about minimize the KL-divergence, as it is easier to minimize KL divergence.

In the notes for Lecture 8, we introduce a way to maximize the variational inference ELBO -- through coordinate ascent. In the derivation of the updates for coordinate ascent, there is a place where we reframed an maximization of the ELBO problem as an equivalent minimization of a KL-divergence problem. Write down the exact form of this equivalence (the two expressions are separated in the derivation by a bunch of lines, you'll need to identify both parts that you need). In this case, why was the KL-divergence easier to minimize and the ELBO harder to maximization (use the instantiation of the update for Gaussian Mixture Models in Lecture 8 to help support your answer)?

**In coordinate ascent, now why we choose minimizing KL-divergence?**

Maximize ELBO is equivalent to minimize the KL-divergence in coordinate ascent.

$\underset{\lambda_i}{\max}\mathbb{E}_{\psi_i \sim q(\psi_i|\lambda_i)}\left[\mathbb{E}_{\psi_{-i} \sim q(\psi_{-i}|\lambda_{-i})} \left[\log \left(\frac{p(\psi,data)}{q(\psi|\lambda)}\right)\right]\right] $
$\equiv \underset{\lambda_i}{\min} D_{\text{KL}}\left[q(\psi_i|\lambda_i)|| z \exp \{\mathbb{E}_{\psi_{-i} \sim q(\psi_{-i}|\lambda_{-i})} [\log p(\psi_i,data|\psi_{-i}]\}\right]$

For the following, we will use the following notation:

\begin{aligned}
\lambda_{-i} &= [\lambda_1\; \ldots\; \lambda_{i-1}\; \lambda_{i+1}\; \ldots\; \lambda_{I}]\\
\psi_{-i} &= [\psi_1\; \ldots\; \psi_{i-1}\; \psi_{i+1}\; \ldots\; \psi_{I}]\\
q(\psi_{-i}|\lambda_{-i}) &= \prod_{j\neq i}q(\psi_{j}|\lambda_{j})\\
\end{aligned}

From lecture 10, we have shown that
    $$\mathbb{E}_{\psi \sim q(\psi|\lambda)}[\ldots] = \mathbb{E}_{\psi_i \sim q(\psi_i|\lambda_i)}\left[\mathbb{E}_{\psi_{-i} \sim q(\psi_{-i}|\lambda_{-i})}[\ldots] \right]$$

and the exact form of the equivalence:

\begin{aligned}
\underset{\lambda_i}{\max} ELBO(\lambda) &= \underset{\lambda_i}{\max} \mathbb{E}_{\psi_i \sim q(\psi_i|\lambda_i)}\left[\mathbb{E}_{\psi_{-i} \sim q(\psi_{-i}|\lambda_{-i})} \left[\log \left(\frac{p(Y_1, \ldots, Y_N, \psi_i | \psi_{-i})}{q(\psi_{i}|\lambda_{i})} \right)\right]\right]\\
&= \underset{\lambda_i}{\min}D_{\text{KL}} \left[ q(\psi_{i}|\lambda_{i})\| \mathcal{Z}\exp\left\{\mathbb{E}_{\psi_{-i} \sim q(\psi_{-i}|\lambda_{-i})} \left[\log \left(p(Y_1, \ldots, Y_N, \psi_i | \psi_{-i}) \right)\right]\right\}\right].
\end{aligned}

As the last set of questions, if we set $q(\psi_{i}|\lambda_{i})$ equal to $\mathcal{Z}\exp\left\{\mathbb{E}_{\psi_{-i} \sim q(\psi_{-i}|\lambda_{-i})} \left[\log \left(p(Y_1, \ldots, Y_N, \psi_i | \psi_{-i}) \right)\right]\right\}$, we can solve the optimization. This will be easier than if we compute the gradient of of $ELBO$. In Gaussian Mixture models, in lec10 derivation, terms with subscript $-i$ will become constant after iterations.

Based on this analysis, can you draw some general conclusions about when we'd prefer to minimize the KL-divergence versus when we'd prefer to maximize the ELBO?

**Conclusion**

1. Look for complexity of mathematics when implementing the right method

2. We use KL divergence when we know one of the distributions exactly - then it is clear that the to minimize the divergence we just have to equate the other distribution to the other distribution

3. When the form of probability $P(A|B)$ is known and not intractible, we can use KL divergence minimization. When $P(A|B)$ is unknown and hard to compute, we can choose to work with ELBO, but this still require the joint of $A$ and $B$ to be tractable.

3. **(The Mean Field Assumption and Coordinate Ascent)**  Describe exactly when and how the mean field assumption is used in the derivation of the coordinate ascent updates.

**Answer:**

We assume that the joint $q(\psi)$ factorizes completely over each dimension of $\psi$, i.e. $q(\psi)= \prod_{i=1}^I q(\psi_i | \lambda_i)$. This is called the ***mean field assumption***. This assumes that each dimension of $q(\psi)$ is independent from the others, which may or may not be valid, but it helps to simplify the expression of $q(\psi)$, making it easier to find the ELBO. Thus it is a design choice to make, and it could also go wrong with this design choice.

The coordinate ascent algorithm maximize an objective function ELBO by iteractively maximizing over one dimension, while holding other dimension constant. That's why it requires us to be able to use mean field assumption and factorize $q(\psi)$ into individual pieces.

To use mean field assumption, we will maximize the $ELBO$ by following steps in the derivation of the coordinate ascent updates.

1. Use the mean-field assumption to break up the expectation $\mathbb{E}_{\psi \sim q(\psi|\lambda)}$ into an iterated expectation $\mathbb{E}_{\psi_i \sim q(\psi_i|\lambda_i)}\mathbb{E}_{\psi_{-i} \sim q(\psi_{-i}|\lambda_{-i})}$
2. We will rewrite the outer expectation $\mathbb{E}_{\psi_i \sim q(\psi_i|\lambda_i)}$ as a negative KL-divergence
3. We will then maximize the negative KL-divergence by setting the two arguments of the divergence equal to each other

4. **(Generalizability of CAVI)** Summarize what kind of derivations/math is needed in order instantiate Coordinate Ascent Variational Inference (CAVI) for a given new model (look at what we did for Gaussian Mixture Models in Lecture 8 and predict what you'd need to do for a new model). Based on this, discuss the potential draw backs of using CAVI for Bayesian inference in general. What do these draw backs imply about the practicality of variational inference as an inference method?

**Answer:**

We see that

$$
D_{\text{KL}} \left[ q(\psi_{i}|\lambda_{i})\| \mathcal{Z}\exp\left\{\mathbb{E}_{\psi_{-i} \sim q(\psi_{-i}|\lambda_{-i})} \left[\log \left(p(Y_1, \ldots, Y_N, \psi_i | \psi_{-i}) \right)\right]\right\}\right]
$$

is minimized when

$$
 q(\psi_{i}|\lambda_{i})\propto \exp\left\{\mathbb{E}_{\psi_{-i} \sim q(\psi_{-i}|\lambda_{-i})} \left[\log \left(p(Y_1, \ldots, Y_N, \psi_i | \psi_{-i}) \right)\right]\right\}.
$$

This is also exactly where the ELBO is maximized.

**Derivations/Math needed:**

1. We need mean field assumption to derive the variational distribution of each variable or parameter independently and separately.
2. For each i, we need to set $q(\psi_{i}|\lambda_{i})$ equal to $\mathcal{Z}\exp\left\{\mathbb{E}_{\psi_{-i} \sim q(\psi_{-i}|\lambda_{-i})} \left[\log \left(p(Y_1, \ldots, Y_N, \psi_i | \psi_{-i}) \right)\right]\right\}$ to solve for ELBO iteractively in CAVI.
3. We need lengthy algebraic manipulation to expand the expectation term
4. Computational wise also expensive, need to iterate through $\lambda_i$

**Drawbacks**
1. It may never converge
2. We will never be able to capture the full complexity of target distribution by sampling from posterior q
3. We are trading off fidelity for computational tracibility.
4. Computational heavy, we need to derive different $q(\psi_{i}|\lambda_{i})$ at each iteration for different models.
5. We need conduct lengthy posterior derivations for each latent variable / paramter which may result in intractable posterior distributions

**Particability**

It is still practical but it may be need some mathematical derivations when iteractively minimize KL. In each step, variety of complex algorithms and sampling methods can be used.

5. **(Generalizability of EM)** Summarize what kind of derivations/math is needed in order instantiate Expectation Maximization (EM) for a given new model (look at what we did for Gaussian Mixture Models in Lecture 9 and predict what you'd need to do for a new model). Based on this, discuss the potential draw backs of using EM for MLE inference in general. What do these draw backs imply about the practicality of EM as an inference method?

**Answer:**

0. **Initialization:** Pick $\theta_0$, $\phi_0$.
1. Repeat $i=1, \ldots, I$ times:

  **E-Step:**
$$q_{\text{new}}(Z_n) = \underset{q}{\mathrm{argmax}}\; ELBO(\theta_{\text{old}}, \phi_{\text{old}}, q) = p(Z_n|Y_n, \theta_{\text{old}}, \phi_{\text{old}})$$

  **M-Step:**
  \begin{aligned}
  \theta_{\text{new}}, \phi_{\text{new}} &= \underset{\theta, \phi}{\mathrm{argmax}}\; ELBO(\theta, \phi, q_{\text{new}})\\
  &= \underset{\theta, \phi}{\mathrm{argmax}}\; \sum_{n=1}^N\mathbb{E}_{Z_n\sim p(Z_n|Y_n, \theta_{\text{old}}, \phi_{\text{old}})}\left[\log \left( p(y_n, Z_n | \phi, \theta\right) \right].
\end{aligned}

For the **E-Step**, we just simply calculate the conditional distribution of each $Z_n|Y_n, \theta_\text{old}, \phi_\text{old}$. It is usually obtained by employing Bayesian Rule. (The $Z_n$ here is of discrete type.)

\begin{align}
p(Z_n|Y_n, \theta_\text{old}, \phi_\text{old}) = \frac{p(y_n|Z_n = k, \theta_\text{old})p(Z_n=k | \phi_{\text{old}})}{\sum_{k=1}^K p(y_n|Z_n = k,\theta_\text{old})p(Z_n=k | \phi_{\text{old}})}
\end{align}

For the **M-Step**, it usually involves expanding the expectation $\mathbb{E}$, splitting into two parts by property of conditional probability and optimizting two parts seperately to get the solution.

\begin{align}
\theta^*, \phi^*
&= \underset{\theta, \phi}{\mathrm{argmax}}\;\sum_{n=1}^N \mathbb{E}_{Z_n\sim p(Z_n|Y_n, \theta_\text{old}, \phi_\text{old})} \left[ \log\left(p(y_n, z_n|\theta, \phi)\right)\right] \\
&= \underset{\theta, \phi}{\mathrm{argmax}} \;\sum_{n=1}^N \mathbb{E}_{Z_n\sim p(Z_n|Y_n, \theta_\text{old}, \phi_\text{old})} \left[ \log p(y_n|z_n, \theta) + \log p(z_n|\phi)\right] \\
&= \underset{\theta, \phi}{\mathrm{argmax}} \;\sum_{n=1}^N \mathbb{E}_{Z_n\sim p(Z_n|Y_n, \theta_\text{old}, \phi_\text{old})} \log p(y_n|z_n, \theta) + \sum_{n=1}^N \mathbb{E}_{Z_n\sim p(Z_n|Y_n, \theta_\text{old}, \phi_\text{old})} \log p(z_n|\phi)
\end{align}

Since we know the $p(Z_n|Y_n, \theta_\text{old}, \phi_\text{old})$ in the previous E-Step, we can plug it in and take gradient to solve for $\theta_\text{new}$ and $\phi_\text{new}$.

For for a given new model, we need to re-derive the equation for $q(z)$ which is the posterior and the update function for M-step which needs taking the gradients in order to instantiate Expectation Maximization (EM). Also, we need to choose $q(z)$ smartly in order to easily re-derive the math.

The potential draw backs are that,
1. It has slow convergence.
2. It makes convergence to the local optima only.
3. It requires both the probabilities, forward and backward (numerical optimization requires only forward probability), to be tractable so we can actually implement it.

Hence, in order to make EM easy to implement for any given latent variable model, we need to ensure that the original distribution has tractable form so that we can update something in the M step. We also should make sure that the posterior is easy to compute so we can use it easily in the E step.

Although EM has drawbacks like those listed above, we can still use it and it is heavily used in fields like Biostatistics. And EM is a pretty straight-forward method to solve a latent variable model.

# Problem Description: Modeling Kidney Cancer Data
In this problem, we will continue to work with the US Kidney Cancer Data set, `kcancer.csv`. This is a dataset of kidney cancer frequencies across the US over 5 years on a per county basis.

**In this homework, we focus on comparing different types of models for this data set.**

## Part I: Empirical Bayes
Let $N$ be the number of counties; let $y_j$ the number of kidney cancer case for the $j$-th county, $n_j$ the population of the $j$-county and $\theta_j$ be the underlying kidney cancer rate for that county. The following is a Bayesian model for our data:

\begin{aligned}
y_j | \theta_j &\sim Poisson(5 \cdot n_j \cdot \theta_j), \quad j = 1, \ldots, N\\
\theta_j &\sim Gamma(\alpha, \beta), \quad j = 1, \ldots, N
\end{aligned}

where $\alpha, \beta$ are hyper-parameters of the model.

1. **(Visualize the raw cancer rates)** Produce a scatter plot of the raw cancer rates (pct mortality) vs the county population size (in log scale). Highlight the top 300 raw cancer rates in red. Highlight the bottom 300 raw cancer rates in blue. What can you say about the counties with the highest and lowest raw cancer rates.

```python
df = pd.read_csv("kcancer.csv")
y = df['dc'].values
n = df['pop'].values

```

```python
#plt.figure(figsize = (10,10))
scatter_plot_cancer_rates(df)
plt.title("raw cancer rates vs the county population size")
plt.show()
```

**Answer:**

The countries with highest and lowest cancer rate are those countries with low populations, located on the left side of the scatterplot. Both the highest rates mortality and the lowest rates mortality counties are not the points have large population.

This could happen due to drawbacks of MLE estimates, it tends to overfit when data is scarce. Countries with less population may only have few data points, thus resulting in overfitting, exaggerated effect and extreme pct_mortality rate.

2. **(Empirical Bayes)** Using Empirical Bayes and moment matching, choose values for the hyperparameters $\alpha, \beta$ based on your data. Use these values of $\alpha$ and $\beta$ to obtain posterior distributions for each county.

***Hint:*** You'll first need to derive the fact that the ***evidence*** for a Poisson-Gamma model has a Negative Binomial distribution.

From the last homework, we first derived $p(\theta_j | \theta_{-j}, y, \alpha, \beta)= Ga(\theta_j; \alpha+y_j, 5n_j+\beta)$

**Part I.** By Bayes' Theorm,
$$
\begin{align}
p(\theta_j | \theta_{-j}, y, \alpha, \beta) &= \frac{p(\theta| y, \alpha, \beta)}{p( \theta_{-j}| y, \alpha, \beta)} \propto p(\theta| y, \alpha, \beta) = \prod_{i=1}^N p(\theta_i|y, \alpha, \beta) \\
\end{align}
$$

By holding $\theta_{-j}, y, \alpha, \beta$ constant, we could see that $p(\theta| y, \alpha, \beta)$ only depends on $$p(\theta_j| y_j, \alpha, \beta) =\frac{ p(\theta_j, y_j|\alpha, \beta)}{p(\alpha, \beta)}\propto p(\theta_j, y_j|\alpha, \beta) = p(y_j|\theta_j, \alpha, \beta)p(\theta_j|\alpha,\beta)$$.

Then, we could compute the joint distribution of $p(\theta_j, y_j|\alpha, \beta)$ as the product of the two distributions.

$$
\begin{align}
p(\theta_j, y_j|\alpha, \beta) &=p(y_j|\theta_j, \alpha, \beta)p(\theta_j|\alpha,\beta)\\ &= Poisson(y_j; 5\cdot n_j \cdot \theta_j)\cdot(\theta_j; \alpha, \beta)\\
&= \frac{(5n_j\theta_j)^{y_j}e^{-5n_j\theta_j}}{y_j!}\cdot\frac{\beta^\alpha\theta_j^{\alpha-1}e^{-\beta\theta_j}}{\Gamma(\alpha)}\\
&\propto \theta_j^{\alpha+y_j-1}e^{-(5n_j+\beta)\theta_j}\\
&= Ga(\theta_j; \alpha+y_j, 5n_j+\beta)
\end{align}
$$
Q.E.D.

Thus, we derived that $p(\theta_j | \theta_{-j}, y, \alpha, \beta)= Ga(\theta_j; \alpha+y_j, 5n_j+\beta)$

Then we prove $p(y_j) = NB\left(r, p\right)$ using the fact: $\Gamma(\alpha) = (\alpha - 1)!$
\begin{aligned}
p(y_j) &= \frac{p(y_j | \theta_j) p(\theta_j)}{p(\theta_j | y_j)}\\
    &= \frac{Poiss(y_j; 5 \cdot n_j \cdot \theta_j) Ga(\theta_j; \alpha, \beta)}{Ga(\alpha + y_j, 5 n_j  + \beta)}\\
&= \frac{\frac{1}{y_j!} Exp[5 \cdot n_j \cdot \theta_j] \frac{\beta^\alpha}{\Gamma(\alpha)} \theta_j^{\alpha - 1} Exp[-\beta\theta_j]}{\frac{\left(5 n_j  + \beta\right)^{\left( \alpha + y_j\right)}}{\Gamma\left(\alpha + y_j\right)} \theta_j^{\alpha + y_j - 1} Exp[-\left(5 n_j  + \beta\right) \theta_j]}\\
&= \frac{\beta^\alpha\Gamma\left(\alpha + y_j\right)  \left(5 \cdot n_j\right)^{y_j} \theta_j^{\alpha + y_j - 1}Exp[-\left(5 \cdot n_j  + \beta\right)\theta_j]}{y_j!\Gamma(\alpha) \left(5 n_j  + \beta\right)^{\left( \alpha + y_j\right)}\theta_j^{\alpha + y_j - 1} Exp[-\left(5 n_j  + \beta\right) \theta_j]}\\
&= \frac{\beta^\alpha\Gamma\left(\alpha + y_j\right)  \left(5 \cdot n_j\right)^{y_j}}{y_j!\Gamma(\alpha) \left(5 n_j  + \beta\right)^{\left( \alpha + y_j\right)}}\\
&= \frac{\Gamma\left(\alpha + y_j\right)}{y_j!\Gamma(\alpha)}\frac{\beta^\alpha}{\left(5 n_j  + \beta\right)^{\alpha}} \frac{\left(5 \cdot n_j\right)^{y_j}}{\left(5 n_j  + \beta\right)^{y_j}}\\
&= \frac{\Gamma\left((\alpha + y_j - 1) + 1\right)}{y_j!\Gamma(\alpha)} \left( \frac{\beta}{5 n_j  + \beta}\right)^\alpha \left( \frac{5 \cdot n_j}{5 n_j  + \beta}\right)^{y_j}\\
&= \frac{(\alpha + y_j - 1)!}{y_j!\Gamma(\alpha)}\left( \frac{\beta}{5 n_j  + \beta}\right)^\alpha \left( \frac{5 \cdot n_j}{5 n_j  + \beta}\right)^{y_j}\\
&= \frac{(\alpha + y_j - 1)!}{y_j!(\alpha - 1)!}\left( \frac{\beta}{5 n_j  + \beta}\right)^\alpha \left( \frac{5 \cdot n_j}{5 n_j  + \beta}\right)^{y_j}\\
&= \binom{\alpha + y_j - 1}{y_j}\left( \frac{\beta}{5 n_j  + \beta}\right)^\alpha \left( \frac{5 \cdot n_j}{5 n_j  + \beta}\right)^{y_j}\\
&= NB\left(\alpha, \frac{5 n_j}{ 5 n_j + \beta}\right)
\end{aligned}

The theoritical mean and variance for negative binomial distribution:
\begin{align}
\mathbb{E}\left[y_j\right] &=  \frac{\alpha5n_j}{\beta} \\
\text{Var}\left[y_j\right] &= \frac{\alpha 5 n_j (5n_j+\beta)}{\beta^2}
\end{align}

Then, the empirical counterparts adjusted by the population of each county:
\begin{aligned}
\widehat{\mathbb{E}}\left[\frac{y_j}{n_j}\right] &= \frac{5\alpha}{\beta}\\
\widehat{\text{Var}}\left[\frac{y_j}{n_j}\right] &= \frac{5 \alpha (5 n_j+\beta)}{n_j\beta^2}  = \frac{5 \alpha(5\bar{n}+\beta)}{\bar{n}\beta^2}
\end{aligned}
For the variance case we shall put population mean instead of $n_j$

Solve and get:
\begin{aligned}
\alpha &= \frac{\bar{pop}\widehat{\mathbb{E}}^2 }{\bar{pop} \widehat{\text{Var}} - \widehat{\mathbb{E}}} \\ \\
\beta &= \frac{5 \bar{pop}  \widehat{ \mathbb{E}} }{\bar{pop}\widehat{\text{Var}} - \widehat{\mathbb{E}}}
\end{aligned}

```python
e_hat = np.mean(y/n)
var_hat = np.var(y/n)
pop_mean = np.mean(n)

alpha = pop_mean*e_hat**2/(pop_mean*var_hat - e_hat)
beta = 5*pop_mean*e_hat/(pop_mean*var_hat - e_hat)

print("empirical cancer rate population adjusted mean: = {}".format(e_hat))
print("empirical cancer rate population adjusted var = {}".format(var_hat))
print("empirical population mean = {}".format(pop_mean))
print('')

print("alpha = {}".format(alpha))
print("beta = {}".format(beta))

```

    empirical cancer rate population adjusted mean: = 5.786552354108626e-05
    empirical cancer rate population adjusted var = 2.527523010812238e-09
    empirical population mean = 160512.41021522647

    alpha = 1.5451734918796929
    beta = 133514.17193888978

Knowing from the derivation that $$\theta_i|y_i \sim Ga(\alpha + y_j, 5 n_j  + \beta)$$
we can sample $\theta$ from $Ga(1.5451734918796929 + y_j, 5 n_j  + 133514.17193888978)$ and compute the mean or mode from the samples:

```python
# update estimate
thetas = []
for i in range(5000):
    thetas.append(np.random.gamma(alpha + df['dc'].values, 1. / (beta + 5 * n)))
thetas = np.array(thetas)
thetas_mean = np.mean(thetas, axis=0)
```

3. **(Posterior Means)** Produce a scatter plot of the raw cancer rates (pct mortality) vs the county population size (in log scale). Highlight the top 300 raw cancer rates in red. Highlight the bottom 300 raw cancer rates in blue. Finally, on the same plot again, scatter plot the posterior mean cancer rate estimates vs the county population size, highlight these means in green.

  Using the scatter plot, explain why using the posterior means (from our model) to estimate cancer rates is preferable to studying the raw rates themselves.

```python
scatter_plot_cancer_rates(df, estimates=thetas_mean)
plt.legend()
plt.show()
```

**Answer:**

Comparing to previous plot, we can observe that the posterior means to estimate cancer rate produce less extreme pct_mortality result. For countries with low population, it no longer has extreme high or low cancer rate, instead, they trend towards population average. It is preferable because the mean estimate mitigates the effect of small population size and overfitting problem.

## Part II: Hierarchical Bayes
Rather than choosing fixed constants for the hyperparameters $\alpha, \beta$, following the Bayesian philosophy, we typically put additional priors on quantities of which we are uncertain. That is, we model the kidney cancer rates using a ***hierarchical model***:

\begin{aligned}
y_j| \theta_j &\sim Poisson(5 \cdot n_j \cdot \theta_j), \quad j = 1, \ldots, N\\
\theta_j | \alpha, \beta &\sim Ga(\alpha, \beta), \quad j = 1, \ldots, N\\
\alpha &\sim Ga(a, b)\\
\beta &\sim Ga(c, d)
\end{aligned}
where $a, b, c, d$ are hyperparameters.

1.  **(Posterior Marginal Means)** Produce a scatter plot of the raw cancer rates (pct mortality) vs the county population size (in log scale). Highlight the top 300 raw cancer rates in red. Highlight the bottom 300 raw cancer rates in blue. Finally, on the same plot again, scatter plot the mean of the posterior marginal distribution over $\theta_j$, i.e. $p(\theta_j|y_1, \ldots, y_N)$, vs the county population size (in log scale), highlight these means in orange.

  You should use `pymc3` to sample from the posterior. Compare `pymc3`'s sampler with your sampler from the previous homework, what is the difference (if any) in the performance of these samplers?

```python
# import relevant package
import pymc3 as pm
from pymc3 import model_to_graphviz
```

```python
# hyperparameters
a, b, c, d = 9, 6, 9, 0.00001
burn_in = 0.1
thinning = 10

y = df['dc'].values
n = df['pop'].values
average_pop = n.mean()

```

```python
#define hierarchical model in pymc3
with pm.Model() as hierarchical_model:
    #priors on alpha
    alpha = pm.Gamma('alpha', alpha=a, beta=b)
    #priors on beta
    beta = pm.Gamma('beta', alpha=c, beta=d)
    #priors on theta
    theta = pm.Gamma('theta', alpha=alpha, beta=beta, shape=len(y))

    #convert rate into number of disease incidents
    mu = theta * 5 * n

    #likelihood
    y_obs = pm.Poisson('y', mu=mu, observed=y)

```

```python
#draw graphical model for the thing we just defined
# model_to_graphviz(hierarchical_model)

```

```python
with hierarchical_model:
    # using default sampler
    trace = pm.sample(1000, tune=1000, target_accept = 0.9)
    print(f'DONE')
```

    <ipython-input-13-995cae4c87d5>:3: FutureWarning: In v4.0, pm.sample will return an `arviz.InferenceData` object instead of a `MultiTrace` by default. You can pass return_inferencedata=True or return_inferencedata=False to be safe and silence this warning.
      trace = pm.sample(1000, tune=1000, target_accept = 0.9)
    Auto-assigning NUTS sampler...
    Initializing NUTS using jitter+adapt_diag...
    Multiprocess sampling (4 chains in 4 jobs)
    NUTS: [theta, beta, alpha]

  <progress value='8000' class='' max='8000' style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [8000/8000 00:34<00:00 Sampling 4 chains, 0 divergences]

    Sampling 4 chains for 1_000 tune and 1_000 draw iterations (4_000 + 4_000 draws total) took 48 seconds.
    The number of effective samples is smaller than 25% for some parameters.

    DONE

```python
N = len(trace['theta'])
theta_trace = trace['theta'][int(burn_in * N)::thinning]
pymc_post_mean = np.mean(theta_trace, axis=0)
```

```python
# make scatter plot
scatter_plot_cancer_rates_orange(df,estimates = pymc_post_mean )
plt.title("pct_mortality vs pop in kcancer dataset")
plt.legend()

```

**Interpretation**:

From this graph, we could observe that mean estimate from the posterior marginal distribution that sampled by hierarchical model using pymc3 now shrink to population mean, with more shrinkage for low population counties.

```python
fig, ax = plt.subplots(1, 2, figsize=(30, 5))
#N = len(trace['theta'].T[county_1])
alpha_trace = trace['alpha'][int(burn_in * N)::thinning]
beta_trace = trace['beta'][int(burn_in * N)::thinning]

ax[0].plot(range(len(alpha_trace)), alpha_trace, color='gray')
ax[0].set_title('Trace Plot for Alphas')
ax[0].set_ylabel('alpha')
ax[1].plot(range(len(beta_trace)), beta_trace, color='gray')
ax[1].set_title('Trace Plot for Betas')
ax[1].set_ylabel('beta')
plt.show()
```

```python
#select the county to visualize the traceplot for theta
#same countries to compare with last homework
county_1 = 0
county_2 = 1

#plot the traceplots for one theta and alpha and beta
fig, ax = plt.subplots(1, 2, figsize=(30, 5))
N = len(trace['theta'].T[county_1])
theta_trace = trace['theta'][int(burn_in * N)::thinning].T[county_1]
ax[0].plot(range(len(theta_trace)), theta_trace, color='gray', alpha=0.5)
ax[0].set_title('trace plot for theta_{}'.format(county_1))

theta_trace = trace['theta'][int(burn_in * N)::thinning].T[county_2]
ax[1].plot(range(len(theta_trace)), theta_trace, color='gray', alpha=0.5)
ax[1].set_title('trace plot for theta_{}'.format(county_2))
```

**Interpretation**:

Comparing to last homework where visual graph show clearly evidence of non-convergence of $\alpha$ and $\beta$ after multiple rounds of Gibbs-MH sampling, this graph shows a considerable level of convergence, and looks better than last time.

I selected two counties that is the same as last time in homework 4 and compare for posterior marginal distribution of $\theta$. It seems they have converged and are fluctuating within a small range too. It seems this sampler perform better than our Gibbs MH sampler implemented. This could be due to a better choice of proposal alpha beta distribution and hyperparameters, and a more complex implementation of sampling method.

2.  **(Hierarchical Bayes vs Empirical Bayes)** Compare the shrinkage of the posterior marginal means of the hierarchical model to the shrinkage of the posterior means from the Bayesian model with empirical Bayes estimates for $\alpha, \beta$. What is the difference in shrinkage between the full hierarchical model and the Bayesian model with empirical Bayes?

```python
scatter_plot_cancer_rates(df, estimates=thetas_mean)
plt.title("pct_mortality vs pop in kcancer dataset (empirical bayes)")
plt.legend()
plt.show()

scatter_plot_cancer_rates_orange(df, estimates = pymc_post_mean)
plt.title("pct_mortality vs pop in kcancer dataset (hierachical bayes)")
plt.legend()
plt.show()
```

**Interpretation**:

Comparing two plots with green shrinkage (empirical bayes) and the orange shrinkage (hierachical bayes) of mean estimates, we could observe a larger shrinkage for the counties with least population in hierachical bayes estimations than empirical bayes estimates.

The hierarchy model pulls together the pct_mortality drastically for counties with small population size, while empirical bayes still shows strips of trends of pct_mortality that are not shrinked to a population mean for counties with small population size. The variations observed in hierachial bayes estimate are that there exists higher variance in the middle parts of population size. However, this doesn't seem to correlated with population size, nor due to the result of overfitting. It could be random variation instead of systematic variation. With hierachical bayes we can safely conclude that the cancer rate is almost the same for all counties regardless of population size, which also makes much more sense in real life.

## Part III: Broader Impact Analysis

Starting in 2020, major machine learning conferences are beginning to ask authors as well as reviewers to explicitly consider the broader impact of new machine learning methods. To properly evaluate the potential good or harm that a piece of technology (AI or not) can do to the general public, we need to be aware that no technology is deployed in ideal conditions or in perfectly neutral contexts. In order to assess the potential broader impact of technology, we need to analyze the social systems/institutions of which these technologies will become a part.

To help you analyze the broader impact of your technology, begin by considering the following questions:

I. Identify the relevant socio-technical systems
  - In what social, political, economic system could the tech be deployed?
  - How would the tech be used in these systems (what role will it take in the decision making processes)?

II. Identify the stakeholders
  - Who are the users?
  - Who are the affected communities (are these the users)?

    ***Hint:*** users are typically decision makers who will use the technology as decision aids (e.g. doctors), whereas affected communities may be folks who are impacted by these decisions but who are not represented in the decision making process (e.g. patients).

III. What types of harm can this tech do?
  - What kinds of failures can this tech have?
  - What kinds of direct harm can these failures cause?
  - What kinds of harm can the socio-technical system cause?

    ***Hint:*** many technical innovations have niche applications, they may sit in a long chain of decision making in a complex system. As such, it may seem, at first glance, that these technologies have no immediate real-life impact. In these cases, it’s helpful to think about the impact of the entire system and then think about how the proposed innovations aid, hamper or change the goals or outcomes of this system.

IV. What types of good can this tech do?
  - What kinds of needs do these users/communities have?
  - What kinds of constraints do these users/communities have?

1. **(Computational Footprint)**  In Homework #4, we considered the broader impact of the this hierarchical model for kidney cancer: we examined under what circumstances a hierachical model for kidney cancer is more preferable to a MLE model or a Bayesian model with hand-picked priors. In this problem, we compare hierarchical Bayes and empirical Bayes.

  In practical terms, what are the real-life advantages and drawbacks of implementing an empirical Bayesian model compared with an hierarchical Bayesian model?

  **Hint:** for example, compare how tractable it is to implement the inference algorithm for each model, compare how easy it is to for the inference method for each model to converge (would you know that your training has converged?). Consider also: how easily could a practitioner diagnose the training process and how could they evaluate the model fit and what kinds of computational resurces would be required to train each model.

**Answers**:

When put this model for kidney cancer and health care resources, we are sampling and estimating $\theta$, which is the underlying kidney cancer rate for that county. The other components of the decision system in which the model will deployed will be citizens living in that county, patients receiving kidney cancer related healthcare or treatments, government, resources department, hospital, doctors etc.

End user will be government, resources department, hospital, doctors etc;
Affected communities will be citizens living in that county, patients receiving kidney cancer related healthcare or treatments.

Whether it is more preferable to use a hiearchical model, an MLE model, a Bayesian model with hand-picked priors, or here, an empirical model depends more on a downstream task.

* Real life advantages:
   * Easy to implement and compute, more tractable to implement the inference algorithm for empirical Bayesian model compared with an hierarchical Bayesian.
   * As we only need to look at evidence moments, it is more easy to understand and interpret to end user and community.
   * We don't need to hand pick prior that incorporate our prior belief if we have no experience in the field. It avoids the risk of a poor choice of prior.
   * It will converge when data size gets larger.

**Drawbacks of implementing an empirical Bayesian model compared with an hierarchical Bayesian model:**
   * It may heavily depend on data quality and data size. Unlike how little poor choice of hierarchical model was, it has less effect on estimatation $\theta$. However, the data quality in empirical model heavily decides our estimation.
   * It may not converge, or the mean estimate we observed may not shrink to a range. There's still overfitting in low population or small size data points. Those data points are unable to borrow statistical power from data points with more populations like in hierarchial models.
   * If not converged or shrink to a population mean, this will leads to difficulty for government and agency to figure out where to fund, which county has higher kidney cancer rate.
   * The needs for evidence and samples will leads to questions about where to put clinical trials, how many trials shall we obtain in terms of data points; Healthcare data points is unlike other data points to collect, it needs a lot of resource and invests to obtain.
   * If we overestimate the cancer rate, it's not serving anything or communities; it's not serving people who needed; The actual death rate is not that high; If we underestimate the cancer rate, we might under devote health care resources to needed people and county;
   * We might also missing out envrionment factors that affect the rate.
   * The algorithm may not always be tractable. If the population is non-conjugate prior, then it would be hard to sample from population to get needed data points for empirical bayesian model.
   * It's hard to diagnose issue within the data after we obtain the estimation.

2. **(Mitigation of Potential Negative Impacts)** Based on your answers for the previous question and your broader impact analysis from HW#4, what information would the model designer/engineer need to disclose to the end-users to mitigate potentially negative impacts?

  **Hint:** consider how your end-user would be able to validate the model (how would they know if the model is working or has ceased to work once it has been deployed), how should the end-user choose between implementing a hiearchical Bayesian model or an empirical Bayesian model, how could the end-user identify and mitigate negative impacts (if they occur).

**Answers**:

The potential downside and negative impacts, as mentioned briefly above, could be that
- If we overestimate the cancer rate, it's not serving anything or communities; it's not serving people who needed; The actual death rate is not that high; If we underestimate the cancer rate, we might under devote health care resources to needed people and county;
- If there exists systematic inequality, the model are helping to exaggerated it. For example, in rural counties they might already have low income, bad infrastructure, and a bad sampling model will help to systematically exaggerating the circumstance.

To mitigate negative impacts, end users should be able to validate the models by convergence check (traceplots etc), cross validation with human experts and experienced professionals. They should also treat end result with care and involve human decison making instead of fully taking computed results.

For patients and other affected communities, end users should give full disclosure of how model is being used and deployed, giving advice per treatment. Besides, other environmental factors should also be reviewed before making decision.

The next thing we would like to consider is that if any of these mistakes or negative impacts happening at random?

- Data inbalance in healthcare is a huge problem (For example, gender data inequality, in many cases there's seldom woman data points, which is regarded as a commonly underpresented group.) The model may exaggerate this problem, and also overfit to one sub-population while be harmful to predictions in other subpopulation.
- Model for these data are qualitatively different. Studies in man doesn't generalize to women.
- For subgroups that are previously undersample, we will likely to remove them, as they look like outliers (Small data inside big data problem).
- Thus, maybe consider using a mixture of several models to compare and cross check for estimation, and also allow small data size points to borrow statistical power from large data size points like what hierarchial model does would be a preferable choice.


```python
%matplotlib inline

import matplotlib
import numpy as np
import matplotlib.pyplot as plt

plt.rcParams["figure.figsize"] = (12, 9)
plt.rcParams["font.size"] = 18
```

# Radioactivity

Learning Objectives:

- Explain how radioactivity was discovered
- Explain the nuclear physical reason for radioactive decay
- List the sources of natural and man made radiation
- Read and understand a decay diagram
- Calculate Q values for various types of decay
- Describe the physics of various types of decay
- State the radioactive decay law
- Derive the radioactive decay law
- Understand how incorporating sources impacts decay calcuations
- Calculate decay with production for simple cases

## Discovery of Radioactivity
- Radioactivity was first discovered in 1896 by  Henri Becquerel, while working on phosphorescent materials.
- These materials glow in the dark after exposure to light, and he thought that the glow produced in cathode ray tubes by X-rays might be connected with phosphorescence.
- He wrapped a photographic plate in black paper and placed various phosphorescent minerals on it.
- All results were negative until he used uranium salts.

Becquerel in the lab

Photographic plate made by Henri Becquerel showing effects of exposure to radioactivity.

**What is nuclear decay?**
A spontaneous process where the protons and neutrons in a given nucleus are rearranged into a lower energy state.
The transition may involve levels of the same nucleus (gamma emission, internal conversion) or levels of a different nucleus (alpha, beta decay).

**Why do nuclei decay?** A nucleus decays when it is **unstable**.  It undergoes decay in order to become more stable (lower energy state).

**Where do the unstable nuclei come from?**

## Sources of Radiation

> A chart of the public's exposure to ionizing radiation (displayed below)
> shows that people generally receive a total annual dose of about 620 millirem.
> Of this total, natural sources of radiation account for about 50 percent,
> while man-made sources account for the remaining 50 percent. -- US NRC

## Natural Sources

### Terrestrial

Uranium, thorium, radium, etc. are all present naturally inside rocks and soil and are part of the four major decay chains. Others are 'primordial.' Primoidal radioactive isotopes include about 20 isotopes that are long lived and not part of any decay chain (K-40, Rubidium-87) Potassium has a quite radioactive isotope, $^{40}K$. (bananas)

Evolution of the earth's radiogenic heat (in the mantle).

### Internal

Mostly $^{40}K$ and $^{14}C$ inside your body.

### Cosmic
Commonly, cosmic radiation includes $^{14}C$, tritium ($^3H$), and others.

CC BY-SA 3.0, Link

Air showers ensuing from very-high-energy cosmic rays can enter Earth’s atmosphere from multiple directions. Credit: Simon Swordy/NASA.

## Man-Made Radiation

Of approximately 3200 known nuclides:
- **266** are stable,
- **65** long-lived radioactive isotopes are found in nature,
- and the remaining **~2900** have been made by humans.

One of the heaviest named elements is Livermorium (Z=116), its most stable isotope $^{293}Lv$ has half-life $t_{1/2} = 60 ms$.

### Types of Decay

```python
# The below IFrame displays Page 121 of your textbook:
# Shultis, J. K. (2016). Fundamentals of Nuclear Science and Engineering Third Edition,
# 3rd Edition. [Vitalsource]. Retrieved from https://bookshelf.vitalsource.com/#/books/9781498769303/

from IPython.display import IFrame
IFrame("https://bookshelf.vitalsource.com/books/9781498769303/pageid/121", width=1000, height=1000)

```

All elements smaller than Z=83 (Bismuth) have at least one stable isotope.
Exceptions: Technetium (Z=43) and Promethium (Z=61).

Once nucleus gets past a certain size, it is unstable
The largest stable nucleus is Pb-208, all isotopes larger than this are unstable

Nuclear conservation laws apply. The following are conserved:
- charge
- number of nucleons (proton + neutron)
- total energy (mass + energy)
- linear momentum (in inertial frame of reference)
- angular momentum (spin)
  - alternatively: leptons (electrons + neutrinos)

```python
# The below IFrame displays Page 100 of your textbook:
# Shultis, J. K. (2016). Fundamentals of Nuclear Science and Engineering Third Edition,
# 3rd Edition. [Vitalsource]. Retrieved from https://bookshelf.vitalsource.com/#/books/9781498769303/
from IPython.display import IFrame
IFrame("https://bookshelf.vitalsource.com/books/9781498769303/pageid/122", width=1000, height=1000)
```

## Energetics of Decay

### Alpha ($\alpha$) Decay

An $\alpha$ particle is emitted.
The daughter is left with 2 fewer neutrons and 2 fewer protons than the parent.

\begin{align}
^{A}_{Z}P \longrightarrow ^{A-4}_{Z-2}D^{2-} + ^4_2\alpha
\end{align}

\begin{align}
\frac{Q}{c^2} = M\left(^{A}_{Z}P\right)-\left[ M\left(^{A-4}_{Z-2}D\right) + M\left(^{4}_{2}He\right)\right]
\end{align}

### Gamma ($\gamma$) Decay

An excited nucleus decays to its ground state by the emission of a gamma photon.

\begin{align}
^{A}_{Z}P^* \longrightarrow ^{A}_{Z}P + \gamma
\end{align}

\begin{align}
Q = E^*
\end{align}

### Negatron ($\beta -$) Decay
A neutron changes into a proton in the nucleus (nuclear weak force). An electron ($\beta -$) and an antineutrino ($\bar{\nu}$) are emitted.

\begin{align}
^{A}_{Z}P \longrightarrow ^{A}_{Z+1}D^{+} + ^0_{-1}e + \bar{\nu}
\end{align}

\begin{align}
\frac{Q}{c^2} = M\left(^{A}_{Z}P\right)- M\left(^{A}_{Z+1}D\right)
\end{align}

### Positron ($\beta +$) Decay

A proton changes into a neutron in the nucleus (nuclear weak force). A positron ($\beta +$) and a neutrino ($\nu$) are emitted.

\begin{align}
^{A}_{Z}P \longrightarrow ^{A}_{Z-1}D^{-} + ^0_{+1}e + \nu
\end{align}

\begin{align}
\frac{Q}{c^2} = M\left(^{A}_{Z}P\right)- \left[ M\left(^{A}_{Z-1}D\right) + 2m_e\right]
\end{align}

### Electron Capture
1. An orbital electron is absorbed by the nucleus,
2. converts a nuclear proton into a neutron and a neutrino ($\nu$),
3. and typically leaves the nucleus in an excited state.

\begin{align}
^{A}_{Z}P + \left(^0_{-1}e\right) \longrightarrow ^{A}_{Z-1}D + \nu
\end{align}

\begin{align}
\frac{Q}{c^2} = M\left(^{A}_{Z}P\right)-  M\left(^{A}_{Z-1}D\right)
\end{align}

### Proton Emission
A proton is ejected from the nucleus.

\begin{align}
^{A}_{Z}P \longrightarrow ^{A-1}_{Z-1}D^{-} + ^1_1p
\end{align}

\begin{align}
\frac{Q}{c^2} = M\left(^{A}_{Z}P\right)-\left[ M\left(^{A-1}_{Z-1}D\right) + M\left(^{1}_{1}H\right)\right]
\end{align}

The decay of a proton rich nucleus A populates excited states of a daughter nucleus B by β+ emission or electron capture (EC). Those excited states that lie below the separation energy for protons (Sp) decay by γ emission towards the groundstate of daughter B. For the higher excited states a competitive decay channel of proton emission to the granddaughter C exists, called β-delayed proton emission.

### Neutron Emission
A neutron is ejected from the nucleus.

\begin{align}
^{A}_{Z}P \longrightarrow ^{A-1}_{Z}P + ^1_0n
\end{align}

\begin{align}
\frac{Q}{c^2} = M\left(^{A}_{Z}P\right)-\left[ M\left(^{A-1}_{Z}P\right) + m_n\right]
\end{align}

### Internal Conversion

The excitation energy of a nucleus is used to eject an orbital electron (typically a K-shell) electron.

\begin{align}
^{A}_{Z}P^* \longrightarrow ^{A-1}_{Z}P^+ + ^0_{-1}e
\end{align}

\begin{align}
Q = E* - BE^K_e
\end{align}

### An Aside on The Nuclear Weak Force

Beta ($\beta \pm$) decay is a consequence of the **weak force**, which is characterized by long decay times (thousands of years). Nucleons are composed of up quarks and down quarks, and **the weak force allows a quark to change type** by the exchange of a W boson and the creation of an electron/antineutrino or positron/neutrino pair. For example, **a neutron, composed of two down quarks and an up quark, decays to a proton composed of a down quark and two up quarks.**

## Radioactive Decay Law

**The Law:** The probability that an unstable parent nucleus will decay spontaneously into one or more particles of lower mass/energy is independent of the past history of the nucleus and is the same for all radionuclides of the same type.

### Decay Constant
Radioactive decay takes place stochastically in a single atom, and quite predictably in a large group of radioactive atoms of the same type.

The probability that any one of the radionuclides in the sample decays in $\Delta t$ is $\frac{\Delta N}{N}$. This **decay probability** $\frac{\Delta N}{N}$ per unit time for a time interval $\Delta t$ should vary smoothly for large $\Delta t$.

The statistically averaged **decay probability per unit time**, in the limit of infinitely small $\Delta t$, approaches a constant $\lambda$.
Thus:

\begin{align}
       \lambda &= \mbox{decay constant}\\
        &\equiv \lim_{\Delta t \to 0} \frac{\left(\Delta N/N\right)}{\Delta t}\\
N(t) &= \mbox{expected number of nuclides in a sample at time t}\\
\implies \mbox{ } -dN &= \mbox{decrease in number of radionuclides}\\
                      &= \lambda N(t) dt\\
\implies \mbox{ } \frac{dN(t)}{dt} &=-\lambda N(t)\\
\end{align}

The above is a differential equation. The solution to this differential equation give a definition of N as a function of t. We can now describe the change in radioactive nuclides over time as:

\begin{align}
        \frac{dN}{dt} &= -\lambda N \\
       \Rightarrow N_i(t) &= N_i(0)e^{-\lambda t}\\
\end{align}

where

\begin{align}
        N_i(t) &= \mbox{number of isotopes i adjusted for decay}\\
        N_i(0)&= \mbox{initial condition}\\
        \end{align}

```python
import math
def n_decay(t, n_initial=100, lam=1):
    """This function describes the decay of an isotope"""
    return n_initial*math.exp(-lam*t)

# This code plots the decay of an isotope
import numpy as np
y = np.arange(6.0)
x = np.arange(6.0)
for t in range(0,6):
    x[t] = t
    y[t] = n_decay(t)

# creates a figure and axes with matplotlib
fig, ax = plt.subplots()
scatter = plt.scatter(x, y, color='blue', s=y*20, alpha=0.4)
ax.plot(x, y, color='red')

# adds labels to the plot
ax.set_ylabel('N_i(t)')
ax.set_xlabel('Time')
ax.set_title('N_i')

# adds tooltips
import mpld3
labels = ['{0}% remaining'.format(i) for i in y]
tooltip = mpld3.plugins.PointLabelTooltip(scatter, labels=labels)
mpld3.plugins.connect(fig, tooltip)

mpld3.display()
```

## Half Life

All dynamic processes which decay (or grow) exponentially can be characterized by their half life (or doubling time). In the case of radioactive decay

\begin{align}
        \tau_{1/2}&= \mbox{half-life}\\
               &=\frac{ln(2)}{\lambda}  \\
        t &= \mbox{time elapsed [s]}\\
        \tau_{1/2} &= \mbox{half-life [s]} \\
\end{align}

```python
# This code converts decay constant to half life

def half_life(lam):
    return math.log(2)/lam
lam = 1.0

# This code plots the decay of an isotope for various half lives
import numpy as np
y = np.arange(8.0)
x = np.arange(8.0)

lives = np.arange(half_life(lam), 9*half_life(lam), half_life(lam))
for i, t in enumerate(lives):
    x[i] = float(t)
    y[i] = n_decay(t)

# creates a figure and axes with matplotlib
fig, ax = plt.subplots()
scatter = plt.scatter(x, y, color='blue', s=y*20, alpha=0.4)
ax.plot(x, y, color='red')

# adds labels to the plot
ax.set_ylabel('N_i(t)')
ax.set_xlabel('Time')
ax.set_title('N_i')

# adds tooltips
import mpld3
labels = ['{0}% remaining'.format(i) for i in y]
tooltip = mpld3.plugins.PointLabelTooltip(scatter, labels=labels)
mpld3.plugins.connect(fig, tooltip)

mpld3.display()
```

After n half lives, only $\frac{1}{2^n}$ of the original sample remain.
\begin{align}
N(n\tau_{1/2})= \frac{1}{2^n}N_0
\end{align}

In a sample, $N_0$ has has been reduced to a fraction of $N_0$, $\epsilon$. How many half lives have passed?

\begin{align}
n = \frac{-ln\epsilon}{ln2}
\end{align}

Finally, the radioactive decay law can be expressed using the half-life:

\begin{align}
N(t) = N_0\left(\frac{1}{2}\right)^{t/\tau_{1/2}}
\end{align}

## Decay by competing processes

\begin{align}
\frac{dN(t)}{dt} &= -\lambda_1N(t) - \lambda_2N(t) - \cdots \lambda_nN(t)\\
 &= -\sum_{i=1}^n \lambda_iN(t)\\
 &\equiv-\lambda N(t)
\end{align}

A nuclide will decay by the $i^{th}$ mode with probability $f_i$.

\begin{align}
f_i &= \frac{\mbox{decay rate by ith mode}}{\mbox{decay rate by all modes}}\\
    &=\frac{\lambda_i}{\lambda}
\end{align}

## Decay with Production

In reactors, isotopes decay into one another and still others are born from fission. Thus, if there is production, we can rewrite the standard decay differential equation as:

\begin{align}
\frac{dN(t)}{dt} &= -\mbox{rate of decay} + \mbox{rate of production}\\
\implies N(t) &= N_0 e^{-\lambda t} + \int_0^t dt'Q(t')e^{-\lambda (t-t')}\\
\end{align}

If the production rate is constant, this simplifies:

\begin{align}
N(t) &= N_0 e^{-\lambda t} + \frac{Q_0}{\lambda}\left[1-e^{-\lambda t}\right]\\
\end{align}

### Fuel Depletion

Decays, fissions, and absorptions compete throughout the life of the reactor.

#### Aside: Reaction Rates

In a reactor, this Q can be characterized via reaction rates.

- The microscopic cross section $\sigma_{i,j}$ is just the likelihood of the event per unit area.
- The macroscopic cross section $\Sigma_{i,j}$is just the likelihood of the event per unit area of a certain density of target isotopes.
- The reaction rate is the macroscopic cross section times the flux of incident neutrons.

\begin{align}
R_{i,j}(\vec{r}) &= N_j(\vec{r})\int dE \phi(\vec{r},E)\sigma_{i,j}(E)\\
R_{i,j}(\vec{r}) &= \mbox{reactions of type i involving isotope j } [reactions/cm^2s]\\
N_j(\vec{r}) &= \mbox{number of nuclei participating in the reactions }\\
E &= \mbox{energy}\\
\phi(\vec{r},E)&= \mbox{flux of neutrons with energy E at position i}\\
\sigma_{i,j}(E)&= \mbox{cross section}\\
\end{align}

We said this can be written more simply as $R_x = \sigma_x I N$, where I is intensity of the neutron flux. In the notation of the above equation, we can describe the production of an isotope by neutron absorption by another isotope as :

\begin{align}
\mbox{isotope i production via neutron absorption in m} = f_{im}\sigma_{am}N_m  \phi
\end{align}

### Total composition evolution

\begin{align}
\frac{dN_i}{dt} &= \sum_{m=1}^{M}l_{im}\lambda_mN_m + \phi\sum_{m=1}^{M}f_{im}\sigma_mN_m - (\lambda_i + \phi\sigma_i + r_i - c_i)N_i + F_i\Big|_{i\in [1,M]}\\
\end{align}
\begin{align}
N_i &= \mbox{atom density of nuclide i}\\
M &= \mbox{number of nuclides}\\
l_{im} &= \mbox{fraction of decays of nuclide m that result in formation of nuclide i}\\
\lambda_i &= \mbox{radioactive decay constant of nuclide i}\\
\phi &= \mbox{neutron flux, averaged over position and energy}\\
f_{im} &= \mbox{fraction of neutron absorption by nuclide m leading to the formation of nuclide i}\\
\sigma_m &= \mbox{average neutron absorption cross section of nuclide m}\\
r_i &= \mbox{continuous removal rate of nuclide i from the system}\\
c_i &= \mbox{continuous feed rate of nuclide i}\\
F_i &= \mbox{production rate of nuclide i directly from fission}\\
\end{align}

### Example: $^{135}Xe$

**Discussion: What is interesting about Xenon?**

$^{135}Xe$ is produced directly by fission and from the decay of iodine.

\begin{align}
\frac{dN_{xe}}{dt} &= \sum_{m=1}^{M}l_{Xem}\lambda_mN_m + \phi\sum_{m=1}^{M}f_{Xem}\sigma_mN_m - (\lambda_{Xe} + \phi\sigma_{Xe} + r_{Xe} - c_{Xe})N_{Xe} + F_{Xe}\\
 &= -\lambda_{Xe}N_{Xe}  - \sigma_{aXe}\phi N_{Xe} + \lambda_IN_I + F_{Xe}\\
 &= -\lambda_{Xe}N_{Xe}  - \sigma_{aXe}\phi N_{Xe} + \lambda_IN_I + \gamma_{Xe}\Sigma_f\phi\\
 \gamma_{Xe} &= 0.003\\
 \gamma_{I} &= 0.061\\
\end{align}

### Example: $^{239}Pu$

\begin{align}
\frac{dN_{Pu}}{dt} &= \sum_{m=1}^{M}l_{Pum}\lambda_mN_m + \phi\sum_{m=1}^{M}f_{Pum}\sigma_mN_m - (\lambda_{Pu} + \phi\sigma_{Pu} + r_{Pu} - c_{Pu})N_{Pu} + F_{Pu}\\
\end{align}

Let's formulate this equation together.

$$\mathrm{^{238}_{\ 92}U \ + \ ^{1}_{0}n \ \longrightarrow \ ^{239}_{\ 92}U \ \xrightarrow [23.5\ min]{\beta^-} \ ^{239}_{\ 93}Np \ \xrightarrow [2.3565\ d]{\beta^-} \ ^{239}_{\ 94}Pu}$$

- Decay of what nuclides result in the formation of $^{239}Pu$?
- Does $^{239}Pu$ decay?
- Is there a nuclide that becomes  $^{239}Pu$ after it absorbs a neutron?
- Does  $^{239}Pu$ ever absorb neutrons?
- Is  $^{239}Pu$ ever produced directly from fission?

## Burnable Poisons

- Gadolinia ($Gd_2O_3$) or erbia ($Er_2O_3$) common
- Natural Erbium consists of Er166, Er167, Er168  and Er170 primarily.  Er167 has large thermal  cross section.
- Gd is an early life burnable poison, typically gone by 10‐20 GWd
- Boron also used widely.
- Can be mixed with the fuel or a coating on the pellet.

\begin{align}
\frac{dN^P(t)}{dt} &= -g(t)\sigma_{aP}N^P(t)\phi\\
g(t) &= \frac{\mbox{average flux inside BP}}{\mbox{average flux in core}}\\
\sigma_{aP} &=\mbox{neutron absorption cross section of the BP}\\
N^P(t) &= \mbox{number of atoms of the BP at time t}
\end{align}

```python

```