# Click "Edit App" to see the code

Python is a very powerful object oriented programming language, which is used in most fields of science both for computing and for data analysis.
This document is a very simplistic introduction to Python written by a non-Python expert, and is designed to be a quick introduction to Python for CHEM2000 students.
In many places the nomenclature is likely to be inaccurate and you are encouraged to consult more rigorous resources developed by real Python programmers.

This document is aimed at providing enough knowledge to solve all the numerical laboratories that are part of the CHEM2000 unit using Python, but it is far from comprehensive. There are many different ways to solve problems numerically, different libraries/packages can be used and here we are providing only one (or a few) possible way of doing things.

The entire document and the examples therein have been developed using a **Jupyter Notebook**, and work best there.

A Jupyter notebook consists of a series of cells that can be either _text_ or _code_, the cell can then be _executed_ by pressing the **run** button or _Shift+Return_ (if you want to look like a pro). If things go pear-shaped, restarting the kernel and rerunning all the cells may help; this can be done by pressing the _fast forward_ button.

Text cells accept _Markdown_ syntax and once executed will render the text. Markdown provides a simple way of producing formatted text and it accept _LaTeX_ commands for equations, which look much nicer that those made with the equation editor in MS Word. For a Markdown tutorial click [here](https://www.markdowntutorial.com)

The code cells contain Python 3 commands. Note that some Python 3 is different from Python 2, _e.g._ the **print** command works differently.

# My first Jupyter Notebook
In general you would use the first _code_ cell to import all the packages that we need to use in the remainder of the code. We can import entire packages, part of the packages and assign aliases to the package name, _e.g._

```python
# python packages
import pandas as pd # Dataframes and reading CSV files
import numpy as np # Numerical libraries
import matplotlib.pyplot as plt # Plotting library
from lmfit import Model # Least squares fitting library
```

this is particularly important because it ensures that if we define a variable that has the same name of a function belonging to that package there is no confusion about what the code does.
```python
# This is a variable
mean = 10

# This is the NumPy function to compute the mean of an array of numbers
np.mean([1,2,3])
```

Things to keep in mind when programming in Pythons are

1. It is Case sensitivE

```python
average = 0
```
is different from
```python
aVeRagE = 0
```

2. Variables' names cannot have spaces. This line would give you a _syntax error_
```ptyhon
number of values = 10
```

3. Spaces between operators are ignored, _1+2_ is the same as _1 + 2_

```python
print(1 + 2)
print(1+2)
```

4. \# is a comment and everything on its right is ignored

```python
# This is a comment
print("Hello world") # This is also a comment
```

5. A single command can spread over multiple lines. If we split the line between variables no continuation line character is required. Otherwise we can use the "\" character. Note the different results of the two commands below.

```python
print("Hello",
      "world")
print("Hello \
       world")
```

6. Indentation matters! Indentation is used to define the content of loops, functions... This will be more clear when we start using functions.

# Good programming practices

1. Use meaningful name for your variables so that you know what's inside, there is no character limit.
2. Use a consistent style and convention for your variables, the code will look neater. I like to use the _camel case_ style (numberOfValues) that allows me to have separation between words without using spaces. Alternatively you can use the _Pascal case_ style (NumberOfValues) or use the underscore character to separate words (number_of_variables) or create your own style.
3. Comment your code well, it may be obvious what it does when you write it, but it won't be so obvious after a year or more.

# Python as a simple calculator

Let's start by doing some simple mathematical operations using this Jupyter Notebook; addition, multiplication, division $\dots$, just to get familiar with the Jupyter Notebook.

```python
2 + 3
```

```python
4 * 3
```

```python
12 / 3
```

```python
2**3
```

The same operations can be done using variables. We can first define two variables _a_ and _b_ and then use them in in the following cells

```python
a = 12
b = 3
a + b
```

This is not a very efficient way of working because the result of the operation is not available to the rest of the code after the cell has been executed. Typically, we want to create new variables and than write the result using the _print_ command.

```python
a = 10
b = 20
c = a + b
print("Result :",c)
```

# Python as a scientific calculator
Not all scientific operators are natively available in Python, but they can be accessed through optional packages that are loaded at the beginning of your notebook, _e.g._ **NumPy**.
NumPy is one of the most commonly used Python libraries, it contains all the operators for square root, logarithm, exponential, the trigonometric function, a suite of constants and much more.
For more information see [https://numpy.org](https://numpy.org).

In the examples below we show how to use NumPy to access some of these functions.
* Note that we access the NumPy function through using the _np_ prefix because of the way we imported the NumPy library at the beginning of this notebook.

```python
print("The approximate value of pi is                     :",np.pi)
print("The approximate value of the Euler constant (e) is :",np.e)
print("The square root of two is                          :",np.sqrt(2))
print("The natural logarithm of two is                    :",np.log(2))
print("The logarithm base 10 of two is                    :",np.log10(2))
print("The sine of pi is                                  :",np.sin(np.pi))
print("The cosine of pi is                                :",np.cos(np.pi))
```

Even more useful than variables are arrays, which allow us to store many values in one place. Arrays can be created by hand or be the output of other Python functions.
They can contain numbers, strings or other variables, or mixed types

```python
arrayOfNumbers = [300, 2, 3.2]
print("Three dimensional array of numbers :",arrayOfNumbers)
arrayOfStrings = ["Temperature" , "Pressure" , "Volume"]
print("Three dimensional array of strings :",arrayOfStrings)
mixedArray = ["temperature" , 300]
print("The mixed array is                 :",mixedArray)
```

We can easily access one of the elements of the array, by specifying its location in the array.
* Note that Python starts counting from zero!

```python
print("Second element of the array of numbers :",arrayOfNumbers[1])
print("Second element of the array of strings :",arrayOfStrings[1])
```

We can also construct a loop to cycle over the elements of the array.
If we are interested in cycling over one array only we can use the **in** operator.
* It's important to note here that the indentation of the code determines where the loop finished

```python
for value in arrayOfNumbers:
    print("--- This is inside the loop ----",value)
print("--- This is outside the loop ---")
```

Alternatively we can create a loop over the indices of the array using the **range** iterator.
* Note the use of the function **len** to compute the size of the array!
* Note that the upper limit of the **range** iterator is not included!

```python
numberOfElements = len(arrayOfNumbers)
print("Number of elements :",numberOfElements)
for index in range(0,numberOfElements):
    print(index,                   # index
          arrayOfStrings[index],   # sting
          arrayOfNumbers[index])   # number
```

**range** is a special thing of python 3, and it produces a list of indices only when part of a loop. This is at variance with the NumPy **arange** function, which instead will produce an array that we can use normally.
Both the **range** and **arange** function typically take three arguments, the lower limit of the range (included) the upper limit of the range (not included) and the step. The main difference is that **range** being an iterator works only with integer numbers, while **arange** can also work with floating point numbers.
If the step is omitted, one is assumed.
Let's have a look at a couple of examples.

```python
for i in range(0,4):
    print(i)
```

```python
for i in range(0,4,2):
    print(i)
```

```python
for i in np.arange(0,4,2):
    print(i)
```

```python
for i in np.arange(0,4,0.5):
    print(i)
```

Let's now see how we can use the **range** iterator and the **np.arange** function to generate arrays of equally spaced values. While this is straightforward with the NumPy function, it is more complicated with **range** iterator, and we need to _recast_ the output into a list.

```python
values = np.arange(1.1,4,0.7)
print(values)

values = list(range(0,5,2))
print(values)
```

# Operations with arrays
As an illustrative example of using arrays and variables we can now compute the sum and average of all the elements in an array of numbers. In order to do that we initialise a variable to zero, and progressively add the array elemnts to it.
* Note how we can increment the value of the variable using the += operator. These two commands are equivalent
```python
summ += value
summ = summ + value
```
Those two line are also equivalent to the following, where we use a temporary variable, to explicitly show what the code does
```python
tmp = summ + value
summ = tmp
```
The operators -=, \*= and /= have analogous meanings.

```python
summ = 0.
for value in arrayOfNumbers:
    summ += value

print("Result of += :",summ)
```

The average can then be computed directly inside the print statement.

```python
print("Average :",summ/len(arrayOfNumbers))
```

Many simple operations on arrays can however be more efficiently performed using libraries such as NumPy, _e.g._ summation, average, standard deviation, etc.
Using these function will also make your code slimmer and easier to read.

```python
tally = np.sum(arrayOfNumbers)
average = np.mean(arrayOfNumbers)
StDev = np.std(arrayOfNumbers)

print("Sum :",tally)
print("Average :",average)
print("Standard Deviation :",StDev)
```

Unfortunately there is no NumPy function for computing the standard error, but we can easily compute that from its definition

\begin{equation}
StdErr = \frac{\sigma}{\sqrt{N}}
\end{equation}
where $\sigma$ is the standard deviation and $N$ the number of values used in the calculation.

```python
print("Standard Error :",StDev/np.sqrt(len(arrayOfNumbers)))
```

# Using DataFrames
Dataframes are powerful objects that are part of the _pandas_ package. The most simplistic description of a dataframe is that it is a multi-dimension mixed array. Dataframes are more than that, as they also include functions that operate on the DataFrame content.
This definition of DataFrames would probably horrify a Python programmer, but it would suffice for the purpose of this course.

Dataframes can be defined by hand, or created by other functions, _e.g._ by reading a Comma Separated Values file (.csv). Let's see first how we can create a an empty DataFrame, with three columns named "Temperature", "Volume" and "Pressure"; with their units.

```python
# This array is used to define the names of the columns
header = ["Temperature (K)" , "Volume (L)" , "Pressure (bar)"]

# This is our new dataframe
df = pd.DataFrame(data=None, columns=header)
print(df)
```

Let's now fill the dataframe using the ideal gas law

\begin{equation}
pV = nRT
\end{equation}

where $p$ is the pressure, $V$ the volume, $T$ the temperature and $R=8.314\ J/mol/K$ is the ideal gas constant, each expressed with the units specified in the header of the dataframe.

Let's compute the volume of an ideal gas at different pressures and temperatures.

For simplicity we fix the number of moles to 1.
* Note how we use the **range** function to create an array of integers and the NumPy **arange** function to create a an array of _floating point numbers_.
* The variable _index_ is used to count the number of elements that we already have in the DataFrame, and to add the next one. This works because Python starts counting from zero.
* Note we used the **loc** function to added an array to the DataFrame at a specific position.
* Note how we also created an array to store all the temperatures we generate, the array is created empty using **= []** and then we append elements to it using the **.append()** function

```python
R = 8.314 # kJ/mol
n = 1
# Conversion factor between J/bar to litre
conversionFactor = 0.01

listOfTemperatures = []
for T in range(100,301,50):
    listOfTemperatures.append(T)

    for p in np.arange(0.1,1,0.02):
        V = (n * R * T / p) * conversionFactor
        index = len(df.index)
        df.loc[index]  = [T , V , p] # a vector is added to the DataFrame

print(df)
```

There are many ways to access the data in a DataFrame. Here we'll show you two; one to quickly get an entire column of the array, and one to get selected chunks of data using the **iloc** function.

```python
print(df["Temperature (K)"])
```

For the DataFrame that we have, the **iloc** function takes two arguments, the row and columns indices.

```python
print(df.iloc[0,1])
```

We can then use "**:**" to specify a range of elements that we want to use.
* Note that the lower limit of the range is included while the upper limit is not !
* Note that if one limit of the range is missing, the start/end of the array is assumed
* Note how we have used **.values** to cast the output data in an array.

```python
print(df.iloc[0:3 , 1  ].values) # the first three volumes
print(df.iloc[0   , 0:3].values) # the first row
print(df.iloc[0   ,  : ].values) # the first row
print(df.iloc[0   , 1: ].values) # the last two elements of the first row
print(df.iloc[0   ,  :2].values) # the first two elements of the first row
```

# Making a graph
Let's now make a graph with these data using the **mathplotlib** library.
As a start we can make a plot of the entire DataFrame.

The **subplots** function creates two objects, the _figure_ and the _axes_ of the figure itself.
Each of those objects contain functions that we can use to customise the final plot. More in another tutorial.

```python
# Create the figure and axes objects
fig , ax = plt.subplots()
# Add the data to the plot from the DataFrame
ax.scatter(df["Pressure (bar)"] , df["Volume (L)"])
# Diplay the figure
plt.show()
```

Let's now do some data manipulation to make a better plot, using a line for each isotherm.
There are many ways of doing this, but here we'll take an educational approach and use _conditional_ statements to select parts of the dataframe, add them to arrays and plot them.

What we are going to do is to choose a temperature, _e.g._ 100K, and create two arrays (p,v) with the corresponding pressures and volumes, than we will use them for plotting.
* Note that we used the **scatter** and **plot** functions to plot the data as circles with a line overlaid.
* Note that we set the **set** function for to add the labels to the axes (_ax_)

```python
T=100
pressure = []
volume = []
for index in range(0,len(df.index)):
    if df.iloc[index,0] == T:
        pressure.append(df.iloc[index,2])
        volume.append(df.iloc[index,1])

print("Pressure array :",pressure[0:3],"...")
print("Volume array :",volume[0:3],"...")

fig , ax = plt.subplots()
ax.scatter(pressure , volume)
ax.plot(pressure , volume)

# Let's add the labels to the axes
ax.set(xlabel="Pressure (bar)")
ax.set(ylabel="Volume (L)")

plt.show()
```

If we want then to plot all isotherm in one graph we can wrap the code above in a loop over all the temperatures we have created.

* Note the indentation of the **for** loops and **if** conditional statement.
* Note how in this example we select portions of the DataFrame including a conditional statement in **[]** when we call the DataFrame
* Note that we used **.values** to transform the DataFrame into an array

```python
# First we have to create the figure
fig , ax = plt.subplots()

for T in listOfTemperatures:
    pressure = df[df["Temperature (K)"] == T]["Pressure (bar)"].values
    volume = df[df["Temperature (K)"] == T]["Volume (L)"].values

    # add one line to the plot for each temperature
    ax.plot(pressure , volume, label=T)

# Let's add the labels to the axes
ax.set(xlabel="Pressure (bar)")
ax.set(ylabel="Volume (L)")

# Let's also add the legend
ax.legend()

plt.show()
```


```python
import matplotlib.pyplot as plt
import plotly as py
import plotly.graph_objs as go
import numpy as np
import math
import ipywidgets as widgets
from IPython.display import display, Math, Latex, HTML, IFrame
from astropy.table import Table, Column
from ipywidgets import interact, interactive

py.offline.init_notebook_mode(connected=True)
%matplotlib inline

font = {'family' : 'sans-serif',
        'weight' : 'normal',
        'size'   : 14}

plt.rc('font', **font)

'''Above, we are importing all the necessary modules in order to run the notebook.
Numpy allows us to define arrays of values for our variables to plot them
matplotlib is what we use to create the figures
the display and widgets are to make the notebook look neat
'''

HTML('''
<form action="javascript:code_toggle()"><input type="submit" id="toggleButton" value="Show Code"></form>''')

```

***

  **Light and Optics**

***

Gif taken from https://giphy.com/gifs/fandor-sun-eclipse-3o7OsM9vKFH2ESl0KA/links, August 1st, 2018.
 Figure 1: For hundreds of years, scientists have tried to understand the nature of light. With advances in technology, and inventions like telescopes, we have been able to see farther than ever before.

***

## Introduction

Throughout most of history, humans did not understand light as we do today. As science and technology have progressed over time, so too has our knowledge of the nature of light.

In this lesson, when we say the word "light", we will be referring to visible light (light that comes from the sun, lightbulbs etc.). We will go over how a few key experiments kickstarted a new way of thinking, and a few of the ways that we are able to manipulate light. We will also talk about how our eyes enable us to see.

## Background

If you had to describe to someone what light is, you may have a hard time. Some people think of light as the absence of darkness, but even that doesn't say much about light itself.

Our understanding of light truly began around the 17th century, when a few individuals started to realize that light was not a mystical substance. Scientists (or "natural philosophers", as they were called during that time) recognized that certain properties of light were measurable, and that some properties could be manipulated. Sir Isaac Newton and Ole Rømer were among the first scientists to take a step in this direction.

> ### Isaac Newton's Prism Experiment

Sir Isaac Newton has made contributions to many fields of science and mathematics. In 1666, while spending time at his childhood home in Lincolnshire, England, Newton began experimenting with light.

Using a small slit in his window shutters, Newton passed a narrow beam of sunlight through a glass prism. The light travelled through the prism, and projected a rainbow of color on the other side!

Picture taken from http://lightingmatters.com.au/wp/what-is-the-colour-of-white/white-light-prism-experiment/, July 30th, 2018.
 Figure 2: This picture shows how a prism can create a spectrum of color. This is what Newton would have seen in 1666.

Later on, scientists determined that the prism was actually splitting light into its component parts. This phenomenon is called **dispersion**.

Through this experiment, Newton demonstrated that white light was actually made up of all the individual colors of the rainbow!

> ### Ole Rømer and the Speed of Light

For many years, people thought that if somebody lit a match, the light from that match would be instantly visible to everyone, no matter how far away they were. However, in 1676 Ole Rømer proved that this is not the case.

Rømer spent a long time studying the orbit of Io, one of Jupiter's moons. As part of his study, he began predicting the times when Io should be hidden behind Jupiter's shadow (these periods are called eclipses). However, Rømer saw that his predictions for when these eclipses should occur were not always accurate.

Gif taken from https://giphy.com/gifs/timelapse-DXIa1beDspYRy, August 1st, 2018.
 Figure 3: Here we can see Jupiter as it looks through a telescope. You might be able to see a black spot move from the left to the right across Jupiter's surface. This is actually one of Jupiter's many moons!

Rømer then realized that these errors may be because the distance between Io and the Earth was always changing. Rømer thought that when the distance between Io and the Earth increased, it might take a longer time for light coming from Io to reach Earth. If this were the case, then the light must be travelling at a finite speed!

After taking many measurements and using some clever mathematics, Rømer calculated the speed of light to be roughly 220,000,000 m/s, or 792,000,000 km/h.

Today, we have measured the speed of light to be 299,792,458 m/s. Although he was not exactly right, Rømer provided one of the first mathematical calculations for the speed of light.

***
Since the time of Rømer and Newton, scientists have made many new discoveries about the nature of light. While not all of these discoveries agree with one another, here are two things we know for sure:
- Light is made up of a spectrum of color
- Light travels at a speed of 299,792,458 m/s

Now let's talk about some of the ways we can manipulate light.
***

## Reflection

We are all familiar with reflection; chances are, you look at your reflection more than once a day. But have you ever stopped to wonder what is really going on?

Reflection is the term used to describe how light can change direction when it comes into contact with certain surfaces.

When incoming light rays encounter a reflective surface, they bounce off the surface and continue moving in a new direction. The new direction in which it moves is determined by the **law of reflection**.

\begin{equation}
\rm Law\: of\: Reflection: Angle\: of\: Incidence = Angle\: of\: Reflection
\end{equation}

On the animation below, click on the flashlight to turn it on, and move your mouse to change the angle of incidence.

```python
IFrame('Animations/reflect.html',width=500,height=320)
```

As seen above, the **normal** is what we call the line that forms a 90$^{\circ}$ angle with  the surface. The **angle of incidence** is what we call the angle between the flash lights beam and the normal. Similarly, the **angle of reflection** is the angle that the newly reflected light beam makes with the normal. The law of reflection states that these two angles will always be equal.

## Refraction

Have you ever tried to reach down and grab an object sitting at the bottom of a pool of water? If you have, you may have noticed that the object isn't actually in the location that you thought it was.

 Image taken from http://legacy.sciencelearn.org.nz/Contexts/Light-and-Sight/Sci-Media/Video/Refraction/(quality)/hi on August 3rd, 2018.
 Figure 4: When you are looking into a body of water from above, the objects you see beneath the surface are not actually where they appear to be.

This phenomenon occurs because the light travelling to your eyes from the bottom of the pool **refracts**, or changes its direction of travel, when it transitions from water to air.

The **index of refraction** is a value that we use to show how much light will bend when travelling through a substance. For example, the index of refraction for air is approximately 1.00, and the index of refraction for water is about 1.33. Because these indexes are different, light will bend when passing from water to air, or vice versa.

Use the animation below to see how light refracts when passing from air to water. Click on the flashlight to turn it on.

```python
IFrame('Animations/refract.html',width=520,height=320)
```

Mathematically, reflection can be described using the following equation, known as Snell's Law:

\begin{equation}
\textrm{Snells Law:}\: n_1\sin(\theta_1) = n_2\sin(\theta_2)
\end{equation}

where $n_1$ is the index of refraction for the first medium, $\theta_1$ is the incident angle, $n_2$ is the index of refraction for the second medium, and $\theta_2$ is the angle of refraction.

Light will bend *towards* the normal when travelling from a medium with a *lower* index of refraction to one with a *higher* index of refraction, and vice versa.

***
Some of the most beautiful sights in nature are caused by reflection and refraction. Here are a couple of examples:

### Rainbows

Rainbows are a result of both reflection and refraction. As its raining, each water droplet acts like a tiny prism, just like the one we saw in Figure 2. The water droplets split visible light into colors, and these colors are then reflected back towards our eyes.

 Image taken from https://waterstories.nestle-waters.com/environment/how-does-a-rainbow-form/ on August 3rd, 2018.
 Figure 5: Water droplets use reflection and refraction to create the beautiful rainbows that we see while it is raining.

### Mirages

Have you ever been driving on a sunny day, and up ahead it looks as though a stream of water is running across the road? You are really seeing a mirage.
Mirages also occur because of refraction, but they do not result in a display of color like a rainbow. This type of refraction occurs due to a difference in temperature between separate layers of air.

As we were describing before, refraction occurs when light travels from one substance to another. Well, it turns out that hot air and cold air are actually different enough to act as different substances. Therefore, light will refract when passing through one to the other.

 Image taken from https://edexcellence.net/articles/what-the-mirage-gets-wrong-on-teacher-development on August 3rd, 2018.
 Figure 6: Although it may look like water running across the road, it is actually a mirage. These commonly occur in desert areas, where the road can become very hot.

When you are looking at a mirage, it can look as though the air is wavy and fluid, which is why it is common to think that you are looking at water. This appearance occurs when layers of hot and cold air are mixing together, and light passing through these layers is constantly being refracted in different directions.

You may see a mirage appear on top of a hot roadway, behind the exhaust pipe of a plane or car, or around any other source of heat.

## Applications of Reflection and Refraction

### Lenses

If you have glasses, or contact lenses, then you are constantly using refraction in order to help you see! Lenses use refraction to point light in specific directions.

Generally speaking, there are two types of lenses: **convex** and **concave**.

To see how each type of lense affects light, use the following animation.

```python
IFrame('Animations/convex.html',width=520,height=420)
```

As seen above, a convex lens focuses light towards a specific point, while a concave lens will spread light away from a point. These lenses can be combined in many ways in order to produce different effects. For example, a camera lens uses a series of both convex and concave lenses in order to direct incoming light towards the back of the camera.

 Image taken from https://www.reddit.com/r/pic/comments/3o3b7w/camera_lens_cut_in_half/ on August 3rd, 2018.
 Figure 5: This is what the inside of a camera lens looks like. The photographer can adjust how they want the picture to look by changing the distance between the individual lenses.

## Vision

Our eyes are very complex organs, but the process that enables us to see is actually pretty simple. The basic steps are as follows:

1. Light enters the eye through the **pupil**
2. The convex **lens** behind the pupil directs incoming light towards the **retina**, which is like a screen at the back of our eye.
3. The retina then sends this image to the brain.
4. The brain then interprets the image.

 Image taken from https://openclipart.org/detail/261647/eye-diagram on August 3rd, 2018.
 Figure 6: This diagram shows some of the key components of the eye that enable us to see.

However, the image that is projected onto the retina is actually upside down!

 Image taken from https://www.eetimes.com/author.asp?section_id=14&doc_id=1282795 on August 3rd, 2018.
 Figure 7: The convex lens at the front of our eye actually flips images upside down.

So the retina actually sends an upside down image to the brain, and the brain automatically flips the image rightside up.

Use the following link to see an animation showing how a convex lens flips images upside down:https://phet.colorado.edu/sims/geometric-optics/geometric-optics_en.html.

## Technology & Inventions

### The Telescope

The first telescope was made by Hans Lippershey in 1608, but it was Galileo Galilee who became famous by using it for astronomy. There are many different types of telescopes, but they all use reflection and refraction to make far away objects appear closer.

A telescope uses a large opening to collect incoming light, and then directs this light towards your eye by using mirrors and lenses.

 Image taken from https://www.skyandtelescope.com/press-releases/tips-for-first-time-telescope-buyers/ on August 3rd, 2018.
 Figure 8: Telescopes come in many different shapes and sizes.

The reason why things look bigger when looking through a telescope is because of the lenses.

### The Microscope

- talk about invention of microscope
- why it works

## Conclusion

Our understanding of light is the result of hundreds of years of research and innovation. Along the way, we have created incredible new technologies that have allowed us to look further than ever before.

 Image taken from https://xenlife.com.au/hubble-space-telescope-important/ on August 3rd, 2018.
 Figure 9: The Hubble Space Telescope has shown us pictures of galaxies that are billions of light years away.

[](https://github.com/callysto/curriculum-notebooks/blob/master/LICENSE.md)


```python
%matplotlib widget

import matplotlib
import numpy as np
import matplotlib.pyplot as plt
import mplcursors
```

```python
# Scale plot to fit page
plt.rcParams["figure.figsize"] = (8, 6)
plt.rcParams["font.size"] = 18
```

# Radiation Interatctions with Matter

### Learning Objectives

- Define uncollided flux
- Define linear interaction coefficient
- Apply linear interaction coefficients to a slab problem
- Identify the units of intensity, flux density, fluence, reaction rate
- Compare linear interaction coefficient and cross section
- Calculate uncollided flux in a medium
- Calculate mean free path of a particle in a medium
- Define the half thickness in a medium
- Apply the concept of buildup factor to attenuation in a slab
- Define microscopic cross section
- Calculate macroscopic cross sections, given a microscopic cross section
- Calculate the mass interaction coefficients of mixtures
- Calculate flux density
- Calculate Reaction Rate Density
- Recognize the dependence of flux on energy, position, and time
- Define radiation fluence
- Calculate uncollided flux density from isotropic point sources
- Apply the Kelin-Nishina formula to Compton Scattering
- Compare energy dependence of photon interaction cross sections
- Describe energy dependence of neutron interaction cross sections
- Recognize the comparative range of heavy vs. light particles
- Recognize the comparative range of charged particles

## Linear Interaction Coefficient

- The interaction of radiation with matter is always statistical in nature, and, therefore, must be described in probabilistic terms.

Consider a particle travelling through a homogeneous material.

\begin{align}
P_i(\Delta x) &= \mbox{probability the particle, causes a reaction of type i in distance }\Delta x\\
\end{align}

Empirically, we find that this probability becomes constant as $\Delta x \longrightarrow 0$. Thus:

\begin{align}
\mu_i &= \lim_{\Delta x \rightarrow 0}\frac{P_i(\Delta x)}{\Delta x}\\
\end{align}

Facts about $\mu_𝑖$:

- $\mu_i$ is an *intrinsic* property of the material for a given incident particle and interaction.
- $\mu_i$ is independent of the path length traveled prior to the interaction.
- $\mu_i$ may represent many types of interaction (scattering: $\mu_s$, absorption: $\mu_a$, ...)
- $\mu_i$ typically depends on particle energy

The probability, per unit path length, that a neutral particle undergoes some sort of reaction, is the sum of the probabilities, per unit path length of travel, for each type :

\begin{align}
\mu_t(E) = \sum_i \mu_i(E)
\end{align}

## Think Pair Share:

What are the units of the linear interaction coefficient?

### Attenuation of Uncollided Flux

Imagine a plane of neutral particles strike a slab of some material, normal to the surface.

We can describe this using $\mu_t$ or, equivalently, the macroscopic total cross section $\Sigma_t$.

\begin{align}
I(x) &= I_0e^{-\mu_t x}\\
I(x) &= I_0e^{-\Sigma_t x}\\
\end{align}

where

\begin{align}
        I(x) &= \mbox{uncollided intensity at distance x}\\
        I_0 &= \mbox{initial uncollided intensity}\\
        \mu_t &= \mbox{total linear interaction coefficient} \\
        \Sigma_t &= \mbox{macroscopic total cross section} \\
        x &= \mbox{distance into material [m]}\\
\end{align}

```python
import math
def attenuation(distance, initial=100, sig_t=1):
    """This function describes neutron attenuation into the slab"""
    return initial*math.exp(-sig_t*distance)

```

Rather than intensity, one can find the probability density:

We have a strong analogy between decay and attenuation, as above. In the case of decay the probability of decay in a time interval dt is:

\begin{align}
P(t)dt &= \lambda e^{-\lambda t}dt\\
 &= \mbox{probability of decay in interval dt}
\end{align}

From this, one can find the mean lifetime of a neutron before decay:

\begin{align}
\bar{t} &= \int_0^\infty t'P(t')dt'\\
        &= \int_0^\infty t'\lambda e^{-\lambda t'}dt'\\
        &= \frac{1}{\lambda}
\end{align}

In the case of attenuation:
\begin{align}
P(x)dx &= \Sigma_te^{-\Sigma_tx}dx
\end{align}

Such that:

\begin{align}
P(x)dx &= \mu_t e^{-\mu_t x}dx\\
 &= \Sigma_t e^{-\Sigma_t x}dx\\
 &= \mbox{probability of interaction in interval dx}
\end{align}

So, the mean free path is:

\begin{align}
\bar{l} &= \int_0^\infty x'P(x')dx'\\
        &= \int_0^\infty x'\Sigma_te^{-\Sigma_t x'}dx'\\
        &= \frac{1}{\Sigma_t}
\end{align}

Or, equivalently in $\mu_t$ notation:

\begin{align}
\bar{x} &= \int_0^\infty x'P(x')dx'\\
        &= \int_0^\infty x'\mu_te^{-\mu_t x'}dx'\\
        &= \frac{1}{\mu_t}
\end{align}

```python
def prob_dens(distance, initial=100, sig_t=1):
    return sig_t*attenuation(distance, initial=100, sig_t=1)

```

```python
sig_t = 0.2
i_0 = 100

# This code plots attenuation
import numpy as np
z = np.arange(24)
y = np.arange(24)
x = np.arange(24)
for h in range(0,24):
    x[h] = h
    y[h] = attenuation(h, initial=i_0, sig_t=sig_t)
    z[h] = prob_dens(h, initial=i_0, sig_t=sig_t)

# creates a figure and axes with matplotlib
fig, ax = plt.subplots()
scatter = plt.scatter(x, y, color='blue', s=y*20, alpha=0.4)
ax.plot(x, y, color='red')
ax.plot(x, z, color='green')

# adds labels to the plot
ax.set_ylabel('Percent of Neutrons')
ax.set_xlabel('Distance into slab')
ax.set_title('Attenuation')

# Add mpl widget for interactivity
labels = ['{0}% intensity'.format(i) for i in y]
mplcursors.cursor(ax).connect(
    "add", lambda sel: sel.annotation.set_text(labels[sel.index]))

plt.show()
```

## Half-thickness

In another analog to decay, the **half-thickness** of a material is the distance required for half of the incident radiation to interact with a medium:

\begin{align}
\frac{I(x_{1/2})}{I(0)} &= e^{-\mu_t x_{1/2}}\\
\implies x_{1/2} &= \frac{\ln{2}}{\mu_t}
\end{align}

## Think pair share:
What is the concept in the context of decay that is analogous to the half-thickness?

## Microscopic and Macroscopic Cross Sections

- The microscopic cross section $\sigma_i$ is  the likelihood of the event per unit area.
- The macroscopic cross section $\Sigma_i$ is  the likelihood of the event per unit area of a certain density of target isotopes.
- The macroscopic cross section $\Sigma_i$ is equivalent to the linear interaction coefficient $\mu_i$, but we tend to use $\Sigma_i$ in nuclear interactions, reserving $\mu_i$ for photon interactions.

\begin{align}
\mu_i &= \mbox{linear interaction coefficient}\\
\Sigma_i &= \mbox{macroscopic cross section}\\\\
 &= \sigma_i N\\
 &= \sigma_i \frac{\rho N_a}{A}\\
 \mbox{where }& \\
 N &= \mbox{atom density of medium}\\
 \rho &= \mbox{mass density of the medium}\\
 N_a &= \mbox{Avogadro's number}\\
 A &= \mbox{atomic weight of the medium}
\end{align}

```python
def macroscopic_xs(micro, N):
    """Returns the macroscopic cross section [cm^2] or [barns]

    Parameters
    ----------
    micro: double
        microscopic cross section [cm^2] or [barns]
    N: double
        atom density in the medium [atoms/cm^3]
    """
    return micro*N
```

```python
def NA():
    """Returns Avogadro's number
    6.022x10^23 atoms per mole
    """
    return 6.022E23

def num_dens_from_rho(rho, na, a):
    """The atomic number density.
    That is, the concentration of atoms or molecules per unit volume (V)

    Parameters
    -----------
    rho : double
        material density (in units like g/cm^3 or kg/m^3) of the sample
    na : double
        Avogadro's number
    a : double
        The atomic or molecular weight of the atom or molecule of interest
    """
    return rho*na/a
```

## Example:
Imagine a beam of neutrons striking a body of water, $H_2O$. Many will be absorbed by the hydrogen in the water, particularly $^1H$.

Find the macroscopic absorption cross section

```python
# Find the macroscpic absorption cross section
# of the 1H in H2O
sig_1h = 0.333 # barns

# First, molecular density of water
rho_h2o = 1 # g/cm^3
a_h2o = 18.0153 # g/mol
n_h2o = num_dens_from_rho(rho_h2o, NA(), a_h2o) # molecules water / cm^3
n_h2o_barn = n_h2o/10**(24) # 10^24 molecules water / cm^3
print('n_h2o [1/cm^3] = ', n_h2o)
print('n_h2o [10^(24)/cm^3] = ', n_h2o_barn)

# Now, there are two Hydrogens in each molecule of water, so:
macroscopic_h1 = macroscopic_xs(sig_1h, 2*n_h2o_barn)
print('absorption in water from 1H = ', macroscopic_h1)
```

    n_h2o [1/cm^3] =  3.342714248444378e+22
    n_h2o [10^(24)/cm^3] =  0.033427142484443784
    absorption in water from 1H =  0.02226247689463956

### Mixtures
In a medium that is a mixture of isotopes (e.g. $H_2O$), we can calculate the total macroscopic cross section based on individual microscopic cross sections and number densities for each component of the mixture. We may need to include information about relative isotopic abundances (f).

For the same problem as above (neutrons striking a body of water) we can calculate the absorption by *all* isotopes in the $H_2O$.

\begin{align}
\mu^{H_2O} \equiv \Sigma^{H_2O} &= N^1\sigma_a^1 + N^2\sigma_a^2 + N^{16}\sigma_a^{16}
+ N^{17}\sigma_a^{17}  + N^{18}\sigma_a^{18}\\
&= f^1N^H\sigma_a^1 + f^2N^H\sigma_a^2 + f^{16}N^O\sigma_a^{16} + f^{17}N^O\sigma_a^{17} + f^{18}N^O\sigma_a^{18}
\end{align}

Superscripts 1, 2, 16, 17, and 18 indicate isotopes $^1H$, $^2H$, $^{16}O$,$^{17}O$, and $^{18}O$.

\begin{align}
N^H = 2N^{H_2O}\\
N^{O} = N^{H_2O}\\
N^{H_2O} = \frac{\rho^{H_2O}N_a}{A^{H_2O}}
\end{align}

Thus:
\begin{align}
\mu^{H_2O} \equiv \Sigma^{H_2O} &= N^{H_2O}\left[2f^1\sigma_a^1 + 2f^2\sigma_a^2 + f^{16}\sigma_a^{16} + f^{17}\sigma_a^{17} + f^{18}\sigma_a^{18}\right]
\end{align}

```python
# We need a lot of data

# Abundances
f_1 = 0.99985
f_2 = 0.00015
f_16 = 0.99756
f_17 = 0.00039
f_18 = 0.00205

# Then, microscopic absorption cross sections
sig_1 = 0.333
sig_2 = 0.000506
sig_16 = 0.000190
sig_17 = 0.239
sig_18 = 0.000160

macroscopic_h2o = n_h2o_barn*(2*f_1*sig_1
                              + 2*f_2*sig_2
                              + f_16*sig_16
                              + f_17*sig_17
                              + f_18*sig_18)
print('absorption in water from all isos = ', macroscopic_h2o,"\n",
     'while absorption in water from 1H = ', macroscopic_h1,"\n",
     'Thus, absorption in water is mostly from 1H.')
```

    absorption in water from all isos =  0.02226860496564809
     while absorption in water from 1H =  0.02226247689463956
     Thus, absorption in water is mostly from 1H.

### Reaction Rates

- The microscopic cross section is just the likelihood of the event per unit area.
- The macroscopic cross section is just the likelihood of the event per unit area of a certain density of target isotopes.
- The reaction rate is the macroscopic cross section times the flux of incident neutrons.

\begin{align}
R_{i,j}(\vec{r}) &= N_j(\vec{r})\int dE \phi(\vec{r},E)\sigma_{i,j}(E)\\
R_{i,j}(\vec{r}) &= \mbox{reactions of type i involving isotope j } [reactions/cm^3s]\\
N_j(\vec{r}) &= \mbox{number of nuclei participating in the reactions } [\#/cm^3]\\
E &= \mbox{energy} [MeV]\\
\phi(\vec{r},E)&= \mbox{flux of neutrons with energy E at position i } [\#/cm^2s]\\
\sigma_{i,j}(E)&= \mbox{cross section } [cm^2]\\
\end{align}

This can be written more simply as $R_x = \Sigma_x \phi$.

Using flux notation, the density of ith type of neutron interaction with isotope j, per unit time is:

\begin{align}
R_{i,j}(\vec{r}) = \Sigma_{i,j}\phi(\vec{r})
\end{align}

## Flux density from Point Source
Finding $\phi(\vec{r}0$ generally requires *particle transport calculations.*

However, in some simple practical situations, the flux density can be approximated by the flux density of uncollided source particles.

### Point Source in Vacuum

Consider a source of particles:

- it emits $S_p$ particles per unit time
- all particles have energy E
- and they are emitted radially outward into an infinite vacuum
- isotropically (equally in all directions)
- from a single point in space

### Think-pair share:

- How many interactions occur?

### At a radius r:
Because the source is isotropic, each unit area on an imaginary spherical shell of radius $r$ has the same number of particles crossing it. Thus:

\begin{align}
\phi^o(r) &= \mbox{uncollided flux at radius r in any direction}\\
&= \frac{S_p}{4\pi r^2}
\end{align}

```python
def phi_o_r(r, s):
    """Returns the uncolided flux at radius r
    due to an isotropic point source in a vacuum

    Parameters
    -----------
    r : double
        radius away from the point [length]
    s : double
        point source strength [particles/time]
    """
    return s/(4*math.pi*pow(r,2))
```

```python
s=200
plt.clear()
plt.plot(range(1,10), [phi_o_r(r, s) for r in range(1,10)])
plt.show
```

The plot above, this $1/r^2$ reduction in flux and reaction rate, is called **geometric attenuation**

### Think-pair-share

What other phrase do you think we use for attenuation in a medium?

## Point Source in an Attenuating Medium
So, the unollided flux is
\begin{align}
\phi^o(r) &= \frac{S_p}{4\pi r^2}
\end{align}

### A small volume

At a distance r, we place a homogeneous mass with a volume $\Delta V_d$. The interaction rate $R_d$ in the mass is:

\begin{align}
&R^o(r)=\mu_d(E)\Delta V_d\frac{S_p}{4\pi r^2}\\
\mbox{where}&\\
&\mu_d(E)=\mbox{linear interaction coefficient in the volume}
\end{align}

### An inifinite volume

From this, we can imagine the point source embedded in an infinite medium of this material. A detector is at distance r in the volume:

\begin{align}
&\phi^o(r) = \frac{S_p}{4\pi r^2}e^{-\mu r}\\
\mbox{where}&\\
&e^{-\mu r}=\mbox{material attenuation} \\
&\frac{S_p}{4\pi r^2}=\mbox{geometric attenuation}
\end{align}

### A slab shield

Imagine a slab shield, thickness t, at a distance r, between the point source and a detector.

\begin{align}
&\phi^o(r) = \frac{S_p}{4\pi r^2}e^{-\mu t}\\
\mbox{where}&\\
&t=\mbox{thickness of the slab}
\end{align}

If it were made of a series of materials $i$, with coefficients $\mu_i$, and thicknesses $t_i$:

\begin{align}
&\phi^o(r) = \frac{S_p}{4\pi r^2}e^{\sum_i -\mu_i t_i}\\
\mbox{where}&\\
&\mu_i=\mbox{linear interaction coefficient of ith slab}\\
&t_i=\mbox{thickness of ith slab}
\end{align}

### Heterogeneous Medium

An arbitrary heterogeneous medium can be described as having an interaction coefficient $\mu(\vec{r})$ at any point $\vec{r}$ in the medium, a funciton of position in the medium.

\begin{align}
&\phi^o(r) = \frac{S_p}{4\pi r^2}e^{\left[-\int_0^r \mu(s) ds\right]}\\
\end{align}

# Announcements
* If you had trouble tagging pages based on problems for HW06, this should be fixed by the end of class. If you already submitted, please edit your submission to tag problems appropriately.
* There will be a HW assigned next Friday. It will be entirely review material based on things that were difficult on previous assignments and the exam.
* The quiz and HW are posted. The quiz is still due on Monday.

## Polyenergetic Point Source

- Previous examples assume a **monoenergetic** point source (particles of a single energy, E).
- But, a single source can emit particles at several discrete energies, or even a continuum of energies.

Question: From last lecture, what do we expect of fission neutrons?

Let's define some variables:

\begin{align}
f_i &= \mbox{fraction of the source emitted with energy }E_i\\
E_i &= \mbox{discrete energy of }f_iS_p\mbox{ particles}\\
S_p &= \mbox{still the number of particles emitted from the point source}
\end{align}

The total interaction rate caused by uncollided particles streaming through a small volume mass at distance r from the source is the following, **for some set of i discrete energies**.

\begin{align}
R^o(r)=\sum_i\frac{S_p f_i\mu_d(E_i) \Delta V_d}{4\pi r^2}e^{\left[-\int_0^r \mu(s,E_i) ds\right]}\\
\end{align}

If the source emits a continuum of energies, it's best to define the fraction $f_i$ as a differential probability:

\begin{align}
N(E)dE\mbox{ the probability that a source particle is emitted with energy in dE about E}
\end{align}

With this definition, the sum over discrete energies becomes an integral.

\begin{align}
R^o(r)=\int_o^\infty \left[\frac{S_p N(E)\mu_d(E) \Delta V_d}{4\pi r^2}e^{\left[-\int_0^r \mu(s,E) ds\right]}\right]dE\\
\end{align}

Please note, you may see many nuclear texts list the dE first in the integral... don't be bamboozled. This is equivalent to the above:

\begin{align}
R^o(r)=\int_o^\infty dE\frac{S_p N(E)\mu_d(E) \Delta V_d}{4\pi r^2}e^{\left[-\int_0^r \mu(s,E) ds\right]}
\end{align}

### Example 7.4 from your book (Shultis & Faw)

A point source with an activity of 500 Ci emits 2-MeV photons with a frequency of 70% per decay.

\begin{align}
S_p = 500 Ci\\
f_2 = 0.7\\
\end{align}

What is the flux density of 2-MeV photons 1 meter from the source?

```python
s_p = 500 # Ci
f_2 = 0.7 # fraction emitted at 2MeV
mu = 1.0/187.0 # mean free path of 2MeV photon in air is 187m

# first, convert S_p is in number of particles per decay (Bq)
bq_to_ci = 3.7e10 # Bq/Ci
s_p = s_p*bq_to_ci

# Now, find uncollided flux of 2MeV photons at 1 m
r = 1.0 #m
s = s_p*f_2 # just want 2MeV photons
phi = phi_o_r(r, s)
print("Uncollided flux is : ", phi)

# Uh oh, we forgot the material attenuation!
phi = phi_o_r(r, s)*math.exp(-mu*r)
print("Flux after material attenuation is : ", phi)
```

    Uncollided flux is :  1030528256520.0223
    Flux after material attenuation is :  1025032118881.2917

### Think Pair Share

What are the units of $\phi^o$, above?

# Photon Interactions

**Recall:**

\begin{align}
c &= \mbox{speed of light}\\
  &=2.9979\times10^8\left[\frac{m}{s}\right]\\
E &= \mbox{photon energy}\\
  &=h\nu\\
  &=\frac{hc}{\lambda}\\
h &= \mbox{Planck's constant}\\
  &= 6.62608\times10^{−34} [J\cdot s] \\
\nu &=\mbox{photon frequency}\\
\lambda &= \mbox{photon wavelength}
\end{align}

**Nota bene:**
- **10eV - 20MeV** photons are important in radiation sheilding
- At **10eV - 20MeV**, only photoelectric effect, pair production, and Compton Scattering are significant

Figure from: "Radiation Interactions with Tissue." Radiology Key. Jan 8 2016.

Figure from: Cullen, D. E. 1994. "Photon and Electron Interaction Databases and Their Use in Medical Applications." UCRL-JC--117419. Lawrence Livermore National Lab. http://inis.iaea.org/Search/search.aspx?orig_q=RN:26035330.

## Klein Nishina

The total Compton cross section, per atom with Z electrons, based on the free-electron approximation, is given by the well-known Klein-Nishina formula [Evans 1955]:

\begin{align}
\sigma_c(E) =\pi Zr_e^2\lambda\left[(-2\lambda - 2\lambda^2)\ln{\left(1+\frac{2}{\lambda}\right)} + \frac{2(1+9\lambda + 8\lambda^2 + 2\lambda^3)}{(\lambda + 2)^2}\right]
\end{align}

Here $\lambda \equiv \frac{m_ec^2}{E}$, a dimensionless quantity, and $r_e$ is the classical electron radius. The value of $r_e$ is given by:

\begin{align}
r_e &\equiv \frac{e^2}{4\pi\epsilon_om_ec^2}\\
&= 2.8179\times10^{-13}cm
\end{align}

### Think pair share:
Conceptually, in the above equation:

- what is $r_e$?
- what is $e$?
- what is $\epsilon_o$?
- what is $m_ec^2$?

### Total Photon Cross Section
Various types of incoherent scattering, including Compton, are actually present in that intermediate energy range. It is occaisionally important to correct for all types of incoherent scattering, but it can typically be assumed to be primarily Compton scattering.

For photons, then $\mu$ becomes:

\begin{align}
\mu(E)&\equiv N\left[\sigma_{ph}(E) + \sigma_{inc}(E) + \sigma_{pp}(E)\right]\\
      &\simeq N\left[\sigma_{ph}(E) + \sigma_{c}(E) + \sigma_{pp}(E)\right]\\
    N &= \mbox{atom density}\\
      &= \frac{\rho N_a}{A}
\end{align}

It is common to denote this as the total mass interaction coefficient:

\begin{align}
\frac{\mu}{\rho} &= \frac{N_a}{A}\left[\sigma_{ph}(E) + \sigma_{c}(E) + \sigma_{pp}(E)\right]\\
&= \frac{N_a}{A}\left[\frac{\mu_{ph}(E)}{\rho} + \frac{\mu_{c}(E)}{\rho} + \frac{\mu_{pp}(E)}{\rho}\right]
\end{align}

## Neutron Interactions

Photons tend to interact with electrons in a target atom. **Neutrons tend to interact with the nucleus.**

Neutron cross sections:

- Vary rapidly with the incident neutron energy,
- Vary erratically from one element to another
- Even vary dramatically between isotopes of the same element.

There are lots of sources of neutron cross sections. The best place to start is the Brookhaven National Laboratory National Nuclear Data Center [https://www.nndc.bnl.gov/](https://www.nndc.bnl.gov/).

Your book has a clever table (7.1) listing some of the data needed for high and low energy interaction calculations. These include:

- Elastic scattering cross sections
- Angular distribution of elastically scattered neutrons
- Inelastic scattering cross sections
- Angular distribution of inelastically scattered neutrons
- Gamma-photon yields from inelastic neutron scattering
- Resonance absorption cross sections
- Thermal-averaged absorption cross sections
- Yield of neutron-capture gamma photons
- Fission cross sections and associated gamma-photon and neutron yields

# Total cross sections

**For light nuclei** ($A<25$) and $E<1keV$, the cross section typically varies as:

\begin{align}
\sigma_t = \sigma_1 + \frac{\sigma_2}{\sqrt{E}}
\mbox{where}&\\
\sigma_1 \mbox{ and }\sigma_2 \mbox{ are constants}& \\
\sigma_1&=\mbox{elastic scattering}\\
\frac{\sigma_2}{\sqrt{E}}&=\mbox{radiative capture}\\
\end{align}

**For solids** at energies less than about 0.01 eV, Bragg cutoffs apply. These are energies below which no coherent scattering is possible from the material's crystalline planes.

**For heavy nuclei**, the total cross section has a $\frac{1}{\sqrt{E}}$ behavior with low energy, narrow resonances and high energy broad resonances:

\begin{align}
\sigma_t \propto \frac{1}{\sqrt{E}}
\end{align}

### Recall fission cross sections :

```python

```


```python
%matplotlib inline
import numpy as np
import pandas as pd
import scipy.stats
import matplotlib.pyplot as plt

from matplotlib import animation
from matplotlib import rcParams
rcParams['figure.dpi'] = 120
from IPython.display import HTML
from IPython.display import YouTubeVideo
from functools import partial
YouTubeVideo_formato = partial(YouTubeVideo, modestbranding=1, disablekb=0,
                               width=640, height=360, autoplay=0, rel=0, showinfo=0)
```

# Estadística inferencial

La inferencia busca

> Extraer **conclusiones** a partir de **hechos u observaciones** a través de un **método o premisa**

En el caso particular de la **inferencia estadística** podemos realizar las siguientes asociaciones

- Hechos: Datos
- Premisa: Modelo probabilístico
- Conclusión: Una cantidad no observada que es interesante

Y lo que buscamos es

> Cuantificar la incerteza de la conclusión dado los datos y el modelo

La inferencia estadística puede dividirse en los siguientes tres niveles

1. Ajustar un modelo a nuestros datos
1. Verificar que el modelo sea confiable
1. Responder una pregunta usando el modelo

En esta lección estudiaremos las herramientas más utilizadas asociadas a cada uno de estos niveles

1. **Estimador de máxima verosimilitud**
1. **Bondad de ajuste** e **Intervalos de confianza**
1. **Test de hipótesis**

## Ajuste de modelos: Estimación de máxima verosimilitud

En este nivel de inferencia se busca **ajustar** un modelo teórico sobre nuestros datos. En esta lección nos enfocaremos en **modelos de tipo parámetrico**. Un modelo parámetrico es aquel donde **se explicita una distribución de probabilidad**.

Recordemos que una distribución tiene **parámetros**. Por ejemplo la distribución Gaussiana (univariada) se describe por su media $\mu$ y su varianza $\sigma^2$. Luego ajustar una distribución Gaussiana corresponde a encontrar el valor de $\mu$ y $\sigma$ que hace que el modelo se parezca lo más posible a la distribución empírica de los datos.

A continuación veremos los pasos necesarios para ajustar una distribución a nuestros datos

### ¿Qué distribución ajustar?

Antes de ajustar debemos realizar un supuesto sobre la distribución para nuestro modelo. En general podemos ajustar cualquier distribución pero un mal supuesto podría invalidar nuestra inferencia

Podemos usar las herramientas de **estadística descriptiva** para estudiar nuestros datos y tomar esta decisión de manera informada

En el siguiente ejemplo, un histograma de los datos revela que un modelo gaussiano no es una buena decisión

¿Por qué? La distribución empírica es claramente asimétrica, su cola derecha es más pesada que su cola izquierda. La distribución Gaussiana es simétrica por lo tanto no es apropiada en este caso ¿Qué distribución podría ser más apropiada?

### ¿Cómo ajustar mi modelo? Estimación de máxima verosimilitud

A continuación describiremos un procedimiento para ajustar modelos paramétricos llamado *maximum likelihood estimation* (MLE)

Sea un conjunto de datos $\{x_1, x_2, \ldots, x_N\}$

**Supuesto 1** Los datos siguen el modelo $f(x;\theta)$ donde $f(\cdot)$ es una distribución y $\theta$ son sus parámetros

$$
f(x_1, x_2, \ldots, x_N |\theta)
$$

**Supuesto 2** Las observaciones son independientes e idénticamente distribuidas (iid)

- Si dos variables son independientes se cumple que $P(x, y) = P(x)P(y)$
- Si son además idénticamente distribuidas entonces tienen **la misma distribución y parámetros**

Usando esto podemos escribir

$$
\begin{align}
f(x_1, x_2, \ldots, x_N |\theta) &= f(x_1|\theta) f(x_2|\theta) \ldots f(x_N|\theta) \nonumber \\
& = \prod_{i=1}^N f(x_i|\theta) \nonumber \\
& = \mathcal{L}(\theta)
\end{align}
$$

donde $\mathcal{L}(\theta)$ se conoce como la verosimilitud o probabilidad inversa de $\theta$

Si consideramos que los datos son fijos podemos buscar el valor de $\theta$ de máxima verosimilitud

$$
\begin{align}
\hat \theta &= \text{arg} \max_\theta \mathcal{L}(\theta) \nonumber \\
&= \text{arg} \max_\theta \log \mathcal{L}(\theta) \nonumber \\
&= \text{arg} \max_\theta \sum_{i=1}^N \log f(x_i|\theta)
\end{align}
$$

El segundo paso es valido por que el máximo de $g(x)$ y $\log(g(x))$ es el mismo. El logaritmo es monoticamente creciente. Además aplicar el logaritmo es muy conveniente ya que convierte la multiplicatoria en una sumatoria.

Ahora sólo falta encontrar el máximo. Podemos hacerlo

- Analíticamente, derivando con respecto a $\theta$ e igualando a cero
- Usando técnicas de optimización iterativas como gradiente descedente

**Ejemplo:** La pesa defectuosa

Su profesor quiere medir su peso pero sospecha que su pesa está defectuosa. Para comprobarlo mide su peso $N$ veces obteniendo un conjunto de observaciones $\{x_i\}$. ¿Es posible obtener un estimador del peso real $\hat x$ a partir de estas observaciones?

Modelaremos las observaciones como

$$
x_i = \hat x + \varepsilon_i
$$

donde $\varepsilon_i$ corresponde al ruido o error del instrumento y asumiremos que $\varepsilon_i \sim \mathcal{N}(0, \sigma_\varepsilon^2)$, es decir que el ruido es **independiente** y **Gaussiano** con media cero y **varianza** $\sigma_\varepsilon^2$ **conocida**

Entonces la distribución de $x_i$ es

$$
f(x_i|\hat x) = \mathcal{N}(\hat x, \sigma_\varepsilon^2)
$$

Para encontrar $\hat x$, primero escribimos el logaritmo de la **verosimilitud**

$$
\begin{align}
\log \mathcal{L}(\hat x) &=  \sum_{i=1}^N \log f(x_i|\hat x) \nonumber \\
&= \sum_{i=1}^N  \log \frac{1}{\sqrt{2\pi\sigma_\varepsilon^2}}  \exp \left ( - \frac{1}{2\sigma_\varepsilon^2} (x_i - \hat x)^2 \right)  \nonumber \\
&= -\frac{N}{2}\log(2\pi\sigma_\varepsilon^2)  - \frac{1}{2\sigma_\varepsilon^2}  \sum_{i=1}^N  (x_i - \hat x)^2  \nonumber
\end{align}
$$

Luego debemos resolver

$$
\begin{align}
\hat \theta &= \text{arg} \max_\theta \log \mathcal{L}(\theta) \nonumber \\
&= \text{arg} \max_\theta - \frac{1}{2\sigma_\varepsilon^2}  \sum_{i=1}^N  (x_i - \hat x)^2
\end{align}
$$

donde podemos ignorar el primer término de la verosimilitud ya que no depende de $\theta$. Para encontrar el máximo derivamos la expresión anterior e igualamos a cero

$$
-\frac{1}{2\sigma_\varepsilon^2} \sum_{i=1}^N 2(x_i - \hat x ) = 0.
$$

Finalmente si despejamos llegamos a que

$$
\hat x = \frac{1}{N} \sum_{i=1}^N x_i,
$$

que se conoce como el estimador de máxima verosimilitud **para la media de una Gaussiana**

Recordemos que podemos comprobar que es un máximo utilizando la segunda derivada

### Estimación MLE con `scipy`

Como vimos en la lección anterior el módulo [`scipy.stats`](https://docs.scipy.org/doc/scipy/reference/stats.html) provee de un gran número de distribuciones teóricas organizadas como

- continuas de una variable
- discretas de una variable
- multivariadas

Las distribuciones comparten muchos de sus métodos, a continuación revisaremos los más importantes. A modo de ejemplo consideremos la distribución Gaussiana (Normal)

```python
from scipy.stats import norm
dist = norm() # Esto crea una Gaussiana con media 0 y desviación estándar (std) 1
dist = norm(loc=2, scale=2) # Esto crea una Gaussiana con media 2 y std 2
```

**Crear una muestra aleatoria con `rvs`**

Luego de crear un objeto distribución podemos obtener una muestra aleatoria usando el método el atributo `rvs`

```python
dist = norm(loc=2, scale=2)
dist.rvs(size=10, # Cantidad de números aleatorios generados
         random_state=None #Semilla aleatoria
        )
```

Esto retorna un arreglo de 10 números generados aleatoriamente a partir de `dist`

**Evaluar la función de densidad de probabilidad**

La función de densidad de la Gaussiana es

$$
f(x; \mu, \sigma^2) = \frac{1}{\sqrt{2\pi \sigma^2}} \exp \left( -\frac{1}{2\sigma^2} (x-\mu)^2 \right)
$$

La densidad de un objeto distribución continuo puede obtenerse con el método `pdf` el cual es función de `x`

```python
dist = norm(loc=2, scale=2)
p = dist.pdf(x # Un ndrray que representa x en la ecuación superior
            )
plt.plot(x, p) # Luego podemos graficar la fdp
```

De forma equivalente, si deseamos la función de densidad acumulada usamos el método `cdf`

Para objetos distribución discretos debemos usar el atributo `pmf`

**Ajustar los parámetros con MLE**

Para hacer el ajuste se usa el método `fit`

```python
params = norm.fit(data # Un ndarray con los datos
                 )
```

En el caso de la Gaussiana el vector `params` tiene dos componentes `loc` y `scale`. La cantidad de parámetros depende de la distribución que estemos ajustando. También es importante notar que para ajustar se usa `norm` (clase abstracta) y no `norm()` (instancia)

Una vez que tenemos los parámetros ajustados podemos usarlos con

```python
dist = norm(loc=params[0], scale=params[1])
```

Para distribuciones que tienen más de dos parámetros podemos usar

```python
dist = norm(*params[:-2], loc=params[-2], scale=params[-1])
```

### Ejercicio

Observe la siguiente distribución y reflexione ¿Qué características resaltan de la misma? ¿Qué distribución sería apropiado ajustar en este caso?

```python
df = pd.read_csv('../data/cancer.csv', index_col=0)
df = df[["diagnosis", "radius1", "texture1"]]
x = df["radius1"].values
fig, ax = plt.subplots(figsize=(5, 3), tight_layout=True)
ax.hist(x, bins=20, density=True)
ax.set_xlabel('Radio del nucleo');
```

- Seleccione una distribución de `scipy.stats`  ajustela a los datos
- Grafique la pdf teórica sobre el histograma

```python

```

## Verificación de modelos: Tests de bondad de ajuste

Una vez que hemos ajustado un modelo es buena práctica verificar que tan confiable es este ajuste. Las herramientas más típicas para medir que tan bien se ajusta nuestra distribución teórica son

- el [test de Akaike](https://en.wikipedia.org/wiki/Akaike_information_criterion)
- los [gráficos cuantil-cuantil](https://es.wikipedia.org/wiki/Gr%C3%A1fico_Q-Q) (QQ plot)
- el test no-paramétrico de Kolmogorov-Smirnov (KS)

A continuación revisaremos el test de KS para bondad de ajuste

**El test de Kolmogorov-Smirnov**

Es un test no-paramétrico que compara una muestra de datos estandarizados (distribución empírica) con una distribución de densidad acumulada (CDF) teórica. Este test busca refutar la siguiente hipótesis

> **Hipótesis nula:** Las distribuciones son idénticas

Para aplicar el test primero debemos **estandarizar** los datos. Estandarizar se refiere a la transformación

$$
z = \frac{x - \mu_x}{\sigma_x}
$$

es decir los datos estándarizados tienen media cero y desviación estándar uno

Esto puede hacerse fácilmente con NumPy usando

```python
z = (x - np.mean(x))/np.std(x)
```

### Test de KS con `scipy`

Podemos realizar el test de KS con la función [`scipy.stats.kstest`](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.kstest.html) donde

```python
scipy.stats.kstest(rvs, # Una muestra de observaciones estandarizadas
                   cdf, # Una distribución acumulada teórica, por ejemplo scipy.stats.norm.cdf
                   ...
                  )
```

Esta función retorna el valor del estadístico de KS y su *p-value* asociado. Mientras más cerca de cero sea el estadístico de KS mejor es el ajuste.

Más adelante haremos un repaso de tests de hipótesis en detalle. De momento recordemos que si el *p-value* es menor que una confianza $\alpha=0.05$ entonces rechazamos la hipótesis nula con confianza $1-\alpha = 0.95$ o $95\%$

### Ejercicio

Considere la muestra de datos anterior
- Seleccione un conjunto de distribuciones teóricas
- Encuentra la que tiene mejor ajuste usando `kstest`

```python

```

## Responder preguntas con nuestro modelo: Test de hipótesis

Se aplica un tratamiento nuevo a una muestra de la población

- ¿Es el tratamiento efectivo?
- ¿Existe una diferencia entre los que tomaron el tratamiento y los que no?

El test de hipótesis es un procedimiento estadístico para comprobar si el resultado de un experimento es significativo en la población

Para esto formulamos dos escenarios cada uno con una hipótesis asociada

- Hipótesis nula ($H_0$): Por ejemplo
    - "El experimento no produjo diferencia"
    - "El experimento no tuvo efecto"
    - "Las observaciones son producto del azar"
- Hipótesis alternativa ($H_A$): Usualmente el complemento de $H_0$

> El test de hipótesis se diseña para medir que tan fuerte es la evidencia **en contra** de la hipótesis nula

### Algoritmo general de un test de hipótesis

El siguiente es el algoritmo general de un test de hipótesis paramétrico

1. Definimos $H_0$ y $H_A$
1. Definimos un estadístico $T$
1. Asumimos una distribución para $T$ dado que $H_0$ es cierto
1. Seleccionamos un nivel de significancia $\alpha$
1. Calculamos el $T$ para nuestros datos $T_{data}$
1. Calculamos el **p-value**
    - Si nuestro test es de una cola:
        - Superior: $p = P(T>T_{data})$
        - Inferior: $p = P(T<T_{data})$
    - Si nuestro test es dos colas: $p = P(T>T_{data}) + P(T<T_{data})$

Finalmente:

`Si`  $p < \alpha$

> Rechazamos la hipótesis nula con confianza (1-$\alpha$)

`De lo contrario`

> No hay suficiente evidencia para rechazar la hipótesis nula

El valor de $\alpha$ nos permite controlar el **[Error tipo I](https://es.wikipedia.org/wiki/Errores_de_tipo_I_y_de_tipo_II)**, es decir el error que cometemos si rechazamos $H_0$ cuando en realidad era cierta (falso positivo)

Tipicamente se usa $\alpha=0.05$ o $\alpha=0.01$

**Errores de interpretación comunes**

Muchas veces se asume que el p-value es la probabilidad de que $H_0$ sea cierta dado nuestras observaciones

$$
p = P(H_0 | T> T_{data})
$$

Esto es un **grave error**. Formálmente el **p-value** es la probabilidad de observar un valor de $T$ más extremo que el observado, es decir

$$
p = P(T> T_{data} | H_0)
$$

Otro error común es creer que no ser capaz de rechazar $H_0$ es lo mismo que aceptar $H_0$

No tener suficiente evidencia para rechazar no es lo mismo que aceptar

### Un primer test de hipótesis: El t-test de una muestra

Sea un conjunto de $N$ observaciones iid $X = {x_1, x_2, \ldots, x_N}$ con media muestral $\bar x = \sum_{i=1}^N x_i$

El t-test de una muestra es un test de hipótesis que busca verificar si $\bar x$ es significativamente distinta de la **media poblacional** $\mu$, en el caso de que **no conocemos la varianza poblacional** $\sigma^2$

Las hipótesis son

- $H_0:$ $\bar x = \mu$
- $H_A:$ $\bar x \neq \mu$ (dos colas)

El estadístico de prueba es

$$
t = \frac{\bar x - \mu}{\hat \sigma /\sqrt{N-1}}
$$

donde $\hat \sigma = \sqrt{ \frac{1}{N} \sum_{i=1}^N (x_i - \bar x)^2}$ es la desviación estándar muestral (sesgada)

Si asumimos que $\bar x$ se distribuye $\mathcal{N}(\mu, \frac{\sigma^2}{N})$ entonces
$t$ se distribuye [t-student](https://en.wikipedia.org/wiki/Student%27s_t-distribution) con $N-1$ grados de libertad

- Para muestras iid y $N$ grande el supuesto se cumple por teorema central del límite
- Si $N$ es pequeño debemos verificar la normalidad de los datos

### Aplicación de t-test para probar que la regresión es significativa

En un modelo de regresión lineal donde tenemos $N$ ejemplos

$$
y_i = x_i \theta_1 + \theta_0, ~ i=1, 2, \ldots, N
$$

Podemos probar que la correlación entre $x$ es $y$ es significativa con un test sobre $\theta_1$

Por ejemplo podemos plantear las siguientes hipótesis

- $H_0:$ La pendiente es nula $\theta_1= 0$
- $H_A:$ La pendiente no es nula: $\theta_1\neq 0$ (dos colas)

Y asumiremos que $\theta_1$ es normal pero que desconocemos su varianza. Bajo este supuesto se puede formular el siguiente estadístico de prueba

$$
t = \frac{(\theta_1-\theta^*) }{\text{SE}_{\theta_1}/\sqrt{N-2}} = \frac{ r\sqrt{N-2}}{\sqrt{1-r^2}},
$$

donde $r$ es el coeficiente de correlación de Pearson (detalles más adelante) y la última expresión se obtiene reemplazando  $\theta^*=0$ y $\text{SE}_{\theta_1} = \sqrt{ \frac{\frac{1}{N} \sum_i (y_i - \hat y_i)^2}{\text{Var}(x)}}$.

El estadístico tiene distribución t-student con dos grados de libertad (modelo de dos parámetros)

## Ejercicio formativo: Regresión lineal

En lecciones anteriores estudiamos el modelo de regresión lineal el cual nos permite estudiar si existe correlación entre variables continuas. También vimos como ajustar los parámetros del modelo usando el método de mínimos cuadrados. En este ejercicio formativo veremos como verificar si el modelo de regresión ajustado es correcto

Luego de revisar este ejercicio usted habrá aprendido

- La interpretación probabilística de la regresión lineal y la relación entre mínimos cuadrados ordinarios y la estimación por máxima verosimilitud
- El estadístico $r$ para medir la fuerza de la correlación entre dos variables
- Un test de hipótesis para verificar que la correlación encontrada es estadística significativa

Usaremos el siguiente dataset de consumo de helados. Referencia: [A handbook of small datasets](https://www.routledge.com/A-Handbook-of-Small-Data-Sets/Hand-Daly-McConway-Lunn-Ostrowski/p/book/9780367449667), estudio realizado en los años 50

```python
df = pd.read_csv('../data/helados.csv', header=0, index_col=0)
df.columns = ['consumo', 'ingreso', 'precio', 'temperatura']
display(df.head())
```

El dataset tiene la temperatura promedio del día (grados Fahrenheit), el precio promedio de los helados comprados (dolares), el ingreso promedio familiar semanal de las personas que compraron helado (dolares) y el consumo ([pintas](https://en.wikipedia.org/wiki/Pint) per capita).

A continuación se muestra un gráfico de dispersión del consumo en función de las demás variables. ¿Cree usted que existe correlación en este caso?

```python
fig, ax = plt.subplots(1, 3, figsize=(8, 3), tight_layout=True, sharey=True)
for i, col in enumerate(df.columns[1:]):
    ax[i].scatter(df[col], df["consumo"], s=10)
    ax[i].set_xlabel(col)
ax[0].set_ylabel(df.columns[0]);
```

### Interpretación probabilística y MLE de la regresión lineal

Sea $y$ el consumo y $x$ la temperatura.

Asumiremos errores gaussianos iid

$$
y_i = \hat y_i + \epsilon_i, \epsilon_i \sim \mathcal{N}(0, \sigma^2),
$$

y un modelo lineal de **dos parámetros** (linea recta)

$$
\hat y_i = \theta_0 + \theta_1 x_i
$$

Bajo estos supuestos el estimador de máxima verosimilitud es

$$
\begin{align}
\hat \theta &= \text{arg}\max_\theta \log \mathcal{L}(\theta) \nonumber \\
&=\text{arg}\max_\theta  - \frac{1}{2\sigma^2} \sum_{i=1}^N (y_i - \theta_0 - \theta_1 x_i)^2 \nonumber
\end{align}
$$

Es decir que el estimador de máxima verosimilitud es equivalente al de mínimos cuadrados ordanrios $\hat \theta= (X^T X)^{-1} X^T y$ que vimos anteriormente

**Importante:** Cuando utilizamos la solución de mínimos cuadrados estamos asumiendo implicitamente que las observaciones son iid y que la verosimilitud es Gaussiana

Derivando con respecto a los parámetros e igualado a cero tenemos que

$$
\begin{align}
\sum_i y_i  - N\theta_0 - \theta_1  \sum_i x_i &= 0 \nonumber \\
\sum_i y_i x_i - \theta_0 \sum_i x_i - \theta_1 \sum_i x_i^2 &= 0 \nonumber
\end{align}
$$

Finalmente podemos despejar

$$
\begin{align}
\theta_0 &= \bar y - \theta_1 \bar x \nonumber \\
\theta_1 &= \frac{\sum_i x_i y_i - N \bar x \bar y}{\sum_i x_i^2 - M \bar x^2}  \nonumber \\
&= \frac{ \sum_i (y_i - \bar y)(x_i - \bar x)}{\sum_i (x_i - \bar x)^2}  = \frac{\text{COV}(x, y)}{\text{Var}(x)}
\end{align}
$$

de donde reconocemos las expresiones para la covarianza entre $x$ e $y$ y la varianza de $x$

### Coeficiente de correlación de Pearson

La fuerza de la correlación se suele medir usando

$$
r^2 = 1 - \frac{\sum_i ( y_i - \hat y_i)^2}{\sum_i ( y_i - \bar y)^2} = 1 - \frac{\frac{1}{M} \sum_i (y_i - \hat y_i)^2}{\text{Var}(y)} = \frac{\text{COV}^2(x, y)}{\text{Var}(x) \text{Var}(y)}
$$

donde $r = \frac{\text{COV}(x, y)}{\sqrt{\text{Var}(x) \text{Var}(y)}} \in [-1, 1]$ se conoce como [coeficiente de correlación de Pearson](https://en.wikipedia.org/wiki/Pearson_correlation_coefficient)

donde

- si $r=1$ existe una correlación lineal perfecta
- si $r=-1$ existe una anticorrelación lineal perfecta
- si $r=0$ no hay correlación lineal entre las variables

En general un $r>0.5$ se considera una correlación importante

**Calculando $r$ con y los parámetros de la regresión lineal**

Podemos usar el atributo de dataframe

```python
df.corr()
```

Que retorna la matriz de correlaciones lineales

```python
df.corr()
```

Si queremos también el valor de los parámetros podemos usar la función de scipy

```python
scipy.stats.linregress(x, # Variable independiente unidimensional
                       y # Variable dependiente unidimensional
                      )
```

Esta función retorna una tupla con

- Valor de la pendiente: $\theta_1$
- Valor de la intercepta: $\theta_0$
- Coeficiente de correlación $r$
- p-value
- Error estándar del ajuste

```python
fig, ax = plt.subplots(1, 3, figsize=(8, 3), tight_layout=True, sharey=True)
ax[0].set_ylabel(df.columns[0]);

for i, col in enumerate(df.columns[1:]):
    res = scipy.stats.linregress(df[col], df["consumo"])
    x_plot = np.linspace(np.amin(df[col]), np. amax(df[col]), num=100)
    ax[i].scatter(df[col], df["consumo"], label='datos', s=10)
    ax[i].plot(x_plot, res.slope*x_plot + res.intercept, lw=2, c='r', label='modelo');
    ax[i].set_xlabel(col)
    ax[i].set_title(f"$r$: {res.rvalue:0.5f}")
    ax[i].legend()
```

Es decir que visualmente parece existir

- una correlación positiva alta entre consumo y temperatura
- una correlación negativa moderada entre consumo y precio
- una correlación cercana a cero entre consumo e ingreso

### Test de hipótesis y conclusiones

La función `linregress` implementa el t-test sobre $\theta_1$ que vimos anteriormente. Usemos estos resultados para verificar si las correlaciones son estadísticamente significativas

```python
alpha = 0.05

for i, col in enumerate(df.columns[1:]):
    res = scipy.stats.linregress(df[col], df["consumo"])
    print(f"{col}: \t p-value:{res.pvalue:0.4f} \t ¿Menor que {alpha}?: {res.pvalue < alpha}")
```

Como complemento visualizemos

- las distribuciones bajo la hipótesis nula: linea azul
- los límites dados por $\alpha$: linea punteada negra
- El valor del observado para cada una de las variables: linea roja

```python
fig, ax = plt.subplots(1, 3, figsize=(8, 2), tight_layout=True, sharey=True)
ax[0].set_ylabel(df.columns[0]);

N = df.shape[0]
t = np.linspace(-7, 7, num=1000)
dist = scipy.stats.t(loc=0, scale=1, df=N-2) # dos grados de libertad

for i, col in enumerate(df.columns[1:]):
    res = scipy.stats.linregress(df[col], df["consumo"])
    t_data = res.rvalue*np.sqrt(N-2)/np.sqrt(1.-res.rvalue**2)
    ax[i].plot(t, dist.pdf(t))
    ax[i].plot([dist.ppf(alpha/2)]*2, [0, np.amax(dist.pdf(t))], 'k--')
    ax[i].plot([dist.ppf(1-alpha/2)]*2, [0, np.amax(dist.pdf(t))], 'k--')
    ax[i].plot([t_data]*2, [0, np.amax(dist.pdf(t))], 'r-')
    ax[i].set_xlabel(col)
```

**Conclusión**

Basado en los p-values y considerando $\alpha=0.05$

¿Qué podemos decir de las correlaciones con el consumo de helados?

> Rechazamos la hipótesis nula de que no existe correlación entre temperatura y consumo con un 95% de confianza

Para las variables ingreso y precio no existe suficiente evidencia para rechazar $H_0$

### Reflexión final

En el ejercicio anterior usamos t-test para una regresión lineal entre dos variables ¿Qué prueba puedo usar si quiero hacer regresión lineal multivariada?

> Se puede usar [ANOVA](https://pythonfordatascience.org/anova-python/)

¿Qué pasa si...

- mis datos tienen una relación que no es lineal?
- $\theta_1$ no es Gaussiano/normal?
- si el ruido no es Gaussiano?
- si el ruido es Gaussiano pero su varianza cambia en el tiempo?

> En estos casos no se cumplen los supuestos del modelo o del test, por ende el resultado no es confiable

Si mis supuestos no se cumplen con ninguna prueba parámetrica, la opión es utilizar pruebas no-paramétricas

## Prueba no-paramétrica: *Bootstrap*

Podemos estimar la incerteza de un estimador de forma no-paramétrica usando **muestreo tipo *bootstrap***

Esto consiste en tomar nuestro conjunto de datos de tamaño $N$ y crear $T$ nuevos conjuntos que "se le parezcan". Luego se calcula el valor del estimador que estamos buscando en los $T$ conjuntos. Con esto obtenemos una distribución para el estimador como muestra el siguiente diagrama

Para crear los subconjuntos podríamos suponer independencia y utilizar **muestreo con reemplazo**. Esto consiste en tomar $N$ muestras al azar permitiendo repeticiones, como muestra el siguiente diagrama

Si no es posible suponer indepdencia se puede realizar bootstrap basado en residuos y bootstrap dependiente. Puedes consultar más detalles sobre [*bootstrap*](https://www.stat.cmu.edu/~cshalizi/402/lectures/08-bootstrap/lecture-08.pdf) [aquí](http://homepage.divms.uiowa.edu/~rdecook/stat3200/notes/bootstrap_4pp.pdf) y [acá](https://www.sagepub.com/sites/default/files/upm-binaries/21122_Chapter_21.pdf). A continuación nos enfocaremos en el clásico muestreo con reemplazo y como implementarlo en Python

### Implementación con Numpy y Scipy

La función `numpy.random.choice` permite remuestrear un conjunto de datos

Por ejemplo para la regresión lineal debemos remuestrar las parejas/tuplas $(x_i, y_i)$

Luego calculamos y guardamos los parámetros del modelo para cada remuestreo. En este ejemplo haremos $1000$ repeticiones del conjunto de datos

```python
df = pd.read_csv('../data/helados.csv', header=0, index_col=0)
df.columns = ['consumo', 'ingreso', 'precio', 'temperatura']

x, y = df["temperatura"].values, df["consumo"].values
params = scipy.stats.linregress(x, y)

def muestreo_con_reemplazo(x, y):
    N = len(x)
    idx = np.random.choice(N, size=N, replace=True)
    return x[idx], y[idx]

def boostrap_linregress(x, y, T=100):
    # Parámetros: t0, t1 y r
    params = np.zeros(shape=(T, 3))
    for t in range(T):
        res = scipy.stats.linregress(*muestreo_con_reemplazo(x, y))
        params[t, :] = [res.intercept, res.slope, res.rvalue]
    return params

boostrap_params = boostrap_linregress(x, y, T=1000)
```

### Intervalos de confianza empíricos

Veamos la distribución empírica de $r$ obtenida usando bootstrap

En la figura de abajo tenemos

- Histograma azul: Distribución bootstrap de $r$
- Linea roja: $r$ de los datos
- Lineas punteadas negras: Intervalo de confianza empírico al 95%

```python
r_bootstrap = boostrap_params[:, 2]

fig, ax = plt.subplots(figsize=(4, 3), tight_layout=True)
hist_val, hist_lim, _ = ax.hist(r_bootstrap, bins=20, density=True)

ax.plot([params.rvalue]*2, [0, np.max(hist_val)], 'r-', lw=2)
IC = np.percentile(r_bootstrap, [2.5, 97.5])
ax.plot([IC[0]]*2, [0, np.max(hist_val)], 'k--', lw=2)
ax.plot([IC[1]]*2, [0, np.max(hist_val)], 'k--', lw=2)

print(f"Intervalo de confianza al 95% de r: {IC}")
```

De la figura podemos notar que el 95% de la distribución empírica esta sobre $r=0.5$

También podemos notar que la distribución empírica de $r$ no es simétrica, por lo que aplicar un t-test parámetrico sobre $r$ no hubiera sido correcto

### Visualizando la incerteza del modelo

Usando la distribución empírica de los parámetros $\theta_0$ y $\theta_1$ podemos visualizar la incerteza de nuestro modelo de regresión lineal

En la figura de abajo tenemos
- Puntos azules: Datos
- Linea roja: Modelo de regresión lineal en los datos
- Sombra rojo claro: $\pm 2$ desviaciones estándar del modelo en base a la distribución empírica

```python
fig, ax = plt.subplots(figsize=(4, 3), tight_layout=True)
ax.set_ylabel('Consumo')
ax.set_xlabel('Temperatura')
ax.scatter(x, y, zorder=100, s=10, label='datos')

def model(theta0, theta1, x):
    return x*theta1 + theta0

ax.plot(x_plot, model(params.intercept, params.slope, x_plot),
        c='r', lw=2, label='mejor ajuste')

dist_lines = model(boostrap_params[:, 0], boostrap_params[:, 1], x_plot.reshape(-1, 1)).T
mean_lines, std_lines = np.mean(dist_lines, axis=0), np.std(dist_lines, axis=0)
ax.fill_between(x_plot,
                mean_lines - 2*std_lines,
                mean_lines + 2*std_lines,
                color='r', alpha=0.25, label='incerteza')
plt.legend();
```

```python

```


```python
# Mount Google Drive
from google.colab import drive # import drive from google colab

ROOT = "/content/drive"     # default location for the drive
print(ROOT)                 # print content of ROOT (Optional)

drive.mount(ROOT,force_remount=True)
```

    /content/drive
    Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&response_type=code&scope=email%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdocs.test%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive.photos.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fpeopleapi.readonly

    Enter your authorization code:
    ··········
    Mounted at /content/drive

#  Neuromatch Academy: Week 1, Day 1, Tutorial 2
# Model Types: "How" models
__Content creators:__ Matt Laporte, Byron Galbraith, Konrad Kording

__Content reviewers:__ Dalin Guo, Aishwarya Balwani, Madineh Sarvestani, Maryam Vaziri-Pashkam, Michael Waskom

___
# Tutorial Objectives
This is tutorial 2 of a 3-part series on different flavors of models used to understand neural data. In this tutorial we will explore models that can potentially explain *how* the spiking data we have observed is produced

To understand the mechanisms that give rise to the neural data we save in Tutorial 1,  we will build simple neuronal models and compare their spiking response to real data. We will:
- Write code to simulate a  simple "leaky integrate-and-fire" neuron model
- Make the model more complicated — but also more realistic — by adding more physiologically-inspired details

```python
#@title Video 1: "How" models
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='PpnagITsb3E', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=PpnagITsb3E

# Setup

```python
import numpy as np
import matplotlib.pyplot as plt
from scipy import stats
```

```python
#@title Figure Settings
import ipywidgets as widgets #interactive display

%matplotlib inline
%config InlineBackend.figure_format = 'retina'
plt.style.use("https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle")

```

```python
#@title Helper Functions
def histogram(counts, bins, vlines=(), ax=None, ax_args=None, **kwargs):
  """Plot a step histogram given counts over bins."""
  if ax is None:
    _, ax = plt.subplots()

  # duplicate the first element of `counts` to match bin edges
  counts = np.insert(counts, 0, counts[0])

  ax.fill_between(bins, counts, step="pre", alpha=0.4, **kwargs)  # area shading
  ax.plot(bins, counts, drawstyle="steps", **kwargs)  # lines

  for x in vlines:
    ax.axvline(x, color='r', linestyle='dotted')  # vertical line

  if ax_args is None:
    ax_args = {}

  # heuristically set max y to leave a bit of room
  ymin, ymax = ax_args.get('ylim', [None, None])
  if ymax is None:
    ymax = np.max(counts)
    if ax_args.get('yscale', 'linear') == 'log':
      ymax *= 1.5
    else:
      ymax *= 1.1
      if ymin is None:
        ymin = 0

  if ymax == ymin:
    ymax = None

  ax_args['ylim'] = [ymin, ymax]

  ax.set(**ax_args)
  ax.autoscale(enable=False, axis='x', tight=True)

def plot_neuron_stats(v, spike_times):
  fig, (ax1, ax2) = plt.subplots(ncols=2, figsize=(12, 5))

  # membrane voltage trace
  ax1.plot(v[0:100])
  ax1.set(xlabel='Time', ylabel='Voltage')
  # plot spike events
  for x in spike_times:
    if x >= 100:
      break
    ax1.axvline(x, color='red')

  # ISI distribution
  isi = np.diff(spike_times)
  n_bins = np.arange(isi.min(), isi.max() + 2) - .5
  counts, bins = np.histogram(isi, n_bins)
  vlines = []
  if len(isi) > 0:
    vlines = [np.mean(isi)]
  xmax = max(20, int(bins[-1])+5)
  histogram(counts, bins, vlines=vlines, ax=ax2, ax_args={
    'xlabel': 'Inter-spike interval',
    'ylabel': 'Number of intervals',
    'xlim': [0, xmax]
  })
  plt.show()
```

# Section 1: The Linear Integrate-and-Fire Neuron

How does a neuron spike?

A neuron charges and discharges an electric field across its cell membrane. The state of this electric field can be described by the _membrane potential_. The membrane potential rises due to excitation of the neuron, and when it reaches a threshold a spike occurs. The potential resets, and must rise to a threshold again before the next spike occurs.

One of the simplest models of spiking neuron behavior is the linear integrate-and-fire model neuron. In this model, the neuron increases its membrane potential $V_m$ over time in response to excitatory input currents $I$ scaled by some factor $\alpha$:

\begin{align}
  dV_m = {\alpha}I
\end{align}

Once $V_m$ reaches a threshold value a spike is produced, $V_m$ is reset to a starting value, and the process continues.

Here, we will take the starting and threshold potentials as $0$ and $1$, respectively. So, for example, if $\alpha I=0.1$ is constant---that is, the input current is constant---then $dV_m=0.1$, and at each timestep the membrane potential $V_m$ increases by $0.1$ until after $(1-0)/0.1 = 10$ timesteps it reaches the threshold and resets to $V_m=0$, and so on.

Note that we define the membrane potential $V_m$ as a scalar: a single real (or floating point) number. However, a biological neuron's membrane potential will not be exactly constant at all points on its cell membrane at a given time. We could capture this variation with a more complex model (e.g. with more numbers). Do we need to?

The proposed model is a 1D simplification. There are many details we could add to it, to preserve different parts of the complex structure and dynamics of a real neuron. If we were interested in small or local changes in the membrane potential, our 1D simplification could be a problem. However, we'll assume an idealized "point" neuron model for our current purpose.

#### Spiking Inputs

Given our simplified model for the neuron dynamics, we still need to consider what form the input $I$ will take. How should we specify the firing behavior of the presynaptic neuron(s) providing the inputs to our model neuron?

Unlike in the simple example above, where $\alpha I=0.1$, the input current is generally not constant. Physical inputs tend to vary with time. We can describe this variation with a distribution.

We'll assume the input current $I$ over a timestep is due to equal contributions from a non-negative ($\ge 0$) integer number of input spikes arriving in that timestep. Our model neuron might integrate currents from 3 input spikes in one timestep, and 7 spikes in the next timestep. We should see similar behavior when sampling from our distribution.

Given no other information about the input neurons, we will also assume that the distribution has a mean (i.e. mean rate, or number of spikes received per timestep), and that the spiking events of the input neuron(s) are independent in time. Are these reasonable assumptions in the context of real neurons?

A suitable distribution given these assumptions is the Poisson distribution, which we'll use to model $I$:

\begin{align}
  I \sim \mathrm{Poisson}(\lambda)
\end{align}

where $\lambda$ is the mean of the distribution: the average rate of spikes received per timestep.

### Exercise 1: Compute $dV_m$

For your first exercise, you will write the code to compute the change in voltage $dV_m$ (per timestep) of the linear integrate-and-fire model neuron. The rest of the code to handle numerical integration is provided for you, so you just need to fill in a definition for `dv` in the `lif_neuron` function below. The value of $\lambda$ for the Poisson random variable is given by the function argument `rate`.

The [`scipy.stats`](https://docs.scipy.org/doc/scipy/reference/stats.html) package is a great resource for working with and sampling from various probability distributions. We will use the `scipy.stats.poisson` class and its method `rvs` to produce Poisson-distributed random samples. In this tutorial, we have imported this package with the alias `stats`, so you should refer to it in your code as `stats.poisson`.

```python
def lif_neuron(n_steps=1000, alpha=0.01, rate=10):
  """ Simulate a linear integrate-and-fire neuron.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    rate (int): The mean rate of incoming spikes

  """
  # precompute Poisson samples for speed
  exc = stats.poisson(rate).rvs(n_steps)

  v = np.zeros(n_steps)
  spike_times = []

  ################################################################################
  # Students: compute dv, then comment out or remove the next line
  #raise NotImplementedError("Excercise: compute the change in membrane potential")
  ################################################################################

  for i in range(1, n_steps):

    dv = alpha*exc[i]

    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times

# Uncomment these lines after completing the lif_neuron function
v, spike_times = lif_neuron()
plot_neuron_stats(v, spike_times)
```

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W1D1_ModelTypes/solutions/W1D1_Tutorial2_Solution_f8960ca1.py)

*Example output:*

## Interactive Demo: Linear-IF neuron
Like last time, you can now explore how various parametes of the LIF model influence the ISI distribution.

```python
#@title

#@markdown You don't need to worry about how the code works – but you do need to **run the cell** to enable the sliders.

def _lif_neuron(n_steps=1000, alpha=0.01, rate=10):
  exc = stats.poisson(rate).rvs(n_steps)
  v = np.zeros(n_steps)
  spike_times = []
  for i in range(1, n_steps):
    dv = alpha * exc[i]
    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0
  return v, spike_times

@widgets.interact(
  n_steps=widgets.FloatLogSlider(1000.0, min=2, max=4),
  alpha=widgets.FloatLogSlider(0.01, min=-2, max=-1),
  rate=widgets.IntSlider(10, min=5, max=20)
)
def plot_lif_neuron(n_steps=1000, alpha=0.01, rate=10):
  v, spike_times = _lif_neuron(int(n_steps), alpha, rate)
  plot_neuron_stats(v, spike_times)
```

    interactive(children=(FloatLogSlider(value=1000.0, description='n_steps', min=2.0), FloatLogSlider(value=0.01,…

```python
#@title Video 2: Linear-IF models
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='QBD7kulhg4U', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=QBD7kulhg4U

# Section 2: Inhibitory signals

Our linear integrate-and-fire neuron from the previous section was indeed able to produce spikes. However, our ISI histogram doesn't look much like empirical ISI histograms seen in Tutorial 1, which had an exponential-like shape. What is our model neuron missing, given that it doesn't behave like a real neuron?

In the previous model we only considered excitatory behavior -- the only way the membrane potential could decrease was upon a spike event. We know, however, that there are other factors that can drive $V_m$ down. First is the natural tendency of the neuron to return to some steady state or resting potential. We can update our previous model as follows:

\begin{align}
  dV_m = -{\beta}V_m + {\alpha}I
\end{align}

where $V_m$ is the current membrane potential and $\beta$ is some leakage factor. This is a basic form of the popular Leaky Integrate-and-Fire model neuron (for a more detailed discussion of the LIF Neuron, see the Appendix).

We also know that in addition to excitatory presynaptic neurons, we can have inhibitory presynaptic neurons as well. We can model these inhibitory neurons with another Poisson random variable:

\begin{align}
I = I_{exc} - I_{inh} \\
I_{exc} \sim \mathrm{Poisson}(\lambda_{exc}) \\
I_{inh} \sim \mathrm{Poisson}(\lambda_{inh})
\end{align}

where $\lambda_{exc}$ and $\lambda_{inh}$ are the average spike rates (per timestep) of the excitatory and inhibitory presynaptic neurons, respectively.

### Exercise 2: Compute $dV_m$ with inhibitory signals

For your second exercise, you will again write the code to compute the change in voltage $dV_m$, though now of the LIF model neuron described above. Like last time, the rest of the code needed to handle the neuron dynamics are provided for you, so you just need to fill in a definition for `dv` below.

```python
def lif_neuron_inh(n_steps=1000, alpha=0.5, beta=0.1, exc_rate=10, inh_rate=10):
  """ Simulate a simplified leaky integrate-and-fire neuron with both excitatory
  and inhibitory inputs.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    beta (float): The membrane potential leakage factor
    exc_rate (int): The mean rate of the incoming excitatory spikes
    inh_rate (int): The mean rate of the incoming inhibitory spikes
  """

  # precompute Poisson samples for speed
  exc = stats.poisson(exc_rate).rvs(n_steps)
  inh = stats.poisson(inh_rate).rvs(n_steps)

  v = np.zeros(n_steps)
  spike_times = []

  ###############################################################################
  # Students: compute dv, then comment out or remove the next line
  #raise NotImplementedError("Excercise: compute the change in membrane potential")
  ################################################################################

  for i in range(1, n_steps):

    dv = (exc[i]-inh[i])*alpha -beta*v[i-1]
    #use v[i-1] because we're calculating v[i] in this step, so it's based on v[i-1]

    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times

# Uncomment these lines do make the plot once you've completed the function
v, spike_times = lif_neuron_inh()
plot_neuron_stats(v, spike_times)
```

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W1D1_ModelTypes/solutions/W1D1_Tutorial2_Solution_4d9a2677.py)

*Example output:*

## Interactive Demo: LIF + inhibition neuron

```python
#@title
#@markdown **Run the cell** to enable the sliders.
def _lif_neuron_inh(n_steps=1000, alpha=0.5, beta=0.1, exc_rate=10, inh_rate=10):
  """ Simulate a simplified leaky integrate-and-fire neuron with both excitatory
  and inhibitory inputs.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    beta (float): The membrane potential leakage factor
    exc_rate (int): The mean rate of the incoming excitatory spikes
    inh_rate (int): The mean rate of the incoming inhibitory spikes
  """
  # precompute Poisson samples for speed
  exc = stats.poisson(exc_rate).rvs(n_steps)
  inh = stats.poisson(inh_rate).rvs(n_steps)

  v = np.zeros(n_steps)
  spike_times = []
  for i in range(1, n_steps):
    dv = -beta * v[i-1] + alpha * (exc[i] - inh[i])
    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times

@widgets.interact(n_steps=widgets.FloatLogSlider(1000.0, min=2.5, max=4),
                  alpha=widgets.FloatLogSlider(0.5, min=-1, max=1),
                  beta=widgets.FloatLogSlider(0.1, min=-1, max=0),
                  exc_rate=widgets.IntSlider(12, min=10, max=20),
                  inh_rate=widgets.IntSlider(12, min=10, max=20))
def plot_lif_neuron(n_steps=1000, alpha=0.5, beta=0.1, exc_rate=10, inh_rate=10):
  v, spike_times = _lif_neuron_inh(int(n_steps), alpha, beta, exc_rate, inh_rate)
  plot_neuron_stats(v, spike_times)
```

```python
#@title Video 3: LIF + inhibition
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='Aq7JrxRkn2w', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=Aq7JrxRkn2w

#Summary

In this tutorial we gained some intuition for the mechanisms that produce the observed behavior in our real neural data. First, we built a simple neuron model with excitatory input and saw that it's behavior, measured using the ISI distribution, did not match our real neurons. We then improved our model by adding leakiness and inhibitory input. The behavior of this balanced model was much closer to the real neural data.

# Bonus

### Why do neurons spike?

A neuron stores energy in an electric field across its cell membrane, by controlling the distribution of charges (ions) on either side of the membrane. This energy is rapidly discharged to generate a spike when the field potential (or membrane potential) crosses a threshold. The membrane potential may be driven toward or away from this threshold, depending on inputs from other neurons: excitatory or inhibitory, respectively. The membrane potential tends to revert to a resting potential, for example due to the leakage of ions across the membrane, so that reaching the spiking threshold depends not only on the amount of input ever received following the last spike, but also the timing of the inputs.

The storage of energy by maintaining a field potential across an insulating membrane can be modeled by a capacitor. The leakage of charge across the membrane can be modeled by a resistor. This is the basis for the leaky integrate-and-fire neuron model.

### The LIF Model Neuron

The full equation for the LIF neuron is

\begin{align}
C_{m}\frac{dV_m}{dt} = -(V_m - V_{rest})/R_{m} + I
\end{align}

where $C_m$ is the membrane capacitance, $R_M$ is the membrane resistance, $V_{rest}$ is the resting potential, and $I$ is some input current (from other neurons, an electrode, ...).

In our above examples we set many of these parameters to convenient values ($C_m = R_m = dt = 1$, $V_{rest} = 0$) to focus more on the general behavior of the model. However, these too can be manipulated to achieve different dynamics, or to ensure the dimensions of the problem are preserved between simulation units and experimental units (e.g. with $V_m$ given in millivolts, $R_m$ in megaohms, $t$ in milliseconds).