Probabilistic Programming
=====
and Bayesian Methods for Hackers
========

##### Version 0.1

`Original content created by Cam Davidson-Pilon`

`Ported to Python 3 and PyMC3 by Max Margenot (@clean_utensils) and Thomas Wiecki (@twiecki) at Quantopian (@quantopian)`
___

Welcome to *Bayesian Methods for Hackers*. The full Github repository is available at [github/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers](https://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers). The other chapters can be found on the project's [homepage](https://camdavidsonpilon.github.io/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/). We hope you enjoy the book, and we encourage any contributions!

Chapter 1
======
***

The Philosophy of Bayesian Inference
------

> You are a skilled programmer, but bugs still slip into your code. After a particularly difficult implementation of an algorithm, you decide to test your code on a trivial example. It passes. You test the code on a harder problem. It passes once again. And it passes the next, *even more difficult*, test too! You are starting to believe that there may be no bugs in this code...

If you think this way, then congratulations, you already are thinking Bayesian! Bayesian inference is simply updating your beliefs after considering new evidence. A Bayesian can rarely be certain about a result, but he or she can be very confident. Just like in the example above, we can never be 100% sure that our code is bug-free unless we test it on every possible problem; something rarely possible in practice. Instead, we can test it on a large number of problems, and if it succeeds we can feel more *confident* about our code, but still not certain.  Bayesian inference works identically: we update our beliefs about an outcome; rarely can we be absolutely sure unless we rule out all other alternatives.

### The Bayesian state of mind

Bayesian inference differs from more traditional statistical inference by preserving *uncertainty*. At first, this sounds like a bad statistical technique. Isn't statistics all about deriving *certainty* from randomness? To reconcile this, we need to start thinking like Bayesians.

The Bayesian world-view interprets probability as measure of *believability in an event*, that is, how confident we are in an event occurring. In fact, we will see in a moment that this is the natural interpretation of probability.

For this to be clearer, we consider an alternative interpretation of probability: *Frequentist*, known as the more *classical* version of statistics, assume that probability is the long-run frequency of events (hence the bestowed title). For example, the *probability of plane accidents* under a frequentist philosophy is interpreted as the *long-term frequency of plane accidents*. This makes logical sense for many probabilities of events, but becomes more difficult to understand when events have no long-term frequency of occurrences. Consider: we often assign probabilities to outcomes of presidential elections, but the election itself only happens once! Frequentists get around this by invoking alternative realities and saying across all these realities, the frequency of occurrences defines the probability.

Bayesians, on the other hand, have a more intuitive approach. Bayesians interpret a probability as measure of *belief*, or confidence, of an event occurring. Simply, a probability is a summary of an opinion. An individual who assigns a belief of 0 to an event has no confidence that the event will occur; conversely, assigning a belief of 1 implies that the individual is absolutely certain of an event occurring. Beliefs between 0 and 1 allow for weightings of other outcomes. This definition agrees with the probability of a plane accident example, for having observed the frequency of plane accidents, an individual's belief should be equal to that frequency, excluding any outside information. Similarly, under this definition of probability being equal to beliefs, it is meaningful to speak about probabilities (beliefs) of presidential election outcomes: how confident are you candidate *A* will win?

Notice in the paragraph above, I assigned the belief (probability) measure to an *individual*, not to Nature. This is very interesting, as this definition leaves room for conflicting beliefs between individuals. Again, this is appropriate for what naturally occurs: different individuals have different beliefs of events occurring, because they possess different *information* about the world. The existence of different beliefs does not imply that anyone is wrong. Consider the following examples demonstrating the relationship between individual beliefs and probabilities:

- I flip a coin, and we both guess the result. We would both agree, assuming the coin is fair, that the probability of Heads is 1/2. Assume, then, that I peek at the coin. Now I know for certain what the result is: I assign probability 1.0 to either Heads or Tails (whichever it is). Now what is *your* belief that the coin is Heads? My knowledge of the outcome has not changed the coin's results. Thus we assign different probabilities to the result.

-  Your code either has a bug in it or not, but we do not know for certain which is true, though we have a belief about the presence or absence of a bug.

-  A medical patient is exhibiting symptoms $x$, $y$ and $z$. There are a number of diseases that could be causing all of them, but only a single disease is present. A doctor has beliefs about which disease, but a second doctor may have slightly different beliefs.

This philosophy of treating beliefs as probability is natural to humans. We employ it constantly as we interact with the world and only see partial truths, but gather evidence to form beliefs. Alternatively, you have to be *trained* to think like a frequentist.

To align ourselves with traditional probability notation, we denote our belief about event $A$ as $P(A)$. We call this quantity the *prior probability*.

John Maynard Keynes, a great economist and thinker, said "When the facts change, I change my mind. What do you do, sir?" This quote reflects the way a Bayesian updates his or her beliefs after seeing evidence. Even — especially — if the evidence is counter to what was initially believed, the evidence cannot be ignored. We denote our updated belief as $P(A |X )$, interpreted as the probability of $A$ given the evidence $X$. We call the updated belief the *posterior probability* so as to contrast it with the prior probability. For example, consider the posterior probabilities (read: posterior beliefs) of the above examples, after observing some evidence $X$:

1\. $P(A): \;\;$ the coin has a 50 percent chance of being Heads. $P(A | X):\;\;$ You look at the coin, observe a Heads has landed, denote this information $X$, and trivially assign probability 1.0 to Heads and 0.0 to Tails.

2\.   $P(A): \;\;$  This big, complex code likely has a bug in it. $P(A | X): \;\;$ The code passed all $X$ tests; there still might be a bug, but its presence is less likely now.

3\.  $P(A):\;\;$ The patient could have any number of diseases. $P(A | X):\;\;$ Performing a blood test generated evidence $X$, ruling out some of the possible diseases from consideration.

It's clear that in each example we did not completely discard the prior belief after seeing new evidence $X$, but we *re-weighted the prior* to incorporate the new evidence (i.e. we put more weight, or confidence, on some beliefs versus others).

By introducing prior uncertainty about events, we are already admitting that any guess we make is potentially very wrong. After observing data, evidence, or other information, we update our beliefs, and our guess becomes *less wrong*. This is the alternative side of the prediction coin, where typically we try to be *more right*.

### Bayesian Inference in Practice

 If frequentist and Bayesian inference were programming functions, with inputs being statistical problems, then the two would be different in what they return to the user. The frequentist inference function would return a number, representing an estimate (typically a summary statistic like the sample average etc.), whereas the Bayesian function would return *probabilities*.

For example, in our debugging problem above, calling the frequentist function with the argument "My code passed all $X$ tests; is my code bug-free?" would return a *YES*. On the other hand, asking our Bayesian function "Often my code has bugs. My code passed all $X$ tests; is my code bug-free?" would return something very different: probabilities of *YES* and *NO*. The function might return:

>    *YES*, with probability 0.8; *NO*, with probability 0.2

This is very different from the answer the frequentist function returned. Notice that the Bayesian function accepted an additional argument:  *"Often my code has bugs"*. This parameter is the *prior*. By including the prior parameter, we are telling the Bayesian function to include our belief about the situation. Technically this parameter in the Bayesian function is optional, but we will see excluding it has its own consequences.

#### Incorporating evidence

As we acquire more and more instances of evidence, our prior belief is *washed out* by the new evidence. This is to be expected. For example, if your prior belief is something ridiculous, like "I expect the sun to explode today", and each day you are proved wrong, you would hope that any inference would correct you, or at least align your beliefs better. Bayesian inference will correct this belief.

Denote $N$ as the number of instances of evidence we possess. As we gather an *infinite* amount of evidence, say as $N \rightarrow \infty$, our Bayesian results (often) align with frequentist results. Hence for large $N$, statistical inference is more or less objective. On the other hand, for small $N$, inference is much more *unstable*: frequentist estimates have more variance and larger confidence intervals. This is where Bayesian analysis excels. By introducing a prior, and returning probabilities (instead of a scalar estimate), we *preserve the uncertainty* that reflects the instability of statistical inference of a small $N$ dataset.

One may think that for large $N$, one can be indifferent between the two techniques since they offer similar inference, and might lean towards the computationally-simpler, frequentist methods. An individual in this position should consider the following quote by Andrew Gelman (2005)[1], before making such a decision:

> Sample sizes are never large. If $N$ is too small to get a sufficiently-precise estimate, you need to get more data (or make more assumptions). But once $N$ is "large enough," you can start subdividing the data to learn more (for example, in a public opinion poll, once you have a good estimate for the entire country, you can estimate among men and women, northerners and southerners, different age groups, etc.). $N$ is never enough because if it were "enough" you'd already be on to the next problem for which you need more data.

### Are frequentist methods incorrect then?

**No.**

Frequentist methods are still useful or state-of-the-art in many areas. Tools such as least squares linear regression, LASSO regression, and expectation-maximization algorithms are all powerful and fast. Bayesian methods complement these techniques by solving problems that these approaches cannot, or by illuminating the underlying system with more flexible modeling.

#### A note on *Big Data*
Paradoxically, big data's predictive analytic problems are actually solved by relatively simple algorithms [2][4]. Thus we can argue that big data's prediction difficulty does not lie in the algorithm used, but instead on the computational difficulties of storage and execution on big data. (One should also consider Gelman's quote from above and ask "Do I really have big data?")

The much more difficult analytic problems involve *medium data* and, especially troublesome, *really small data*. Using a similar argument as  Gelman's above, if big data problems are *big enough* to be readily solved, then we should be more interested in the *not-quite-big enough* datasets.

### Our Bayesian framework

We are interested in beliefs, which can be interpreted as probabilities by thinking Bayesian. We have a *prior* belief in event $A$, beliefs formed by previous information, e.g., our prior belief about bugs being in our code before performing tests.

Secondly, we observe our evidence. To continue our buggy-code example: if our code passes $X$ tests, we want to update our belief to incorporate this. We call this new belief the *posterior* probability. Updating our belief is done via the following equation, known as Bayes' Theorem, after its discoverer Thomas Bayes:

\begin{align}
 P( A | X ) = & \frac{ P(X | A) P(A) } {P(X) } \\\\[5pt]
& \propto P(X | A) P(A)\;\; (\propto \text{is proportional to })
\end{align}

The above formula is not unique to Bayesian inference: it is a mathematical fact with uses outside Bayesian inference. Bayesian inference merely uses it to connect prior probabilities $P(A)$ with an updated posterior probabilities $P(A | X )$.

##### Example: Mandatory coin-flip example

Every statistics text must contain a coin-flipping example, I'll use it here to get it out of the way. Suppose, naively, that you are unsure about the probability of heads in a coin flip (spoiler alert: it's 50%). You believe there is some true underlying ratio, call it $p$, but have no prior opinion on what $p$ might be.

We begin to flip a coin, and record the observations: either $H$ or $T$. This is our observed data. An interesting question to ask is how our inference changes as we observe more and more data? More specifically, what do our posterior probabilities look like when we have little data, versus when we have lots of data.

Below we plot a sequence of updating posterior probabilities as we observe increasing amounts of data (coin flips).

```python
"""
The book uses a custom matplotlibrc file, which provides the unique styles for
matplotlib plots. If executing this book, and you wish to use the book's
styling, provided are two options:
    1. Overwrite your own matplotlibrc file with the rc-file provided in the
       book's styles/ dir. See http://matplotlib.org/users/customizing.html
    2. Also in the styles is  bmh_matplotlibrc.json file. This can be used to
       update the styles in only this notebook. Try running the following code:

        import json
        s = json.load(open("../styles/bmh_matplotlibrc.json"))
        matplotlib.rcParams.update(s)

"""

# The code below can be passed over, as it is currently not important, plus it
# uses advanced topics we have not covered yet. LOOK AT PICTURE, MICHAEL!
%matplotlib inline
from IPython.core.pylabtools import figsize
import numpy as np
from matplotlib import pyplot as plt
figsize(11, 9)

import scipy.stats as stats

dist = stats.beta
n_trials = [0, 1, 2, 3, 4, 5, 8, 15, 50, 500]
data = stats.bernoulli.rvs(0.5, size=n_trials[-1])
x = np.linspace(0, 1, 100)

# For the already prepared, I'm using Binomial's conj. prior.
for k, N in enumerate(n_trials):
    sx = plt.subplot(len(n_trials)//2, 2, k+1)
    plt.xlabel("$p$, probability of heads") \
        if k in [0, len(n_trials)-1] else None
    plt.setp(sx.get_yticklabels(), visible=False)
    heads = data[:N].sum()
    y = dist.pdf(x, 1 + heads, 1 + N - heads)
    plt.plot(x, y, label="observe %d tosses,\n %d heads" % (N, heads))
    plt.fill_between(x, 0, y, color="#348ABD", alpha=0.4)
    plt.vlines(0.5, 0, 4, color="k", linestyles="--", lw=1)

    leg = plt.legend()
    leg.get_frame().set_alpha(0.4)
    plt.autoscale(tight=True)

plt.suptitle("Bayesian updating of posterior probabilities",
             y=1.02,
             fontsize=14)

plt.tight_layout()
```

The posterior probabilities are represented by the curves, and our uncertainty is proportional to the width of the curve. As the plot above shows, as we start to observe data our posterior probabilities start to shift and move around. Eventually, as we observe more and more data (coin-flips), our probabilities will tighten closer and closer around the true value of $p=0.5$ (marked by a dashed line).

Notice that the plots are not always *peaked* at 0.5. There is no reason it should be: recall we assumed we did not have a prior opinion of what $p$ is. In fact, if we observe quite extreme data, say 8 flips and only 1 observed heads, our distribution would look very biased *away* from lumping around 0.5 (with no prior opinion, how confident would you feel betting on a fair coin after observing 8 tails and 1 head?). As more data accumulates, we would see more and more probability being assigned at $p=0.5$, though never all of it.

The next example is a simple demonstration of the mathematics of Bayesian inference.

##### Example: Bug, or just sweet, unintended feature?

Let $A$ denote the event that our code has **no bugs** in it. Let $X$ denote the event that the code passes all debugging tests. For now, we will leave the prior probability of no bugs as a variable, i.e. $P(A) = p$.

We are interested in $P(A|X)$, i.e. the probability of no bugs, given our debugging tests $X$. To use the formula above, we need to compute some quantities.

What is $P(X | A)$, i.e., the probability that the code passes $X$ tests *given* there are no bugs? Well, it is equal to 1, for a code with no bugs will pass all tests.

$P(X)$ is a little bit trickier: The event $X$ can be divided into two possibilities, event $X$ occurring even though our code *indeed has* bugs (denoted $\sim A\;$, spoken *not $A$*), or event $X$ without bugs ($A$). $P(X)$ can be represented as:

\begin{align}
P(X ) & = P(X \text{ and } A) + P(X \text{ and } \sim A) \\\\[5pt]
 & = P(X|A)P(A) + P(X | \sim A)P(\sim A)\\\\[5pt]
& = P(X|A)p + P(X | \sim A)(1-p)
\end{align}

We have already computed $P(X|A)$ above. On the other hand, $P(X | \sim A)$ is subjective: our code can pass tests but still have a bug in it, though the probability there is a bug present is reduced. Note this is dependent on the number of tests performed, the degree of complication in the tests, etc. Let's be conservative and assign $P(X|\sim A) = 0.5$. Then

\begin{align}
P(A | X) & = \frac{1\cdot p}{ 1\cdot p +0.5 (1-p) } \\\\
& = \frac{ 2 p}{1+p}
\end{align}
This is the posterior probability. What does it look like as a function of our prior, $p \in [0,1]$?

```python
figsize(12.5, 4)
p = np.linspace(0, 1, 50)
plt.plot(p, 2*p/(1+p), color="#348ABD", lw=3)
#plt.fill_between(p, 2*p/(1+p), alpha=.5, facecolor=["#A60628"])
plt.scatter(0.2, 2*(0.2)/1.2, s=140, c="#348ABD")
plt.xlim(0, 1)
plt.ylim(0, 1)
plt.xlabel("Prior, $P(A) = p$")
plt.ylabel("Posterior, $P(A|X)$, with $P(A) = p$")
plt.title("Are there bugs in my code?");
```

We can see the biggest gains if we observe the $X$ tests passed when the prior probability, $p$, is low. Let's settle on a specific value for the prior. I'm a strong programmer (I think), so I'm going to give myself a realistic prior of 0.20, that is, there is a 20% chance that I write code bug-free. To be more realistic, this prior should be a function of how complicated and large the code is, but let's pin it at 0.20. Then my updated belief that my code is bug-free is 0.33.

Recall that the prior is a probability: $p$ is the prior probability that there *are no bugs*, so $1-p$ is the prior probability that there *are bugs*.

Similarly, our posterior is also a probability, with $P(A | X)$ the probability there is no bug *given we saw all tests pass*, hence $1-P(A|X)$ is the probability there is a bug *given all tests passed*. What does our posterior probability look like? Below is a chart of both the prior and the posterior probabilities.

```python
figsize(12.5, 4)
colours = ["#348ABD", "#A60628"]

prior = [0.20, 0.80]
posterior = [1./3, 2./3]
plt.bar([0, .7], prior, alpha=0.70, width=0.25,
        color=colours[0], label="prior distribution",
        lw="3", edgecolor=colours[0])

plt.bar([0+0.25, .7+0.25], posterior, alpha=0.7,
        width=0.25, color=colours[1],
        label="posterior distribution",
        lw="3", edgecolor=colours[1])

plt.xticks([0.20, .95], ["Bugs Absent", "Bugs Present"])
plt.title("Prior and Posterior probability of bugs present")
plt.ylabel("Probability")
plt.legend(loc="upper left");
```

Notice that after we observed $X$ occur, the probability of bugs being absent increased. By increasing the number of tests, we can approach confidence (probability 1) that there are no bugs present.

This was a very simple example of Bayesian inference and Bayes rule. Unfortunately, the mathematics necessary to perform more complicated Bayesian inference only becomes more difficult, except for artificially constructed cases. We will later see that this type of mathematical analysis is actually unnecessary. First we must broaden our modeling tools. The next section deals with *probability distributions*. If you are already familiar, feel free to skip (or at least skim), but for the less familiar the next section is essential.

_______

## Probability Distributions

**Let's quickly recall what a probability distribution is:** Let $Z$ be some random variable. Then associated with $Z$ is a *probability distribution function* that assigns probabilities to the different outcomes $Z$ can take. Graphically, a probability distribution is a curve where the probability of an outcome is proportional to the height of the curve. You can see examples in the first figure of this chapter.

We can divide random variables into three classifications:

-   **$Z$ is discrete**: Discrete random variables may only assume values on a specified list. Things like populations, movie ratings, and number of votes are all discrete random variables. Discrete random variables become more clear when we contrast them with...

-   **$Z$ is continuous**: Continuous random variable can take on arbitrarily exact values. For example, temperature, speed, time, color are all modeled as continuous variables because you can progressively make the values more and more precise.

- **$Z$ is mixed**: Mixed random variables assign probabilities to both discrete and continuous random variables, i.e. it is a combination of the above two categories.

### Discrete Case
If $Z$ is discrete, then its distribution is called a *probability mass function*, which measures the probability $Z$ takes on the value $k$, denoted $P(Z=k)$. Note that the probability mass function completely describes the random variable $Z$, that is, if we know the mass function, we know how $Z$ should behave. There are popular probability mass functions that consistently appear: we will introduce them as needed, but let's introduce the first very useful probability mass function. We say $Z$ is *Poisson*-distributed if:

$$P(Z = k) =\frac{ \lambda^k e^{-\lambda} }{k!}, \; \; k=0,1,2, \dots $$

$\lambda$ is called a parameter of the distribution, and it controls the distribution's shape. For the Poisson distribution, $\lambda$ can be any positive number. By increasing $\lambda$, we add more probability to larger values, and conversely by decreasing $\lambda$ we add more probability to smaller values. One can describe $\lambda$ as the *intensity* of the Poisson distribution.

Unlike $\lambda$, which can be any positive number, the value $k$ in the above formula must be a non-negative integer, i.e., $k$ must take on values 0,1,2, and so on. This is very important, because if you wanted to model a population you could not make sense of populations with 4.25 or 5.612 members.

If a random variable $Z$ has a Poisson mass distribution, we denote this by writing

$$Z \sim \text{Poi}(\lambda) $$

One useful property of the Poisson distribution is that its expected value is equal to its parameter, i.e.:

$$E\large[ \;Z\; | \; \lambda \;\large] = \lambda $$

We will use this property often, so it's useful to remember. Below, we plot the probability mass distribution for different $\lambda$ values. The first thing to notice is that by increasing $\lambda$, we add more probability of larger values occurring. Second, notice that although the graph ends at 15, the distributions do not. They assign positive probability to every non-negative integer.

```python
figsize(12.5, 4)

import scipy.stats as stats
a = np.arange(16)
poi = stats.poisson
lambda_ = [1.5, 4.25]
colours = ["#348ABD", "#A60628"]

plt.bar(a, poi.pmf(a, lambda_[0]), color=colours[0],
        label="$\lambda = %.1f$" % lambda_[0], alpha=0.60,
        edgecolor=colours[0], lw="3")

plt.bar(a, poi.pmf(a, lambda_[1]), color=colours[1],
        label="$\lambda = %.1f$" % lambda_[1], alpha=0.60,
        edgecolor=colours[1], lw="3")

plt.xticks(a + 0.4, a)
plt.legend()
plt.ylabel("probability of $k$")
plt.xlabel("$k$")
plt.title("Probability mass function of a Poisson random variable; differing \
$\lambda$ values");
```

### Continuous Case
Instead of a probability mass function, a continuous random variable has a *probability density function*. This might seem like unnecessary nomenclature, but the density function and the mass function are very different creatures. An example of continuous random variable is a random variable with *exponential density*. The density function for an exponential random variable looks like this:

$$f_Z(z | \lambda) = \lambda e^{-\lambda z }, \;\; z\ge 0$$

Like a Poisson random variable, an exponential random variable can take on only non-negative values. But unlike a Poisson variable, the exponential can take on *any* non-negative values, including non-integral values such as 4.25 or 5.612401. This property makes it a poor choice for count data, which must be an integer, but a great choice for time data, temperature data (measured in Kelvins, of course), or any other precise *and positive* variable. The graph below shows two probability density functions with different $\lambda$ values.

When a random variable $Z$ has an exponential distribution with parameter $\lambda$, we say *$Z$ is exponential* and write

$$Z \sim \text{Exp}(\lambda)$$

Given a specific $\lambda$, the expected value of an exponential random variable is equal to the inverse of $\lambda$, that is:

$$E[\; Z \;|\; \lambda \;] = \frac{1}{\lambda}$$

```python
a = np.linspace(0, 4, 100)
expo = stats.expon
lambda_ = [0.5, 1]

for l, c in zip(lambda_, colours):
    plt.plot(a, expo.pdf(a, scale=1./l), lw=3,
             color=c, label="$\lambda = %.1f$" % l)
    plt.fill_between(a, expo.pdf(a, scale=1./l), color=c, alpha=.33)

plt.legend()
plt.ylabel("PDF at $z$")
plt.xlabel("$z$")
plt.ylim(0,1.2)
plt.title("Probability density function of an Exponential random variable;\
 differing $\lambda$");
```

### But what is $\lambda \;$?

**This question is what motivates statistics**. In the real world, $\lambda$ is hidden from us. We see only $Z$, and must go backwards to try and determine $\lambda$. The problem is difficult because there is no one-to-one mapping from $Z$ to $\lambda$. Many different methods have been created to solve the problem of estimating $\lambda$, but since $\lambda$ is never actually observed, no one can say for certain which method is best!

Bayesian inference is concerned with *beliefs* about what $\lambda$ might be. Rather than try to guess $\lambda$ exactly, we can only talk about what $\lambda$ is likely to be by assigning a probability distribution to $\lambda$.

This might seem odd at first. After all, $\lambda$ is fixed; it is not (necessarily) random! How can we assign probabilities to values of a non-random variable? Ah, we have fallen for our old, frequentist way of thinking. Recall that under Bayesian philosophy, we *can* assign probabilities if we interpret them as beliefs. And it is entirely acceptable to have *beliefs* about the parameter $\lambda$.

##### Example: Inferring behaviour from text-message data

Let's try to model a more interesting example, one that concerns the rate at which a user sends and receives text messages:

>  You are given a series of daily text-message counts from a user of your system. The data, plotted over time, appears in the chart below. You are curious to know if the user's text-messaging habits have changed over time, either gradually or suddenly. How can you model this? (This is in fact my own text-message data. Judge my popularity as you wish.)

```python
figsize(12.5, 3.5)
count_data = np.loadtxt("data/txtdata.csv")
n_count_data = len(count_data)
plt.bar(np.arange(n_count_data), count_data, color="#348ABD")
plt.xlabel("Time (days)")
plt.ylabel("count of text-msgs received")
plt.title("Did the user's texting habits change over time?")
plt.xlim(0, n_count_data);
```

Before we start modeling, see what you can figure out just by looking at the chart above. Would you say there was a change in behaviour during this time period?

How can we start to model this? Well, as we have conveniently already seen, a Poisson random variable is a very appropriate model for this type of *count* data. Denoting day $i$'s text-message count by $C_i$,

$$ C_i \sim \text{Poisson}(\lambda)  $$

We are not sure what the value of the $\lambda$ parameter really is, however. Looking at the chart above, it appears that the rate might become higher late in the observation period, which is equivalent to saying that $\lambda$ increases at some point during the observations. (Recall that a higher value of $\lambda$ assigns more probability to larger outcomes. That is, there is a higher probability of many text messages having been sent on a given day.)

How can we represent this observation mathematically? Let's assume that on some day during the observation period (call it $\tau$), the parameter $\lambda$ suddenly jumps to a higher value. So we really have two $\lambda$ parameters: one for the period before $\tau$, and one for the rest of the observation period. In the literature, a sudden transition like this would be called a *switchpoint*:

$$
\lambda =
\begin{cases}
\lambda_1  & \text{if } t \lt \tau \cr
\lambda_2 & \text{if } t \ge \tau
\end{cases}
$$

If, in reality, no sudden change occurred and indeed $\lambda_1 = \lambda_2$, then the $\lambda$s posterior distributions should look about equal.

We are interested in inferring the unknown $\lambda$s. To use Bayesian inference, we need to assign prior probabilities to the different possible values of $\lambda$. What would be good prior probability distributions for $\lambda_1$ and $\lambda_2$? Recall that $\lambda$ can be any positive number. As we saw earlier, the *exponential* distribution provides a continuous density function for positive numbers, so it might be a good choice for modeling $\lambda_i$. But recall that the exponential distribution takes a parameter of its own, so we'll need to include that parameter in our model. Let's call that parameter $\alpha$.

\begin{align}
&\lambda_1 \sim \text{Exp}( \alpha ) \\\
&\lambda_2 \sim \text{Exp}( \alpha )
\end{align}

$\alpha$ is called a *hyper-parameter* or *parent variable*. In literal terms, it is a parameter that influences other parameters. Our initial guess at $\alpha$ does not influence the model too strongly, so we have some flexibility in our choice.  A good rule of thumb is to set the exponential parameter equal to the inverse of the average of the count data. Since we're modeling $\lambda$ using an exponential distribution, we can use the expected value identity shown earlier to get:

$$\frac{1}{N}\sum_{i=0}^N \;C_i \approx E[\; \lambda \; |\; \alpha ] = \frac{1}{\alpha}$$

An alternative, and something I encourage the reader to try, would be to have two priors: one for each $\lambda_i$. Creating two exponential distributions with different $\alpha$ values reflects our prior belief that the rate changed at some point during the observations.

What about $\tau$? Because of the noisiness of the data, it's difficult to pick out a priori when $\tau$ might have occurred. Instead, we can assign a *uniform prior belief* to every possible day. This is equivalent to saying

\begin{align}
& \tau \sim \text{DiscreteUniform(1,70) }\\\\
& \Rightarrow P( \tau = k ) = \frac{1}{70}
\end{align}

So after all this, what does our overall prior distribution for the unknown variables look like? Frankly, *it doesn't matter*. What we should understand is that it's an ugly, complicated mess involving symbols only a mathematician could love. And things will only get uglier the more complicated our models become. Regardless, all we really care about is the posterior distribution.

We next turn to PyMC3, a Python library for performing Bayesian analysis that is undaunted by the mathematical monster we have created.

Introducing our first hammer: PyMC3
-----

PyMC3 is a Python library for programming Bayesian analysis [3]. It is a fast, well-maintained library. The only unfortunate part is that its documentation is lacking in certain areas, especially those that bridge the gap between beginner and hacker. One of this book's main goals is to solve that problem, and also to demonstrate why PyMC3 is so cool.

We will model the problem above using PyMC3. This type of programming is called *probabilistic programming*, an unfortunate misnomer that invokes ideas of randomly-generated code and has likely confused and frightened users away from this field. The code is not random; it is probabilistic in the sense that we create probability models using programming variables as the model's components. Model components are first-class primitives within the PyMC3 framework.

B. Cronin [5] has a very motivating description of probabilistic programming:

>   Another way of thinking about this: unlike a traditional program, which only runs in the forward directions, a probabilistic program is run in both the forward and backward direction. It runs forward to compute the consequences of the assumptions it contains about the world (i.e., the model space it represents), but it also runs backward from the data to constrain the possible explanations. In practice, many probabilistic programming systems will cleverly interleave these forward and backward operations to efficiently home in on the best explanations.

Because of the confusion engendered by the term *probabilistic programming*, I'll refrain from using it. Instead, I'll simply say *programming*, since that's what it really is.

PyMC3 code is easy to read. The only novel thing should be the syntax. Simply remember that we are representing the model's components ($\tau, \lambda_1, \lambda_2$ ) as variables.

```python
import pymc as pm

with pm.Model() as model:
    alpha = 1.0/count_data.mean()  # Recall count_data is the
                                   # variable that holds our txt counts
    lambda_1 = pm.Exponential("lambda_1", alpha)
    lambda_2 = pm.Exponential("lambda_2", alpha)

    tau = pm.DiscreteUniform("tau", lower=0, upper=n_count_data - 1)
```

In the code above, we create the PyMC3 variables corresponding to $\lambda_1$ and $\lambda_2$. We assign them to PyMC3's *stochastic variables*, so-called because they are treated by the back end as random number generators.

```python
with model:
    idx = np.arange(n_count_data) # Index
    lambda_ = pm.math.switch(tau > idx, lambda_1, lambda_2)
```

This code creates a new function `lambda_`, but really we can think of it as a random variable: the random variable $\lambda$ from above. The `switch()` function assigns `lambda_1` or `lambda_2` as the value of `lambda_`, depending on what side of `tau` we are on. The values of `lambda_` up until `tau` are `lambda_1` and the values afterwards are `lambda_2`.

Note that because `lambda_1`, `lambda_2` and `tau` are random, `lambda_` will be random. We are **not** fixing any variables yet.

```python
with model:
    observation = pm.Poisson("obs", lambda_, observed=count_data)
```

The variable `observation` combines our data, `count_data`, with our proposed data-generation scheme, given by the variable `lambda_`, through the `observed` keyword.

The code below will be explained in Chapter 3, but I show it here so you can see where our results come from. One can think of it as a *learning* step. The machinery being employed is called *Markov Chain Monte Carlo* (MCMC), which I also delay explaining until Chapter 3. This technique returns thousands of random variables from the posterior distributions of $\lambda_1, \lambda_2$ and $\tau$. We can plot a histogram of the random variables to see what the posterior distributions look like. Below, we collect the samples (called *traces* in the MCMC literature) into histograms.

```python
### Mysterious code to be explained in Chapter 3.
with model:
    step = pm.Metropolis()
    trace = pm.sample(10000, tune=5000, step=step, return_inferencedata=False)
```

    /opt/homebrew/Caskroom/miniforge/base/envs/myenv/lib/python3.8/site-packages/pymc/model.py:984: FutureWarning: `Model.initial_point` has been deprecated. Use `Model.recompute_initial_point(seed=None)`.
      warnings.warn(
    /opt/homebrew/Caskroom/miniforge/base/envs/myenv/lib/python3.8/site-packages/pymc/model.py:984: FutureWarning: `Model.initial_point` has been deprecated. Use `Model.recompute_initial_point(seed=None)`.
      warnings.warn(
    Multiprocess sampling (4 chains in 4 jobs)
    CompoundStep
    >Metropolis: [lambda_1]
    >Metropolis: [lambda_2]
    >Metropolis: [tau]
    /opt/homebrew/Caskroom/miniforge/base/envs/myenv/lib/python3.8/site-packages/pymc/model.py:984: FutureWarning: `Model.initial_point` has been deprecated. Use `Model.recompute_initial_point(seed=None)`.
      warnings.warn(

<div>
    <style>
        /* Turns off some styling */
        progress {
            /* gets rid of default border in Firefox and Opera. */
            border: none;
            /* Needs to be in here for Safari polyfill so background images work as expected. */
            background-size: auto;
        }
        .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {
            background: #F44336;
        }
    </style>
  <progress value='60000' class='' max='60000' style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [60000/60000 00:03<00:00 Sampling 4 chains, 0 divergences]
</div>

    Sampling 4 chains for 5_000 tune and 10_000 draw iterations (20_000 + 40_000 draws total) took 12 seconds.
    The number of effective samples is smaller than 25% for some parameters.

```python
lambda_1_samples = trace['lambda_1']
lambda_2_samples = trace['lambda_2']
tau_samples = trace['tau']
```

```python
figsize(12.5, 10)
#histogram of the samples:

ax = plt.subplot(311)
ax.set_autoscaley_on(False)

plt.hist(lambda_1_samples, histtype='stepfilled', bins=30, alpha=0.85,
         label="posterior of $\lambda_1$", color="#A60628", density=True)
plt.legend(loc="upper left")
plt.title(r"""Posterior distributions of the variables
    $\lambda_1,\;\lambda_2,\;\tau$""")
plt.xlim([15, 30])
plt.xlabel("$\lambda_1$ value")

ax = plt.subplot(312)
ax.set_autoscaley_on(False)
plt.hist(lambda_2_samples, histtype='stepfilled', bins=30, alpha=0.85,
         label="posterior of $\lambda_2$", color="#7A68A6", density=True)
plt.legend(loc="upper left")
plt.xlim([15, 30])
plt.xlabel("$\lambda_2$ value")

plt.subplot(313)
w = 1.0 / tau_samples.shape[0] * np.ones_like(tau_samples)
plt.hist(tau_samples, bins=n_count_data, alpha=1,
         label=r"posterior of $\tau$",
         color="#467821", weights=w, rwidth=2.)
plt.xticks(np.arange(n_count_data))

plt.legend(loc="upper left")
plt.ylim([0, .75])
plt.xlim([35, len(count_data)-20])
plt.xlabel(r"$\tau$ (in days)")
plt.ylabel("probability");
```

### Interpretation

Recall that Bayesian methodology returns a *distribution*. Hence we now have distributions to describe the unknown $\lambda$s and $\tau$. What have we gained? Immediately, we can see the uncertainty in our estimates: the wider the distribution, the less certain our posterior belief should be. We can also see what the plausible values for the parameters are: $\lambda_1$ is around 18 and $\lambda_2$ is around 23. The posterior distributions of the two $\lambda$s are clearly distinct, indicating that it is indeed likely that there was a change in the user's text-message behaviour.

What other observations can you make? If you look at the original data again, do these results seem reasonable?

Notice also that the posterior distributions for the $\lambda$s do not look like exponential distributions, even though our priors for these variables were exponential. In fact, the posterior distributions are not really of any form that we recognize from the original model. But that's OK! This is one of the benefits of taking a computational point of view. If we had instead done this analysis using mathematical approaches, we would have been stuck with an analytically intractable (and messy) distribution. Our use of a computational approach makes us indifferent to mathematical tractability.

Our analysis also returned a distribution for $\tau$. Its posterior distribution looks a little different from the other two because it is a discrete random variable, so it doesn't assign probabilities to intervals. We can see that near day 45, there was a 50% chance that the user's behaviour changed. Had no change occurred, or had the change been gradual over time, the posterior distribution of $\tau$ would have been more spread out, reflecting that many days were plausible candidates for $\tau$. By contrast, in the actual results we see that only three or four days make any sense as potential transition points.

### Why would I want samples from the posterior, anyways?

We will deal with this question for the remainder of the book, and it is an understatement to say that it will lead us to some amazing results. For now, let's end this chapter with one more example.

We'll use the posterior samples to answer the following question: what is the expected number of texts at day $t, \; 0 \le t \le 70$ ? Recall that the expected value of a Poisson variable is equal to its parameter $\lambda$. Therefore, the question is equivalent to *what is the expected value of $\lambda$ at time $t$*?

In the code below, let $i$ index samples from the posterior distributions. Given a day $t$, we average over all possible $\lambda_i$ for that day $t$, using $\lambda_i = \lambda_{1,i}$ if $t \lt \tau_i$ (that is, if the behaviour change has not yet occurred), else we use $\lambda_i = \lambda_{2,i}$.

```python
figsize(12.5, 5)
# tau_samples, lambda_1_samples, lambda_2_samples contain
# N samples from the corresponding posterior distribution
N = tau_samples.shape[0]
expected_texts_per_day = np.zeros(n_count_data)
for day in range(0, n_count_data):
    # ix is a bool index of all tau samples corresponding to
    # the switchpoint occurring prior to value of 'day'
    ix = day < tau_samples
    # Each posterior sample corresponds to a value for tau.
    # for each day, that value of tau indicates whether we're "before"
    # (in the lambda1 "regime") or
    #  "after" (in the lambda2 "regime") the switchpoint.
    # by taking the posterior sample of lambda1/2 accordingly, we can average
    # over all samples to get an expected value for lambda on that day.
    # As explained, the "message count" random variable is Poisson distributed,
    # and therefore lambda (the poisson parameter) is the expected value of
    # "message count".
    expected_texts_per_day[day] = (lambda_1_samples[ix].sum()
                                   + lambda_2_samples[~ix].sum()) / N

plt.plot(range(n_count_data), expected_texts_per_day, lw=4, color="#E24A33",
         label="expected number of text-messages received")
plt.xlim(0, n_count_data)
plt.xlabel("Day")
plt.ylabel("Expected # text-messages")
plt.title("Expected number of text-messages received")
plt.ylim(0, 60)
plt.bar(np.arange(len(count_data)), count_data, color="#348ABD", alpha=0.65,
        label="observed texts per day")

plt.legend(loc="upper left");
```

Our analysis shows strong support for believing the user's behavior did change ($\lambda_1$ would have been close in value to $\lambda_2$ had this not been true), and that the change was sudden rather than gradual (as demonstrated by $\tau$'s strongly peaked posterior distribution). We can speculate what might have caused this: a cheaper text-message rate, a recent weather-to-text subscription, or perhaps a new relationship. (In fact, the 45th day corresponds to Christmas, and I moved away to Toronto the next month, leaving a girlfriend behind.)

##### Exercises

1\.  Using `lambda_1_samples` and `lambda_2_samples`, what is the mean of the posterior distributions of $\lambda_1$ and $\lambda_2$?

```python
print(lambda_1_samples.mean())
print(lambda_2_samples.mean())
```

    17.757078422922426
    22.72724294278267

2\.  What is the expected percentage increase in text-message rates? `hint:` compute the mean of `lambda_1_samples/lambda_2_samples`. Note that this quantity is very different from `lambda_1_samples.mean()/lambda_2_samples.mean()`.

```python
lambda_ratios = np.array([lambda_1 / lambda_2 for lambda_1, lambda_2 in zip(lambda_1_samples, lambda_2_samples)])
lambda_ratios.mean()
```

    0.7825286272057218

3\. What is the mean of $\lambda_1$ **given** that we know $\tau$ is less than 45.  That is, suppose we have been given new information that the change in behaviour occurred prior to day 45. What is the expected value of $\lambda_1$ now? (You do not need to redo the PyMC3 part. Just consider all instances where `tau_samples < 45`.)

```python
lambda_1_cond = lambda_1_samples[np.where(tau_samples < 45)]
print(lambda_1_cond.mean())
```

    17.761947007992113

### References

-  [1] Gelman, Andrew. N.p.. Web. 22 Jan 2013. [N is never large enough](http://andrewgelman.com/2005/07/31/n_is_never_larg).
-  [2] Norvig, Peter. 2009. [The Unreasonable Effectiveness of Data](http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/35179.pdf).
- [3] Salvatier, J, Wiecki TV, and Fonnesbeck C. (2016) Probabilistic programming in Python using PyMC3. *PeerJ Computer Science* 2:e55 <https://doi.org/10.7717/peerj-cs.55>
- [4] Jimmy Lin and Alek Kolcz. Large-Scale Machine Learning at Twitter. Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data (SIGMOD 2012), pages 793-804, May 2012, Scottsdale, Arizona.
- [5] Cronin, Beau. "Why Probabilistic Programming Matters." 24 Mar 2013. Google, Online Posting to Google . Web. 24 Mar. 2013. <https://plus.google.com/u/0/107971134877020469960/posts/KpeRdJKR6Z1>.

```python
from IPython.core.display import HTML
def css_styling():
    styles = open("../styles/custom.css", "r").read()
    return HTML(styles)
css_styling()
```

<style>
    @font-face {
        font-family: "Computer Modern";
        src: url('http://9dbb143991406a7c655e-aa5fcb0a5a4ec34cff238a2d56ca4144.r56.cf5.rackcdn.com/cmunss.otf');
    }
    div.cell{
        width:800px;
        margin-left:16% !important;
        margin-right:auto;
    }
    h1 {
        font-family: Helvetica, serif;
    }
    h4{
        margin-top:12px;
        margin-bottom: 3px;
       }
    div.text_cell_render{
        font-family: Computer Modern, "Helvetica Neue", Arial, Helvetica, Geneva, sans-serif;
        line-height: 145%;
        font-size: 130%;
        width:800px;
        margin-left:auto;
        margin-right:auto;
    }
    .CodeMirror{
            font-family: "Source Code Pro", source-code-pro,Consolas, monospace;
    }
    .prompt{
        display: None;
    }
    .text_cell_render h5 {
        font-weight: 300;
        font-size: 22pt;
        color: #4057A1;
        font-style: italic;
        margin-bottom: .5em;
        margin-top: 0.5em;
        display: block;
    }

    .warning{
        color: rgb( 240, 20, 20 )
        }
</style>

```python

```


Between the text will be code such as the `import` statements below. Please note that the code is just there to support some graphs and numbers. You don't have to understand it to read the text so it can safely be ignored.

If there's something important happening in code then this will be made clear by the text surrounding it.

```python
import numpy as np
import matplotlib.pyplot as plt
import plotutils as pu
from sympy import S, solve, N
%matplotlib inline
```

# complex numbers (are not complex)
As humans needed to solve more and more complex equations we had to move away from *whole numbers* through *rational numbers* arriving at a class of numbers that are nowadays mostly called **complex numbers**.

These numbers are also sometimes known as **imaginary numbers** which is actually a bit of misnomer because these numbers are just as real as any other number and they are essential to a solving a huge number of problems in science. In fact, the complex moniker is kinda stupid as well. They are not complex at all. Just a bit unconventional.

Before we explain complex numbers in more detail it might be useful and take a step back and see why we need them in the first place. We'll sidetrack a bit with *square roots* and *square numbers* get some concepts in place before we get to the really funky stuff.

# the square root
One of the more nasty operations in math is taking the square root of a number. In fact, even before complex numbers people ran into problems with this operation due to the fact [how it's defined](https://en.wikipedia.org/wiki/Square_root):

> In mathematics, a square root of a number $a$ is a number $y$ such that $y^2 = a$, in other words, a number $y$ whose square (the result of multiplying the number by itself, or $y \times y$) is $a$.

As usual, the definition is correct but hard to understand if you're not used to the lingo and format of these things. As an aside, naming one of the variables $a$ doesn't increase the readability of the definition.

Another definition might be:

> Given the hypervolume of some object $P$ in $n$ dimensions and given a hypercube $Q$ in the same $n$ dimensions. What length does each side of the hypercube needs to be represent the volume of $P$.

And to be honest, I just made that up and I hope it's somewhat correct. But it seems to be true if we consider everythig a *hyperthing* where hyper just means its a thing that exists in one or more dimensions.

In math, the square root of a number $x$ is written as $\sqrt{x}$ and the easiest way to understand it is with some examples:

$$
\begin{align}
1 \times 1 = 1 & \implies \sqrt{1} = 1 \\
2 \times 2 = 4 & \implies \sqrt{4} = 2 \\
3 \times 3 = 9 & \implies \sqrt{9} = 3 \\
\cdots \\
x \times x = x^2 & \implies \sqrt{x^2} = x
\end{align}
$$

In the table above we can see that if we take some number $x$ and multiply it with itself we get $x \times x$ which in math is usually written as $x^2$ ($x$ squared). We can also see that if we proceed to take the square root of some number $x^2$ we get back the original number $x$.

In other words, it is the *inverse* of *squaring* a number (multiplying a number with itself). However just by this very definition we get into strange territory very quickly. As long as we are dealing with squares such as $x \times x$ the world is wonderful and we can be sure that any square root we want to find is an *integer* (whole number) just by the way how the operation is defined.

### squared?
Especially in the old times but even today when doing exploratory stuff it's not uncommon to look at things from a *geometric* perspective. Which basically means we try to to think about shapes. When we are dealing with what seemingly is only one dimension it's often useful to look at it from a two-dimensional point of view. Also we can try to understand things by *mapping* them to less dimensions.

# square numbers
So what is a *square number* anyway? If we look at it from a geometric perspective we can say that a square is in fact a rectangle where both sides are equal. Now this is a bit of a recursive definition because we might as well say a rectangle is a square scaled along some kind of axis but lets stick with our basic explanation for now. We know that we can compute the *area* of a rectangle by multiplying its width $x$ and height $y$. So $area = x \times y$.

Now if we consider a number as being some kind of measurement for an abstract *area* we can say that a square number is a number such that we can write $area = x \times y = x \times x = y \times y\implies x = y$ where $area$ is just some number. Which in geometric terms means that some kind of area can either be defined by $x \times y$ (the width times height) or by $x \times x$ (width times width) and that this *implies* that $x$ is equal to $y$. Or in other words, both sides are equal and we're dealing with a square.

This square might be anything, it's not important, after all, with this definition the number $4$ is a square. It's a square of $2 \times 2$ and what it actually represents even in this abstract form is too much to answer in this essay. It can be anything, the important fact is the property that it can be written as a *factor* of two numbers that are the same.

### factor?
A factor is basically when we take a thing and break it up in other using multiplication. So for example if we take $4$ we can factor this into $2 \times 2$ or $1 \times 4$ and we can say that $1$, $2$ and $4$ *are factors* of $4$. More generaly we can say that if a number $x$ can be represented with $n$ and $m$ so that $n \times m = x$ then this number $x$ has factors $n$ and $m$.

So how would we plot square roots? One way is just to plot a graph but we can do something a bit more *geometric*. We can plot square roots (and any root as it turns out) as a rectangle as well.

```python
def plot_rect(ax, p, fmt='b'):
    x, y = p
    ax.plot([0, x], [y, y], fmt) # horizontal line
    ax.plot([x, x], [0, y], fmt) # vertical line

with plt.xkcd():
    fig, axes = plt.subplots(1, 2, figsize=(8, 4))
    for ax in axes: pu.setup_axes(ax, xlim=(-1, 5), ylim=(-1, 5), tickdirection='out')
    for p in [(1, 1), (2, 2), (3, 3)]: plot_rect(axes[0], p)
    plot_rect(axes[0], (2+2/3, 2+2/3), 'r--')
    for p in [(1, 1), (2, 2)]: plot_rect(axes[1], p)
    plot_rect(axes[1], (3, 4), 'r--')
```

We can clearly see all the *easy* roots of $x = 1, 2, 3, \ldots$ but how do our other points compare? In the first case we are between $2$ and $3$ and our width equals our height so $x = y$ and this is still managable but in the second case, how do we find a cube that is the size of a $3 \times 4$ rectangle?

# taking square roots
So now that we have some concepts out of the we way we can look at the square root operation from a slightly different perspective and ask another question:

> Assuming that number $x$ represents an area, how big does my square needs to be to be so its area equals $x$.

This is actually not that easy to answer (without invoking roots) and as such taking square roots is a bit of a funky operation. One thing is that it is somewhat hard for to do for humans. Another thing is that there are actually some people who might argue some square root numbers don't even exist because there is no way to write them down without involving the square root itself. We're not even considering *cube roots* and/or higher here but we should be able to see that finding some kind of number that can be multiplied with itself to make up *any* kind number is somewhat tricky.

### perspective: not squares
After all this talk of squares we could also look at it from a non-square perspective. Let's say we measure a rectangle of $3 \times 4$. It's area would be $3 \times 4 = 12$. Now we walk over to someone and request that we get a square that represents this area. So we request $\sqrt{12}$. It would not be unreasonable to be unable to comply because there isn't much more reasonable ways to write $\sqrt{12}$ in a different way.

Now of course there are exceptions but for the average human finding the square root of some number is not something they can just do. There are various methods in order to get us close but that's the point, we can never ever really get to some actual number with these things. Take for example $\sqrt{2}$ which we'll look at in more detail later. We simply cannot simplify this anymore without losing precision. We cannot write it as a rational number like $\frac{2}{3}$ either. It's just a number that if we would try to write it down as a decimal we would just have to decide to stop somewhere because it has an infinite amount of decimals.

### calculating square roots manually
There's lots of ways to calculate square roots manually and I'm not even sure if the way I'm about to describe is in any way an "official" or even good way but it does make the operation easy to understand if we keep considering a number as some kind of area.

Let's get back to our square of $3 \times 4$ and see if we can figure this out. There's a huge plot incoming but don't worry about it. It's not important to the problem we are trying to figure out, just focus on the graphs below.

```python
with plt.xkcd():
    fig, axes = plt.subplots(1, 2, figsize=(10, 5))
    for ax in axes: pu.setup_axes(ax, xlim=(-1, 5), ylim=(-1, 5), tickdirection='out')
    plot_rect(axes[0], (3, 4), 'b')
    axes[0].plot([0, 3], [3, 3], 'b--')
    axes[0].yaxis.set_ticks([1, 2, 3, 4])
    axes[0].xaxis.set_ticks([1, 2, 3])
    axes[0].yaxis.set_ticklabels(['', '', '3', '4'])
    axes[0].xaxis.set_ticklabels(['', '', 3])
    axes[0].annotate('THE NUMBER 12', (0.5, 4.5))
    axes[0].annotate('A', (0.5, 3.35))
    plot_rect(axes[1], (3, 3), 'b')
    plot_rect(axes[1], (3.461, 3.461), 'b--')
    axes[1].plot([3, 3], [3, 4.2], 'b--')
    axes[1].plot([3, 4.2], [3, 3], 'b--')
    axes[1].yaxis.set_ticks([1, 2, 3, 3.461, 4])
    axes[1].xaxis.set_ticks([1, 2, 3, 3.461, 4])
    axes[1].xaxis.set_ticklabels(['', '', '3', 'x0', '4'])
    axes[1].yaxis.set_ticklabels(['', '', '3', 'x0', '4'])
    axes[1].annotate('THE SQUARE ROOT OF 12', (0.5, 4.5))
    axes[1].annotate('u', (1.5, 3.75))
    axes[1].annotate('w', (3.75, 3.75))
    axes[1].annotate('v', (3.75, 1.5))
    axes[1].annotate('A = u + v + w', (0.5, 1.5))
```

Remember, when we ask for a square root we are basically looking for a square that has an area equal to the number we give. Except this time we started with a non-square number. The area of our triangle is $3 \times 4 = 12$. This is the number that is represented on the left, on the right is a square representing the answer we are looking for: $\sqrt{12}$.

Let's start on the left side and take a look at the area $A$. We can see that:

$A = 3 \times (4 - 3) = 3 \times 1 = 3 \implies A = 3$.

From a geometric perspective we can see that if we want to calculate the biggest possible square we need to look at the shortest side of the rectangle. We can also imagine this holds up for a cube in three dimensions and our guts may tell us it will hold up for any shape in $n$ dimensions. We'll bother with this in a moment though.

For now let's focus on a bit on the right-hand plot. We have this number $x_0 - 3$ which is interesting. Logically we can deduce that in order to produce a square we need to add $(x_0 - 3) \times 3$ to each side. So that's two times that quantity which is $2 \times (x_0 - 3) \times 3$. And then we are left with this little square in the corner and this is just $(x_0 - 3) \times (x_0 - 3) = (x_0 - 3)^2$.

Writing it all out we have:

$$
\begin{align}
A & = (x_0 - 3)^2 + (x_0 - 3) \times 3 + (x_0 - 3) \times 3 \\
& = (x_0 - 3)^2 + 2 \times 3 \times (x_0 - 3) \\
& = (x_0 - 3)^2 + 6 \times (x_0 - 3) \\
& = 3 \\
& = A
\end{align}
$$

So we defined $A$ in some other *terms* and then finally we got back $A$. What have we acomplished? Well first we have shown that $A = 3$ which is a known fact in *this case*. We are trying to find some formula to describe this. So then we composed $A$ in *ratios* of areas we can call $u$, $v$ and $w$.

So now we end up with:

$$
\begin{align}
u & = 3 \times (x_0 - 3) \\
v & = 3 \times (x_0 - 3) \\
w & = (x_0 - 3) \times (x_0 - 3)
\end{align}
$$

What we're really interested in is this number $x_0 - 3$ whatever it might end up to be. And things are getting a bit unwieldly so let's just say $x = (x_0 - 3)$ and we might be able to rewrite this in a more friendly form.

$A = w + u + v = x^2 + 3x + 3x = 3 = x^2 + 6x$

It might not seem much but we're getting somewhere because we can now say that:

$A = x^2 + 6x = 3$.

Let's call in `sympy` in order to try to solve this and see what we get.

```python
A = S('x^2 + 6*x - 3')
solve(A)
```

    [-3 + 2*sqrt(3), -2*sqrt(3) - 3]

Oh great we got two possible answers of $2\sqrt{3} - 3$ and $-2\sqrt{3} - 3$ which doesn't really help us much calculating the square root manually. However the numbers do give a little bit more insight in what is happening. We can notice how prominent the $3$ is in there.

By now we can see that caculating square roots is not something trivial. In fact our best methods just depend on *guestimation*. For example we could just take a look at $x^2 + 6x - 3 = 0$ and by looking at the plot above and the range of the outcome we know it has to be between zero and one. So one way to start is just to put in some values between $0$ and $1$ and see what happens.

Let $f(x) = x^2 + 6x - 3$.

```python
with plt.xkcd():
    fig, axes = plt.subplots(1, 2, figsize=(8, 4))
    for ax in axes: pu.setup_axes(ax)
    f = lambda x: x**2 + (6 * x) - 3
    x1 = np.linspace(0, 1, 100)
    x2 = np.linspace(-50, 50, 1000)
    axes[0].plot(x1, f(x1))
    axes[1].plot(x2, f(x2))
    axes[1].set_xlim(-10, 5)
    axes[1].set_ylim(-15, 2)
```

```python

```

### calculating the biggest possible square
We easily see that the biggest square we can make is $3 \times 3$ but is there a way to calculate this instead? If we have a number 12. What *is* the biggest actual square we can make where the side length is an integer number. Well we *know* that $\sqrt{9} = 3$ so that's the square we are looking for. The next integer square is $4 \times 4$ and that's too big. The number we are looking for is $3$ but how can we figure this out.

Our rectangle is $3 \times 4$ and the square we are looking for is $3 \times 3$ or $3^2$. It seems that given a rectangle, the number we are looking for is the short side. But if we only know an area things are a bit harder, suppose we are only given the number $12$ and we know it's the area of a rectangle but not it's dimensions. The next question we could ponder is, does it even matter for the final outcome?

# sandbox
Everything below is just for playing around and should be deleted or cleaned-up for final publishing.

### sanity check on calculating $\sqrt{12}$

Even though this is jjust a sanity check this is a pretty awesome curve if we consider what it is doing geometrically. How do the negative values even make sense?

```python
# this is the solution we found involing sqrt(12)
f = lambda x: x**2 + 6*x - 3
x = np.linspace(-10, 10, 100)
# let's plot it for sanity
plt.plot(x, f(x))
```

```python
x = 0.46410 # this is our manually calculated value of dx/spread
print(round((x**2) + (x*6) - 3, 2))
print(3 + x)
print(round(np.sqrt(12), 4))
```

    -0.0
    3.4641
    3.4641


# Métodos Numéricos Aplicados à Transferência de Calor

## Introdução

### Sobre o material

* O objetivo desta palestra é **introduzir os principais conceitos empregados em programação e Python**, mais especificamente, no contexto interativo da plataforma Jupyter Notebook;
* Além de demonstrar como **solucionar problemas em transferência de calor** por meio de propostas computacionais;
* Para tanto, o material inclui uma breve **revisão de conceitos fundamentais** e as principais bibliotecas científicas disponíveis. Para maiores detalhes, **pode-se consultar a documentação disponível** ou mesmo as diversas leituras recomendadas que aparecem no decorrer do texto.

### Porque Python?

> Leitura recomendada:
> * [10 motivos para você aprender Python](https://www.hostgator.com.br/blog/10-motivos-para-voce-aprender-python/)

### Porque Jupyter Notebooks?

* Ferramenta web interativa, grátis e de código aberto;
* Exploração de dados. Permite executar o código, ver o que acontece, modificar e repetir, onde temos uma *"conversa"* com os dados disponíveis;
* Útil para a criação de tutoriais interativos;
* Ele fala a nossa língua. Disponível para várias liguagens de programação, como Python, Julia, R, Fortran e muitas outras;
* É possível combinar o código com células `Markdown`, para renderizar equações e tabelas, inserir figuras e explicações sobre o código;
* Facilmente extensível para diversos formatos (PDF, HTML, $\LaTeX$, slides e outros);
* Disponível em [jupyter.org](https://jupyter.org), além de:
    - Acompanhar a instalação do [Anaconda](https://www.anaconda.com/);
    - Ferramenta colaborativa na nuvem com [Google colab](https://colab.research.google.com) ou [binder](https://mybinder.org/).

> Leitura recomendada:
> - [Mastering Markdown](https://guides.github.com/features/mastering-markdown/)
> - [LaTeX/Mathematics](https://en.wikibooks.org/wiki/LaTeX/Mathematics)
> - [Why Jupyter is data scientists’ computational notebook of choice](https://www.nature.com/articles/d41586-018-07196-1)
> - [Why I write with LaTeX (and why you should too)](https://medium.com/@marko_kovic/why-i-write-with-latex-and-why-you-should-too-ba6a764fadf9)
> - [New Developer? You should’ve learned Git yesterday](https://codeburst.io/number-one-piece-of-advice-for-new-developers-ddd08abc8bfa)
> - [12 passos para Navier-Stokes](https://www.fschuch.com/blog/2020/01/12/cfd-com-python-12-passos-para-navier-stokes/)
> - [Jupyter Notebook como uma Poderosa Ferramenta Educacional](https://www.fschuch.com/blog/2021/01/22/jupyter-notebook-como-uma-poderosa-ferramenta-educacional/#formas-de-acessarcompartilhar)

## Programação em Python

As primeiras linhas de código interativas dessa aula (`Shift+Enter` executam o bloco):

```python
"""
Isso é um comentário
"""

print("Olá mundo")

# Isso também é um comentário
```

### Atribuição de variáveis:

```python
i = 5  # inteiro
f = 6.7  # ponto flutuante
g = 1e-2  # notação exponencial
s = "abcdef"  # string
c = 5.0 + 6j  # complexo
```

### Operações matemáticas

Operador | Descrição | Exemplo | Resultado
---------|-----------|---------|----------
`+`   | Soma   | `1 + 1` | `2`
`-`   | Subtração | `2 - 1` | `1`
`*`   | Multiplicação | `6 * 7` | `42`
`/`   | Divisão | `8 / 4` | 2.0
`//`  | Divisão inteira | `10 // 3` | 3
`%`   | Resto da divisão | `10 % 3` | 1
`**`  | Potência | `2 ** 3` | 8

Teste qualquer uma das operações no bloco abaixo:

```python
10 % 7.5
```

```python
a = 10.5
b = 5

c = a * b
c
```

### Operações em laços

Computadores são ótimos para a realização de tarefas repetitivas. Para isso, temos à nossa disposição laços (ou *loops*), que geralmente percorrem um espaço definido pelo seu `valor inicial`, `valor final`, e o tamanho do `incremento`. Veja o exemplo:

```python
inicio = 0  # opcional, será zero se não informado
final = 5
incremento = 1  # opcional, será um se não informado

for i in range(inicio, final, incremento):
    print(i)
    """
    Aqui realizaríamos operações da nossa aplicação
    """
```

**Nota**: Não precisamos indicar o final do laço em Python, porque isso é reconhecido por meio da identação.

**Outra Nota:** sempre que precisar de ajuda para compreender qualquer objeto no Jupyter, digite seu nome seguido de uma interrogação `?`, ou use a função `help()`, veja só:

```python
range?
```

Observe que, em Python:
* **A contagem começa em zero**;
* **O argumento inicial é inclusivo** (ele estará no espaço a percorrer);
* Enquanto **o argumento final é exclusivo** (ele não estará no espaço a percorrer).

Compreenda melhor esses conceitos com exemplos:

```python
for i in range(10):
    print(i, end=" ")
```

```python
for i in range(0, 10, 1):
    print(i, end=" ")
```

```python
for i in range(15, 30, 5):
    print(i, end=" ")
```

```python
for i in range(0, 10, 3):
    print(i, end=" ")
```

```python
for i in range(0, -10, -1):
    print(i, end=" ")
```

```python
for i in range(0):
    print(i, end=" ")
```

**Nota**: Perceba que `range` é apenas uma das diferentes possibilidades que temos para contruir um laço em Python.

### Testes lógicos

Operador | Descrição | Exemplo | Resultado
---------|-----------|---------|----------
`==`     | Igualdade | `1 == 2` | `False`
`!=`     | Diferença | `1 != 2` | `True`
`>`      | Maior que | `1 > 3` | `False`
`<`      | Menor que | `1 < 3` | `True`
`>=`     | Maior ou igual que | `1 >= 3` | `False`
`=<`     | Menor ou igual que | `1 <= 3` | `True`
`and`    | Operador lógico "e" | `True and False` | `False`
`or`     | Operador lógico "ou" | `True or False` | `True`
`not`    | Operador lógico "não" | `not False` | `True`

```python
if 5 <= 3.0:
    """
    Aqui realizaríamos operações da nossa aplicação
    """
    print("Estou no bloco if")
elif 4 != 0:
    """
    Aqui realizaríamos operações da nossa aplicação
    """
    print("Estou no bloco elif")
else:
    """
    Aqui realizaríamos operações da nossa aplicação
    """
    print("estou no blobo else")
```

### Funções

Funções são uma forma de encapsular trechos de código que você porventura queira executar diversas vezes. Argumentos são parâmetros opcionais de entrada, que podem alterar ou controlar o comportamento no interior da função. E elas podem ou não retornar algum valor.

No bloco a seguir, definimos um exemplo didático. Uma função que testa se um dado número de entrada é ímpar, retornando `True`, ou não, retornando `False`. Veja o exemplo:

```python
def testa_se_impar(numero):
    return bool(numero % 2)
```

Agora invocamos e testamos a nossa função:

```python
testa_se_impar(4)
```

```python
testa_se_impar(5)
```

Podemos incrementar a apresentação de nossa função com recursos extras. Por exemplo, atribuir valores padrões aos argumentos, caso eles não sejam informados ao invocar a função. Além disso temos o *type hint*, ou uma dica do tipo, onde podemos anotar na função a tipagem dos argumentos de entrada e saída, para auxiliar quem estiver utilizando nossa função. Finalmente, o comentário inicial é conhecido como *Docstring*, o lugar ideal para uma documentação rápida, que também estará disponível para nossos usuários:

```python
def testa_se_impar_v2(numero: int = 0) -> bool:
    """
    Dado um número inteiro como argumento de entrada,
    retorna True se ele é ímpar e False se ele é par
    """
    return bool(numero % 2)
```

Quando invocada sem argumentos, número será o valor definido como padrão, zero nesse caso, então a função é executada sem erros:

```python
testa_se_impar_v2()
```

O nome dos argumentos podem estar presentes na chamada, oferecendo legibilidade extra ao seu código:

```python
testa_se_impar_v2(numero=67)
```

Note que o *Docstring* é exibido na tela quando solicitamos ajuda:

```python
testa_se_impar_v2?
```

Material complementar:

* [More Control Flow Tools](https://docs.python.org/3/tutorial/controlflow.html)
* [The Python Tutorial - Modules](https://docs.python.org/3/tutorial/modules.html)
* [Data Structures](https://docs.python.org/3/tutorial/datastructures.html)
* [Classes](https://docs.python.org/2/tutorial/classes.html)

### Principais Pacotes

Uma das grandes forças do Python é a enorme gama de pacotes que estão disponíveis, e em contínuo desenvolvimento, nas mais diversas áreas do conhecimento.

A seguir, veremos algumas que são particularmente úteis para aplicações em transferência de calor.

#### SciPy

Ferramentas de computação científica para Python. SciPy refere-se a várias entidades relacionadas, mas distintas:

* O ecossistema SciPy, uma coleção de software de código aberto para computação científica em Python;
* A comunidade de pessoas que usam e desenvolvem essa biblioteca;
* Várias conferências dedicadas à computação científica em Python - SciPy, EuroSciPy e SciPy.in;
* Fazem parte da família os pacotes, que serão melhor descritos a seguir:
    * Numpy;
    * Matplotlib;
    * Sympy;
    * IPython;
    * Pandas.

* Além disso, a própria biblioteca SciPy, um componente do conjunto SciPy, fornecendo muitas rotinas numéricas:
    * Funções especiais;
    * Integração numérica;
    * Diferenciação numérica;
    * Otimização;
    * Interpolação;
    * Transformada de Fourier;
    * Processamento de sinal;
    * Algebra linear e Algebra linear esparsa;
    * Problema de autovalor esparso com ARPACK;
    * Algoritmos e estruturas de dados espaciais;
    * Estatistica;
    * Processamento de imagem multidimensional;
    * I/O de arquivos;

```python
import scipy as sp
import scipy.optimize
import scipy.integrate
```

Material complementar:
* [SciPy](https://www.scipy.org/)
* [Getting Started](https://www.scipy.org/getting-started.html)
* [Scipy Lecture Notes](http://scipy-lectures.org/index.html)

#### Numpy

Numpy é um pacote fundamental para a **computação científica em Python**. Entre outras coisas, destaca-se:
* Objetos em arranjos N-dimensionais
* Funções sofisticadas
* Ferramentas para integrar código C/C++ e Fortran
* Conveniente álgebra linear, transformada de Fourier e capacidade de números aleatórios

Além de seus usos científicos óbvios, o NumPy também pode ser usado como um contêiner multidimensional eficiente de dados genéricos. Tipos de dados arbitrários podem ser definidos. Isso permite que o NumPy integre-se de forma fácil e rápida a uma ampla variedade de bancos de dados.

```python
import numpy as np  # Importando a biblioteca numpy e definindo-a com o codnome de np
```

```python
matriz = np.arange(15).reshape(3, 5)

# exibe na tela
matriz
```

```python
matriz.shape
```

```python
matriz.ndim
```

```python
matriz.dtype.name
```

```python
matriz.size
```

```python
type(matriz)
```

##### Construção e Seleção de Dados

* Criar matrizes completas com valores iniciais em zero ou um:

```python
np.zeros(shape=(3, 4), dtype=np.float64)
```

```python
np.ones(shape=(2, 3, 4), dtype=np.int16)
```

* Definição inicial de um intervalo, de maneira similar a função `range` do Python:

```python
np.arange(10, 30, 5)
```

```python
np.arange(0, 2, 0.3)
```

* Ou ainda um espaço linear:

```python
vetor = np.linspace(start=0.0, stop=2.0, num=9)
vetor
```

A seleção dos dados ocorre de maneira similar às listas, com o número inteiro representando a localização, começando a contagem em zero. Veja os exemplos:

```python
vetor[0]
```

```python
vetor[2]
```

```python
vetor[0:4:2]
```

```python
vetor[-1]
```

No caso em que temos mais dimensões, como na matriz que definimos anteriormente, a mesma ideia se aplica, e separamos cada dimensão por vírgulas:

```python
matriz
```

```python
matriz[0, 0], matriz[0, 1], matriz[1, 0]
```

```python
matriz[0, :]
```

```python
matriz[:, -1]
```

**Cuidado**, pois o sinal de igualdade não cria novas cópias dos tensores, e isso pode confundir os iniciantes:

```python
outro_vetor = vetor
outro_vetor
```

```python
outro_vetor is vetor
```

```python
outro_vetor *= 0

print(vetor)
```

Temos agora duas maneiras de acessar o mesmo vetor na memória, pois tanto `vetor` quanto `outro_vetor` apontam para a mesma posição na memória.

##### Operações Tensoriais

Operações aritméticas e lógicas estão disponíveis para os objetos Numpy, e são propagados para todos os elementos do tensor. Veja os exemplos:

```python
a = np.array([20, 30, 40, 50])
b = np.array([0, 1, 2, 3])
```

```python
a - b
```

```python
b ** 2
```

```python
10 * np.sin(a)
```

```python
a < 35
```

> Leitura recomendada:
> * [NumPy Documentation](https://numpy.org/doc/)
> * [NumPy quickstart](https://numpy.org/doc/1.20/user/quickstart.html)
> * [NumPy: the absolute basics for beginners](https://numpy.org/doc/1.20/user/absolute_beginners.html)
> * [Tutorial: Linear algebra on n-dimensional arrays](https://numpy.org/doc/1.20/user/tutorial-svd.html)
>
> Outros pacotes Python para manipulação de dados:
> * [Pandas](https://pandas.pydata.org/) é uma pacote Python especializado na processamento eficiente de dados tabelados, podendo lidar com arquivos CSV, Excel, SQL, arranjos Numpy e outros;
> * [Xarray](http://xarray.pydata.org/) introduz rótulos na forma de dimensões, coordenadas e atributos sobre os dados brutos dos arranjos em formato NumPy, permitindo uma experiência de desenvolvimento mais intuitiva, consistente e a prova de falhas;
> * [Dask](https://dask.org/) fornece paralelismo avançado para análises, permitindo desempenho em escala para as ferramentas que você adora.

#### **Pandas**

O pandas é um pacote Python que fornece **estruturas de dados rápidas, flexíveis e expressivas**, projetadas para tornar o trabalho com dados “relacionais” ou “rotulados” fáceis e intuitivos. O objetivo é ser o alicerce fundamental de alto nível para a análise prática de dados do mundo real em Python. Além disso, tem o objetivo mais amplo de se tornar a mais poderosa e flexível ferramenta de análise / manipulação de dados de código aberto disponível em qualquer linguagem.

Pandas é bem adequado para muitos tipos diferentes de dados:
* Dados tabulares com colunas de tipos heterogêneos, como em uma **tabela SQL, arquivo `.csv` ou planilha do Excel**;
* Dados de **séries temporais** ordenados e não ordenados (não necessariamente de frequência fixa);
* Dados de matriz arbitrária (homogeneamente digitados ou heterogêneos) com rótulos de linha e coluna;
* Qualquer outra forma de conjuntos de dados observacionais / estatísticos. Os dados realmente não precisam ser rotulados para serem colocados em uma estrutura de dados de pandas.

```python
import pandas as pd
```

```python
df2 = pd.DataFrame({'A': 1.,
                    'B': pd.Timestamp('20130102'),
                    'C': pd.Series(1, index=list(range(4)), dtype='float32'),
                    'D': np.array([3] * 4, dtype='int32'),
                    'E': pd.Categorical(["test", "train", "test", "train"]),
                    'F': 'foo'})
```

```python
df2
```

> Material complementar:
> * [Pandas](https://pandas.pydata.org/)
> * [10 minutes to pandas](https://pandas.pydata.org/pandas-docs/version/0.25.0/getting_started/10min.html)

#### Tqdm

Produz uma barra de progresso. Recurso puramente estético, mas ainda assim, muito útil:

```python
from tqdm.notebook import tqdm
```

```python
for i in tqdm(range(100)):
    ...
```

#### Sympy

SymPy é uma biblioteca Python para **matemática simbólica**. O objetivo é tornar-se um sistema de álgebra computacional (CAS) completo, mantendo o código o mais simples possível para ser compreensível e facilmente extensível. SymPy é escrito inteiramente em Python.

```python
import sympy as sm

sm.init_printing(use_latex="mathjax")  # Para escrever equações na tela
```

```python
x, t = sm.symbols("x t")  # Criando símbolos
```

\begin{equation}
\text{calcular } \int (e^x \sin(x) + e^x \cos(x)) dx
\end{equation}

```python
sm.integrate(sm.exp(x) * sm.sin(x) + sm.exp(x) * sm.cos(x), x)
```

\begin{equation}
\text{calcular a derivada de }\sin(x)e^x
\end{equation}

```python
sm.diff(sm.sin(x) * sm.exp(x), x)
```

\begin{equation}
\text{calcular } \int_{-\infty}^{\infty} \sin(x^2)
\end{equation}

```python
sm.integrate(sm.sin(x ** 2), (x, -sm.oo, sm.oo))
```

\begin{equation}
\text{calcular } \lim_{x \to 0} \dfrac{\sin(x)}{x}
\end{equation}

```python
sm.limit(sm.sin(x) / x, x, 0)
```

\begin{equation}
\text{resolver } x^2 - 2 = 0
\end{equation}

```python
sm.solve(x ** 2 - 2, x)
```

\begin{equation}
\text{resolver a equação diferencial } y'' - y = e^t
\end{equation}

```python
y = sm.Function("y")
eq1 = sm.dsolve(sm.Eq(y(t).diff(t, t) - y(t), sm.exp(t)), y(t))
eq1
```

```python
# Bônus
print(sm.latex(eq1))
```

Material complementar:
* [Sympy](https://www.sympy.org/en/index.html)
* [Documentation](https://docs.sympy.org/latest/index.html)

#### Matplotlib

A Matplotlib é uma biblioteca de plotagem 2D do Python, que produz figuras de qualidade de publicação em uma variedade de formatos impressos e ambientes interativos entre plataformas. O Matplotlib pode ser usado em scripts Python, nos shells do Python e do IPython, no notebook Jupyter, nos servidores de aplicativos da web e em quatro kits de ferramentas de interface gráfica do usuário.

A **Matplotlib tenta tornar as coisas fáceis simples e as coisas difíceis possíveis**. Você pode gerar gráficos, histogramas, espectros de potência, gráficos de barras, gráficos de erros, diagramas de dispersão, etc., com apenas algumas linhas de código.

Como sempre, começamos importando a biblioteca:

```python
import matplotlib.pyplot as plt
```

Agora fazemos nossa primeira figura:

```python
x = np.linspace(start=0, stop=10, num=100)
plt.plot(x, np.sin(x));
```

O nome dos eixos são indispensáveis se você quiser mostrar sua figura para terceiros, um título pode ajudar também. Outro exemplo é como podemos definir os limites de cada eixo do gráfico. Veja nossa nova figura:

```python
x = np.linspace(0, 10, 100)
plt.plot(x, np.sin(x))

plt.xlim([0, 2 * np.pi])
plt.ylim([-2, 2])

plt.xlabel(r"eixo x $\sigma^2$")
plt.ylabel("eixo y")

plt.title("Minha figura");
```

> Leitura recomendada:
> * [Matplotlib](https://matplotlib.org/)
> * [Style sheets reference](https://matplotlib.org/stable/gallery/style_sheets/style_sheets_reference.html)
> * [Gallery](https://matplotlib.org/stable/gallery/index.html)
> * [Gráficos com qualidade de publicação em Python com Matplotlib](https://www.fschuch.com/blog/2020/10/14/graficos-com-qualidade-de-publicacao-em-python-com-matplotlib/)

#### Plotly

A biblioteca de gráficos Python do Plotly cria **gráficos interativos** com qualidade de publicação. As possibilidades de como fazer gráficos são inumeras: de linha, gráficos de dispersão, gráficos de área, gráficos de barras, barras de erro, gráficos de caixa, histogramas, mapas de calor, subplots, eixos múltiplos, gráficos polares e gráficos de bolhas.

```python
import plotly.express as px
import plotly.graph_objects as go
```

```python
px.defaults.template = "ggplot2"
px.defaults.height = 600
```

```python
df = px.data.iris()
fig = px.scatter(df, x="sepal_width", y="sepal_length", color="species")
fig.show()
```

```python
fig = go.Figure(data =
    go.Contour(
        z=[[10, 10.625, 12.5, 15.625, 20],
           [5.625, 6.25, 8.125, 11.25, 15.625],
           [2.5, 3.125, 5., 8.125, 12.5],
           [0.625, 1.25, 3.125, 6.25, 10.625],
           [0, 0.625, 2.5, 5.625, 10]],
        x=[-9, -6, -5 , -3, -1], # horizontal axis
        y=[0, 1, 4, 5, 7] # vertical axis
    ))
fig.show()
```

> Leitura recomendada:
> * [Plotly](https://plotly.com/python/)
> * [Plotly Express in Python](https://plotly.com/python/plotly-express/)
> * [Dash App Gallery](https://dash-gallery.plotly.host/Portal/)

#### Handcalcs

Handcalcs é uma biblioteca para renderizar o código de cálculo Python automaticamente em $\LaTeX$. Como o handcalcs mostra a substituição numérica, os cálculos se tornam significativamente mais fáceis de visualizar e verificar manualmente. A ferramenta é extremamente útil em vários contextos, mas pode-se destacar seu destaque na ramo do ensino, podendo ser empregada tanto por professores produzindo material didático, quanto por alunos preparando trabalhos e relatórios.

```python
import handcalcs.render
```

Nós vamos ver a biblioteca na prática logo mais, caracterizada pelos blocos de código que começam com o comando mágico `%%render`:

```python
%%render
a = 2  # Eu sou um exemplo
b = 3
c = 2 * a + b / 3  # Olhe esse resultado!
```

> Leitura complementar:
> * [Veja no GitHub](www.github.com/connorferster/handcalcs).

#### Pint

Pint é um pacote Python para definir, operar e manipular quantidades físicas: o produto de um valor numérico e uma unidade de medida. Ele permite operações aritméticas entre eles e conversões de e para diferentes unidades.

```python
import pint
```

```python
ureg = pint.UnitRegistry()
```

Veja o exemplo com a combinação de diferentes unidades de medida:

```python
distancia = 3 * ureg("meter") + 4 * ureg("centimeter")
distancia
```

Podemos agora converter essa distância facilmente para outras unidades:

```python
distancia.to("inch")
```

Vamos para um exemplo mais aplicado, com as propriedados do material (prata):

```python
%%render
k = ( 429 * ureg("W/(m*K)") ) # Condutividade térmica
rho = ( 10.5e3 * ureg("kg/m**3") )  # Massa específica
c_p = ( 235 * ureg("J/(kg*K)") ) # Calor específico
```

Agora calculamos a difusividade térmica (perceba a combinação com o Handcalcs):

```python
%%render
alpha = k / (rho * c_p)  # Difusividade térmica
```

Nem sempre a simplificação de unidades é automática, mas podemos acionar manualmente:

```python
alpha.to_base_units()
```

Note que o uso de unidades também é compatível com os arranjos numéricos do NumPy:

```python
np.linspace(0, 10, num=11) * ureg("hour")
```

> Leitura recomendada:
> * [Pint: makes units easy](https://pint.readthedocs.io/en/stable/)

-----

> **Felipe N. Schuch**,<br>
> Pesquisador em Fluidodinâmica Computacional na PUCRS, com interesse em: Escoamentos turbulentos, transferência de calor e massa, e interação fluido-estrutura; Processamento e visualização de dados em Python; Jupyter Notebook como uma ferramenta de colaboração, pesquisa e ensino.<br>
> [felipeschuch@outlook.com](mailto:felipeschuch@outlook.com "Email") [@fschuch](https://twitter.com/fschuch "Twitter") [Aprenda.py](https://fschuch.github.io/aprenda.py "Blog") [@aprenda.py](https://www.instagram.com/aprenda.py/ "Instagram")<br>

-----