\section{Interpretations and Definitions of Probability}

Weirdly enough, every book I open has a different approach to introducing probability. I found \citeasnoun{wasserman2013all} the best resource so far in terms of being clear and applied at the same time. Wasserman's language and notation is congruent with probability theory founded in measure theory. \citeasnoun{lindgren2006lectures} is the best resource so far in introducing mathematical probability in the context of stochastic processes. \citeasnoun{brightsideofmathematics} has an accessible lecture series on measure theory and a very calming German accent.

\subsection{Blitzstein's Naive and Non-Naive Definitions of Probability}
Like most probability texts, \citeasnoun{blitzstein2019introduction} starts with what he calls the \textit{naive} definition of probability, which is to look at the fraction of the event space that corresponds to a particular event. He then introduces a \textit{non-naive} definition of probability, which is congruent with measure theoretic probability.

\subsubsection{Naive Probability, Uniform Probability}
The event space $S$ consists of a collection of equally likely outcomes. The actual outcome $s_{actual} \in S$. The probability of an outcome in a subset $A\subseteq S$, $P(s_{actual}\in A)$ corresponds to the fraction of events in $A$ out of $S$.

\begin{equation}
P_{naive}(A) = \frac{|A|}{|S|} = \frac{\mathrm{number\ of\ outcomes\ favorable\ to\ A}}{\mathrm{total\ number\ of\ outcomes\ in\ S}}
\end{equation}

I've seen this called combinatorial probability or counting probability, presumably because this is how you calculate the probability in situations where events correspond to a finite number of equally likely combinations. I call it \textit{job interview probability}. More helpfully, \citeasnoun{wasserman2013all} calls it the \textit{uniform probability distribution}.

The method of counting the possible outcomes $s_{actual}$ within $S$ and $A$ is not appropriate when the outcomes are not equally likely (because counting them weighs them all equally). Also, the definition is, by itself, unclear on how to deal with infinite probability spaces (for example: $s_outcome \in \mathbb{R}$).

\subsubsection{Blitzstein's Non-Naive Definition of Probability}
This definition is from \cite{blitzstein2019introduction} and is congruent with measure theoretic probability. A probability space consists of a sample space $S$ in addition to a \textit{probability function}. The job of the probability function is to take an event $A\subseteq S$ and map it to a number between $0$ and $1$. I.e.  $P: S \rightarrow [0,1]$.

The function $P$ must satisfy:

\begin{itemize}
\item $P(\emptyset) = 0, P(S) = 1$
\item If $A_1, A_2,...$ are \textit{disjoint} events, then: \begin{equation}P\left(\bigcup^{\infty}_{j=1}A_j \right) = \sum_{j=1}^{\infty}P(A_j)\end{equation}
\end{itemize}

In other words, all you need to work with probability is a set of possible outcomes $S$ and some function to apply to parts of that set.

\subsection{Frequentist Probability}
The \textit{frequentist} interpretation of probability is that it represents the frequency of an outcome of a particular experiment upon running the experiment many times. On one hand, this view on probability is arguably empirically grounded. On the other hand it relies on the impractical notion of identically repeating something a large number of times. \citeasnoun{wasserman2013all} describes \textit{frequentist inference} as inference with guaranteed frequency behavior.

\subsection{Bayesian Probability}
The \textit{Bayesian} view on probability is that it represents a degree of belief in a particular outcome. On one hand, this implies a degree of subjectivity. On the other hand, this notion of probability is much more broadly applicable, because it does not require repeatable experiments. It also permits the introduction of subjective biases through priors. \citeasnoun{wasserman2013all} describes \textit{Bayesian inference} as statistical methods that use data to update belief.

\paragraph{} I'm not sure whether the supposed "battle" between frequentists and Bayesians has ever been of any consequence for me. Also, the frequentist perspective strikes me as inherently contradictory, because supposed empirical proof can strictly speaking never be practically delivered.

In practice, it seems that frequentist approaches tend to be structured around parametrization of solutions in terms of summary statistics, while Bayesians have a tendency to approach problems in terms of full probability distributions. Social scientists seem to be leaning towards frequentist methods, while physicists and engineers seem to be more interested in Bayesian data analysis.

I suspect that an additional source of bias is that frequentist methods tend to be computationally lighter and that many standard accepted recipes exist for common problems like hypothesis testing. In contrast, Bayesian approaches are usually derived ad-hoc in the context of a particular problem. They tend to involve more explicit assumptions about the data generating process.

\subsection{Kolmogorov's Axioms}
\subsection{Cox' Theorems}

% Measure theoretic probability
\section{Measure Theoretic Probability}
The modern approach to probability is rooted in measure theory, and use the following notation and definitions. Kolmogorov's three probability axioms overlap with them. Cox' theorems, an alternative approach to formalizing probability that is rooted in propositional logic, imply them. Conveniently, the other famous child of measure theory happens to be Lebesgue Integration, which allows for rigorous treatment of infinite sample spaces, distribution functions, and so on, all based within the same framework.

\subsection{Sample Space, Outcomes, Events}

\begin{itemize}
\item $\Omega$ is the \textit{state space}, the set of all possible outcomes, and maybe some impossible ones too. It doesn't matter, as long as all the possible outcomes are included. Sometimes also called \textit{event space} or \textit{sample space}.
\item A point or element $\omega \in \Omega$ is an \textit{outcome}. Sometimes also called \textit{element}, \textit{sample outcome} or \textit{realization}.
\item A subset $A \subseteq \Omega$ is an \textit{event}.
\end{itemize}

$\Omega$, $A$ and $\omega$ are sets and elements of sets, and the correct operations on them are set operations like forming unions and intersections. This is different from random variables, which might be numbers that can be added, subtracted, multiplied etc.

% Probability Distributions
\subsection{Probability Distributions}
Probability is a \textit{measure} that assigns a real number between $0$ and $1$ to events within the state space. It turns out that it is not always possible to define a meaningful measure to arbitrary collections subsets of the state space $\Omega$, so that the probability measure is instead assigned to a $\sigma$-algebra $\mathscr{A}$, which is a collection of subsets of $\Omega$ that fulfills criteria that guarantee the existence of a measure. $\sigma$-algebras are also called $\sigma$-fields. The combination $(\Omega, \mathscr{A})$ is called a measurable space.

Given a measurable space $(\Omega, \mathscr{A})$, a \textit{probability distribution} or \textit{probability measure} $\mathbb{P}$ is a measure $\mathbb{P}:\mathscr{A} \rightarrow \mathbb{R}$. The following criteria follow from the definition of a measure:

\begin{itemize}
\item Null-Event: $\mathbb{P}(\emptyset) = 0$. The empty subset $\emptyset \in \mathscr{A}$ is also called the \textit{null event}.
\item Additivity: $\mathbb{P}(\bigcup_i A_i) = \sum_i \mathbb{P}(A_i)$ if $A_i \cap A_j = \emptyset$  when $i \neq j$. The probability of the union of a finite collection of disjoint subsets $A_i \in \mathscr{A}$ is additive.
\item $\sigma$-additivity: $\mathbb{P}(\bigcup^{\infty}_i A_i) = \sum^{\infty}_i \mathbb{P}(A_i)$ if $A_i \cap A_j = \emptyset$  when $i \neq j$. The probability of the union of a countable, possibly infinite collection of disjoint subsets $A_i \in \mathscr{A}$ is also additive. (This derives from a feature of $\sigma$-algebras, which have to be closed under taking the countable union of subsets.) This is Kolmogorov's third axiom.
\end{itemize}

In addition, a probability measure satisfies:

\begin{itemize}
\item Positivity: $\mathbb{P}(A) \geq 0$ for all subsets $A\in\mathscr{A}$. Though positivity is also sometimes included in the definition of a measure in general. Positivity is part of Kolmogorov's first probability axiom, which requires probability to be a positive, real number.
\item Unitarity, Unit Measure: $\mathbb{P}(\Omega) = 1$. This represents certainty that something within the sample space must happen. Additivity ensures that probability is never greater than $1$. This is Kolmogorov's second axiom.
\end{itemize}

The combination of a sample space, a $\sigma$-algebra and a probability measure $(\Omega,\mathscr{A},\mathbb{P})$ is a \textit{probability space}. The important case of the $\sigma$-algebra generated by the real numbers $\sigma(\mathbb{R})$ is called the Borel $\sigma$-algebra, $\mathscr{B}(\mathbb{R})$.

% Random Variables
\subsection{Random Variables}
Random variables are \textit{measurable functions} defined on a probability space that assigns a real number to each outcome. That is, given $(\Omega,\mathscr{A},\mathbb{P})$, a random variable is some function $x(\omega), \omega \in \Omega$, $X:\Omega \rightarrow \mathbb{R}$. The fact that $X(\omega)$ is a \textit{measurable function} (also called \textit{measurable map}) is that both the domain and the range are measurable spaces, and that for each element of the range, the preimage is contained contained within a $\sigma$-algebra, and hence measurable. This allows one to talk about the \textit{distribution} of a random variable by applying the probability measure to subsets of its preimage. For example, one was interested in the probability that $X(\omega)$ will fall on the interval $[0,a]$ for some $a\in \mathbb{R}$, one feeds the subset of outcomes $A_{0\leq X\leq a} = \{\omega : \omega=X^{-1}(x), x \in [0,a]\}$ to the probability measure:

\begin{equation}
\mathbb{P}(0\leq X \leq a) = \mathbb{P}(A_{0\leq X\leq a}) = \mathbb{P}(\{\omega : 0\leq X(\omega) \leq a\})
\end{equation}

\subsection{Cumulative Distribution Function (CDF)}
The cumulative distribution function (CDF) of a random variable $X(\omega), \omega \in \Omega$ is given by $F_X(x) = \mathbb{P}(X\leq x)$. It measures the subset $\{\omega : \omega\in\Omega, X(\omega)\leq x \}$. A CDF uniquely determines its underlying random variable. It satisfies the condition:

\begin{itemize}
\item $F_X$ is normalized: $0\leq F_X(x) \leq 1$, $\lim_{x\rightarrow -\infty} F_X(x) = 0$,  $\lim_{x\rightarrow \infty} F_X(x) = 1$
\item $F_X$ is non-decreasing. That is, $F(x_1) \leq F(x_2)$ if $x_1 \leq x_2$.
\item Right-continuous: $F(x) = F(x^+)$
\end{itemize}

With respect to the probability of $X(\omega)$ assuming particular values, it follows:

\begin{itemize}
\item $\mathbb{P}(X = x) = F_X(x) - F_X(x^-)$ where $F_X(x^{-}) = \lim_{y\uparrow x}F_X(y)$
\item $F_X(b) - F_X(a) = \mathbb{P}(a < x \leq b)$
\item $\mathbb{P}(X > x) = 1-F_X(x)$
\end{itemize}

If $X$ is continuous, then:

\begin{itemize}
\item $F_X(b) - F_X(a) = \mathbb{P}(a<X<b) = \mathbb{P}(a\leq X\leq b) = \mathbb{P}(a\leq X<b) = \mathbb{P}(a<X \leq b)$
\end{itemize}

\subsection{Probability Mass Function (PMF)}
If a random variable $X$ takes on discrete values, then it has a probability mass function (PMF) $f_X(x) = \mathbb{P}(X = x)$. The probability mass function measures the preimage of $x$, which is the subset $\{\omega: \omega \in \Omega, \omega=X^{-1}(x) \}$. The relationship to the CDF is $F_X(x) = \sum_{x_i \leq x} f_X(x_i)$.

\subsection{Probability Density Function (PDF)}
If a random variable $X$ takes on continuous values, then it has a probability density function (PDF) $\mathbb{P}(a\leq X \leq b) = \int_a^b f_X(x) \mathrm{d}x$. It measures the size of the subset $\{\omega: \omega\in\Omega, a\leq X(\omega) \leq b\}$. The relationship to the CDF is $F_X(x) = \int_{-\infty^x f_X(x) \mathrm{d}x}$, or $\frac{\mathrm{d}}{\mathrm{d}x}F(x) = f_X(x)$.

\subsection{Quantile Function (Inverse CDF)}
The quantile function, or inverse CDF, is given by $F_X^{-1}(q) = \inf \{ x: F_X(x) > q \}$. It gives the value of $X(\omega)$ at which the CDF assumes the value $q$.

\begin{itemize}
\item First quartile: $q=1/4$
\item Second quartile / Median: $q =1/2$
\item Third quartile: $q=3/4$
\end{itemize}

\subsection{Joint, Conditional and Marginal Distributions}
Let $X(\omega)$ and $Y(\omega)$ be two random variables defined on some $(\Omega, \mathscr{A},\mathbb{P})$. The joint distribution mesaures the probability of some subset bounded by conditions on both $X$ and $Y$. For example, in the discrete case, the joint distribution $f_{XY}(x,y) = \mathbb{P}(X=x,Y=y)$ measures $\{\omega: X(\omega) = x, Y(\omega) = y\}$. When the value of one of the variables is considered fixed, then the joint distribution is the conditional distribution $f_{X|Y}(x) = \mathbb{P}(X=x|Y=a)$, which measures the subset $\{\omega: X(\omega) = x, Y(\omega) = a\}$. The marginal distribution is the distribution of a single random variable in a setting in which multiple random variables are defined on an event space. Also written $f_{X}$.


\documentclass[12pt]{article}
\usepackage[pdftex]{graphicx}
\newcommand{\kg}{\mathrm{kg}}
\newcommand{\m}{\mathrm{m}}
\newcommand{\mi}{\mathrm{mi}}
\newcommand{\s}{\mathrm{s}}
\newcommand{\h}{\mathrm{h}}
\newcommand{\mps}{\m\,\s^{-1}}
\newcommand{\J}{\mathrm{J}}
\newcommand{\eV}{\mathrm{eV}}
\newcounter{problem}
\begin{document}\thispagestyle{empty}

\section*{NYU General Physics 1---Problem set 7}

\paragraph{Problem~\theproblem:}\refstepcounter{problem}%
Gasoline and olive oil are both substances with great chemical energy
content per unit mass.

\textsl{(a)} In the case of gasoline, the chemical energy is mainly in
carbon bonds.  If you assume that gasoline is \emph{entirely} carbon
atoms, and each one releases $4\,\eV$ of energy when it is combusted, how
much energy per unit mass is there in gasoline?  Get an answer in MJ
per kg and compare to what you find on \textit{Wikipedia}.  How far off
are our assumptions?

\textsl{(b)} Now convert your answer to kcal per g and compare it to
what is written on the ``Nutrition Facts'' label on an olive oil
bottle.  How close are you?  It should be close, I think, because
biofuel is made from things like olive oil!

\textsl{(c)} Now assume that a car moving at speed
$v=75\,\mi\,\h^{-1}$ encounters an air resistance force of
$\rho\,A\,v^2$, where $\rho$ is the density of air and $A$ is the
cross-sectional area of the car, about $2\,\m^2$ How much work does it
take to move the car $30\,\mi$ at this speed?

\textsl{(d)} If a car with these properties was \emph{perfectly
  efficient}, how many miles per gallon would it get?  What does this
make you think about the future of energy-efficient cars?

\paragraph{Problem~\theproblem:}\refstepcounter{problem}%
A (magical) perfect ball of mass $m=0.7\,\kg$ is dropped from a height
$h=0.9\,\m$ onto a hard surface, off of which it bounces perfectly.
It then continues to bounce forever.  Make a plot, labeling carefully
the time and energy axes, of the kinetic energy of the ball as a function of
time, and of the gravitational potential energy of the ball as a
function of time.  Assume that the bounces (the times in contact with
the floor) are extremely short (negligible).  Now also plot the sum of
the kinetic energy and the potential energy (the total energy) as a
function of time.  For all three plots, label relevant times and
energies.  You have to make a choice about the ``zero'' of potential
energy, right?

For extra fun: Make the bounces last a short time $\Delta t$ and plot
also the ``elastic'' potential energy in the ball!

Sanity check: This problem is impossible because in real life, each
bounce will be shorter than the previous bounce.  Why?  Where does
that energy go?

\paragraph{Problem~\theproblem:}\refstepcounter{problem}%
A New York City Bus moving down Broadway at $15\,\mps$ hits a small
elastic rubber ball, which has happens to be very close to at rest
just before the collision.  The bus has a mass of $2\times 10^4\,\kg$
and the ball has a mass of $0.02\,\kg$.

\textsl{(a)} Draw a diagram showing the bus and the ball and their
velocities, immediately \emph{prior} to the collision.  Compute the
kinetic energies and the momenta of the bus and the ball.  For the
momenta, you will have to choose a direction for your coordinate
system.

\textsl{(b)} Draw the same diagram, but now from the point of view of
the driver of the truck next to the bus, who is driving at the exact
same speed.  That is, draw the diagram in the ``reference frame'' in
which the bus is at rest before the collision.  In this frame, the bus
is stationary and the ball is moving.  Again, compute the kinetic
energies and momenta.  Why are they different from what you got in
part \textsl{(a)}?  Aren't energy and momentum conserved?

\textsl{(c)} Staying in this new reference frame, imagine now
how the ball bounces off the bus.  The collision will be elastic, but
you don't really have to calculate anything: What happens when a tiny
ball bounces elastically off a huge bus?  Draw a diagram showing the
bus and the ball and their velocities, in the frame of the bus,
immediately \emph{after} the collision.  Compute the kinetic energies
and momenta.

\textsl{(d)} If you made a certain very useful approximation, then you
probably didn't conserve momentum in part \textsl{(c)}.  Why not?  In
detail, the velocity of the bus is affected by the collision.  By how
much does the velocity of the bus change, approximately, in the
collision?  If you \emph{did} conserve momentum in part \textsl{(c)},
then just report here the change in velocity of the bus.

\textsl{(e)} Now take what you had in part \textsl{(c)} and re-draw it
back in the original reference frame, which is that of the stores on
Broadway.  Compute the kinetic energies and momenta.  How fast is the
rubber ball moving after the collision?  Don't try the experiment.

\textsl{(f)} After the collision, the ball is moving fast, but its
mass is much smaller than that of the bus.  At the end of the problem,
what fraction of the total system momentum and kinetic energy are in
the ball?

\end{document}