\section{Simple Transformation}
\label{sec:transformations-slco:simple}
The transformation~\TSim takes a model and a channel name as input.
If the channel is used for synchronous communication, it is replaced by an asynchronous channel, and the classes of the objects that are connected by the channel are adapted.
It is defined as
%
\[
\begin{array}{l}
\TSim(\it{mn}~\it{class^*}~\it{obj^*}~\it{chan^*}, \it{chn}) = \\
\quad \it{mn} ~
T^S_{\it{CS}}(\it{class^*}, \it{cn_1}, \it{pn_1}, \it{cn_2}, \it{pn_2}) ~
\it{obj^*} \\
\quad \it{chan_1^*}~\it{chn}(\it{type^*})~\textbf{async lossless between}~\it{on_1}.~\it{pn_1}~\textbf{and}~\it{on_2}.~\it{pn_2}~\it{chan_2^*},
\end{array}
\]
if~$\it{chan^*} \equiv \it{chan_1^*}~\it{chn}(\it{type^*})~\textbf{sync}~\textbf{from}~\it{on_1}.~\it{pn_1}~\textbf{to}~\it{on_2}.~\it{pn_2}~\it{chan_2^*}$, $\it{obj^*} \equiv \it{obj_1^*}~\it{on_1}:\it{cn_1}~\it{obj_2^*}$,
and~$\it{obj^*} \equiv \it{obj_3^*}~\it{on_2}:\it{cn_2}~\it{obj_4^*}$,
and it is defined as
\[
\TSim(\it{mn}~\it{class^*}~\it{obj^*}~\it{chan^*}, \it{chn}) = \it{mn}~\it{class^*}~\it{obj^*}~\it{chan^*}
\]
otherwise.

The transformation~$T^S_{\it{CS}}$ defined below takes a sequence of classes, two class names, and two port names as input.
It returns a sequence of classes.
The classes with the names provided as input are modified, and all other classes remain intact.
There are two cases for this transformation.
In one case, the class with name $\it{cn_1}$ is first in the sequence of classes.
In the other case, the class with name $\it{cn_2}$ is first.

\[
\begin{array}{l}
T^S_{\it{CS}}(\it{class^*}, \it{cn_1}, \it{pn_1}, \it{cn_2}, \it{pn_2}) =\\
\quad \it{class_1^*}~T^S_{\it{C}}(\it{class_1}, \it{pn_1})~\it{class_2^*}~T^S_{\it{C}}(\it{class_2}, \it{pn_2})~\it{class_3^*},
\end{array}
\]
if~$\it{class^*} \equiv \it{class_1^*}~\it{class_1}~\it{class_2^*}~\it{class_2}~\it{class_3^*}$, $\it{class_1} \equiv \it{cn_1}~\it{pn_1^*}~\it{var_1^*}~\it{sm_1^*}$, and $\it{class_2} \equiv \it{cn_2}~\it{pn_2^*}~\it{var_2^*}~\it{sm_2^*}$, and
\[
\begin{array}{l}
T^S_{\it{CS}}(\it{class^*}, \it{cn_1}, \it{pn_1}, \it{cn_2}, \it{pn_2}) = \\
\quad \it{class_1^*}~T^S_{\it{C}}(\it{class_2}, \it{pn_2})~\it{class_2^*}~T^S_{\it{C}}(\it{class_1}, \it{pn_1})~\it{class_3^*},
\end{array}
\]
if~$\it{class^*} \equiv \it{class_1^*}~\it{class_2}~\it{class_2^*}~\it{class_1}~\it{class_3^*}$, $\it{class_1} \equiv \it{cn_1}~\it{pn_1^*}~\it{var_1^*}~\it{sm_1^*}$, and~$\it{class_2} \equiv \it{cn_2}~\it{pn_2^*}~\it{var_2^*}~\it{sm_2^*}$.

The transformation~$T^S_{\it{C}}$ defined below takes a class and a port name as input and returns a class with modified state machines.
Only the state machines that send or receive signals over the given port are modified.
%
\[
T^S_{\it{C}}(\it{cn}~\it{pn^*}~\it{var^*}~\it{sm^*}, \it{pn}) = \it{cn}~\it{pn^*}~\it{var^*}~T^S_{\it{SMS}}(\it{sm^*}, \it{pn})
\]

The transformation~$T^S_{\it{SMS}}$ defined below takes a sequence of state machines and a port name as input, and returns a modified sequence of state machines.
Only the state machines that send or receive signals over the given port are modified.
There are two cases for this transformation.
One case deals with empty sequences of state machines and one case with non-empty sequences of state machines.
%
\begin{align*}
T^S_{\it{SMS}}(\epsilon, \it{pn}) & = \epsilon \\
T^S_{\it{SMS}}(\it{sm}~\it{sm^*}, \it{pn}) & = T^S_{\it{SM}}(\it{sm}, \it{pn})~T^S_{\it{SMS}}(\it{sm^*}, \it{pn})
\end{align*}

The transformation~$T^S_{\it{SM}}$ defined below takes a state machine and a port name as input, and returns a state machine with modified transitions and a number of additional states.
Only the transitions that send or receive signals over the given port are modified.
There are two cases for this transformation.
One case deals with state machines without final states, and the other with state machines with final states.
%
\[
T^S_{\it{SM}}(\it{smn}~\it{var^*}~\textbf{initial}~\it{sn}~\it{sn^*}~\it{trans^*}, \it{pn}) =  \it{smn}~\it{var^*}~\textbf{initial}~\it{sn}~\it{sn^*}~\it{sn^*_1}~\it{trans^*_1},
\]
where~$\langle \it{sn^*_1}, \it{trans^*_1} \rangle = T^S_{\it{TS}}(\it{trans^*}, \it{pn})$, and
\[
\begin{array}{l}
T^S_{\it{SM}}(\it{smn}~\it{var^*}~\textbf{initial}~\it{sn}~\it{sn^*}~\textbf{final}~\it{sn^+}~\it{trans^*}, \it{pn}) = \\
\quad \it{smn}~\it{var^*}~\textbf{initial}~\it{sn}~\it{sn^*}~\it{sn^*_1}~\textbf{final}~\it{sn^+}~\it{trans^*_1},
\end{array}
\]
where~$\langle \it{sn^*_1}, \it{trans^*_1} \rangle = T^S_{\it{TS}}(\it{trans^*}, \it{pn})$.
The syntax definition given in Appendix~\ref{ap:sos-slco} is extended as follows to accommodate final states.
%
\[
\it{sm} ::= \it{smn}~\it{var^*}~``\textbf{initial}"~\it{sn}~\it{sn^*}~[``\textbf{final}"~\it{sn^+}]~\it{trans^*}
\]

The transformation~$T^S_{\it{TS}}$ defined below takes a sequence of transitions and a port name as input, and returns a modified sequence of transitions and a set of new states.
Only the transitions that send or receive signals over the given port are modified.
There are two cases for this transformation.
One case deals with empty sequences of transitions and the other with non-empty sequences.
%
\begin{align*}
T^S_{\it{TS}}(\epsilon, \it{pn}) & = \langle \epsilon, \epsilon \rangle \\
T^S_{\it{TS}}(\it{trans}~\it{trans^*}, \it{pn}) & = \langle \it{sn^*_1}~\it{sn^*_2}, \it{trans^*_1}~\it{trans^*_2} \rangle,
\end{align*}
where~$\langle \it{sn^*_1}, \it{trans^*_1} \rangle = T^S_{\it{T}}(\it{trans}, \it{pn})$ and~$\langle \it{sn^*_2}, \it{trans^*_2} \rangle = T^S_{\it{TS}}(\it{trans^*}, \it{pn})$.

The transformation~$T^S_{\it{T}}$ defined below takes a transition and a port name as input, and returns a set of states and a set of transitions.
Only transitions that have a signal reception trigger or a send signal statement are replaced by new transitions.
All other transitions are left unaltered.
There are three cases for this transformation.
One case deals with transitions that send a signal, one case deals with transitions that receive signals, and one case deals with all other transitions.
%
\[
\begin{array}{l}
T^S_{\it{T}}(\it{tn}~\textbf{from}~\it{sn_1}~\textbf{to}~\it{sn_2}~\textbf{send}~\it{sgn}()~\textbf{to}~\it{pn}, \it{pn}) =
\langle \\
\quad \it{sn_3}, \\
\quad \it{tn_1}~\textbf{from}~\it{sn_1}~\textbf{to}~\it{sn_3}~\textbf{send}~\it{sgn_1}()~\textbf{to}~\it{pn} \\
\quad \it{tn_2}~\textbf{from}~\it{sn_3}~\textbf{to}~\it{sn_2}~\textbf{receive}~\it{sgn_2}()~\textbf{from}~\it{pn} \\
\rangle,
\end{array}
\]
where~$\it{sn_3}$ is a fresh state name, $\it{tn_1}$ and~$\it{tn_2}$ are fresh transition names, $\it{sgn_1}~\equiv~``s\_"+\it{sgn}$, and $\it{sgn_2}~\equiv~``a\_"+\it{sgn}$,
\[
\begin{array}{l}
T^S_{\it{T}}(\it{tn}~\textbf{from}~\it{sn_1}~\textbf{to}~\it{sn_2}~\textbf{receive}~\it{sgn}()~\textbf{from}~\it{pn}, \it{pn}) =
\langle \\
\quad \it{sn_3}, \\
\quad \it{tn_1}~\textbf{from}~\it{sn_1}~\textbf{to}~\it{sn_3}~\textbf{receive}~\it{sgn_1}()~\textbf{from}~\it{pn} \\
\quad \it{tn_2}~\textbf{from}~\it{sn_3}~\textbf{to}~\it{sn_2}~\textbf{send}~\it{sgn_2}()~\textbf{to}~\it{pn} \\
\rangle,
\end{array}
\]
where~$\it{sn_3}$ is a fresh state name, $\it{tn_1}$ and~$\it{tn_2}$ are fresh transition names, $\it{sgn_1}~\equiv~``s\_"+\it{sgn}$, and~$\it{sgn_2}~\equiv~``a\_"+\it{sgn}$, and
\[
\begin{array}{l}
T^S_{\it{T}}(\it{trans}, \it{pn}) = \langle \epsilon, \it{trans} \rangle,
\end{array}
\]
for all other transitions.


\chapter{\label{chap:information-theory}Information theory}

The field of information theory is concerned with measurement, storage
and transmission of information.
I has its roots in communication theory,
but applications in many different fields including machine learning
and natural language processing.
In this chapter,
we will briefly introduce some of the main ideas,
and discuss a few important information theoretic measures
that will be used in the rest of the course.

\section{The noisy channel model}

A basic motivation in information theory is communication over
a noisy channel.
In a \emph{noisy channel model},
as the one depicted in Figure~\ref{fig:noisy-channel},
the sender encodes the given message,
and sends it through the channel,
but the receiver receives a possibly corrupted version of the coded message.
\begin{marginfigure}
  \tikzsetnextfilename{noisy-channel}
  \begin{tikzpicture}
    \node (in) {a};
    \node[draw,thick,right=3mm of in, minimum height=4ex] (enc) {encoder};
    \node[draw,thick,right=15mm of enc, minimum height=4ex] (dec) {decoder};
    \node[right=3mm of dec] (out) {a};
    \node[below=0.5mm of enc,xshift=2em,font=\small]
      {100\textcolor{red}{0}0010};
    \node[below=0.5mm of dec,xshift=-2em,font=\small]
      {100\textcolor{red}{1}0010};
    \draw[thick,->] (in) -- (enc);
    \draw[thick,->] (dec) -- (out);
    \draw[thick,decorate, decoration={snake,post length=1mm},->]
      (enc) -- (dec)
      node[midway, above,yshift=2mm,font=\small,text width=1cm,align=center]
        {noisy\\channel};
  \end{tikzpicture}
  \caption{\label{fig:noisy-channel}%
    Schematic description of the noisy-channel model.
    The encoder codes the message and sends through a noisy channel
    to the decoder.
    The encoded message may possibly be corrupted during the transmission.
    The decoder's task is to reconstruct the original message,
    despite the potential noise introduced.
  }
\end{marginfigure}
The task of the decoder is to recover the original message,
even if there are some errors introduced in the noisy channel.
There are two competing objectives within the noisy channel model.
First, we want to use codes that use of the \emph{channel capacity}
as \emph{efficient}ly as possible.
We want coding schemes that result in short coded messages,
to transmit or store.
This is where the strong connection between the information theory
and compression comes into the picture.
Second,
we want to be able to detect and correct the errors introduced by the noisy channel.
An obvious way to detect and correct errors is
to send multiple copies of the code.
As we introduce more redundant copies,
it is more likely to recover the original message.
However,
replication also wastes the bandwidth of the channel.
Coding information efficiently,
while allowing error detection and error correction is fundamental
topics in information theory.
The example code in Figure~\ref{fig:noisy-channel},
is simply the ASCII code followed by an odd parity bit,
which means the last bit is set to \num{1} if the number of \num{1}s
in the code is an odd number,
otherwise,
the last bit is set to \num{0}.
Note that with the given code at hand,
the decoder can detect the error.%
\footnote{Can the decoder correct the error in Figure~\ref{fig:noisy-channel}?}
Here, we will not discuss error-correcting codes,
neither most of the other fascinating topics in information theory.
Interested readers are referred to the textbooks on information theory,
such as \textcite{mackay2003}.

Clearly, the noisy channel model is useful in the study of computer networks.
However, it has many other uses.
Note that the channel does not have to be a network connection.
For example,
the model fits equally well to storing data to a permanent storage.
Permanent storage systems (e.g., hard disks) are fairly accurate,
however, they are not error free.
As a result,
error resilience and efficiency is a concern here too.
The encoder encodes the information,
and sends it to the disk,
the decoder reads code (possibly after a long time) from the disk and
has to make sure that the information it decodes was not corrupted.

Beyond those obvious extensions,
the noisy channel model found its use in many other applications.
For an example close to home,
we often model \emph{speech recognition} with a noisy channel model.
The information here is a linguistic message,
e.g., a sentence.
The speaker codes this information as an acoustic signal,
and the recognizer's task is to decode the sentence
from the acoustic signal (code).
Another common use for the noisy channel model in NLP is
in machine translation, which we will return later.
We will not further discuss
the applications of the noisy channel model here.
However,
we introduce some of the concepts, particularly some measures,
that have very frequent uses in machine learning and NLP.

\section{Entropy and information}

In information theory,
\emph{entropy} is a measure of uncertainty.
The measure is analogous to `physical' entropy measure
in statistical thermodynamics,
but measures the uncertainty of
an information system rather than a physical system.
In ambiguous contexts, it is also called \emph{information entropy} or
\emph{Shannon entropy} after Claude Shannon,
the inventor of the measure and the founder of the field.
Entropy and information are tightly connected concepts.
More concretely,
information in a message
(e.g., in the noisy channel model described above)
is the reduction of entropy after receiving the message.

Before introducing entropy properly,
we will first introduce a related measure \emph{surprisal},
which is also called \emph{self information}
or \emph{information content}.
The information theoretic measure of surprisal of an event $x$ is defined as
\begin{equation}\label{eq:surprisal}
  log \frac{1}{P(x)} = - log P(x) .
\end{equation}
If the probability of the event is \num{1}
(event occurs with certainty)
the surprisal will be \num{0}.
For events with decreasing probabilities,
we will get higher values for surprisal.
The value of surprisal will approach to $\infty$
while the probability of the event approaches to \num{0}.

The base of the logarithm in Equation~\ref{eq:surprisal}
is not very important,
since the logarithms in a different base can be obtained
by multiplication with a constant
(with a linear transformation).
Most common choices include base \num{2} logarithms,
which results in surprisal (or information) measured in \emph{bit}s.
If we use natural logarithm (with base $e$, Euler's number),
than the unit is called \emph{nat}.
In this course we will always use base-\num{2} logarithms,
and measure the information in bits.

The same quantity having names `surprisal' and `self information'
may not sound right at first sight.
The intuition here is that
we learn more from low-probability events.
Low probability events are surprising,
but also have more information content.
There is nothing surprising with
a weather report that tells it will rain in a very rainy country.
It also does not have much information content,
we can already predict it.
But if it predicts a sunny day in the middle of a rainy season,
then it is surprising as well as news-worthy,
it contains more information.

% The inverse relationship between surprisal and probability is intuitive.
% You may wonder, however, the reason for taking the logarithm.
% The main reason seems to be it is more convenient.
% We will try to get the intuition about logarithms with an example.
% Let's assume that we have a language with four letters,
% where occurrence of each letter has probability of \num{1/4},
% that is
%
% or information content of a state.
% High entropy means high unpredictability,
% hence high information content.
% If the outcome of a particular random variable is deterministic,
% then there is no uncertainty.
% Hence, the entropy is \num{0}.

Entropy of a system is the average surprisal.
We define entropy, $H$, of a random variable $X$ as
\begin{equation}\label{eq:entropy}
  H(X) = - \sum_{x \in X} P(x) log P(x)
\end{equation}
where, $x$ ranges over all values of $X$.
The above definition is for discrete variables,
which is much more common in NLP.
The notion of entropy can be extended to continuous random variables,
by replacing the sum with an integral
and $P(x)$ with the appropriate probability density function.
The resulting definition (for continuous random variables)
is called \emph{differential entropy}.

To get a sense of what entropy does,
consider the `guess the number' game,
where the first player picks a number between \num{1} and a larger number $M$,
(say 32),
and the task of the second player is to guess the number.
After every guess,
the first player tells whether the guess was
larger or smaller than the number he/she picked.
Assuming the first player picks the number completely randomly
(samples from the uniform distribution),
and if the second player follows the optimum strategy (binary search),
the first player will need to make $\log_{2} M$ guesses at most.%
\sidenote{What is the best strategy,
  if the number picked randomly,
  but follows a known non-uniform distribution?
}
Which is exactly what you will find if you use Equation~\ref{eq:entropy},
to calculate the entropy of this system.
For $M = 32$,
\[
  H = - \sum_{1 \le x \le 32} \frac{1}{32} \log_{2} \frac{1}{32}
  = - 32 \frac{1}{32} \log_{2} \frac{1}{32} = - \log_{2} \frac{1}{32}
  = 5 .
\]
Note that since the numbers are equally likely,
probabilities for each number is the same ($1/32$).
As a result,
at the beginning we have \num{5} bits of entropy.
You should also see that we reduce the entropy by \num{1} bit for every guess
(unless we guess the correct number).
Hence, information we gain with every guess is \num{1} bit.

It is important to realize that
the entropy, or the information,
increase or decrease proportional to the logarithm of the states (the numbers).
If we double the range of numbers to pick from,
the entropy will not double, but only increase with one bit.
This logarithmic relationship is convenient,
since states of many systems we are interested in grow exponentially.

\begin{margintable}
  \caption{\label{tbl:binary-code-example}
    Example binary coding of an eight letter alphabet.
  }
  \begin{center}
    \begin{tabular}{lS[minimum-integer-digits=3]}
      \toprule
      letter & {code} \\
      \midrule
      a & 000 \\
      b & 001 \\
      c & 010 \\
      d & 011 \\
      e & 100 \\
      f & 101 \\
      g & 110 \\
      h & 111 \\
      \bottomrule
    \end{tabular}
  \end{center}
\end{margintable}
To get a sense of why logarithm is a good idea,
consider we want to code a \num{8}-letter alphabet on a binary medium
(or want to send over a binary channel).
We can use as many bits as the number of letters
(setting one of the bits to \num{1}, and the rest to \num{0},
similar to the one-hot representation discussed in Section~\ref{desc:one-hot}),
but this is wasteful.
The optimum coding would not require \num{8} bits.
We can easily represent \num{8} letters letters with \num{3} bits,
which turns out to be exaclty $\log_{2} 8$.
An example coding of \num{8}-letter alphabet is shown
in Table~\ref{tbl:binary-code-example}.
Note that if we double the letters,
we would not need to double the number of bits used,
all we need to do is add one more bit.
Hence,
the number of bits needed to represent $M$ letters is $log_{2} M$.
This is optimum if the letters we want to transmit are distributed uniformly.
We will soon see that we can do better,
if we have more structure in the distribution of the letters.

\begin{marginfigure}
  \tikzsetnextfilename{bernoulli-entropy}
  \begin{tikzpicture}[
  ]

      \begin{axis}[axis lines=left,
          enlarge x limits=true,
          enlarge y limits=upper,
          samples=50,
          width=1.1\linewidth,
          xlabel={$P(X=1)$},
          ylabel={$H(X)$},
          ylabel style={at={(rel axis cs:0,1)}, anchor=west,rotate=-90}
      ]
          \addplot[smooth, thick, blue, domain=0:1]
            {- x * (ln(x)/ln(2)) - (1-x) * (ln(1-x)/ln(2))};
      \end{axis}
  \end{tikzpicture}
  \caption{\label{fig:bernoulli-entropy}%
    Entropy of Bernoulli random variables (in bits)
    as a function of the parameter $p$.
  }
\end{marginfigure}
For simplicity,
we assumed uniform distribution in the discussion/demonstration above.
What if the distribution of the random variable is not uniform?
To answer this question,
consider a Bernoulli trial, for example,
a coin-toss experiment or
picking letters from a two-letter alphabet like Morse code.
If the letters were equally probable as in our earlier example,
the entropy would be,
\[
%  - \left(P(\mathbf{-}) \log P(\mathbf{-})
%      + P(\mathbf{\cdot}) \log (\mathbf{\cdot})\right)
  H = - \left(0.5 \times \log_{2} 0.5 + 0.5 \times \log_{2} 0.5 \right)
  = - \log_{2} 0.5 = 1.
\]
So, the entropy of a Bernoulli random variable with equal probabilities ($p = 0.5$),
such as outcome of a fair coin toss, is \num{1} bit.
We can also calculate the entropy with non-equal probability values.
For example, entropy of a Bernoulli random variable with $p=0.8$,
\[
%  - \left(P(\mathbf{-}) \log P(\mathbf{-})
%      + P(\mathbf{\cdot}) \log (\mathbf{\cdot})\right)
  H = - \left(0.8 \times \log_{2} 0.8 + 0.2 \times \log_{2} 0.2 \right)
  = \num{0.7219281} .
\]
Note that the entropy reduced from 1 bit to \num{0.7219281} bits.
This should all be intuitive,
since knowing that some of the outcomes are more likely than others reduces
the average surprisal.
If we do the above calculations for the parameter of Bernoulli distribution
$p$ in range $[0, 1]$,
we get the graph in Figure~\ref{fig:bernoulli-entropy}.
If the outcome of the variable is certain with $p = 1$ or $p = 0$,
then the entropy is \num{0},
and the entropy peaks at \num[round-precision=1]{0.5},
with a value of \num{1} bit,
where the uncertainty is at its maximum.

In general, uniform distribution is the distribution
with the maximum entropy
for distributions with the same support
(the set of values/events the distribution is defined on).
If the number of outcomes increase,
the uncertainty and the entropy will increase.
And, as the distribution diverges from the uniform distribution
by assigning higher probabilities to a small set of events,
the entropy will decrease.
Figure~\ref{fig:entropy-demo} demonstrates these two factors
with the categorical distribution.

\begin{figure}
    \tikzset{external/export next=false}
    %\tikzsetnextfilename{entropy-viz1}
    \begin{tikzpicture}[x=1mm,y=1mm]
      \newcounter{x}
      \newcounter{y}
      \def\xstep{22}
      \def\r{8mm}
      \draw[fill=blue!20] (\thex,\they) circle[radius=\r];

      \addtocounter{x}{\xstep}
      \filldraw[fill=green!20!white, draw=green!50!black]
          (\thex,0) ++(0:\r) arc[start angle=0, end angle=180, radius=\r] -- cycle;
      \filldraw[fill=blue!20!white, draw=green!50!black]
          (\thex,0) ++(180:\r) arc[start angle=180, end angle=360, radius=\r] -- cycle;

      \addtocounter{x}{\xstep}
      \filldraw[fill=green!20!white, draw=green!50!black]
          (\thex,0) ++(0:\r) arc[start angle=0, end angle=90, radius=\r] -- (\thex,0) -- cycle;
      \filldraw[fill=blue!20!white, draw=green!50!black]
          (\thex,0) ++(90:\r) arc[start angle=90, end angle=180, radius=\r] -- (\thex,0) --cycle;
      \filldraw[fill=red!20!white, draw=green!50!black]
          (\thex,0) ++(180:\r) arc[start angle=180, end angle=270, radius=\r] -- (\thex,0) --cycle;
      \filldraw[fill=gray!20!white, draw=green!50!black]
          (\thex,0) ++(270:\r) arc[start angle=270, end angle=360, radius=\r] -- (\thex,0) --cycle;

      \addtocounter{x}{\xstep}
      \filldraw[fill=green!20!white, draw=green!50!black]
          (\thex,\they) ++(0:\r) arc[start angle=0, end angle=60, radius=\r] -- (\thex,\they) -- cycle;
      \filldraw[fill=blue!20!white, draw=green!50!black]
          (\thex,\they) ++(60:\r) arc[start angle=60, end angle=120, radius=\r] -- (\thex,\they) --cycle;
      \filldraw[fill=red!20!white, draw=green!50!black]
          (\thex,\they) ++(120:\r) arc[start angle=120, end angle=280, radius=\r] -- (\thex,\they) --cycle;
      \filldraw[fill=gray!20!white, draw=green!50!black]
          (\thex,\they) ++(180:\r) arc[start angle=180, end angle=360, radius=\r] -- (\thex,\they) --cycle;

      \addtocounter{x}{\xstep}
      \filldraw[fill=green!20!white, draw=green!50!black]
          (\thex,\they) ++(0:\r) arc[start angle=0, end angle=30, radius=\r] -- (\thex,\they) -- cycle;
      \filldraw[fill=blue!20!white, draw=green!50!black]
          (\thex,\they) ++(30:\r) arc[start angle=30, end angle=60, radius=\r] -- (\thex,\they) --cycle;
      \filldraw[fill=red!20!white, draw=green!50!black]
          (\thex,\they) ++(60:\r) arc[start angle=60, end angle=90, radius=\r] -- (\thex,\they) --cycle;
      \filldraw[fill=gray!20!white, draw=green!50!black]
          (\thex,\they) ++(90:\r) arc[start angle=90, end angle=360, radius=\r] -- (\thex,\they) --cycle;

      \foreach[count=\i, evaluate=\i as \x using int((\i-1)*\xstep)] \v in {0,1,2,1.79,1.06}{%
        \node[anchor=north] at (\x,-10) {$H = \v$};
      }
    \end{tikzpicture}
    \caption{
      Demonstration of entropy values (in bits)
      with the categorical distribution.
      First three circles from the left represent
      uniform categorical distributions with increasing number of categories.
      Increasing possible outcomes increase the uncertainty, hence the entropy.
      Last three circles demonstrate categorical distributions with equal
      number of possible outcomes.
      As the distribution diverge from the uniform distribution,
      the uncertainty and entropy is reduced.
    }\label{fig:entropy-demo}%
\end{figure}

By now,
you should be able to calculate the entropy of a categorical distribution
over \num{8} symbols,
just like the letters in Table~\ref{tbl:binary-code-example}.
Given that these letters are distributed uniformly
(each having probability of $1/8$),
the entropy of the distribution is \num{3} bits,
which corresponds to how many bits we need for optimally coding the alphabet.
We noted that we can do better if the distribution was not uniform.
Let us assume that,
the letter `a' occurs with probability $1/2$,
the letter `b' occurs with probability $1/4$,
the letter `c' occurs with probability $1/8$,
the letter `d' occurs with probability $1/16$,
and the other letters occur with probability $1/64$.
Now, using trhee bits for all comibinations is just wasteful.
We can do much better with the codes on Table~\ref{tbl:huffman-code-example}.
The type of coding example here is called \emph{Huffmann} coding.
Although the number of bits are variable,
we can unabiguously determine each code with its prefix.
One can easily show that this coding requires less storage or channel bandwidth
on average.
In fact,
this corresponds to the entropy of the distribution,
which is \num{2} bits:
we save \num{1} bit on average.
This is exactly another way to interpret entropy,
it is the average code-length for the best achievable code
for a given distribution.
In other words,
entropy gives us the upper bound on
an optimal encoder (in terms of efficient channel utilization)
in a noisy channel model (Figure~\ref{fig:noisy-channel}).
\begin{margintable}
  \caption{\label{tbl:huffman-code-example}
    Example Huffman coding of an eight letter alphabet.
  }
  \begin{center}
    \begin{tabular}{lrl}
      \toprule
      letter & prob &{code} \\
      \midrule
      a & $1/2$&  \num{0} \\
      b & $1/4$&  \num{10} \\
      c & $1/8$&  \num{110} \\
      d & $1/16$& \num{1110} \\
      e & $1/64$& \num{111100} \\
      f & $1/64$& \num{111101} \\
      g & $1/64$& \num{111110} \\
      h & $1/64$& \num{111111} \\
      \bottomrule
    \end{tabular}
  \end{center}
\end{margintable}

% \begin{margintable}
%   \caption{\label{tbl:cond-prob-table}%
%     Conditional probabilities of $P(\text{letter}\:\vert\:\text{dialect})$.
%   }
%   \begin{center}
%     \setlength{\tabcolsep}{4pt}
%     \begin{tabular}{lS[round-precision=3]S[round-precision=3]S[round-precision=3]}
%       \toprule
%       let. & {east} & {north} & {south} \\
%       \midrule
%       a & 0.2829786& 0.0326985& 0.283765\\
%       b & 0.0413973& 0.0624915& 0.009443\\
%       c & 0.0363760& 0.0465435& 0.116444\\
%       d & 0.0892564& 0.0738920& 0.068607\\
%       e & 0.2458984& 0.4862595& 0.165408\\
%       f & 0.0241540& 0.0131360& 0.067122\\
%       g & 0.0720066& 0.0647505& 0.000000\\
%       h & 0.2079327& 0.2202285& 0.289211\\
%       \bottomrule
%     \end{tabular}
%   \end{center}
% \end{margintable}
\begin{margintable}
  \caption{\label{tbl:marginal-prob-table2}%
    Joint probability table for letters and dialects
    with marginal probabilities.
    $P(D)$ is the (marginal) probability of dialects,
    and $P(L)$ is the probability of letters in the corpus.
    This is the same as Table~\ref{tbl:marginal-prob-table},
    repeated here for convenience.
  }
  \begin{center}
    \setlength{\tabcolsep}{4pt}
    \begin{tabular}{lS[round-precision=2]S[round-precision=2]S[round-precision=2]|S[round-precision=2]}
      \toprule
      let. & {east} & {north} & {south} & {$P(L)$} \\
      \midrule
      a & 0.1980850&0.0065397&0.0283765 & 0.2330012\\
      b & 0.0289781&0.0124983&0.0009443 & 0.0424207\\
      c & 0.0254632&0.0093087&0.0116444 & 0.0464163\\
      d & 0.0624795&0.0147784&0.0068607 & 0.0841186\\
      e & 0.1721289&0.0972519&0.0165408 & 0.2859216\\
      f & 0.0169078&0.0026272&0.0067122 & 0.0262472\\
      g & 0.0504046&0.0129501&0.0       & 0.0633547\\
      h & 0.1455529&0.0440457&0.0289211 & 0.2185197\\
      \midrule
      $P(D)$  & 0.7&0.2&0.1 & 1.0\\
      \bottomrule
    \end{tabular}
  \end{center}
\end{margintable}
To give a more concrete example,
we re-introduce our dialects with eight letters from Chapter~\ref{chap:prob}.
Table~\ref{tbl:marginal-prob-table2} lists the joint probabilities of
letters and dialects (east, north and south),
as well as marginal probabilities.
From this table,
we can easily calculate the entropy of the distributions of letters for the
complete language,
and the entropy of the letter distribution for each dialect.
Now you should take a moment and try to phrase
what high or low entropy means for letters or dialects.
For the sake of making things more concrete,
we will calculate the entropy of dialect distribution in our document set.
The entropy of letter distribution over dialects
\[
  H(d) = -\left(0.7 \times log_{2} 0.7 + 0.2 \times log_{2} 0.2 + 0.1 \times log_{2} 0.1\right)
       = \num{1.15678}
\]
is lower than a uniform distribution of letters over the dialects
which is \num{1.584963}.
In other words,
by knowing $P(D)$, we gain \num{0.428183} bits of information,
in comparison to a setting where the dialects were equally represented
in the data.
The distribution over individual letters is more interesting,
you are encouraged to calculate the entropy of letters ($P(L)$)
and compare it with a uniform distribution.
Note that if you want to calculate entropy of letters within each dilaect,
you need conditional probabilities, e.g., $P(L\given{}D=\text{east})$,
for calculating letter-entropy values of individual dialects.
%\todo{exercise}

\begin{tcolorbox}
  Note that entropy is about complete probability distributions,
  while surprisal is about probabilities of individual events.
\end{tcolorbox}

\section{Mutual Information and conditional entropy}

Another very important quantity from information theory used
widely in computational and corpus linguistics is \emph{mutual information}.
Mutual information measures
the amount of information obtained (reduced entropy) about a random variable,
by knowledge of another random variable.
Mutual information is a measure of dependence between two variables.
If the variables are dependent (provide information about each other),
then the mutual information will be high.
If the variables are independent the mutual information will be \num{0}.

As we did with introduction to entropy,
we will first introduce another relevant measure,
\emph{pointwise mutual information} (PMI),
which measures the (in)dependence of two events.
The PMI value for two events
$X = x$ (or simply $x$) and $Y = y$ (or $y$) is calculated by
\begin{equation}\label{eq:pmi}
  \text{PMI}(x, y) = \log \frac{P(x,y)}{P(x)P(y)}
\end{equation}
A quick study of the formula indicates
that pointwise mutual information is high
if the joint probability of the two events involved is high.
However,
since the high-probability events can cooccur by chance frequently,
the denominator of the term in Equation~\ref{eq:pmi} includes
(marginal) probabilities of both events.
Hence, it discounts the by-chance cooccurrences of the events
due to high marginal probability.
Remember that the joint probability of two independent events is
the product of their probabilities ($P(x,y) = P(x) P(y)$,
for independent events $x$ and $y$).
As a result,
the division within the logarithm in Equation~\ref{eq:pmi} will result in \num{1},
and PMI will be \num{0}, for independent events.
If two events are highly positively associated (they occur together),
then PMI will be positive,
and if events are highly negatively associated
(one does not occur if the other event occurs),
then the PMI value will be negative.

Just for the sake of example,
we will calculate the PMI of letter `e' occurring in the `east' dialect
in the distribution presented in Table~\ref{tbl:marginal-prob-table2}.
\[
  \text{PMI}(\text{east},\text{e})
    = log \frac{P(\text{east}, \text{e})}{P(\text{east})P(\text{e})}
    = log \frac{\num{0.1721289}}{\num{0.7}\times \num{0.2859216}}
    = \num{-0.2175571}
\]
Despite the fact that the joint probability of the two events is rather high,
we find a negative association between them%
---in `e', in fact, occurs less than chance in the eastern dialect.

One of the most common uses of PMI is to find \emph{collocations},
groups of words that occur frequently together.
Using PMI,
it is likely to find linguistically plausible collocations like,
`corpus linguistics',
even though it is much less likely than non-interesting frequent bigrams like `that the'.

The \emph{mutual information} (MI) of two random variables
is a measure of dependence of the variables.
Mutual information is the expected value of (average) PMI.
The mutual information of two discrete random variables $X$ and $Y$ is
\begin{equation}\label{eq:mi}
  \text{MI}(X, Y)
    = \sum_{x\in X} \sum_{y\in Y} P(x,y) \log \frac{P(x,y)}{P(x)P(y)} .
\end{equation}

Similar to PMI,
a positive MI value indicates a positive association between
the random variables,
zero indicates independence (no association),
and negative values indicate negative association.
Like \emph{correlation} (Section~\ref{sec:correlation}),
mutual information measures dependence between two random variables.
However, unlike correlation, it is not limited to linear dependence.

Mutual information is related to entropy
through a measure known as \emph{conditional entropy}.
We note the conditional entropy of a random variable $X$ given
another random variable $Y$ takes the value $y$ as $H(X\given{}Y=y)$.
Intuitively,
if the event $Y=y$ gives information about the random variable $X$,
we expect low conditional entropy.
The conditional entropy of $X$ given $Y$,
noted $H(X\given{}Y)$,
is the average entropy of $X$ given $Y$.
The conditional entropy of $X$ given $Y$ can be calculated by
\begin{equation}\label{eq:conditional-entropy}
  \begin{aligned}
    H(X\given{}Y) &= & &\sum_{y\in Y} P(y) H(X\given{}Y=y)\\
                  &= &- & \sum_{x\in X, y\in Y} P(x,y) \log P(x\given{}y)
  \text{.\footnotemark}\\
  \end{aligned}
\end{equation}%
\footnotetext{You are recommended to derive this equation.
 Meanwhile, you may also want to show that
 \[
    H(X\given{}Y) = \sum_{x\in X, y\in Y} P(x,y) \log \frac{P(y)}{P(x,y)}
 \]
 is also correct.
}

The conditional entropy has also a straightforward interpretation.
The conditional entropy $H(X\given{}Y)$ equals to
the entropy of the variable $X$ ($H(X)$),
if the variables are independent.
As the dependence between variables increase,
the conditional entropy will decrease.

\begin{marginfigure}
  \tikzsetnextfilename{entropy-mi-relation}
  \begin{tikzpicture}
%    \draw[help lines] (0,0) grid (5,5);
    \node[draw,circle,minimum size=3.5cm,inner sep=0pt,fill=blue,opacity=0.2]
      at (2,2) {};
    \node[draw,circle,minimum size=3.5cm,inner sep=0pt,fill=red,opacity=0.2]
      at (3,3) {};
    \node at (0.7,3.7) {$H(X)$};
    \node at (4.3,1.3) {$H(Y)$};

    \node at (1.3, 1.5) {$H(X\given{}Y)$};
    \node at (3.7, 3.5) {$H(Y\given{}X)$};

    \node at (2.5, 2.5) {$MI(X,Y)$};

    \node at (3.5, 0) {$H(X,Y)$};
  \end{tikzpicture}
  \caption{\label{fig:entropy-mi}%
    Relation between conditional entropy entropy and mutual information.
    The total shaded area is the joint entropy of two variables $H(X,Y)$.
  }
\end{marginfigure}

Note that entropy, conditional entropy, and mutual information are all related.
Figure~\ref{fig:entropy-mi} shows the relationship between these measures
schematically using a Venn diagram.
For example, we can see from the figure that $H(X,Y) = H(X) + H(Y) - MI(X,Y)$.
The total entropy associated with two random variables is reduced if the
mutual information (dependence) between them is high.
If one of the variables ($Y$) completely determines the other ($Y$),
the conditional entropy ($H(X|Y)$) will be \num{0},
the joint entropy will be equal to the entropy of one one of the variables
($H(Y)$),
and mutual information will be equal to entropy of the other variable $H(X)$.
The mutual information is symmetric ($MI(X,Y) = MI(Y,X)$),
while conditional entropy is not ($H(X\given{}Y) \ne H(Y\given{}X)$).
Also note that the relationships
between the information theoretic measures are additive,
while the relationship between the corresponding probabilities are multiplicative.

\section{Cross entropy}

\emph{Cross entropy} is another important concept from information theory
that has wide usage, particularly in machine learning.
Remember that entropy gives us the best achievable compression
given a distribution.
In many practical cases we do not know the true distribution of the data,
instead we use an approximate one.
If we do not use the true distribution,
inevitably, our code will be less optimum.
This means that we will get higher entropy.
If the true distribution is $P(x)$ and the its approximation is $\hat{P}(x)$,
the cross entropy is defined as
\begin{equation}
  H(P, \hat{P}) = - \sum_{x} P(x) log \hat{P}(x) .
\end{equation}
The hat-notation used here is used for estimated objects.
If $P$ was the true distribution,
we would note an estimation of it as $\hat{P}$.
Although this is most common context we will see cross entropy used,
the distribution $\hat{P}$ does not have to be an estimate of $P$.
Technically, the only requirement is
that the distribution should have the same support.
Note that the notation $H(X, Y)$ is also used for joint entropy,
but for joint entropy the arguments are random variables
noted like $X$ and $Y$,
and for cross entropy distribution functions
noted with letters like $P$ and $Q$.

The formula above makes sense if you think about it
in terms of the noisy channel model.
Remember that we are coding the data using $\hat{P}$,
hence the code length is determined by the approximate model.
However, the average is taken over the true distribution,
hence, we multiply by $P(x)$ not $\hat{P}(x)$.
Cross entropy is always larger than entropy.
Since we are using the wrong (approximate) distribution to code the data,
the code length will not be optimum.
As $\hat{P}$ gets closer to the $P$,
the cross entropy will approach the entropy of $P$.
Note that cross entropy is not symmetric ($H(P, Q) \ne H(Q, P)$).

A very common use of cross entropy is as a \emph{loss function}
in some machine learning methods.
In many machine learning methods,
training the model is equivalent to minimizing a loss function.
Hence,
as we minimize cross entropy of true distribution
that comes from the training data
and the approximate distribution the machine learning system produces,
we get closer to the true distribution of the answers we are seeking.
That way,
the output of the machine learning model becomes
more similar to the expected output.
We will see example uses of cross entropy later in this course.

\begin{margintable}
  \caption{\label{tbl:east-overall}%
    Probability distributions of letters in our hypothetical dialect data.
    $P(L\given{}D=\text{east})$ is the distribution of letters
    given the dialect is east dialect,
    $P(L)$ is the marginal distribution of letters (for all dialects).
  }
  \begin{center}
    \setlength{\tabcolsep}{4pt}
    \begin{tabular}{lS[round-precision=2]S[round-precision=2]}
      \toprule
      let. & {$P(L\given{}\text{east})$} & {$P(L)$} \\
      \midrule
      a & 0.2829786 & 0.2330012\\
      b & 0.0413973 & 0.0424207\\
      c & 0.0363760 & 0.0464163\\
      d & 0.0892564 & 0.0841186\\
      e & 0.2458984 & 0.2859216\\
      f & 0.0241540 & 0.0262472\\
      g & 0.0720066 & 0.0633547\\
      h & 0.2079327 & 0.2185197\\
      \bottomrule
    \end{tabular}
  \end{center}
\end{margintable}
For a concrete example,
we return to the hypothetical dialects with \num{8}-letter alphabets
in Table~\ref{tbl:marginal-prob-table2}.
Assume that we do not know the distribution of the letters
in the eastern dialect,
and we are using the total distribution
for coding (e.g., compressing) the data from the eastern dialect.
Both distributions are summarized in Table~\ref{tbl:east-overall}.
We simply calculate cross entropy by
\[
  \begin{aligned}
    H\left(P(L\given{}\text{east}), P(L)\right)
      = - \sum_{x\in \text{letters}} P(x\given{}\text{east}) \log_{2} P(x)
      = \num[round-precision=3]{2.577201} .
  \end{aligned}
\]
As expected,
this is larger than entropy of $P(L\given{}\text{east})$,
which is \num[round-precision=3]{2.562475} bits.

\section{Perplexity}

A measure related to entropy,
used often in computational linguistic literature,
is \emph{perplexity} (PP).
Perplexity is simply the exponentiated version of entropy.
If we measure entropy in bits, then the perplexity is
\begin{equation*}
  \text{PP}(X) = 2^{H(X)} .
\end{equation*}

Like entropy, perplexity measures uncertainty.
However, since it measures it in a different scale,
sometime it offers a more intuitive interpretation.
Its main use in NLP is evaluating language models,
which are conditional distributions on words given a history of earlier words.
In this context, the intuitive interpretation is
the average number of words expected after each word.

\section{Kullback--Leibler divergence}

\begin{marginfigure}
  \tikzsetnextfilename{kl-divergence-viz}
  \begin{tikzpicture}[
      declare function={%
        normpdf(\x,\m,\s) = exp(-(\x -\m)^2/(2*\s)^2) /%
                                (\s*(2*pi)^0.5);%
      }
  ]
      \begin{axis}[
          axis lines=middle,
          hide y axis,
          enlargelimits=true,
          samples=50,
          x=4mm,
          y=70mm,
          ymin=0,
          clip=false,
          x axis line style={<->}
      ]
          \addplot[smooth, thick, blue, domain=-6:6,name path=curve1]
            {normpdf(x,-0.5,1)};
          \addplot[smooth, thick, red, domain=-6:6,name path=curve2]
            {normpdf(x,0.5,1.2)};
          \addplot[draw=none, domain=-6:6,name path=xax] {0};
          \addplot[blue,opacity=0.2]
            fill between[of=curve1 and xax,soft clip={domain=-5:5}];
          \addplot[red,opacity=0.2]
            fill between[of=curve2 and xax,soft clip={domain=-5:5}];
          \node[blue] at (-4.2, 0.1) {$P$};
          \node[red] at (3.8, 0.1) {$Q$};
          \node (a) at (2, 0.15) {};
          \node (kpq) at (4, 0.3) {$D_{KL}(P\;\Vert\;Q)$};
          \draw[->] (a) --  (kpq);

          \node (b) at (-1.6, 0.18) {};
          \node (kqp) at (-4, 0.3) {$D_{KL}(Q\;\Vert\;P)$};
          \draw[->] (b) --  (kqp);
      \end{axis}
  \end{tikzpicture}
  \caption{\label{fig:kl-diverngence}%
    Visualization of KL divergences between two distributions.
    Note, however, this is a demonstration to show the asymmetry
    of measure, the areas indicated are not exactly what
    KL-divergence measures.
  }
\end{marginfigure}
Another important quantity we often see during this course is
\emph{Kullback-Leibler divergence} which is also known as
\emph{relative entropy}.
It is often abbreviated as \emph{KL divergence}.
Similar to cross entropy,
KL divergence is a about two probability distributions with the same support.
It measures the amount of information lost,
or extra bits of entropy,
while using a distribution $Q$ instead of using the true distribution $P$.
It is defined as,
\begin{equation}\label{eq:kl-divergence}
  D_{KL} (P\;\Vert\;Q) = - \sum_{x} P(x) \log \frac{P(x)}{Q(x)}.
\end{equation}
It is the average logarithmic difference
between distributions $P$ and $Q$,%
\footnote{%
  Remember that
  \[
    \log \frac{a}{b} = \log(a) - \log(b) .
  \]
}
where average calculated according to $P$.
KL divergence is often used as a measure of difference
between two distributions.
However, it is not symmetric
(as a result, it is not a proper distance measure).
Figure~\ref{fig:kl-diverngence} visualizes the KL divergence
between two continuous distributions.
For continuous distributions,
as usual,
you can simply replace probability mass functions
with probability density functions,
and summation with the integral in Equation~\ref{eq:kl-divergence}.

For a concrete example,
we will return to the question of using an approximate distribution,
the marginal letter distribution,
instead of one of the conditional distributions
in our letter distribution example in Table~\ref{tbl:east-overall}.
As in our cross entropy example,
assume that we do not know the distribution of the eastern dialect,
and approximating it with the marginal distribution.
If we do the calculation,
\[
  D_{KL} \left(P(L\given{}\text{east})\;\Vert\;P(L)\right) = - \sum_{x} P(x|\text{east}) \log \frac{P(x|east)}{P(x)}
  = \num[round-precision=3]{0.0147261}
\]
which is the amount of information we lose by using the marginal distribution
instead of the dialect-specific distribution in bits.

Conceptually,
you should already be expecting a link between
the cross entropy and the KL divergence divergence.
Cross entropy measures the entropy of a distribution ($P$) under
another distribution ($Q$),
while KL divergence measures the amount of additional entropy if one uses one distribution ($Q$) instead of another ($P$).
As a result,
\[
  H(P, Q) = H(P) + D_{KL}(P\;\Vert\;Q).
\]

You can easily verify this
with our running example of dialect letter distributions as well
(maybe with some rounding error).

% \section*{Where to go from here}
%
% \textcite{mackay2003}
%
% \textcite{pierce1980}
%
% \textcite{stone2015}
%
% \textcite{ghahramani2003}
%
% \textcite{shannon1948}
%
% \textcite{seife2007}
%
% \section*{Exercises}
%
% % \begin{question}
%   What is the name of the following measure?
%   \[
%     \log \frac{P(x\given{}y)}{P(x)}
%   \]
% \end{question}

% p = c(0.2829786, 0.0413973, 0.0363760, 0.0892564, 0.2458984, 0.0241540, 0.0720066, 0.2079327)
% q = c(0.2330012, 0.0424207, 0.0464163, 0.0841186, 0.2859216, 0.0262472, 0.0633547, 0.2185197)


\documentclass[12pt]{article}
\usepackage[utf8]{inputenc}
\usepackage[english]{babel}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{bbm}
\usepackage{tikz}
\usepackage{mathtools}
\usepackage{subfigure}
\usepackage[hang]{footmisc}

\setlength{\footnotemargin}{2.5mm}
\usetikzlibrary{shapes,automata,arrows}
\graphicspath{{images/}{../images/}}
\setcounter{tocdepth}{1}

\newcommand{\transpose}[1]{#1^{\mathsf{T}}}
\newcommand{\inverse}[1]{#1^{-1}}
\newcommand{\iterate}[2]{#1^{(#2)}}
\newcommand{\parens}[1]{ \left( #1 \right) }
\newcommand{\prob}{\mathbb{P}}
\newcommand{\dee}{\mathrm{d}}
\renewcommand{\thesubtable}{\Roman{subtable}}
\DeclareMathOperator{\rank}{rank}
\DeclareMathOperator{\Null}{null}
\DeclarePairedDelimiterX{\norm}[1]{\lVert}{\rVert}{#1}
\DeclarePairedDelimiter{\abs}{\lvert}{\rvert}

\title{Exploring PageRank}
\author{Nathanael Gentry}
\date{30 April 2019}

\begin{document}
\maketitle

\section{Motivation}
In 1995, the public Internet contained about 2.5 million pages
\cite{TotalNumberWebsites}, but search techniques relied upon classical
strategies for collections much smaller and less interconnected than the nascent
Internet. By 1998, two Stanford students -- Sergey Brin and Larry Page --
endeavored to bring order to the Web. The students’ graduate project, then hosted at
\texttt{google.stanford.edu}, did not rely on simple heuristics like URL length
or date of publication; their program used the link structure of the Web itself
to decide how relevant a page was to a user's query. Brin and Page thus defined
their problem as a problem of spectral ranking, or eigencentrality. Spectral
ranking provides a tractable definition of an entity's relative
\textit{importance} within a collection and thus gives an excellent heuristic
for its relevance in a search query. In this paper, we explore the foundation of
the Google enterprise: PageRank, the algorithm that provides a spectral ranking
of all pages on the indexed Web.

\section{First Steps}
\begin{figure}[h]
  \centering \scalebox{0.75}{\begin{tikzpicture}[
			scale=0.6,
      > = stealth,
			shorten > = 1pt,
			auto,
			node distance = 3cm,
			semithick]
			\tikzstyle{every state}=[
			draw = black,
			thick,
			fill = white,
			minimum size = 4mm]

			\node[state] (a) {$p_1$};
			\node[state] (b) [above right of=a] {$p_2$};
			\node[state] (c) [right of=a] {$p_3$};
			\node[state] (d) [below right of=a] {$p_4$};
			\node[state] (e) [right of=c] {$p_5$};

			\path[->] (a) edge node {} (b);
			\path[->] (a) edge node {} (d);
			\path[->] (a) edge node {} (c);
			\path[->] (c) edge node {} (b);
			\path[->] (d) edge node {} (c);
			\path[->] (e) edge node {} (b);
			\path[->] (b) edge [bend left=45] node {} (e);
			\path[->] (d) edge node {} (b);
			\path[->] (e) edge node {} (c);
			\path[->] (d) edge node {} (e);
		\end{tikzpicture}}
  \caption{A Miniscule Web}
  \label{fig:web}
\end{figure}
Consider a digraph $\mathcal{W}$, such as that given in Figure \ref{fig:web}.
Each vertex $p$ represents a Web page, and each edge represents the existence of
at least one hyperlink between pages $p_i$ and $p_j$. We seek to calculate the
importance of a Web page, and we can view a hyperlink to that page as an
endorsement of importance from the linking page. The more such votes of support
a page has, the more important it probably is. The votes of importance imparted
by a Web page should be distributed evenly across the pages it endorses. Let us
now formalize this intuition. Let $N^-_i$ represent the set of $p_i$'s
predecessors, and let $N^+_i$ represent the set of $p_i$'s successors. Note that
$|N^+_i|$ gives the out-degree of $p_i$. We thus define the importance score
$\pi_i$ of a page $p_i$:
\begin{equation}\label{eqn:pi_i}
  \pi_i = \sum_{p_j\in N^-_i}{\frac{\pi_j}{|N^+_i|}}.
\end{equation}

Equation \eqref{eqn:pi_i} gives an impredicative definition of an importance
score.\footnote{Logicians disagree about the distinction between
  \textit{recursion} and \textit{impredicativity}. In this paper, we take
  recursion to be constructive -- that is, recursion builds a class by feeding
  the results of a generator into itself. Impredicativity, however, defines an
  impredicative object $X$ as an instantiation of a class which itself contains
  $X$. Equation \eqref{eqn:pi_i} thus gives an impredicative definition of
  $\pi_i$.} We store all the $\pi_i$ in a vector $\pi$, which we take henceforth
as a row vector.\footnote{Some authors denote row vectors by transposing a
  column vector every time, but we have found this notation wearying. In this
  paper the probability distribution $\pi$ will always be given by a row vector,
  without indication of transposition.} Take some initial score vector
$\iterate{\pi}{0}$, say a uniform distribution $\pi = [1/|\mathcal{W}|]$. Then,
we define an iterative calculation for an iterator $t\geq 1$ to refine the
initial distribution:

\begin{equation}
  \label{eqn:pi_t}
  \iterate{\pi}{t+1}_i = \sum_{p_j\in N^-_i}{\frac{\iterate{\pi}{t}_j}{|N^+_i|}}.
\end{equation}
We encode the system given by Equation \eqref{eqn:pi_i} over $\mathcal{W}$ in a hyperlink matrix:
\begin{equation*}
  H(\mathcal{W})_{ij}=\begin{cases}
    1/|N^+_i|, & \text{if } p_i \in N^-_j; \\
    0, & \text{otherwise}.
  \end{cases}
\end{equation*}

Thus, we have the following iterative calculation:
\begin{equation}
  \iterate{\pi}{k+1} = \iterate{\pi}{k} H.
\end{equation}
The hyperlink matrix gives an adjacency matrix of $\mathcal{W}$ modified to
ensure that each nonzero row provides a probability distribution. By
construction, $H$ is nonnegative; that is, $H_{ij}\geq 0$ for all components.
\begin{figure}[t!]
  \centering
  \begin{subfigure}
    \centering
    $\begin{bmatrix}
      0 & 1/3 & 1/3 & 1/3 & 0 \\
      0 & 0 & 0 & 0 & 1 \\
      0 & 1 & 0 & 0 & 0 \\
      0 & 1/3 & 1/3 & 0 & 1/3 \\
      0 & 1/2 & 1/2 & 0 & 0
    \end{bmatrix}$
    \caption{Hyperlink Matrix for Figure \ref{fig:web}}
    \label{fig:hyperlink}
  \end{subfigure}
  \vspace{2em}
  \begin{subfigure}
    \centering
    \begin{tabular}{|c||c|c|c|c|c|}
      \hline
      $k$ & $\pi_1$ & $\pi_2$ & $\pi_3$ & $\pi_4$ & $\pi_5$ \\
      \hline\hline
      0 & 0.2 & 0.2 & 0.2 & 0.2 & 0.2 \\
      1 & 0 & 0.433 & 0.233 & 0.067 & 0.267 \\
      2 & 0 & 0.389 & 0.156 & 0 & 0.456 \\
      3 & 0 & 0.383 & 0.228 & 0 & 0.390 \\
      4 & 0 & 0.422 & 0.194 & 0 & 0.383 \\
      \hline
    \end{tabular}
    \caption{Score Iteration for Figure \ref{fig:web}}
    \label{fig:example_calculation}
  \end{subfigure}
\end{figure}

Figures \ref{fig:hyperlink} and \ref{fig:example_calculation} show the
hyperlink matrix and some iterative calculations for the digraph given in
Figure \ref{fig:web}. We simply rank the pages by the distribution given in
$\pi$.

Observe that $p_2$ has the highest rank, as most other pages point into it
and it only links to one page. However, this page $p_5$ also receives a high
score, since an important page directly links to it. Our na{\"i}ve approach
shuts out $p_1$ and $p_4$ entirely, which destroys the uniqueness of the
ranking. There are other problems with our formulation. Note that the zero
rows of $H$ correspond to dangling pages, i.e. vertices with zero
out-degree. These rank sinks will break our current calculation, but we
cannot simply exclude them. Some sinks -- for instance, published PDF
documents -- might be highly cited and quite important. Consider also the
trouble posed by cyclic graphs, which would eternally shift their importance
around their cycles. If we continued the iterative calculation in Figure
\ref{fig:example_calculation}, we would find the scores of $p_2$ and $p_5$
keep oscillating as the number of iterations grew. We will discuss the
problem of cyclicity later, but we easily fix can rank sinks by making $H$
fully row-stochastic.

\section{Markov Chains}

\subsection{Stochasticity}
Consider a vector $d$, where $d_i=1$ if page $p_i$ dangles and $d_i=0$
otherwise. Let $v$ be a distribution over all pages in a Web graph
$\mathcal{W}$. We stochasticize $H$ thusly:
\begin{equation}
  \label{eqn:S}
  S=H + d\transpose{v}.
\end{equation}

If an ideal ``random surfer'' on the Web hits a dangling page, it chooses based
upon the distribution $v$ what page to next visit. Let $\mathbbm{1}$ represent a
vector of ones, and note that the row-stochasticity of $S$ implies
$\Vert S \Vert_{\infty} = 1$, i.e. $S \mathbbm{1} = \mathbbm{1}$. Thus,
$(1, \mathbbm{1})$ provides a right eigenpair for every stochastic matrix. Now
consider an eigenpair $(v, \lambda)$ of a stochastic matrix $S$ and suppose
$|\lambda| > 1$. Then, the vector given by $|\lambda|^n v = S^n v$ grows
exponentially, so at least one $(S^n)_{ij} > 1$ for some $n$. This, however,
contradicts the stochasticity of $S$. Thus, we conclude that $|\lambda| \leq 1$,
and since we have already 1 as an eigenvalue, we conclude the spectral radius
$\rho(S)=1$. This result can be proved directly from the Gershgorin circle
theorem, but we will not explore that here.

\subsection{Transition Probability}
As noted above, we can view $S_{ij}$ as the probability that a random surfer on
a Web jumps from page $p_i$ to page $p_j$ at a given time. Thus, we can model
our spectral ranking problem as a Markov chain. In this conversation, we
consider homogeneous Markov chains -- those that have transition probabilities
independent of time. This probabalistic insight leads us to view $S$ as the
1-transition matrix of a finite-state, discrete-time Markov chain. Recall that a
discrete-time stochastic process $\{X_t\}_{t \in T}$ for a countable set $T$
over a finite $n$-sized state space $\Omega = \{s_i\}_{i\leq n}$ is a Markov chain if, for
each $t$, the Markov property holds. Markov chains are colloquially described by
\textit{memorylessness}, which the Markov property formalizes:
\begin{equation*}
  \prob(X_{t+1} = s_j \mid X_t = s_{i_t}, X_{t-1} = s_{i_{t-1}},\cdots, X_0 =
s_{i_0}) = \prob(X_{t+1} = s_j \mid X_t=s_{i_{t}}).
\end{equation*}
We encode the $k$-step transition probabilities for a homogeneous Markov chain
in a stochastic matrix of dimension $|\Omega| \times |\Omega|$:
\begin{equation}
  \iterate{P}{k} = [ \iterate{p}{k}_{ij} ] = [ \prob(X_{t+k} = s_j \mid X_t =
s_i) ] = [ \prob(X_{t} = s_j \mid X_0 = s_i) ].
\end{equation}
Note how the memorylessness of the Markov chain enables the powerful final
equality above. By the definition of the transition matrix, we see
$ [ \iterate{p}{1}_{ij} ] = P$. Now, suppose inductively that
$\iterate{p}{\ell}_{ij} = \parens{\iterate{P}{\ell}}_{ij}$ for all
$i, j \in \Omega$. Then, observe the following:
\begin{align*}
  \iterate{p}{1+\ell}_{ij} &= \prob(X_{\ell+1} = s_j \mid X_0 = s_i) \\
                           &= \sum_{s_k\in\Omega}{\prob(X_1 = s_k \mid X_0 = s_i) \prob(X_{1+\ell} = s_j \mid
                             X_1 = s_k)} && \text{(total probability)} \\
                           &= \sum_{s_k\in\Omega}{\prob(X_1 = s_k \mid X_0 = s_i) \prob(X_{\ell} = s_j \mid X_0 = s_k)} &&
                                                                                \text{(homogeneity)} \\
                           &= \sum_{k\leq
                             |\Omega|}{\parens{\iterate{p}{1}_{ik}\iterate{p}{\ell}_{kj}}}
                           && \text{(definition of transition matrix)}\\
                           &= p^{1+\ell}_{ij}. && \text{(matrix multiplication)}
\end{align*}
From this result, we derive the Chapman-Kolmogorov equations:
\begin{equation}
  \iterate{p}{m+n}_{ij} = \sum_{k\leq |\Omega|}{\parens{\iterate{p}{n}_{ik} \iterate{p}{m}_{kj}}}.
\end{equation}

We will denote the $k$-step transition probability from state $s_i$ to $s_j$ by
directly referencing the components of the transition matrix. Since Markov
chains are memoryless, we simply take each successive iteration as if it were
the first; in this way, we understand that iteration of transitions is given by
matrix multiplication in a homogenous chain. Recall that $k$-paths of graphs are
also given by the $k^\text{th}$ power of the adjacency matrix.

With this in mind, consider a distribution $\iterate{v}{\ell}$ over $\Omega$,
defined by $\iterate{v}{\ell}_i = \prob(X_\ell = i)$. Then, by the total law of
probability, we calculate the following:
\begin{align*}
  \iterate{v}{\ell}_i &= \sum_{s_k\in\Omega}{\prob(X_0 = s_k) \prob(x_\ell = s_i \mid X_0 = s_k)} \\
                      &= \sum_{k\leq |\Omega|}{\parens{\iterate{v}{0}_k \iterate{p}{\ell}_{ki}}}.
\end{align*}

From these results, we conclude that
$\iterate{v}{\ell} = \iterate{v}{0} \iterate{P}{\ell} = \iterate{v}{0} P^\ell$,
for some initial iterate $\iterate{v}{0}$. We have now more clearly justified
the construction of equation \eqref{eqn:pi_t}.

\subsection{Stationarity}
We may take the stochastic process $\{X_t\}_{t \in T}$ underlying a Markov chain
as a categorical distribution over $\mathcal{S}$, the state space.\footnote{The
  categorical (multinoulli) distribution is a generalization of the Bernoulli
  distribution, where each random variable may take one of $k$ states.} A chain
distribution $\pi$ gives a simplex of dimension $|S| - 1$. Choose such a
$P$-invariant simplex. By Brouwer's fixed point theorem, within this simplex
must be a $P$-invariant distribution, i.e., a
$P$-eigenvector.\footnote{Brower's fixed point theorem leads to an elegeant
  argument for Perron's theorem of positive matrices. Michael Brin -- Sergey's
  father -- first advanced such topological arguments for Perron-Frobenius
  theory in 1993. We encourage the reader to consult
  \cite{boyleBasicPerronFrobeniusTheory} for more details. \vspace{0.5em} \\
  Note further that the simplexical argument for the stationary distribution is
  interesting but not at all constructive. We can, in fact, construct an exact
  definition of $\pi$ from the expected number of visits to each state in a
  positive recurrent chain. See \cite{freedmanConvergenceTheoremFinite} for an
  excellent summary of these probabalistic arguments. In this paper, we take a
  linear-algebraic approach to convergence via the Perron-Frobenius theorem. In
  hindsight, we see that our route of exposition might have been the messier
  option. However, since Google determines PageRank through power iteration on
  the transition matrix, our conversation shows more clearly how PageRank works
  in the real world.} We denote this fixed distribution $\pi$ such that
$\pi P = \pi$. That is, $\pi_i = \prob(X_t = s_i)$ for all $t\in T$ over all
states. This time independence defines the stationarity of a distribution $\pi$,
and it thus gives an eigenequation for spectral analysis. Consider a stationary
distribution $\pi$ that is also a limiting distribution, so that we have the
following:
\begin{equation*}
  \pi = \lim_{k\to\infty}{{\iterate{\pi}{k}}} = \lim_{k\to\infty}{\parens{{\iterate{\pi}{0}} P^k}}.
\end{equation*}

In a Markov chain with a limiting distribution, any initial distribution
$\iterate{\pi}{0}$ will eventually converge upon $\pi$. However, a stationary
distribution does not itself imply a limiting distribution. For instance,
consider the Markov chain given by
$P=\begin{bsmallmatrix} 0 & 1 \\ 1 & 0 \end{bsmallmatrix}$. This standard
example corresponds to turning a coin over, so the opposite state is reached
with certainty at each step. However, the probability of revealing a certain
face at a given step depends entirely upon the initial state -- what face of the
coin was initially facing up. Thus, this chain does not have a limiting
distribution. However, we have an easy stationary distribution:
$v = [\frac{1}{2}, \frac{1}{2}]$, corresponding to choosing the initial state by
flipping a fair coin. The stationary distribution hides the chain's dependence
upon its initial state. However, if we have a biased coin, say one for which
$\iterate{v}{0} = [\frac{3}{5}, \frac{4}{5}]$, the transition probabilities at
each turn of the coin clearly depend upon the initial state of the coin.

We can also show that no limiting distribution exists by examining the
transition matrix. As $k\to\infty$, the iterate $\iterate{P}{k}$ swaps certainty
of transition between states; it alternates between $P$ and $I$. Thus, the limit
of the transition matrix does not exist, and so a limiting distribution does not
exist.

The nature of our spectral ranking problem thus becomes stunningly clear, for
the time-independent probabilities within $\pi$ yield a unique metric of the
importance we have sought to codify. We can predict, from the structure of the
Web itself, how much time a random Web surfer will spend on a given page, and we
can take these probabilities as evidence for how important a page actually is.
Analyzing the asymptopic behavior of $P$ will show us when such a limiting
distribution exists, and we will thus know when we have a unique solution to the
stationary eigenequation. We now answer the vital question of when a Markov
chain has such distribution by exploring a fundamental theory of dynamical
systems.

\section{Perron-Frobenius Theory}

\subsection{Irreducibility}
In a Markov chain, states $s_i$ and $s_j$ communicate if
$\iterate{P}{k}_{ij} > 0$, for some iterate $k$. That is, there is a path of
nonzero probability (a random walk) between states $s_i$ and $s_j$.
Communicativity is easily shown to be an equivalence relation.

Consider partitioning a Markov state space $\Omega$ of size $n$ into partitions
$\Omega_1 = \{ s_i \}_{i\leq k}$ and $\Omega_2 = \{ s_j \}_{k < j \leq n}$.
Moreover, suppose there are random walks from $\Omega_2$ to $\Omega_1$ but none
the other way. Note that the chain eventually never visits states in $\Omega_2$;
when $\Omega_1$ is a communicating class, the chain reduces to the states of
$\Omega_1$. We model such reducibility of a Markov chain in a lower-triangular
canonical form:
\begin{equation*}
  \label{eqn:reducible}
  P =
  \begin{bmatrix}
    \omega_{1\to 1} & 0 \\
    \omega_{2 \to 1} & \omega_{2 \to 2}
  \end{bmatrix},
\end{equation*}
where $\omega_{1\to 1}$ and $\omega_{2\to 2}$ are substochastic square matrices
representing transition probabilities within $\Omega_1$ and $\Omega_2$,
respectively, and $\omega_{2 \to 1}$ holds the transition probabilities from
$\Omega_2$ to $\Omega_1$. Note that when $\omega_{2\to 1} = 0$, the chain is
disconnected and so \textit{completely} reducible.

We say a Markov chain is irreducible if and only if it contains a single
communicating class. Equivalently, a digraph is irreducible when it has no
strongly connected components (SCCs) smaller than itself.\footnote{Note that a
  topological sort on a reducible graph can factorize it into its disjoint
  strongly-connected components.} An irreducible matrix is a matrix
corresponding to such irreducible objects. Note that an irreducible matrix is
nonnegative; the transition matrix motivation above shows the classes of
reducibility and irreducibility arise through the arrangement of zeroes within
the matrix. To find reducbility in a chain-graph, we simply attempt to permute
the state-vertices in the underlying matrix to obtain the above canonical form.

Consider further what reducibility means in a Markov chain. As time passes, the
probability of visiting the pages in $\Omega_2$ drops to zero. To our iterative
importance calculation on Web pages, these pages will eventually appear not at
all important, but there is no reason why they shouldn't be. We must not permit
this artifact of the Web structure to corrupt our importance scores. For now, we
take reducibility of a Markov chain as a hypothetical difficulty; later, we will
see that the real Web is quite reducible and requires a less naïve model to give
convergent and unique importance scores.

\subsection{Periodicity}
Consider a Markov chain with states $s_i \in \mathcal{S}$. Let
$D(s_i) = \{n\in \mathbb{Z}_{\geq 0} : \iterate{p}{n}_{ii} > 0 \} \subset
\mathbb{N}$. Note that $D(s_i)$ is trivially nonempty, as
$\iterate{p}{0}_{ii} = 1$ for all $s_i$. We observe that $D(s_i)$ forms a
numerical semigroup under addition. We define the period of a state by
$d(s_i)=\text{gcd}\{D(s_i)\}$. In other words, a random walk (a path where each
step has nonzero probability) from state $s_i$ can only return to $s_i$ in
multiples of $d(s_i)$ steps. Consider states $s_i$ and $s_j$ in the same
communicating class. Thus, $\iterate{p}{m}_{ij} > 0$ and
$\iterate{p}{n}_{ji} > 0$ for some nonnegative iterators. Otherwise stated,
consider a random $m$-path from $s_i$ to $s_j$ and a a random $n$-path from
$s_j$ to $s_i$.
\begin{figure}[h]
  \centering \scalebox{0.75}{\begin{tikzpicture}[
			scale=0.6,
      > = stealth,
			shorten > = 1pt,
			auto,
			node distance = 3cm,
			semithick]

			\tikzstyle{every state}=[
			draw = black,
			thick,
			fill = white,
			minimum size = 4mm]

			\node[state] (a) {$s_i$};
			\node[state] (b) [right of=a] {$s_j$};

			\path[->] (a) edge [bend left=45] node {$m$-step} (b)
      (a) edge [loop above]   node {$\ell$-step} (a)
      (b) edge [bend left=45] node {$n$-step} (a);
		\end{tikzpicture}}
  \caption{Communicating States}
  \label{fig:markov}
\end{figure}

Figure \ref{fig:markov} illustrates these cycles. Note that there are random
cycles of length $\ell \equiv m+n \pmod{d(s_i)}$ on state $s_i$. The
communicativity of $s_i$ and $s_j$ implies $\iterate{p}{n+m}_{jj} > 0$ and
$\iterate{p}{n + \ell + m}_{jj} > 0$. That is, there is a random $\ell$-cycle on
$s_j$; moreover, there is a random cycle on $s_j$ that goes from $s_j$ to $s_i$
and returns to $s_i$ before terminating at $s_j$. Thus, $d(s_j) \mid \ell$.
However, since we have shown that $s_i$ has a random $\ell$-cycle, we know that
$d(s_j) \mid d(s_i)$. By the symmetry of $s_i$ and $s_j$, we find
$d(s_i) \mid d(s_j)$, and so $d(s_i) = d(s_j)$ whenever $s_i$ and $s_j$
communicate. Thus, when $P$ is irreducible, $d(P)$ is well-defined; every state
has the same period. When $d(P) = 1$, we say the Markov chain is aperiodic.

\subsection{Primitivity}
Much has been said on the conditional convergence of periodic Markov chains, but
we confine this conversation to the aperiodic case. Suppose we have an aperiodic
irreducible Markov chain, and recall $D(s_i)$, the numerical semigroup given
above.\footnote{Markovian semigroups form an important part of ergodic theory,
  which we barely explore here.} For a semigroup $V$ with $d=\gcd(V)$, it can be
shown that there exists an integer $N$ such that $v\equiv n \pmod{d}$ for any
$v\in V$ and all $n \geq N$. Since we have supposed an aperiodic Markov chain,
we have a coprime numerical semigroup. Thus, there is some bound $t_i$ beyond
which $\iterate{p}{t}_{ii} > 0$ for all $t \geq t_i$ over all states. By the
irreducibility of the chain, we have a $k = k_{(i,j)}$ for which
$\iterate{p}{k}_{ij} > 0$. Thus, for $t \geq t_i + k$, the Chapman-Kolmogorov
equations yield the following:
\begin{align*}
  \iterate{p}{t}_{ij} \geq \iterate{p}{t-k}_{ii} \iterate{P}{k}_{ij} > 0.
\end{align*}
Let $t'_i = t_i + \max_{j\in\Omega}\{ k_{(i,j)} \}$. Thus, for $t \geq t'_i$, we
have $\iterate{p}{t}_{xy} > 0$ for all $y\in\Omega$. Then, for
$t \geq \max_{i\in\Omega}\{ t'_i \}$, we conclude that, for an irreducible
aperiodic chain, there is a $t$ that gives $\iterate{p}{t}_{ij} > 0$ over all
states, and so $P^t > 0$. We take this result as the definition of matrix
primitivity.

Primitivity is irreducibility plus aperiodicity. Recall that a merely
irreducible Markov chain requires an iterate-function of state pairs. However, a
primitive chain guarantees that \textit{all} states communicate -- not just that
they are a part of the same communicating class -- after $t$ transitions.

Consider a primitive Markov chain with eigenpair $(v, \lambda)$. The underlying
primitivity of the transition matrix $P$ gives $P^k v = \lambda^k v > 0$ for
some iterate $k$ and forever thereafter, so we see the eigenpair must be
positive. Take another eigenpair $(w, \gamma)$ such that $\norm{w} < \norm{v}$
and $\gamma = \rho(P)$. Then for all iterates $n$, we have the following:
\begin{equation*}
  \gamma^n w = P^n w \leq P^n v = \lambda^n v.
\end{equation*}
However, since we supposed $\norm{w} < \norm{v}$, we cannot have
$\beta < \lambda$. By the inequality above, we must conclude $\beta = \lambda$,
and so a primitive $P$ has only one eigenvalue on its spectral circle, namely
the spectral radius itself. This eigenvalue is the dominant eigenvalue of $P$.
Since all stochastic matrices have spectral radius 1, we conclude that primitive
Markov chains have an eigenpair $(1, \pi)$. This is exactly what we want to
satisfy our limiting distribution condition $\pi = \pi P$. The following
spectral properties of primitive Markov chains should not be terribly surprising
by now, but they are are quite beautiful.

\subsection{Perron-Frobenius Theorem}
Perron's theorem, proved in 1907, provided spectral properties of positive
matrices -- most notably, that such matrices have only one eigenvalue on their
spectral circle, the dominant eigenvalue $\lambda_1=\rho(A)$. However, Perron's
results results did not generally apply to nonnegative matrices. Frobenius
subsequently realized that irreducible matrices behave like positive matrices,
except that they may have multiple eigenvalues that are roots of unity of
$d(P)$. Primitive matrices, however, do have a dominant eigenvalue, as we have
seen above. The Perron-Frobenius theorem, proved in 1912, gives the following
properties for a primitive square matrix:
\begin{enumerate}
\item The spectral radius $\rho$ equals some real eigenvalue $\lambda_1$;
\item This eigenvalue (the Perron root) is simple, and it is the only eigenvalue on the spectral circle;
\item The Perron root has unique, strictly positive left and right eigenvectors (Perron vectors);
\item These left and right Perron eigenvectors are, up to normalization, the only nonnegative eigenvectors of $A$.
\end{enumerate}
Note that the Perron root is a dominant eigenvalue; it is the unique $\lambda_1$
such that $|\lambda_1|>|\lambda_2|\geq\cdots\geq |\lambda_{n}|$ for all
$\lambda \in \sigma(A)$. Recall that $A$ and $\transpose{A}$ share a spectrum,
and a left eigenvector of $A$ is a right eigenvector of $\transpose{A}$.

We will not prove the complete Perron-Frobenius theorem in this paper -- its
full statement contains far more than we have given here -- although we have
heretofore outlined some basic arguments for the properties we find useful for
spectral ranking.

\section{Limiting Distributions}

\subsection{Spectral Resolution\protect\footnote{The results in this section
    summarize the extended treatment of matrix functions and convergence given
    in \cite{meyerMatrixAnalysisApplied2000}}.}
Recall that, for a a diagonalizable matrix $A=PD\inverse{P}$, the columns of the
similarity transformation $P$ correspond to distinct right eigenvectors of $A$.
Consider partitioning $P$ into bases $\beta_{\mathcal{E}_\lambda}$ for distinct
right eigenspaces of $A$. Similarly, we partition the rows of $\inverse{P}$ into
bases $\gamma_{\mathcal{E}_\lambda}$ for distinct left eigenspaces. Thus, for a
matrix $A$ with with spectrum $\sigma$, we have an outer product formulation of
the spectral decomposition:
\begin{equation}
  \label{eqn:spectral_decomposition}
  A = PD\inverse{P} = \sum_{\lambda\in\sigma}{\lambda [\beta_{\mathcal{E}_\lambda}]\transpose{[\gamma_{\mathcal{E}_\lambda}]}}.
\end{equation}

Note that
$E_\lambda =
[\beta_{\mathcal{E}_\lambda}]\transpose{[\gamma_{\mathcal{E}_\lambda}]}$
projects onto the eigenspace $\mathcal{E}_\lambda$. In fact, $A$ is
diagonalizable if and only if it can be written as a sum of such spectral
projectors. Consider $x$ and $\transpose{y}$ as the handed eigenvectors of a
simple eigenvalue $\lambda_s$. We see this elegant formula for the corresponding
eigenprojection:
\begin{equation}
  \label{eqn:simple_eigenprojection}
  E_{\lambda_s} = \frac{x \transpose{y}}{\transpose{y} x},
\end{equation}
where the inner product in the denominator comes from normalization to ensure projection properties.

\subsection{Matrix Functions}
Consider generalizing a scalar function $f$ to be a matrix function. As the
well-known matrix exponential illustrates, we often do not define matrix
functions as scalar functions applied entrywise, for we then might lose the nice
properties associated with the scalar functions. Using the Taylor expansion of a
sufficiently-differentiable $f$, we first define such a function about a Jordan
block $J_k(\lambda)$:
\begin{equation*}
  f\big( J_k(\lambda) \big)=f(\lambda)I + f'(\lambda)(J_k(\lambda) - \lambda I) + \frac{f'(\lambda)(J_k(\lambda) - \lambda I)^2}{2!} + \cdots.
\end{equation*}
However, since $(J_k(\lambda) - \lambda I)$ is nilpotent of order $k$, the
Taylor expansion reduces nicely:
\begin{equation}
  \label{eqn:f_jordan_block}
  f\big( J_k(\lambda) \big) = \sum_{i=0}^{k - 1}\parens{\frac{f^{[i]}(\lambda)}{i!} (J_k(\lambda) - \lambda I)^i}.
\end{equation}

Thus, for $f$ with at least $(k - 1)$ derivatives that exist at $\lambda$, we
have the definition of a function applied to a Jordan block:
\begin{equation}\label{eq:f_jordan_block}
  f\big( J_k(\lambda) \big) =
  \begin{bmatrix}
    f(\lambda) & f'(\lambda) & \frac{f''(\lambda)}{2!} & \ldots & \frac{f^{[k - 1]}(\lambda)}{(k-1)!} \\
    0 & f(\lambda) & f'(\lambda) & \ldots & \frac{f^{[k - 2]}(\lambda)}{(k-2)!} \\
    0 & \vdots & \vdots & \ddots & \vdots \\
    0 & 0 & 0 & \ldots & f(\lambda)
  \end{bmatrix}.
\end{equation}

For a diagonalizable matrix $A$ as above, note that we can we apply Equation
\eqref{eqn:spectral_decomposition} to obtain a nice result:
\begin{align*}
  f(A) = Pf(D)\inverse{P} = \sum_{\lambda \in \sigma}{f(\lambda) E_\lambda}.
\end{align*}

We can extend $f$ to a Jordan form $J$ by requiring that $f$ has at least
$(\iota - 1)$ derivatives, where $\iota = \iota_\lambda$ is the index of
$\lambda$, i.e., the size of the largest $\lambda$-block. Then, for $A$ with
spectrum $\sigma$, not necessarily diagonalizable, we have this function
definition:
\begin{equation*}
  f(A) = P f(J) \inverse{P} = \sum_{\lambda \in \sigma}\parens{ [\beta_{\mathcal{E}_\lambda}]\, f\big( J_{k_\lambda}(\lambda) \big)\, \transpose{[\gamma_{\mathcal{E}_\lambda}]} }.
\end{equation*}
Expanding the definition of the function and applying the eigenprojection gives
this form:
\begin{equation}
  f(A) = \sum_{\lambda \in \sigma}\sum_{j=0}^{\iota_{\lambda} - 1}\parens{ \frac{f^{[j]}(\lambda)}{i!} (J_{k_\lambda}(\lambda) - \lambda I)^j E_{\lambda} }.
\end{equation}

\subsection{Matrix Convergence}
Consider the function $f(x) = x^n$ for some real $n$. Note that when we apply
$f$ to a Jordan block, we can easily give the coefficients of differentiation as
binomial coefficients.
\begin{equation}\label{eq:power_jordan_block}
  \big( J_k(\lambda) \big)^n =
  \begin{bmatrix}
    \lambda^n & \binom{n}{1}(\lambda^{n-1}) & \binom{n}{2}(\lambda^{n-2}) & \ldots & \binom{n}{k-1}(\lambda^{n-(k+1)}) \\
    0 & \lambda^n & \binom{n}{1}(\lambda^{m-1}) & \ldots & \binom{n}{k-2}(\lambda^{m-(k+2)}) \\
    0 & \vdots & \vdots & \ddots & \vdots \\
    0 & 0 & 0 & \ldots & \lambda^n
  \end{bmatrix}.
\end{equation}
In general, then, we define $f(A)$ thusly:
\begin{equation}
  A^n = \sum_{\lambda \in \sigma}\sum_{j=0}^{\iota_{\lambda} - 1}\parens{ \binom{n}{j} \lambda^{n-j} (J_{k_\lambda}(\lambda) - \lambda I)^j E_{\lambda} }.
\end{equation}

Suppose $| \lambda | < 1$. Then, for each $j \in [1, n-(k+1)]$, we have
\begin{equation*}
  \abs*{\binom{n}{j}\lambda^{m-j}} \leq \frac{k^j}{j!}{\abs*{\lambda^{m-j}}}.
\end{equation*}
Note that as $m \to \infty$, the numerator $k^j$ diverges polynomially since $j$
is fixed. However, since $| \lambda | < 1$, $\abs{\lambda^{m-j}}$ converges to
zero exponentially. Thus, we conclude the following:
\begin{equation*}
  \lim_{n\to\infty}{\big( J_k(\lambda) \big)^n} = 0.
\end{equation*}
We can extend this argument to show that, for a square matrix $A$,
$\lim_{n\to\infty}{\parens{A^n}} = 0$ if and only if $\rho(A) < 1$, and similar
reasoning shows the limit does not exist when $\rho(A) > 1$.

We now study the convergence of a matrix with $\rho(A) = 1$, such as we have
with all stochastic (and primitive) matrices. Suppose we have a primitive Markov
chain with transition matrix $P$. By the Perron-Frobenius theorem, we know $P$
has dominant eigenvalue $\lambda_1 = 1$. As all non-dominant eigenvalues have
modulus less than 1, we simply have the following:
\begin{equation*}
  \lim_{n\to\infty}{A^n} = E_{\lambda_1}.
\end{equation*}
Powers of a primitive matrix will eventually converge upon the dominant
eigenprojector. For primitive Markov chains, we have a left eigenvector $\pi$,
the stationary distribution. Moreover, we previously showed that all stochastic
matrices have right eigenvector $\mathbbm{1}$. Thus, using the limit we just
discovered, we have a limiting distribution:
\begin{equation}
  \lim_{k\to\infty}{{\iterate{\pi}{k}}} = \lim_{k\to\infty}{\parens{{\iterate{\pi}{0}} P^k}} = \iterate{\pi}{0} \parens{\frac{\mathbbm{1} \pi}{\pi \mathbbm{1}}} = \parens{\iterate{\pi}{0} \mathbbm{1}} \pi = \pi.
\end{equation}
Thus, primitivity guarantees a distribution completely independent of the
chain's starting state. The convergence of a primitive chain is a rather
remarkable result, one often called the fundamental theorem of Markov chains.

\subsection{Power Iteration}
Applying Perron-Frobenius theory to primitive Markov chains gives a classical
method for calculating the limiting distribution. Techniques based on Gaussian
elimination would not be a smart choice for calculating a limiting distribution
$\pi$ of the Internet, since this family of alogrithms generally has arithmetic
complexity of order $O(n^3)$.

Here, we explore the power method -- the simplest iterative calculation method.
Suppose a square matrix $A$ has $n$ eigenpairs $\{ (\lambda_i,x_i) \}_{i\leq n}$
with $\lambda_1$ as a dominant eigenvalue. Consider a starting vector
$x^{(0)}=\sum_i{\gamma_i x_i}$ for scalars $\gamma_i$. Then, we again consider
an iterative calculation:
\begin{equation}
  \iterate{x}{k} = A^k \iterate{x}{0} = \sum_{i=1}^{k}{\gamma_i \lambda_i^k x_i} = \lambda_1^k\parens{\gamma_1 x_1 + \sum_{i=2}^{k}{\gamma_i \frac{\lambda_i}{\lambda_1} x_i}} \to \lambda_1^k \gamma_1 x_1.
\end{equation}
Since $\lambda_i<\lambda_1$ for $i>1$, the sum approaches 0 as $k\to\infty$. The
speed of convergence clearly depends upon the subdominant eigenvalue
$\lambda_2$. Power method applications often normalize the the estimated
eigenvector after each iteration to avoid underflow or overflow under successive
powers of the eigenvalue, but we need not worry about this since primitive
Markov chains have dominant eigenvalue $\lambda_1 = 1$.

\section{The Google Matrix}
Deriving a limiting distribution from the Web graph still requires a bit more
work, for the real Web is quite reducible.
\begin{figure}
  \centering
  \includegraphics[scale=0.6]{bow-tie}
  \caption{The Bow Tie \cite{broderGraphStructureWeb}}
  \label{fig:bow-tie}
\end{figure}
In 1999, \cite{broderGraphStructureWeb} crawled over 1.5 billion links to
generate a comprehensive map of the Web. Their macroscopic figure, given in
Figure \ref{fig:bow-tie}, stylizes the Web super-graph as a rather eldritch bow
tie. Note that the figure has the form of a directed acyclic graph given from a
topological sort. In the middle is the ``bow knot'' -- the largest
strongly-connected component (SCC) of the Web. In 2012, when the last public Web
mapping occured, about half of mapped Web pages were strongly connected. The
``thistle ends'' of the tie, denoted \texttt{IN} and \texttt{OUT}, respectively
reference pages from which surfers can link into (out of) the SCC but cannot
return by the same route. Pages within the tendrils and tubes are likewise
unidirectional on this macroscopic scale: They be reached from \texttt{IN} and
\texttt{OUT}, but these components cannot be reached from within them. Although
the numbers in Figure \ref{fig:bow-tie} are obsolete, its bow-tie model wtill
describes the Web well.

We previously eliminated individual dangling pages, but we could view these
entire tendrils and tubes again as ``dangling pages,'' and we can again correct
for them. In our Markov discussion, we interpreted the stochastic Web matrix $S$
as the matrix of probabilities that an ideal random surfer follows links from a
page or chooses another randomly when it hits a dead end. However,
\textit{human} Web surfers often get bored. They will not follow a linear path
through the Web; they might suddenly elect to teleport -- shift course
altogether and visit a page on whim. Let $\mathbbm{1}$ represent a vector of
ones, and again consider a distribution $u$ over all Web pages. We model this
teleportation behavior with a rank-one matrix $T = \mathbbm{1}\transpose{u}$.

Now, consider a matrix $G$ formed from a convex combination of $S$ and $T$:
\begin{equation}
  \label{eqn:google_matrix}
  G = \alpha S + (1-\alpha)T.
\end{equation}
Every page now communicates with every other page, and our process is still
stochastic. In fact, $G$ is trivially primitive, as each page has a random loop.
In this model, the random surfer on page $p_i$ follows the Web's link structure
about $\alpha s_i$ percent of the time and teleports with probability
$(1-\alpha) u_j$. Note that the teleportation matrix need not give equal
deference to each Web page. External factors -- a user's search preferences or
browsing habits or a search engine's fiat against suspected abusers -- can
influence the teleportation probabilities. Here, however, we take a rank-one
distribution. The teleportation factor $\alpha$ dampens the wild, raw Web graph
into a more friendly model; it is the damping factor.

Note that the teleportation adjustment provides all we need for a guaranteed
limiting distribution, from which we can directly rank pages. In sum, $G$ simply
represents a lazy random walk on the Web graph $\mathcal{W}$. The matrix $G$ is
the Google matrix -- the original foundation of PageRank calculations.

Although the primitive Google matrix is completely dense, we can expand $S$ and
$T$ into their constituent distributions $v$ and $u$ to obtain a power
iteration:
\begin{align*}
  \iterate{\pi}{k+1} &= \iterate{\pi}{k} G
                     = \alpha \iterate{\pi}{k} (H + d \transpose{v}) + (1 - \alpha) \iterate{\pi}{k} (\mathbbm{1} \transpose{u}).
\end{align*}

Note that the rank-one reduction given above does not require us to actually
generate the dense component matrices $S$ and $G$. Unlike decompositional
methods (e.g. UR factorization and GMRES), we do not ruin the sparsity of $H$.
As we see above, we must only store the current iterate $\iterate{\pi}{k}$ and
the dangling page vector $d$ to apply the power method; we do not need $O(n^2)$
matrix multiplication. We need only perform vector-matrix multiplication on the
very sparse $H$, and sparse multiplications have approximate complexity
$O(\eta(H))$, where $\eta(H)$ is the number of nonzero elements in $H$.
Empirical data shows the average Web page has about 10 outgoing links, so
$\eta(H) \approx 10n$.

When we apply the power method to a primitive matrix, we have an
essentially linear algorithm. The power method beats more complex algorithms
here because it gives only the simple result we want: the dominant eigenvalue of
a primitive matrix. However, the convergence of a primitive Markov chain depends
upon how quickly powers of its subdominant eigenvalue converge to zero. We now
examine this eigenvalue more closely, and also investigate an optimum value for
$\alpha$.

\section{Network Topology}
\begin{figure}
  \centering
  \includegraphics[scale=1.8]{scale-free-network}
  \caption{A Scale-Free Network (Yeast Proteome) \cite{wangKineticConformationalCharacterization}}
  \label{fig:scale-free}
\end{figure}

\subsection{Near-Reducibility}
Consider a Markov chain given by
$P=\begin{bsmallmatrix} 1 & 0 \\ 0 & 1 \end{bsmallmatrix}$. The chain is fully
reducible, and equivalently, the underlying graph is disconnected. Clearly, $P$
will only have eigenvalue 1 with multiplicity 2. Suppose now we have a chain
with
$P'=\begin{bsmallmatrix} 1 - \epsilon_1 & \epsilon_1 \\ \epsilon_2 & 1 -
  \epsilon_2 \end{bsmallmatrix}$, for $\epsilon \in (1,0)$. Thus, the spectrum
$\sigma(P') =P' \{ 1, 1-\epsilon_1 - \epsilon_2 \}$.

Nearly-reducible Markov chains have strong interactions within their clusters,
but relatively weak interactions between clusters.\footnote{The formal study of
  clustering in Markov chains is a well-developed field much more clearly
  understood through the probabilistic convergence theorem, which we have
  eschewed in this paper for a linear-algebraic focus. Powerful analysis of
  graph clustering and subdominant eigenvalues can also be done through the
  graph Laplacian, but we will not expound that tool here.} Considering the
canonical form of reducibility in Equation \eqref{eqn:reducible}, we note that
an irreducible Markov chain with subdominant eigenvalue $\lambda_2 \approx 1$
will be nearly reducible. However, the existence of such a subdominant
eigenvalue does not itself ensure near-reducibility.

\subsection{Scale-Free Networks}
In 1959, Erdős and Rényi proposed generating random graphs by considering a
Poisson distribution over the out-degree of vertices. In such a random graph
(also called an exponential network), most nodes share a similar number of
out-links, and there are very few nodes that deviate from this mean. A random
network is thus very democratic and has a steep bell curve. Most natural
networks, however, do not follow a random structure.
\begin{figure}
  \centering
  \includegraphics[scale=0.15]{random-graph}
  \caption{A Random Graph \cite{RandomGeometricGraph}}
  \label{fig:random}
\end{figure}
Consider Figure \ref{fig:scale-free}, which maps the partial proteome of a
yeast. The red nodes, which denote proteins most essential for life, tend to
bind together clusters of proteins of lesser importance. Note our use of the
term \textit{importance} here. If we only had the proteome graph, sans color, we
would likely be able to determine which proteins were most ``important'' for
life by computing a spectral rank like the PageRank. Indeed, we have assumed
from the start that the most connected nodes are the most important nodes. In
fact, HITS -- another spectral rank developed at IBM contemporaneously with
PageRank -- makes the pursuit more explicit by splitting Web pages into hubs and
authorities; it uses the more complex heuristic that good hubs link to good
authorities and vice versa. In a random graph, however, such a definition of
importance breaks down.

A music theorist analyzes a piece of classical music by first determining what
chords are most important -- that is, what chords are most tonally connected to
other chords. However, an atonal piece by Schönberg defies such straightforward
analysis. A twelve-tone series ensures an equal distribution of tones so no few
tones or chords distinguish themselves as tonally important, just as a random
graph ensures no few nodes stand out.\footnote{Note that this does not mean
  atonal music is ``random" in the standard sense; just like an Erdős-Rényi
  random graph, an atonal piece provides little information about the
  connections between tones. Thus, from a tonal perspective, we cannot easily
  determine which chords are more important than others. Tonality concerns
  itself about relations between chords, and these relations are evenly
  distributed by design in a twelve-tone piece. The progressions in such a piece
  mostly sound alike to our tonal ears, except for a few serendipitous ones that
  give a fleeting sense of tonality. This perhaps gives
  insight into why Schönberg abhored the label \textit{atonal} and much
  preferred to call his music \textit{pan-tonal}.}

Networks such as these follow a power law. The probability that a node has $k$
outlinks is proportional to $1/k^\gamma$, for some power $\gamma$. Although most
nodes have only a few links to others, a few have thousands. Removing just these
most important nodes (the hubs) can catastrophically fragment the network into
its isolated components disrupt the network. These networks also reveal a
beautiful fractal structure, so we we call them scale-free networks.
Sociologists have long known of these networks. The Erdős number and the ``six
degrees of Kevin Bacon'' work because a few people are highly connected; they
bridge groups of people who would otherwise be unrelated. Income distribution in
Western countries can be similarly modeled with a scale-free network. Note that
scale-free networks will also be very reducible, as they have strong
interactions within their clusters but relatively weak interactions among them.

Both the physical and logical Internet have been shown to be scale-free
networks. The Internet is thus highly resilient to random attacks, but
destroying just about 3\% of hubs -- the most highly connected ones -- would be
enough to fragment the Internet into dozens of isolated components
\cite{barabasiNetworkScienceScaleFree}. This suggests that even the large SCC of
Figure \ref{fig:web} is vulnerable to targeted attacks. By contrast, the
strongly-connected components of a random graph would be quite strong, as a
random graph does not encode much information in unique locations.

As we have seen, only scale-free networks have a sufficiently heterogeneous,
information-rich structure to permit a meaningful spectral importance rank.
Thus, Markov chains over scale-free networks will have a subdominant eigenvalue
quite close to $\lambda_1 = 1$. Although the teleportation adjustment ensured
the primitivity of the Google matrix, it still tends to leave the dampened Web
only weakly irreducible -- that is, nearly reducible.

\subsection{Damping Factor}
Recall $\alpha$, the damping factor used in Equation \eqref{eqn:google_matrix}.
This probability $\alpha$ controls the fidelity of the model to the raw Web
structure. As $\alpha \to 1$, the Google matrix becomes more nearly reducible,
its subdominant eigenvalue $\lambda \to 1$, and the power method converges ever
more slowly. Moreover, as $\alpha \to 1$, the limiting distribution becomes far
more sensitive -- both in terms of importance score and the consequent PageRanks
-- to link changes even within minor clusters; further approaching 1 magnifies
these effects. We can explore these results more by analyzing the vector
derivative $\dee\pi(\alpha)/\dee\alpha$, but we will not do so here.

We learn in \cite{haveliwalaSecondEigenvalueGoogle} that a stochastic matrix $S$
with $\sigma_S = \{ 1, \lambda_2, \lambda_3, \cdots, \lambda_n \}$ gives a
Google matrix with
$\sigma_G = \alpha\sigma_S = \{ \alpha, \alpha\lambda_2, \alpha\lambda_3,
\cdots, \alpha\lambda_n \}$. From this, we see $\lambda_2 \leq \alpha$ for the
Google matrix. Thus, the power method converges at the rate which
$\alpha^k \to 0$ as $k$ grows. With $\alpha = 0.85$, Google's formulation of the
power method converges within a thousandth by approximately 50 iterations. Note
that the power method scales well to immense datasets like the Web graph -- it
requires at most 100 iterations to converge upon a unique ranking all pages on
the indexed Web.

\section{Evaluation}
Since PageRank incorporates the entire Web into its calculation, it is
query-independent. The refinements given in
\cite{haveliwalaAnalyticalComparisonApproaches2003} adapt the stochasticity and
teleportation distributions to provide a personalized PageRank. Query-dependent
algorithms like HITS beat PageRank in personalized searching, as they run a
power method on small components of the Web. However, the more serious induced
deficiencies -- like more imprecise ranking and susceptibility to link spamming
-- have made PageRank the dominant spectral ranking algorithm on the Web.
Moreover, the definition of importance built into PageRank means it can predict
backlinks (citations) well; Brin and Page \cite{brinPageRankCitationRanking1998}
showed that PageRank determines an optimal path for Web crawlers, who should
index the most important parts of the Web quickly and not be trapped by
closely-linked local chains.

Google still relies heavily upon the traditional vector space relevance models
and sophisticated analysis of anchor text, font style, and capitalization.
PageRank, after all, addresses a specific problem of structural relevance; other
systems must decide the a page's contextual relevance to a query. PageRank,
however, forms the central part of Google's search strategy -- a system designed
to scale even to a Web with $10^{100}$ pages -- a \textit{googol}.

\newpage
\bibliographystyle{ieeetr}
\bibliography{references}
\end{document}