\chapter{Variables, expressions and statements}

One of the most powerful features of a programming language
is the ability to manipulate {\bf variables}.  Broadly
speaking, a variable is a name that refers to a value. It
might be more accurate to say that a variable is a container
that has a name and holds a value.
\index{variable}

\section{Assignment statements}
\label{variables}
\index{assignment!statement}
\index{statement!assignment}
\index{$=$ assignment operator}
\index{operator!assignment}

An {\bf assignment statement} uses the equal {\tt =} sign
and  gives a value to a variable, but, before you can assign
a value to a variable, you first need to
create the variable by declaring it if does not already
exist:

\begin{verbatim}
> my $message;          # variable declaration, no value yet
> $message = 'And now for something completely different';
And now for something completely different
> my $number = 42;      # variable declaration and assignment
42
> $number = 17;         # new assignment
17
> my $phi = 1.618033988;
1.618033988
>
\end{verbatim}
%
\index{assignment}
\index{statement!assignment}
\index{declaration!variable}
\index{variable!declaration}

This example makes four assignment statements.  The first
assigns a string to a new variable named {\tt \$message};
the second assigns the integer {\tt 42} to {\tt \$number}; the
third reassigns the integer 17 to {\tt \$number}; the fourth
assigns the (approximate) value of the golden ratio to {\tt \$phi}.

There are two important syntax features to understand here.

First, in Perl, variable names start with a so-called
\emph{sigil}, i.e. a special non-alphanumeric character such
as \verb'$', \verb'@', \verb'%', \verb'&' and some others.
This special character
tells us and the Perl compiler (the program that reads
the code of our program and transforms it into computer
instructions) which kind of variable it is. For example, the
\$ character indicates that the variables above all are \emph{scalar variables}, which means that they can contain only one value at any given time. We'll see later other types of variables that may contain more than one value.
\index{sigil}
\index{scalar}
\index{variable!scalar}
\index{scalar}

Second, notice that all three variables above are first introduced by the keyword {\tt my}, which is a way of declaring a new variable. Whenever you create a new variable in Perl, you need to \emph{declare} it, i.e. to tell Perl that you're going to use that new variable, and this is most commonly done  with the {\tt my} keyword, which declares a \emph{lexical} variable: we will explain later what a lexical variable is, let's just say for the time being that this enables you to make your variable local to a limited part of your code. One of the good consequences of the requirement to declare variables before you use them is that, if you accidentally make a typo when writing a variable name, the compiler will usually be able to tell you that you are using a variable which has not been declared previously and thus help you find your error. This has other far-reaching implications which we will examine later.
\index{my}
\index{declaring variables}
\index{variable!declaration}
\index{lexical}
\index{variable!lexical}

When we wrote at the beginning of this section that a variable has
to be declared before it is used (or just when it is used), it plainly
means that the declaration has to be before (or at the point of) the
variable(s first use in the text file containing the program. We will see later that
programs don't necessarily run from top to bottom in the order in which
the lines or code appear in the program file; still, the variable
declaration must be before its use in the text file containing the program.

If you neglect to declare a variable, you get a syntax error:
\index{syntax error}

\begin{verbatim}
> $number = 5;
===SORRY!=== Error while compiling <unknown file>
Variable '$number' is not declared
at <unknown file>:1
------> <BOL><HERE>$number = 5;
>
\end{verbatim}
%
Please remember that you may obtain slightly different
error messages depending on the version of Rakudo you
run. The above message was obtained in February 2016;
with a newer version (October 2016), the same error
is now displayed somewhat more cleanly as:
\begin{verbatim}
>
> $number = 5;
===SORRY!=== Error while compiling:
Variable '$number' is not declared
at line 2
------> <BOL><HERE>$number = 5;
>
\end{verbatim}

\index{state diagram}
\index{diagram!state}

A common way to represent variables on paper is to write the name with
an arrow pointing to its value.  This kind of figure is
called a {\bf state diagram} because it shows what state each of the
variables is in (think of it as the variable's state of mind).
Figure~\ref{fig.state2} shows the result of the previous example.

\begin{figure}
\centerline
{\includegraphics[scale=0.6]{figs/test_5.png}}
\caption{State diagram.}
\label{fig.state2}
\end{figure}

\section{Variable names}
\index{variable}

Programmers generally choose names for their variables that
are meaningful---they document what the variable is used for.

Variable names can be as long as you like.  They can contain
both letters and numbers, but user-defined variable names
can't begin with a number. Variable names are case-sensitive,
i.e. {\tt \$message} is not the same variable as {\tt \$Message}
or {\tt \$MESSAGE}. It is legal to use uppercase letters, but
it is conventional to use only lower case for most variables
names. Some people nonetheless like to use {\tt \$TitleCase}
for their variables or even pure {\tt \$UPPERCASE} for
some special variables.
\index{lower case}
\index{upper case}
\index{title case}
\index{case!lower}
\index{case!upper}
\index{case!title}

\index{Unicode}
\index{ASCII}
Unlike most other programming languages, Perl~6 does not require letters
and digits used in variable names to be plain ASCII.
You can use all kinds of Unicode letters, i.e.
letters from almost any language in the world, so that, for example,
{\tt \$brücke}, {\tt \$payé} or {\tt \$niño} are
valid variable names, which can be useful for non-English
programmers (provided that these Unicode characters are
handled correctly by your text editor and your
screen configuration). Similarly, instead of using
\verb"$phi" for the name of the golden ratio variable,
we might have used the \emph{Greek small letter phi}, \verb'φ'
(Unicode code point U+03C6), just as we could have used
the \emph{Greek small letter pi}, \verb'π',  for the well-known
circle circumference to diameter ratio:
\index{golden ratio}
\index{Unicode}
\index{phi}
\index{pi}

\begin{verbatim}
> my $φ = (5 ** .5 + 1)/2;       # golden ratio
1.61803398874989
> say 'Variable $φ = ', $φ;
variable $φ = 1.61803398874989
> my $π = 4 * atan 1;
3.14159265358979
> # you could also use the pi or π built-in constant:
> pi
3.14159265358979
\end{verbatim}

The underscore character, \verb"_", can appear anywhere in a variable name.
It is often used in names with multiple words, such as
\verb"$your_name" or \verb"$airspeed_of_unladen_swallow".
\index{underscore character}

You may even use dashes to create so-called
``kebab case''\footnote{Because the
words appear to be skewered like pieces of food prepared for
a barbecue.} and name those variables
\verb"$your-name" or \verb"$airspeed-of-unladen-swallow", and this
might make them slightly easier to read: a dash \- is
valid in variable names provided it is immediately followed by an
alphabetical character and preceded by an alphanumerical character.
For example, \verb"$double-click" or \verb"$la-niña" are legitimate
variable names. Similarly, you can use an apostrophe \' (or single
quote) between letters, so \verb"$isn't" or \verb"$o'brien's-age"
are valid identifiers.
\index{dash}
\index{apostrophe}
\index{single quote}
\index{quote!single}
\index{kebab case}
\index{case!kebab}

If you give a variable an illegal name, you get a syntax error:
\index{syntax error}

\begin{verbatim}
> my $76trombones = 'big parade'
===SORRY!=== Error while compiling <unknown file>
Cannot declare a numeric variable
at <unknown file>:1
------> my $76<HERE>trombones = "big parade";
>
> my $more§ = 100000;
===SORRY!=== Error while compiling <unknown file>
Bogus postfix
at <unknown file>:1
------> my $more<HERE>§ = 100000;
(...)
\end{verbatim}
%
{\tt \$76trombones} is illegal because it begins with a number.
{\tt more§} is illegal because it contains an illegal character, {\tt
§}.

If you've ever used another programming language and stumbled across a terse message such as {\tt"SyntaxError: invalid syntax"}, you will notice that the Perl designers have made quite a bit of effort to provide detailed, useful and meaningful error messages.
\index{error message}

Many programming languages have \emph{keywords} or \emph{reserved words} that are part of the syntax, such as {\tt if}, {\tt while} or {\tt for}, and thus cannot be used for identifying variables because this would create an ambiguity. There is no such problem in Perl: since variable names start with a sigil, the compiler is always able to tell the difference between a keyword and a variable. Names such as {\tt \$if} or {\tt \$while} are syntactically valid variable identifiers in Perl (whether such names make sense is a different matter).
\index{sigil}
\index{keyword}
\index{reserved word}

\section{Expressions and statements}
\label{expr_and_statements}

An {\bf expression} is a combination of terms and operators.
Terms may be variables or literals, i.e. constant values such as a number or a string. A value all by itself is considered an expression, and so is
a variable, so the following are all legal expressions:
\index{expression}
\index{term}
\index{literal}

\begin{verbatim}
> 42
42
> my $n = 17;
17
> $n;
17
> $n + 25;
42
>
\end{verbatim}
%
When you type an expression at the prompt, the interpreter
{\bf evaluates} it, which means that it finds the value of
the expression.
In this example, {\tt \$n} has the value 17 and
{\tt \$n + 25} has the value 42.
\index{evaluate}

A {\bf statement} is a unit of code that has an effect, like
creating a variable or displaying a value, and usually needs to end
with a semi-colon {\tt ;} (but the semi-colon can sometimes be omitted
as we will see later).
\index{statement}
\index{semi-colon}

\begin{verbatim}
> my $n = 17;
17
> say $n;
17
\end{verbatim}
%

The first line is an assignment statement that gives a value to
{\tt \$n}.  The second line is a print statement that displays the
value of {\tt \$n}.

When you type a statement and then press {\tt Enter}, the
interpreter {\bf executes} it, which means that it does
whatever the statement says.
\index{execute}

An assignment can be combined with expressions using arithmetic
operators. For example, you might write:
\index{operator}

\begin{verbatim}
> my $answer = 17 + 25;
42
> say $answer;
42
\end{verbatim}
%

The \verb'+' symbol is obviously the addition operator
and, after the assignment statement, the \verb'$answer'
variable contains the result
of the addition. The terms on each side of the operator
(here 17 and 25) are sometimes called the \emph{operands}
of the operation (an addition in this case).
\index{operand}
\index{addition operator}
\index{assignment}

Note that the REPL actually displays the result of the
assignment (the first line with ``42''), so that the
print statement was not really necessary in this
example \emph{under the REPL}; from now on, for the sake of
brevity, we will generally omit the print statements in the
examples where the REPL displays the result.
\index{REPL}

In some cases, you want to add something to a variable
and assign the result to that same variable. This could
be written:

\begin{verbatim}
> my $answer = 17;
17
> $answer = $answer + 25;
42
\end{verbatim}
%

Here, \verb"$answer" is first declared with a value of 17. The next
statement assigns to \verb"$answer" the current value of
\verb"$answer" (i.e. 17) + 25. This is such a common operation
that Perl, like many other programming languages, has a
shortcut for this:

\begin{verbatim}
> my $answer = 17;
17
> $answer += 25;
42
\end{verbatim}
%

\index{$+=$ augmented assignment operator}
The \verb"+=" operator combines the arithmetic addition operator
and the assignment operator to modify a value and apply the result
to a variable in one go, so that \verb"$n += 2" means: take
the current value of \verb"$n", add 2 and assign the result to
\verb"$n". This syntax works with all other arithmetic operators.
For example, \verb"-=" performs similarly a subtraction and an
assignment, \verb"*=" a multiplication and an assignment, etc. It
can even be used with operators other than arithmetic operators,
such as the string concatenation operator, that we will see later.

Adding 1 to a variable is a very common version of this, so that
there is a shortcut to the shortcut, the \emph{increment} operator,
which increments its argument by one, and returns the incremented value:
\index{increment operator}
\index{++ increment operator}
\index{operator!++ (increment)}

\begin{verbatim}
> my $n = 17;
17
> ++$n;
18
> say $n;
18
\end{verbatim}
%
This is called the prefix increment operator, because the \verb"++"
operator is placed before the variable to be incremented. There is
also a postfix version, \verb"$n++", which first returns the current
value and then increments the variable by one. It would not make
a difference in the code snippet above, but the result can be very different
in slightly more complex expressions.

There is also a \verb"--"
decrement operator, which decrements its argument by one and also
exists in a prefix and a postfix form.
\index{decrement operator}
\index{\verb'--' decrement operator}
\index{operator!$--$ (decrement)}

\section{Script mode}

So far we have run Perl in {\bf interactive mode}, which
means that you interact directly with the interpreter (the
REPL). Interactive mode is a good way to get started,
but if you are working with more than a few lines of code,
it can be clumsy and even tedious.
\index{interactive mode}

The alternative is to use a text editor and save code in a file
called a {\bf script} and then run the interpreter in {\bf script mode}
to execute the script.  By convention, Perl~6 scripts have names that
end with {\tt .pl}, {\tt .p6} or {\tt .pl6}.
\index{script}
\index{script mode}

Please make sure that you're really using a \emph{text editor}
and not a \emph{word processing program} (such as MS Word,
OpenOffice or LibreOffice Writer). There is a very large
number of text editors available for free. On Linux, you might use
\emph{vi} (or \emph{vim}), \emph{emacs}, \emph{gEdit} or
\emph{nano}. On Windows, you may use \emph{notepad} (very limited)
or \emph{notepad++}. There are also many cross-platform editors
or integrated development environments (IDEs) providing a
text editor functionality, including \emph{padre}, \emph{eclipse}
or \emph{atom}. Many of these provide various syntax highlighting
capabilities, which might help you use correct syntax (and
find some syntax errors).
\index{syntax!highlighting}
\index{text editor!emacs}
\index{text editor!vi}
\index{text editor!vim}
\index{text editor!gEdit}
\index{text editor!padre}
\index{text editor!eclipse}
\index{text editor!nano}
\index{text editor!notepad++}
\index{text editor!atom}

Once you've saved your code into a file (say, for example,
\verb'my_script.pl6'), you can run the program by issuing
the following command at the operating system prompt (for example
in a Linux console or in a \verb'cmd' window under Windows):
\begin{verbatim}
perl my_script.pl6
\end{verbatim}

Because Perl provides both modes,
you can test bits of code in interactive mode before you put them
in a script.  But there are differences between interactive mode
and script mode that can be confusing.
\index{interactive mode}
\index{script mode}

For example, if you are using Perl~6 as a calculator, you might type

\begin{verbatim}
> my $miles = 26.2;
26.2
> $miles * 1.61;
42.182
\end{verbatim}

The first line assigns a value to {\tt \$miles} and displays that value.
The second line is an expression, so the
interpreter evaluates it and displays the result.  It turns out that a
marathon is about 42~kilometers.

But if you type the same code into a script and run it, you get no
output at all.  In script mode, an expression, all by itself, has no
visible effect.  Perl actually evaluates the expression, but it doesn't
display the value unless you tell it to:

\begin{verbatim}
my $miles = 26.2;
say $miles * 1.61;
\end{verbatim}

This behavior can be confusing at first. Let's examine why.

A script usually contains a sequence of statements.  If there
is more than one statement, the results appear one at a time
as the print statements execute.

For example, the script

\begin{verbatim}
say 1;
my $x = 2;
say $x;
\end{verbatim}
%
produces the output

\begin{verbatim}
1
2
\end{verbatim}
%
The assignment statement produces no output.

To check your understanding, type the following statements in the
Perl interpreter and see what they do:

\begin{verbatim}
5;
my $x = 5;
$x + 1;
\end{verbatim}

Now put the same statements in a script and run it.  What
is the output?  Modify the script by transforming each
expression into a print statement and then run it again.

\section{One-liner mode}

Perl also has a one-liner mode, which enables you to type directly a very
short script at the operating system prompt. Under Windows, it might
look like this:
\index{one-liner mode}
\label{one-liner mode}

\begin{verbatim}
C:\Users\Laurent>perl6 -e "my $value = 42; say 'The answer is ', $value;"
The answer is 42

\end{verbatim}

The {\tt -e} option tells the compiler that the script to be run is not
saved in a file but instead typed at the prompt between quote marks immediately after
this option.

Under Unix and Linux, you would replace double quote marks with apostrophes (or single quotes)
and apostrophes with double quote marks:
\index{apostrophe}
\index{quote mark}

\begin{verbatim}
$  perl6 -e 'my $value = 42; say "The answer is $value";'
The answer is 42

\end{verbatim}

The one-liner above may not seem to be very useful, but
throw-away one-liners can be very practical to perform
simple one-off operations, such as quickly modifying
a file not properly formatted, without having to save a script
into a separate file before running it.

We will not give any additional details about the one-liner
mode here, but will give some more useful examples
later in this book; for example,
Subsection~\ref{one-liner-example},
Subsection~\ref{rot13_oneliner} (solving the ``rot-13'' exercise), or
Subsection~\ref{sol_cartalk} (solving the exercise on
consecutive double letters).

\section{Order of operations}
\index{order of operations}
\index{PEMDAS}
\index{operator precedence}
\index{precedence!operator}

When an expression contains more than one operator, the order of
evaluation depends on the {\bf order of operations} or \emph{operator precedence}.
For mathematical operators, Perl follows mathematical convention.
The acronym {\bf PEMDAS}\footnote{US students are sometimes taught
to use the "Please Excuse My Dear Aunt Sally" mnemonics to remember
the right order of the letters in the acronym} is a useful
way to remember the rules:

\begin{itemize}

\item {\bf P}arentheses have the highest (or tightest) precedence and can be used
to force an expression to evaluate in the order you want. Since
expressions in parentheses are evaluated first, {\tt 2 * (3-1)} is 4,
and {\tt (1+1)**(5-2)} is 8. You can also use parentheses to make an
expression easier to read, as in {\tt (\$minute * 100) / 60}, even
if it doesn't change the result.
\index{$()$ parenthesis operator}

\item {\bf E}xponentiation has the next highest precedence, so
{\tt 1 + 2**3} is 9 (1 + 8), not 27, and {\tt 2 * 3**2} is 18, not 36.
\index{$**$ exponentiation operator}
\index{operator!$**$ (exponentiation)}

\item {\bf M}ultiplication and {\bf D}ivision have higher precedence
  than {\bf A}ddition and {\bf S}ubtraction.  So {\tt 2*3-1} is 5, not
  4, and {\tt 6+4/2} is 8, not 5.
\index{$*$ multiplication operator}
\index{operator!$*$ (multiplication)}
\index{$/$ division operator}
\index{operator!$/$ (division)}

\item Operators with the same precedence are usually evaluated
from left to right (except exponentiation).  So in the expression
{\tt \$degrees / 2 * pi}, the division happens first and the
result is multiplied by {\tt pi}, which is not the expected
result. (Note that {\tt pi} is not a variable, but a predefined
constant in Perl~6, and therefore does not require a sigil).  To
divide by $2 \pi$, you can use parentheses:
\index{sigil}
\index{pi}

\begin{verbatim}
my $result = $degrees / (2 * pi);
\end{verbatim}

or write
  {\tt \$degrees / 2 / pi} or {\tt \$degrees / } $2 / \pi$, which
  will divide \verb'$degrees' by 2, and then divide the result of
  that operation by $\pi$ (which is equivalent \verb'$degrees' by
   $2 \pi$.

\end{itemize}

We don't work very hard to remember the precedence of
operators.  If we can't tell by looking at the expression, we use
parentheses to make it obvious. If we don't know for sure which of two operators
has the higher precedence, then the next person reading or maintaining
our code may also not know.
\index{precedence}
\index{parentheses}

\section{String operations}
\label{string_operations}
\index{string!operation}
\index{operator!string}
\index{coercion}
\index{type!coercion}

In general, you can't perform mathematical operations on strings, unless
the strings look so much like numbers that Perl can transform or \emph{coerce} them into numbers and still make sense, so the
following are illegal:

\begin{verbatim}
'2'-'1a'    'eggs'/'easy'    'third'*'a charm'
\end{verbatim}
%

For example, this produces an error:

\begin{verbatim}
> '2'-'1a'
Cannot convert string to number: trailing characters after number
in '1?a' (indicated by ?)
  in block <unit> at <unknown file>:1
\end{verbatim}
%

But the following expressions are valid because these strings
can be coerced to numbers without any ambiguity:
\begin{verbatim}
> '2'-'1'
1
> '3'/'4'
0.75
\end{verbatim}
%

The \verb'~' operator performs {\bf string concatenation}, which means
it joins the strings by linking them end-to-end.  For example:
\index{string concatenation}
\index{string!concatenation}

\begin{verbatim}
> my $first = 'throat'
throat
> my $second = 'warbler'
warbler
> $first ~ $second
throatwarbler
\end{verbatim}
%
The {\tt x} operator also works on strings; it performs repetition.
For example:
\index{string repetition}

\begin{verbatim}
> 'ab' x 3;
ababab
> 42 x 3
424242
> 3 x 42
333333333333333333333333333333333333333333
\end{verbatim}

Notice that, although the {\tt x} operator somewhat looks like
the multiplication operator when we write it by hand, {\tt x}
is obviously not commutative, contrary to the {\tt *}
multiplication operator. The first operator is a string or is
\emph{coerced} to a string (i.e. transformed into a string:
{\tt 42} is coerced to {\tt '42'}), and the second operator
has to be a number or something that can be transformed
into a number.
\index{commutativity}
\index{coercion}

\section{Comments}
\index{comment}

As programs get bigger and more complicated, they get more difficult
to read.  Formal languages are dense, and it is often difficult to
look at a piece of code and figure out what it is doing, or why.

For this reason, it is a good idea to add notes to your programs to explain
in natural language what the program is doing.  These notes are called
{\bf comments}, and they start with the \verb"#" symbol:

\begin{verbatim}
# compute the percentage of the hour that has elapsed
my $percentage = ($minute * 100) / 60;
\end{verbatim}
%
In this case, the comment appears on a line by itself.  You can also put
comments at the end of a line:

\begin{verbatim}
$percentage = ($minute * 100) / 60;     # percentage of an hour
\end{verbatim}
%
Everything from the {\tt \#} to the end of the line is ignored---it
has no effect on the execution of the program.

Comments are most useful when they document non-obvious features of
the code.  It is reasonable to assume that the reader can figure out
{\em what} the code does; it is more useful to explain {\em why}.

This comment is redundant with the code and useless:

\begin{verbatim}
my $value = 5;        # assign 5 to $value
\end{verbatim}
%
This comment, by contrast, contains useful information that
is not in the code:

\begin{verbatim}
my $velocity = 5;     # velocity in meters/second.
\end{verbatim}
%
Good variable names can reduce the need for comments, but
long names can make complex expressions hard to read, so there is
a tradeoff.
\index{variable name}

\section{Debugging}
\index{debugging}
\index{bug}

Three kinds of errors can occur in a program: syntax errors, runtime
errors, and semantic errors.  It is useful
to distinguish between them in order to track them down more quickly.

\begin{description}

\item[Syntax error:] ``Syntax'' refers to the structure of a program
  and the rules about that structure.  For example, parentheses have
  to come in matching pairs, so {\tt (1 + 2)} is legal, but
{\tt 8)} is a \emph{syntax error}. \footnote{We are using here
``syntax error'' as a quasi synonym for ``compile-time error'';
they are not exactly the same thing (you may in theory have
syntax errors that are not compile-time errors and the other way
around), but they can be deemed to be the same for our
practical purpose here. In Perl~6, compile-time error
have the ``===SORRY!==='' string at the beginning of
the error message.}

\index{error!syntax}
\index{error message}
\index{syntax}
\index{syntax!error}

If there is a syntax error
anywhere in your program, Perl displays an error message and quits
without even starting to run your program, and you will
obviously not be able to run the program.  During the first few
weeks of your programming career, you might spend a lot of
time tracking down syntax errors.  As you gain experience, you will
make fewer errors and find them faster.

\item[Runtime error:] The second type of error is a runtime error, so
  called because the error does not appear until after the program has
  started running.  These errors are also called \emph{exceptions}
  because they usually indicate that something exceptional (and bad)
  has happened.  \index{runtime error} \index{error!runtime}
  \index{exception} \index{safe language} \index{language!safe}

Runtime errors are rare in the simple programs you will see in the
first few chapters, so it might be a while before you encounter one.
We have seen one example of such errors, though, at the beginning
of Section~\ref{string_operations} (p.~\pageref{string_operations}),
when we tried to subtract \verb"'2'-'1a'".

\item[Semantic error:] The third type of error is ``semantic'', which
  means related to meaning.  If there is a semantic error in your
  program, it will run without generating error messages, but it will
  not do the right thing.  It will do something else.  Specifically,
  it will do what you \emph{told} it to do, but not what you
  \emph{intended} it to do.
  \index{semantic error}
  \index{error!semantic} \index{error message}

Identifying semantic errors can be tricky because it requires you to work
backward by looking at the output of the program and trying to figure
out what it is doing.

\end{description}

\section{Glossary}

\begin{description}

\item[variable:]  Informally, a name that refers to a value. More
accurately, a variable is a container that has a name and holds a value.
\index{variable}

\item[assignment:]  A statement that assigns a value to a variable.
\index{assignment}

\item[state diagram:]  A graphical representation of a set of variables and the
values they refer to.
\index{state diagram}

\item[keyword:]  A reserved word that is used to parse a
program; in many languages, you cannot use keywords like
{\tt if}, {\tt  for}, and {\tt while} as variable names.
This problem usually does not occur in Perl because variable
names begin with a \emph{sigil}.
\index{keyword}
\index{sigil}

\item[operand:]  One of the values on which an operator operates.
\index{operand}

\item[term:]  A variable or a literal value.
\index{term}

\item[expression:]  A combination of operators and terms that
represents a single result.
\index{expression}

\item[evaluate:]  To simplify an expression by performing the operations
in order to yield a single value.
\index{evaluate}

\item[statement:]  A section of code that represents a command or action.  So
far, the statements we have seen are assignments and print statements. Statements usually end with a semi-colon.
\index{statement}

\item[execute:]  To run a statement and do what it says.
\index{execute}

\item[interactive mode (or interpreter mode):] A way of using the Perl
interpreter by typing code at the prompt.
\index{interactive mode}

\item[script mode:] A way of using the Perl interpreter to read
code from a script and run it.
\index{script mode}

\item[one-liner mode:] A way of using the Perl interpreter to read
code passed at the operating system prompt and run it.
\index{one-liner mode}

\item[script:] A program stored in a file.
\index{script}

\item[order of operations:]  Rules governing the order in which
expressions involving multiple operators and operands are evaluated.
It is also called operator precedence.
\index{order of operations}
\index{operator precedence}
\index{precedence!operator}

\item[concatenate:]  To join two operands end-to-end.
\index{concatenation}

\item[comment:]  Information in a program that is meant for other
programmers (or anyone reading the source code) and has no effect on the
execution of the program.
\index{comment}

\item[syntax error:]  An error in a program that makes it impossible
to parse (and therefore impossible to compile and to run).
\index{syntax!error}

\item[exception:]  An error that is detected while the program is running.
\index{exception}

\item[semantics:]  The meaning of a program.
\index{semantics}

\item[semantic error:]   An error in a program that makes it do something
other than what the programmer intended.
\index{semantic error}

\end{description}

\section{Exercises}

\begin{exercise}

Repeating our advice from the previous chapter, whenever you learn
a new feature, you should try it out in interactive mode (under
the REPL) and make errors on purpose to see what goes wrong.
\index{interactive mode}
\index{REPL}

\begin{itemize}

\item We've seen that {\tt \$n = 42} is legal.  What about {\tt 42 = \$n}?

\item How about {\tt \$x = \$y = 1}? (Hint: note that you will
have to declare both variables, for example with a statement
such as {\tt my \$x; my \$y;} or possibly {\tt my (\$x, \$y);},
before you can run the above.)

\item In some languages, statements don't have to end with a semi-colon,
{\tt ;}. What happens in script mode if you omit a semi-colon at the end
of a Perl statement?
\index{semi-colon}

\item What if you put a period at the end of a statement?

\item In math notation you can multiply $x$ and $y$ like this: $x y$.
What happens if you try that in Perl?

\end{itemize}

\end{exercise}

\begin{exercise}

Practice using the Perl interpreter as a calculator:
\index{calculator}

\begin{enumerate}

\item The volume of a sphere with radius $r$ is $\frac{4}{3} \pi r^3$.
  What is the volume of a sphere with radius 5?

\item Suppose the cover price of a book is \$24.95, but bookstores get a
  40\% discount.  Shipping costs \$3 for the first copy and 75 cents
  for each additional copy.  What is the total wholesale cost for
  60 copies?

\item If I leave my house at 6:52 am and run 1 mile at an easy pace
  (8:15 per mile), then 3 miles at tempo (7:12 per mile) and 1 mile at
  easy pace again, what time do I get home for breakfast?
\index{running pace}

\end{enumerate}
\end{exercise}


\documentclass{article}

\usepackage[T1]{fontenc}
\usepackage[osf]{libertine}
\usepackage[scaled=0.8]{beramono}
\usepackage[margin=1.5in]{geometry}
\usepackage{url}
\usepackage{booktabs}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{nicefrac}
\usepackage{microtype}
\usepackage{bm}

\usepackage{sectsty}
\sectionfont{\large}
\subsectionfont{\normalsize}

\usepackage{titlesec}
\titlespacing{\section}{0pt}{10pt plus 2pt minus 2pt}{0pt plus 2pt minus 0pt}
\titlespacing{\subsection}{0pt}{5pt plus 2pt minus 2pt}{0pt plus 2pt minus 0pt}

\usepackage{pgfplots}
\pgfplotsset{
  compat=newest,
  plot coordinates/math parser=false,
  tick label style={font=\footnotesize, /pgf/number format/fixed},
  label style={font=\small},
  legend style={font=\small},
  every axis/.append style={
    tick align=outside,
    clip mode=individual,
    scaled ticks=false,
    thick,
    tick style={semithick, black}
  }
}

\pgfkeys{/pgf/number format/.cd, set thousands separator={\,}}

\usepgfplotslibrary{external}
\tikzexternalize[prefix=tikz/]

\newlength\figurewidth
\newlength\figureheight

\setlength{\figurewidth}{8cm}
\setlength{\figureheight}{6cm}

\setlength{\parindent}{0pt}
\setlength{\parskip}{1ex}

\newcommand{\acro}[1]{\textsc{\MakeLowercase{#1}}}
\newcommand{\given}{\mid}
\newcommand{\mc}[1]{\mathcal{#1}}
\newcommand{\data}{\mc{D}}
\newcommand{\intd}[1]{\,\mathrm{d}{#1}}

\begin{document}

{\large \textbf{CSE 515T (Spring 2015) Assignment 1 Solutions}} \\

\begin{enumerate}
\item
  (Barber.)
  Suppose that a study shows that 90\% of people who have contracted
  Creutzfeldt--Jakob disease (``mad cow disease'') ate hamburgers
  prior to contracting the disease.  Creutzfeldt--Jakob disease is
  incredibly rare; suppose only one in a million people have the
  disease.

  If you eat hamburgers, should you be worried?  Does this depend on
  how many other people eat hamburgers?
\end{enumerate}

\subsection*{Solution}
Let CJ be the random variaable ``has Creutzfeldt--Jakob disease,'' and
let H be the random variable ``eats hamburgers.''  From the problem,
we know $\Pr(\mathrm{H} \given \mathrm{CJ}) = 0.9$ and
$\Pr(\mathrm{CJ}) = 10^{-6}$.  From Bayes' theorem, we may compute the
posterior probability of having Creutzfeldt--Jakob disease given that
you eat hamburgers:
\begin{equation*}
  \Pr(\mathrm{CJ} \given \mathrm{H})
  =
  \frac
      {\Pr(\mathrm{H} \given \mathrm{CJ}) \Pr(\mathrm{CJ})}
      {\Pr(\mathrm{H})}.
\end{equation*}
The denominator, $\Pr(\mathrm{H})$, is the probability that a random
person eats hamburgers, so whether you should be worried does depend
on this value.  I estimate $\Pr(\mathrm{H})$ might be around
\nicefrac{1}{2}; plugging in this value, the posterior probability
of having Creutzfeldt--Jakob disease is
\begin{equation*}
  \frac{0.9 \times 10^{-6}}{\nicefrac{1}{2}}
  =
  1.8 \times 10^{-6},
\end{equation*}
so your probability of having the disease has only increased from
$10^{-6}$ to $1.8 \times 10^{-6} = 0.0000018$ as a result of eating
hamburgers.  You can sleep safe.

If hamburger eating were rare, say $\Pr(\mathrm{H}) = 10^{-5}$, then
we would instead have $\Pr(\mathrm{CJ} \given \mathrm{H}) = 9\%$, and
maybe you should be worried!

\clearpage

\begin{enumerate}
\setcounter{enumi}{1}
\item
  (O'Hagan and Forster.)
  Suppose $x$ has a Poisson distribution with unknown mean
  $\theta$:
  \begin{equation*}
    p(x \given \theta) = \frac{\theta^x}{x!} \exp(-\theta),
    \qquad
    x = 0, 1, \dotsc
  \end{equation*}
  Let the prior for $\theta$ be a gamma distribution:
  \begin{equation*}
    p(\theta \given \alpha, \beta)
    =
    \frac{\beta^\alpha \theta^{\alpha - 1}}{\Gamma(\alpha)} \exp(-\beta\theta),
    \qquad \theta > 0
  \end{equation*}
  where $\Gamma$ is the gamma function.  Show that, given an
  observation $x$, the posterior $p(\theta \given x, \alpha, \beta)$
  is a gamma distribution with updated parameters $(\alpha', \beta') =
  (\alpha + x, \beta + 1)$.
\end{enumerate}

\subsection*{Solution}
From Bayes' theorem, we have:
\begin{align*}
  p(\theta \given x)
  &\propto
  p(x \given \theta)
  p(\theta)
  \\
  &\propto
  \bigl(\theta^{x} \exp(-\theta)\bigr)
  \bigl(\theta^{\alpha - 1} \exp(-\beta\theta)\bigr)
  \\
  &=
  \theta^{x + \alpha - 1}
  \exp\bigl(-(\beta + 1)\theta\bigr)
  \\
  &\propto
  \mc{G}(\alpha + x, \beta + 1).
\end{align*}

Here we exploit a common trick: we manipulate the numerator, ignoring
constants independent of $\theta$.  If we can recognize the functional
form as belonging to a distribution family we know, we can simply
identify the parameters and trust that the distribution normalizes!

\clearpage

\begin{enumerate}
\setcounter{enumi}{2}
\item
  (Optimal \emph{Price is Right} bidding.)
  Suppose you have a standard normal belief about an unknown parameter
  $\theta$, $p(\theta) = \mc{N}(\theta; 0, 1^2)$.  You are asked to
  give a point estimate $\hat{\theta}$ of $\theta$, but are told that
  there is a heavy penalty for guessing too high.  The loss function is
  \begin{equation*}
    \ell(\hat{\theta}, \theta; c)
    =
    \begin{cases}
      (\theta - \hat{\theta})^2 & \hat{\theta}  <   \theta; \\
      c                         & \hat{\theta} \geq \theta
    \end{cases},
  \end{equation*}
  where $c > 0$ is a constant cost for overestimating.  What is the
  Bayesian estimator in this case?  How does it change as a function
  of $c$?
\end{enumerate}

\subsection*{Solution}
Let $\phi(x) = \mc{N}(x; 0, 1^2)$ be the standard normal \acro{PDF}
evaluated at $x$, and let $\Phi(x) = \int_{-\infty}^x \phi(x)
\intd{x}$ be the standard normal \acro{CDF} evaluated at $x$.  If we
fix a point $\hat{\theta}$, the expected loss is:
\begin{align*}
  \mathbb{E}\bigl[\ell(\hat{\theta}, \theta; c)\bigr]
  &=
  \int
  \ell(\hat{\theta}, \theta; c)
  p(\theta)
  \intd{\theta}
  \\
  &=
  \int_{-\infty}^{\hat{\theta}}
  c
  \phi(\theta)
  \intd{\theta}
  +
  \int_{\hat{\theta}}^{\infty}
  (\theta - \hat{\theta})^2
  \phi(\theta)
  \intd{\theta}.
\end{align*}
The first term is proportional to the standard normal \acro{CDF}
$\Phi$ evaluated at $\hat{\theta}$:
\begin{equation*}
  \int_{-\infty}^{\hat{\theta}}
  c
  \phi(\theta)
  \intd{\theta}
  =
  c\,\Phi(\hat{\theta}).
\end{equation*}

We may also compute the second integral using the following
antiderivatives:\footnote{\url{http://en.wikipedia.org/wiki/List_of_integrals_of_Gaussian_functions}
  is useful here! (Wolfram alpha also works.)}
\begin{equation*}
  \int \theta \phi(\theta) \intd{\theta} = -\phi(\theta) + C;
  \qquad
  \int \theta^2 \phi(\theta) \intd{\theta} = \Phi(\theta) - \theta \phi(\theta) + C.
\end{equation*}
Using the fundamental theorem of calculus, we may use these to calculate
\begin{equation*}
  \int_{\hat{\theta}}^{\infty}
  (\theta - \hat{\theta})^2
  \phi(\theta)
  \intd{\theta}
  =
  (\hat{\theta}^2 + 1) (1 - \Phi(\hat{\theta}))
  - \hat{\theta}\phi(\hat{\theta}).
\end{equation*}
Finally, the entire expected loss is
\begin{equation*}
  \mathbb{E}\bigl[\ell(\hat{\theta}, \theta; c)\bigr]
  =
  (c - \hat{\theta}^2 - 1)\Phi(\hat{\theta})
  - \hat{\theta}\phi(\hat{\theta})
  + \hat{\theta}^2 + 1.
\end{equation*}
The derivative with respect to $\hat{\theta}$ is
\begin{equation*}
  \frac{\partial \mathbb{E}\bigl[\ell(\hat{\theta}, \theta; c)\bigr]}
       {\partial \hat{\theta}}
  =
  2 \hat{\theta} \bigl(1 - \Phi(\hat{\theta})\bigr)
  + (c - 2) \phi(\hat{\theta}).
\end{equation*}
Unfortunately, I do not believe we may find a root explicitly, so we
would have to rely on numerical root finding.  For $c = 1$, the
minimial expected loss is achieved at $\hat{\theta} = 0.612$; for $c =
10$, the Bayes action is $\hat{\theta} = -1.0615$; for $c = 100$, the
Bayes action is $\hat{\theta} = -2.1167$.  In general, the larger $c$,
the smaller your estimate should be, due to the potentially high cost
of overestimation.  For $c = 0$, there is no Bayes action, because we
may continue to decrease the expected loss when taking $\hat{\theta}
\to \infty$ (there is no reason not to!).

\clearpage

\begin{enumerate}
\setcounter{enumi}{3}
\item
  (Maximum-likelihood estimation.)
  Suppose you flip a coin with unknown bias $\theta$, $\Pr(x =
  \text{H}) = \theta$, three times and observe the outcome HHH.  What
  is the maximum likelihood estimator for $\theta$?  Do you think this
  is a good estimator?  Would you want to use it to make predictions?

  Consider a Bayesian analysis of $\theta$ with a beta prior $p(\theta
  \given \alpha, \beta) = \mc{B}(\theta; \alpha, \beta)$.  What is the
  posterior mean of $\theta$?  What is the posterior mode?  Consider
  $(\alpha, \beta) = (\nicefrac{1}{5}, \nicefrac{1}{5})$.  Plot the
  posterior density in this case.  Is the posterior mean a good
  summary of the distribution?
\end{enumerate}

\subsection*{Solution}
The likelihood of HHH is proportional to $\theta^3$, which over the
domain $\theta \in [0, 1]$ is maximized at $\theta = 1$.  Whether you
think this is a good estimator is subjective; however, I certainly
wouldn't use it for prediction, because it completely discounts the
(in my opinion, still rather likely) possiblity of a tails event.

With a $\mc{B}(\alpha, \beta)$ prior, the posterior is $\mc{B}(\alpha
+ 3, \beta)$.  We may calculate the mean of an arbitrary beta
distribution:
\begin{equation*}
  \mathbb{E}[\theta \given \alpha, \beta]
  =
  \int
  \theta
  \mc{B}(\theta \given \alpha, \beta)
  \intd{\theta}
  =
  \frac{1}{B(\alpha, \beta)}
  \int_{0}^{1}
  \theta
  \bigl(
  \theta^{\alpha - 1}
  (1 - \theta)^{\beta - 1}
  \bigr)
  \intd{\theta}
  =
  \frac{B(\alpha + 1, \beta)}{B(\alpha, \beta)}
  =
  \frac{\alpha}{\alpha + \beta}.
\end{equation*}

Therefore the posterior mean is $\frac{\alpha + 3}{\alpha + \beta +
  3}$.  The posterior mode is $\frac{\alpha + 2}{\alpha + \beta +
  1}$.\footnote{\url{http://goo.gl/zigQER} -- this is how I would
  answer this type of question!} Note that the posterior mode only
exists for $\alpha, \beta > 1$.

The prior and posterior distributions are plotted below.  An
interesting point about the prior is that its mean and median are both
$\nicefrac{1}{2}$, but this is simultaneously the anti-mode!  It's an
unusual estimator.

\begin{figure}[h]
  \centering
  \input{figures/problem_4.tex}
  \caption{The prior and posterior distributions over $\theta$ for the
    coin-flipping problem.}
\end{figure}

\clearpage

\begin{enumerate}
\setcounter{enumi}{4}
\item
  (Gaussian with unknown mean.)
  Let $\bm{x} = \{ x_i \}_{i = 1}^N$ be independent, identically
  distributed real-valued random variables with distribution $p(x_i
  \given \theta) = \mc{N}(x_i; \theta, \sigma^2)$.  Suppose the
  variance $\sigma^2$ is known but the mean $\theta$ is unknown with
  prior distribution $p(\theta) = \mc{N}(\theta; 0, 1^2)$.
  \begin{itemize}
  \item
    What is the likelihood of the full observation vector $p(\bm{x}
    \given \theta)$?
  \item
    After observing $\bm{x}$, what is the posterior distribution of
    $\theta$, $p(\theta \given \bm{x}, \sigma^2)$?  (Note: you might
    find it more convenient in this case to work with the
    \emph{precision} $\tau = \sigma^{-2}$.)
  \item
    Interpret how the posterior changes as a function of $N$.  What
    happens if $N = 0$?  What happens if $N \to \infty$?  Does this
    agree with your intuition?
  \end{itemize}
\end{enumerate}

\subsection*{Solution}
In the first part, we use the definition of independence:
\begin{equation*}
  p(\bm{x} \given \theta)
  =
  \prod_{i = 1}^N
  p(x_i \given \theta)
  =
  \prod_{i = 1}^N
  \mc{N}(x_i; \theta, \sigma^2).
\end{equation*}

We will consider the slightly more general case with an arbitrary
Gaussian prior on $\theta$: $p(\theta \given m, s^2) = \mc{N}(\theta;
m, s^2)$.  For convenience, we will parameterize the Gaussians on the
$\{x_i\}$ and $\theta$ with the \emph{precision} parameters $\tau =
\sigma^{-2}; t = s^{-2}$.

By Bayes' theorem, we have
\begin{align*}
  p(\theta \given \bm{x})
  &\propto
  p(\bm{x} \given \theta)
  p(\theta)
  \\
  &=
  \prod \mc{N}(x_i; \theta, \tau)
  \,
  \mc{N}(\theta; m, t)
  \\
  &\propto
  \exp\Biggl(
  -\frac{1}{2}
  \biggl(
  \tau
  \sum_{i = 1}^N (x_i - \theta)^2
  +
  t
  (\theta - m)^2
  \biggr)
  \Biggr).
\end{align*}
This is an exponentiated, negative quadratic function of $\theta$ and
is therefore proportional to a Gaussian distribution over $\theta$.
Our goal is to identify the mean and precision of this distribution.

We expand:
\begin{equation*}
  \tau
  \sum_{i = 1}^N (x_i - \theta)^2
  +
  t
  (\theta - m)^2
  =
  (\tau N + t)\theta^2
  - 2(\tau \textstyle\sum x_i + tm) \theta
  + (\tau \sum x_i^2 + t m^2).
\end{equation*}
Notice that $\sum x_i = N \bar{x}$, where $\bar{x}$ is the sample mean
of the measurements.  Notice also that the last term is a constant
independent of $\theta$, which will be absorbed into the normalizing
constant.  To arrive at a more-familiar form, we ``complete the
square:''
\begin{equation*}
  (\tau N + t)\theta^2
  - 2(\tau N \bar{x} + tm) \theta
  =
  (\tau N + t) \biggl(
  \theta -
  \frac{\tau N \bar{x} + tm}{\tau N + t}
  \biggr)^2
  +
  c,
\end{equation*}
where $c$ is another constant independent of $\theta$.  Therefore:
\begin{align*}
  p(\theta \given \bm{x})
  &\propto
  \exp\bigl(
  \textstyle -\frac{t'}{2}
  (\theta - m')^2
  \bigr),
\end{align*}
which is a Gaussian distribution with mean and precision
\begin{equation*}
  m' = \frac{\tau N \bar{x} + t m}{\tau N + t};
  \qquad
  t' = \tau N + t,
\end{equation*}
respectively.  We recognize that the new precision is the sum of the
precisions of each measurement (the $N$ independent measurements
$\{x_i\}$ with precision $\tau$ aplus one additional prior
``measurement.'' with precision $t$).  The posterior mean can be
recognized as a precision-weighted average of the measurements,
including the ``measurement'' at the prior mean $m$.

As $N \to \infty$, the sample mean $\bar{x}$ dominates, and the
influence of the prior diminishes to zero.  If $N = 0$, we rely only
on our prior knowledge.

\clearpage

\begin{enumerate}
\setcounter{enumi}{5}
\item
  (Spike and slab priors.)
  Suppose $\theta$ is a real-valued random variable that is expected
  to either be near zero (with probability $\pi$) or to have a wide
  range of potential values (with probability $(1 - \pi)$).  Such
  scenarios happen a lot in practice: for example, $\theta$ could be
  the coefficient of a feature in a regression model.  We either
  expect the feature to be useless for predicting the output (and have
  a value close to zero) or to be useful, in which case we expect a
  value with larger magnitude but can't say much else.

  A common approach in this case is to use a so-called \emph{spike and
    slab prior.} Let $f \in \{0, 1\}$ be a discrete random variable
  serving as a flag.  We define the following conditional prior:
  \begin{equation*}
    p(\theta \given f, \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2)
    =
    \begin{cases}
      \mc{N}(\theta; 0, \sigma_{\text{spike}}^2) & f = 0 \\
      \mc{N}(\theta; 0, \sigma_{\text{slab}}^2)  & f = 1,
    \end{cases}
  \end{equation*}
  where $\sigma_{\text{spike}}$ is the width of a narrow ``spike'' at
  zero, and $\sigma_{\text{slab}} > \sigma_{\text{spike}}$ is the
  width of a ``slab'' supporting values with larger magnitude.

  In practice, we will never observe the flag variable $f$; instead,
  we must infer it or marginalize it, as required.
  \begin{itemize}
  \item
    Suppose we choose a prior $\Pr(f = 1) = \pi = \nicefrac{1}{2}$,
    expressing no \emph{a priori} preference for the spike or the
    slab.  What is the marginal prior $p(\theta \given
    \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2$)?  Plot the
    marginal prior distribution for $(\sigma_{\text{spike}}^2,
    \sigma_{\text{slab}}^2) = (1^2, 10^2)$.
  \item
    Suppose that we can make a noisy observation $x$ of $\theta$, with
    distribution $p(x \given \theta, \omega^2) = \mc{N}(x; \theta,
    \omega^2)$, with known variance $\omega^2$.  Given $x$, what is
    the posterior distribution of the flag parameter, $\Pr(f = 1
    \given x, \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2,
    \omega^2)$?  Plot this distribution as a function of $x$.  What
    observation would teach us the most about $f$?  What teaches us
    the least?
  \item
    Given an observation $x$ as in the last part, what is the
    posterior distribution of $\theta$, $p(\theta \given x,
    \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2, \omega^2)$?
    (Hint: use the sum rule to eliminate $f$ and use the result
    above.)
  \item
    Suppose the noise variance is $\omega^2 = 0.1^2$ and we make an
    observation $x = 3$.  Plot the posterior distribution of $\theta$,
    using the parameters from the first part.
  \end{itemize}
\end{enumerate}

\subsection*{Solution}
For the first part, we use the sum rule:
\begin{equation*}
  p(\theta \given \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2)
  =
  \sum_f
  \Pr(f)p(\theta \given f, \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2)
  =
  \frac{1}{2}\mc{N}(\theta; 0, \sigma_{\text{spike}}^2) +
  \frac{1}{2}\mc{N}(\theta; 0, \sigma_{\text{slab}}^2).
\end{equation*}
The prior density is plotted below.

\begin{figure}[h]
  \centering
  \input{figures/problem_6_prior.tex}
  \caption{The prior distribution over $\theta$ for the spike-and-slab
    prior problem.}
\end{figure}

For the second part, we have from Bayes' theorem that:
\begin{equation*}
  \Pr(f = 1 \given x, \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2, \omega^2)
  =
  \frac{p(x \given f = 1, \sigma_{\text{slab}}^2, \omega^2)\Pr(f = 1)}
       {\sum_f p(x \given f, \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2, \omega^2)\Pr(f)}.
\end{equation*}

We must derive the likelihood $p(x \given f, \sigma_{\text{spike}}^2,
\sigma_{\text{slab}}^2)$.  We apply the sum rule:
\begin{equation*}
  p(x \given f, \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2, \omega^2)
  =
  \int
  p(x \given \theta, \omega^2)
  p(\theta \given f, \sigma_{\text{slab}}^2, \omega^2)
  \intd{\theta}.
\end{equation*}
For $f = 1$, this becomes:
\begin{align*}
  p(x \given f = 1, \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2, \omega^2)
  &=
  \int
  p(x \given \theta, \omega^2)
  p(\theta \given f = 1, \sigma_{\text{slab}}^2, \omega^2)
  \intd{\theta}
  \\
  &=
  \int
  \mc{N}(x; \theta, \omega^2)
  \mc{N}(\theta; 0, \sigma_{\text{slab}}^2)
  \intd{\theta}
  \\
  &=
  \mc{N}(x; 0, \sigma_{\text{slab}}^2 + \omega^2).
\end{align*}
A similar expression holds for $f = 0$.  The final result is
\begin{equation*}
  \Pr(f = 1 \given x, \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2, \omega^2)
  =
  \frac{\mc{N}(x; 0, \sigma_{\text{slab}}^2 + \omega^2)}
       {\mc{N}(x; 0, \sigma_{\text{spike}}^2 + \omega^2) + \mc{N}(x; 0, \sigma_{\text{slab}}^2 + \omega^2)}.
\end{equation*}
This value is plotted as a function of $x$ on the range $x \in [-10,
  10]$ below for $\omega^2 = 0.1^2$.  Extreme values of $x$ teach us
the most, because we can conclude with near certainty that the
observation came from the slab.  The observations that would teach us
the least are $x \approx \pm 2.165$, either of which result in a
posterior slab probability of $\nicefrac{1}{2}$ -- the same as our
prior!

\begin{figure}[h]
  \centering
  \input{figures/problem_6_f_posterior.tex}
  \caption{The posterior probability for the slab flag $(f = 1)$ for
    the spike-and-slab prior problem as a function of the observation
    $x$.}
\end{figure}

For the third part, we use the sum rule:
\begin{equation*}
  p(\theta \given x, \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2, \omega^2)
  =
  \sum_f
  p(\theta \given f, x, \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2, \omega^2)
  \Pr(f \given x, \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2, \omega^2).
\end{equation*}
We need to compute the posterior on $\theta$ given $f$ and $x$.  Here
we may use the result from the last problem:
\begin{align*}
  p(\theta \given f = 0, x, \sigma_{\text{spike}}^2, \omega^2)
  &=
  \mc{N}(\theta;
  \tau_{\text{spike}}^{-1} \omega^{-2} x,
  \tau_{\text{spike}}^{-1}) \\
  p(\theta \given f = 1, x, \sigma_{\text{slab}}^2, \omega^2)
  &=
  \mc{N}(\theta;
  \tau_{\text{slab}}^{-1} \omega^{-2} x,
  \tau_{\text{slab}}^{-1}),
\end{align*}
where $\tau_{\text{spike}} = (\omega^{-2} +
\sigma_{\text{spike}}^{-2})$, and $\tau_{\text{slab}}$ is defined
similarly.

Finally, the posterior for $\theta$ for the observation $x = 3$ with
$\omega^2 = 0.1^2$ is plotted below.

\begin{figure}[h]
  \centering
  \input{figures/problem_6_theta_posterior.tex}
  \caption{The posterior density for the parameter $\theta$ given
    the example observation $x = 3$, $\omega^2 = 0.1^2$.}
\end{figure}

\end{document}


\documentclass{beamer}

\mode<presentation> { \usetheme{gruvbox} }
\setbeamerfont{frametitle}{size=\huge}

\usepackage{graphicx} % Allows including images
\usepackage{booktabs} % Allows the use of \toprule, \midrule and \bottomrule in tables
%\usepackage{listings}             % Include the listings-package
\usepackage{minted}
\usepackage{tikz}
\usepackage{drawstack}
\usetikzlibrary{calc,shapes.callouts,shapes.arrows,chains,positioning,fit,shapes, arrows.meta, arrows}
\usepackage{verbatimbox}
\usepackage{tcolorbox}

\usemintedstyle{paraiso-dark}

\newcommand{\pointthis}[2]{
    \tikz[remember picture,baseline]{\node[anchor=base,inner sep=0,outer sep=0]%
    (#1) {\underline{#1}};\node[overlay,rectangle callout,%
    callout relative pointer={(0.2cm,0.7cm)},fill=green!50] at ($(#1.north)+(-.5cm,-1.4cm)$) {#2};}%
}%

%----------------------------------------------------------------------------------------
%	TITLE PAGE
%----------------------------------------------------------------------------------------

\title[Cryptography Day 2]{\huge \textbf{Cryptography Day 2}} % The short title appears at the bottom of every slide, the full title is only on the title page

\author{Brandon Hernandez} % Your name
\date{\today} % Date, can be changed to a custom date

\begin{document}

\begin{frame}
    \titlepage % Print the title page as the first slide
\end{frame}

\begin{frame}
    \frametitle{Overview} % Table of contents slide, comment this block out to remove it
    \tableofcontents % Throughout your presentation, if you choose to use \section{} and \subsection{} commands, these will automatically be printed on this slide as an overview of your presentation
\end{frame}

\section{Brief Review}
\begin{frame}
    \frametitle{Brief Review}
    In crypto, we utilize hard problems in mathematics to ensure that breaking the cryptosystem is not trivial.
\end{frame}

\section{Diffie-Hellman}
\begin{frame}
    \frametitle{Diffie-Hellman}
\end{frame}

\section{RSA}
\begin{frame}
    \frametitle{The RSA Cryptosystem}
    Public-key cryptosystem based around the difficulty in factoring a composite integer into primes.

\end{frame}
\subsection{The RSA Problem}
\begin{frame}
    \frametitle{The RSA Problem}
    \begin{itemize}
        \item Consider two large primes, $p$ and $q$
        \item $N = pq$
        \item Consider $e$, $m$, and $c$, where $e,C, m \in \mathbb{Z}$
        \item $C \equiv m^{e} \textrm{(mod N)}$
    \end{itemize}
\end{frame}
\subsection{Construction}
\begin{frame}
    \frametitle{Construction}
    \begin{itemize}
        \item Consider two large primes, p and q
        \item Let our modulus, $N = pq$ ($\mathbb{Z}/N\mathbb{Z}$)
        \item Our public key, $e$, where $gcd(\phi{N},e) = 1$ and $1 < e < \phi{N}$
        \item The private key, $d$, where $d*e \equiv 1 (\textrm{mod} \phi{N})$
    \end{itemize}
\end{frame}

\subsection{Encryption}
\begin{frame}
    \frametitle{Encryption}
    \begin{itemize}
        \item Given $N$, $m$, and $e$
        \item $C \equiv m^{e} (mod N)$
    \end{itemize}
\end{frame}

\subsection{Decryption}
\begin{frame}
    \frametitle{Decryption}
    \begin{itemize}
        \item Given $N$, $C$, and $d$
        \item $m \equiv C^{d} \equiv m^{e*d} (mod N)$
        \item \textbf{Remeber:} $e*d \equiv 1 (mod \phi{N})$
    \end{itemize}
\end{frame}

\subsection{Construction Example}
\begin{frame}
  \frametitle{Construction Example}
\end{frame}
\subsection{}
\begin{frame}
  \frametitle{Basic Exploit Example 1}
  \emph{Refer to Day2/Examples/cube}
\end{frame}

\begin{frame}
  \frametitle{Basic Exploit Example 2 (Factoring)}
  \emph{Refer to Day2/Examples/multiPrime}
\end{frame}

\begin{frame}
  \frametitle{Basic Exploit Example 3 (Hastad)}
  \emph{Refer to Day2/Examples/hastad}
\end{frame}

\section{Closing Thoughts}
\begin{frame}
    \frametitle{What's left?}
    \begin{itemize}
    	\item Elliptic Curve Cryptography
       	\item Post Quantum Cryptography
    	\begin{itemize}
    		\item Lattice-Based Cryptography
    		\item LWE
    		\item Multivariate Cryptograhy
    	\end{itemize}
    \end{itemize}
\end{frame}

\end{document}


\subsection{Heat Engines \& The Carnot Cycle}
Perhaps one of the most useful applications of thermodynamics is the development of heat engines (and heat pumps, which are just their inverse). With everything we have just discussed, we have basically all the tools we need to discuss and understand how heat engines work\footnote{Unintentional pun}. We will start by introducing a very abstract model for a heat engine, and then try to cement our understanding with a more concrete example. \\

In essence, a heat engine is a medium (in the context of Science One, a purely gaseuous medium\footnote{Sorry steam engines... you'll have to wait until second year}) that you can supply energy to in the form of heat, and you will get back some energy in the form of useful work. The end goal of the heat engine is to do as much work for us as possible. Essentially, the process goes as follows:
\begin{enumerate}
    \item The heat engine starts in its original state.
    \item The heat engine accepts some heat $Q_H$ from an external hot bath of temperature $T_H$.
    \item The heat engine does work $W$ from the energy given to it in the form of heat.
    \item The heat engine throws away some "waste heat" $Q_C$ into a cold bath of temperature $T_C$.
    \item The heat engine returns to its original state, and the cycle begins again.
\end{enumerate}
This is demonstrated in a diagram below. Note that this is just a general outline, but really steps 2-4 can occur in a different order, multiple times, and sometimes concurrently. A diagram illustrates this below:
\begin{center}
    \begin{tikzpicture}[scale=3]
    \filldraw[fill=pink] (0,0) rectangle (1,1);
    \draw (1.75,0.25) rectangle (2.25,0.75);
    \filldraw[fill=blue!50] (3,0) rectangle (4,1);
    \draw[->] (1.15,0.5) -- (1.6,0.5);
    \draw[->] (2.4,0.5) -- (2.85,0.5);
    \draw[->] (2,0.2) -- (2,0);
    \node[above] at (1.375,0.5) {$Q_H$};
    \node[above] at (2.625,0.5) {$Q_C$};
    \node[left] at (2,0.1) {$W$};
    \draw (2,0.5) node {Engine};
    \draw (0.5,0.5) node {Hot bath at $T_H$};
    \draw (3.5,0.5) node {Cold bath at $T_C$};=
    \end{tikzpicture}
\end{center}
Of course, we could also flip the direction in which we do things to give the opposite result, which results in a heat pump! Instead of putting heat at a high temperature $T_H$ to get work $W$ out, I can instead supply the heat engine with heat $Q_C$ at a low temperature $T_C$ (which is very cheap) and then supply work to the heat engine to get heat $Q_H$ out at a high temperature.  \\
A measure of how good a heat engine is is given by its efficiency. Clearly, we want to be getting as much useful work $W$ out of the steam engine from the energy/heat $Q_H$ that we are putting into it! Then, a very natural measure of the efficiency becomes:
\begin{equation}
    \eta = \frac{W}{Q_H} = \frac{W_{net}}{Q_{in}}
\end{equation}
We can observe that this efficiency measure is a percentage; It can range from $0$ (where $W=0$ and we have a broken heat engine) to $1$ (where $W=Q_H$ and we have a perfect conversion of the energy we put in to the work we get out). However, we will soon see that a 100\% efficient heat engine is not realistic at all; this would require that no waste heat $Q_C$ is produced at all, which is realistically impossible. In fact, in a little while we will introduce a theoretical maximum for just how efficient a heat engine can be!
\subsubsection{Heat Engines on PV diagrams}
As you might have guessed from all the time we spent setting up how different thermodynamic processes look on PV diagrams, a great way for representing heat engines is to draw them as a closed, non-intersecting curve\footnote{The curve has to end where it began, else the heat engine won't be in the same place where it started after one cycle!} on a PV diagram! Let's illustrate this with an example. Pictured below is a representation of a simple heat engine. For simplicity, let's imagine that the gas medium is 1 mole of monoatomic gas (with 3 degrees of freedom).

\begin{center}
\begin{tikzpicture}
 \draw[stealth-stealth] (0,5) node[below left]{$P$} |- (5,0) node[below left]{$V$};
\filldraw (1,1) circle (2pt);
\filldraw (4,1) circle (2pt);
\filldraw (4,4) circle (2pt);
\filldraw (1,4) circle (2pt);

\draw[thick,->] (1,4) -- (2.5,4);
\draw[thick] (2.5,4) -- (4,4);
\draw[thick,->] (4,4) -- (4,2.5);
\draw[thick] (4,2.5) -- (4,1);
\draw[thick,->] (4,1) -- (2.5,1);
\draw[thick] (2.5,1) -- (1,1);
\draw[thick,->] (1,1) -- (1,2.5);
\draw[thick] (1,2.5) -- (1,4);

\node[above] at (1,4) {$2$};
\node[above] at (4,4) {$3$};
\node[below] at (4,1) {$4$};
\node[below] at (1,1) {$1$};

%\draw[dashed] (0,4) -- (1,4);
%\draw[dashed] (0,1) -- (1,1);
%\draw[dashed] (1,0) -- (1,1);
%\draw[dashed] (4,0) -- (4,1);

\node[below] at (4,0) {$2v$};
\node[below] at (1,0) {$v$};
\node[left] at (0,1) {$p$};
\node[left] at (0,4) {$2p$};
\end{tikzpicture}
\end{center}

What exactly can we get out of this graph? There's a lot of information packed into it, so to start, let's break it down into the individual steps of the heat engine. The gas in the engine starts at state $1$, at pressure $p$ and volume $v$. We heat the gas isochorically, giving it heat, until it reaches state $2$ at pressure $2p$. Then, we add more heat and let it isobarically expand until it reaches state $3$ at volume $2v$, Then, we let the gas isochorically cool and give off heat until it falls back down to pressure $p$. Finally, we isobarically compress the gas (by doing work on it), the gas gives off more heat in the process, until the gas is back to volume $v$ and it has returned to its initial state. \\
So we've broken down the steps qualitatively, but what can we say quantitatively about things like temperature, work and efficiency?

To start, let's figure out the temperature at each state (depending on the information given in the problem, you may or may not always be able to do this straight away). Here, since we know that $n=1\text{ mol}$, and we have an ideal gas/monoatomic gas medium, our job is pretty easy. We can just apply the ideal gas law at each of the four states. Rearranging the ideal gas law, we have:
\[ T = \frac{PV}{nR} \]
So from this, we have that $T_1 = \frac{pv}{R}$, $T_2 = \frac{2pv}{R}$, $T_3 = \frac{4pv}{R}$, and $T_4 = \frac{2pv}{R}$. If I were to ask what was the hottest temperature $T_H$ and coldest temperature $T_C$ of the cycle, you could then say that $T_H = T_3 = \frac{4pv}{R}$, and $T_C = T_1 = \frac{pv}{R}$. Here, their ratio is $4:1$, so the hottest temperature is 4 times that of the coldest. \\

Next, let's figure out the work. One wonderful consequence of the PV diagram representation of the heat engine is that we can actually get the work done by the engine in one cycle just from the area enclosed by the curve! Let's explore why this is true, by again looking at our square heat engine. We know that from $1$ to $3$, the area under the curve represents the \textbf{postive} work that the engine does \textbf{for us}:
\begin{center}
\begin{tikzpicture}
 \draw[stealth-stealth] (0,5) node[below left]{$P$} |- (5,0) node[below left]{$V$};
\filldraw[fill=red] (1,0) rectangle (4,4);
\filldraw (1,1) circle (2pt);
\filldraw (4,4) circle (2pt);
\filldraw (1,4) circle (2pt);

\draw[thick,->] (1,4) -- (2.5,4);
\draw[thick] (2.5,4) -- (4,4);
\draw[thick,->] (1,1) -- (1,2.5);
\draw[thick] (1,2.5) -- (1,4);

\node[above] at (1,4) {$2$};
\node[above] at (4,4) {$3$};
\node[below] at (1,1) {$1$};

%\draw[dashed] (0,4) -- (1,4);
%\draw[dashed] (0,1) -- (1,1);
%\draw[dashed] (1,0) -- (1,1);
%\draw[dashed] (4,0) -- (4,1);

\node[below] at (4,0) {$2v$};
\node[below] at (1,0) {$v$};
\node[left] at (0,1) {$p$};
\node[left] at (0,4) {$2p$};

\draw (2.5,2) node {$+$};
\end{tikzpicture}
\end{center}

And from $3$ back to $1$, the area below the curve represents work that we do \textbf{on} the gas, or the gas doing \textbf{negative} work:

\begin{center}
\begin{tikzpicture}
 \draw[stealth-stealth] (0,5) node[below left]{$P$} |- (5,0) node[below left]{$V$};
 \filldraw[fill=blue] (1,0) rectangle (4,1);
\filldraw (1,1) circle (2pt);
\filldraw (4,1) circle (2pt);
\filldraw (4,4) circle (2pt);

\draw[thick,->] (4,4) -- (4,2.5);
\draw[thick] (4,2.5) -- (4,1);
\draw[thick,->] (4,1) -- (2.5,1);
\draw[thick] (2.5,1) -- (1,1);

\node[above] at (4,4) {$3$};
\node[below] at (4,1) {$4$};
\node[below] at (1,1) {$1$};

%\draw[dashed] (0,4) -- (1,4);
%\draw[dashed] (0,1) -- (1,1);
%\draw[dashed] (1,0) -- (1,1);
%\draw[dashed] (4,0) -- (4,1);

\node[below] at (4,0) {$2v$};
\node[below] at (1,0) {$v$};
\node[left] at (0,1) {$p$};
\node[left] at (0,4) {$2p$};
\draw (2.5,0.5) node {$-$};
\end{tikzpicture}
\end{center}

Therefore, over the whole cycle, the net work that the gas/heat engine does for us is just the sum of the two parts:

\begin{center}
\begin{tikzpicture}
 \draw[stealth-stealth] (0,5) node[below left]{$P$} |- (5,0) node[below left]{$V$};
\filldraw[fill=purple] (1,1) rectangle (4,4);
\filldraw (1,1) circle (2pt);
\filldraw (4,1) circle (2pt);
\filldraw (4,4) circle (2pt);
\filldraw (1,4) circle (2pt);

\draw[thick,->] (1,4) -- (2.5,4);
\draw[thick] (2.5,4) -- (4,4);
\draw[thick,->] (4,4) -- (4,2.5);
\draw[thick] (4,2.5) -- (4,1);
\draw[thick,->] (4,1) -- (2.5,1);
\draw[thick] (2.5,1) -- (1,1);
\draw[thick,->] (1,1) -- (1,2.5);
\draw[thick] (1,2.5) -- (1,4);

\node[above] at (1,4) {$2$};
\node[above] at (4,4) {$3$};
\node[below] at (4,1) {$4$};
\node[below] at (1,1) {$1$};

%\draw[dashed] (0,4) -- (1,4);
%\draw[dashed] (0,1) -- (1,1);
%\draw[dashed] (1,0) -- (1,1);
%\draw[dashed] (4,0) -- (4,1);

\node[below] at (4,0) {$2v$};
\node[below] at (1,0) {$v$};
\node[left] at (0,1) {$p$};
\node[left] at (0,4) {$2p$};
\draw (2.5,2.5) node {$W_{net}$};
\end{tikzpicture}
\end{center}
So in our case, the work the engine does for us is simple; we simply calculate the area of the square, which is just $pv$! Easy! Note that with more complicated looking heat engines (say, with isothermal or adiabatic curves) we might need to be more clever (and either use the formulas we derived\footnote{Which are at heart the same thing as the area argument, because we got those via integration}, or tricks with the first law of thermodynamics and the definition of energy change $\Delta E = nc_v\Delta T$). Of course, there was nothing stopping you from applying those isobaric process work formulas to solve this problem, either. You would find you get an identical result! \\
To figure out the heat that goes in/out of the engine during each process, we can use the formulas that we have derived. Let's start with the heat from $1 \rightarrow 2$. Here, the gas is being isochorically heated, so we know that we are adding heat into the system. We know that its an isochoric process, so there's no work that's done; hence, the only energy change we look at is the heat. Therefore, we have \[ Q_{1\rightarrow 2} = \Delta E  = nc_v \Delta T \]
The temperature difference between $1$ and $2$ is just:
\[\Delta T = T_2 -T_1 = \frac{2pv}{R} - \frac{pv}{R} = \frac{pv}{R} \]
And for a monoatomic gas, we know that the constant volume heat capacity $c_v$ is:
\[ c_v = \frac{\chi}{2}R = \frac{3}{2}R \]
So therefore the heat that we put in during the $1 \rightarrow 2$ process is:
\[Q_{1\rightarrow 2} = \frac{3}{2}R\frac{pv}{R} = \frac{3}{2}pv \]
Now let's consider the heat flow in the $2 \rightarrow 3$ process. We have isobaric expansion, so we know that we'll have to be putting in heat in order to make that happen, and that heat will have value (from our previous derivation):
\[Q = nc_p\Delta T\]
The temperature difference between $2$ and $3$ can be calculated as:
\[\Delta T = T_3 -T_2 = \frac{4pv}{R} - \frac{2pv}{R} = \frac{2pv}{R} \]
And the constant pressure heat capacity is:
\[c_p = c_v + R = \frac{3}{2}R + R = \frac{5}{2}R \]
So the heat that we put in during the $2 \rightarrow 3$ process is:
\[Q_{2\rightarrow 3} = \frac{5}{2}R\frac{2pv}{R} = 5pv \]
I'll leave the heat flow for segments $3 \rightarrow 4$ and $4 \rightarrow 1$ for you! However, for both of those last two segments, heat is coming out of the gas (rather than us putting heat into it) so we have everything we need to calculate the efficiency of the heat engine. We have that:
\[Q_{\text{in}} = Q_{1 \rightarrow 2} + Q_{2 \rightarrow 3} = \frac{3}{2}pv + 5pv = \frac{13}{2}pv \]
And the total work done is just $pv$, so the efficiency of our square heat engine is:
\[ \eta = \frac{W_{net}}{Q_{in}} = \frac{pv}{\frac{13}{2}pv} = \frac{2}{13} \approx 15 \% \]
Wow, our cute boxy heat engine turns out to be pretty crummy at its job. It's only 15\% efficient at converting heat into work. Fortunately, not all heat engines are as bad; we'll introduce a much more efficient heat engine in the next section!

\subsubsection{The Carnot Engine}
The Carnot engine is more than just an engine. Its a limit on the efficiency of real-world thermodynamic processes, and was used to develop the first notions of entropy. That being said, like the example above the Carnot engine is a four step cycle.
\begin{center}
            \begin{tikzpicture}[
          dot/.style = {draw,fill,circle,inner sep=1pt},
          arrow inside/.style = {postaction=decorate,decoration={markings,mark=at position .55 with \arrow{>}}}
          ]
          %\draw[<->] (0,6) node[above right] {$P$} |- (6,0) node[right] {$V$};
          %\draw[->] (0,0) node[above] {$P$} %|- (6,0) node[right] {$V$};
 \draw[stealth-stealth] (0,6) node[below left]{$P$} |- (6,0) node[below left]{$V$};
          \node[dot,label={right:$2$}] (@b) at (4.5,4) {};
          \node[dot,label={left:$1$}] (@a) at (1,4.5) {};
          \node[dot,label={below left:$4$}] (@d) at (1.5,1.5) {};
          \node[dot,label={right:$3$}] (@c) at (5,1) {};
          \draw[fleche={0.5:black} ] (@a) to[looseness=.7,bend right=20] (@b);
          \draw[fleche={0.5:black} ] (@b) to[looseness=.7,bend right=20] (@c);
          \draw[fleche={0.5:black} ] (@c) to[looseness=.7,bend left=20] (@d);
          \draw[fleche={0.5:black} ] (@d) to[looseness=.7,bend left=20] (@a);
        \end{tikzpicture}

\end{center}
If we start at state $1$ in the diagram, the engine first undergoes an isothermal expansion to state $2$, while in contact with the hot heat bath. It then undergoes an adiabatic expansion to state $3$ in contact with no heat bath. This is followed by an isothermal compression to state $4$, which occurs while in contact with a cool heat bath. Finally, the engine undergoes an adiabatic compression to return to state $1$. Note that since the processes $1 \rightarrow 2$ and $3 \rightarrow 4$ are isothermal, the engine has only two temperatures. States $1$ and $2$ are at a high temperature ($T_H$), and states $3$ and $4$ are at a low temperature ($T_C)$. \\
While the optional derivation will be given in the next section, you will want to know that the efficiency of the Carnot engine is given by:
\begin{equation}
    \eta = 1 -\frac{T_C}{T_H}
\end{equation}
Where $T_C$ is the coldest/minimum temperature in the cycle and $T_H$ is the hottest/maximum temperature in the cycle. A brief fact about the maximum and minimum temperatures of the heat engine; often these two parameters are given by the ambient temperature and the combustion temperature of the gas in the engine, respectively. This means that if you were to put a Carnot engine inside of your car (though I'm about to tell you why you can't do that in a few sentences), you would find that during the winter your car will run more efficiently than during the summer! Alternatively, if you were to use a gas medium with a higher combustion temperature, you would see a similar effect.\\
One surprising fact: The Carnot efficiency is the maximum possible efficiency attainable by a heat engine\footnote{An interesting consequence of this is that your heat engine can only be 100\% efficient if it's a Carnot engine, and the lowest temperature is $T_C=0$, or the maximum temperature is $T_H = \infty$. Not very realistic either way.}. This isn't just me trying to challenge you to think of something that could be better; it's actually a theorem, and a consequence of the extremely fundamental second law of thermodynamics, which we will explore in depth in the next section. Unfortunately for the world, a truly isothermal process takes an infinitely long time (we'll explore this more in the next section as well). This means that while the Carnot engine might be the most energy efficient, its power output (i.e. the work it does divided by the time it takes) is zero. So unfortunately, you won't be installing a Carnot engine into your car anytime soon..
\subsubsection{(Optional) Deriving the Carnot Efficiency}
Deriving the efficiency of a Carnot engine is difficult, but doable with the knowledge we have so far. I'd suggest you take a shot at it before looking at this section. \\
I'll label the different states of the Carnot cycle as in the diagram above. The efficiency of a heat engine $\eta$ is defined as
\begin{equation*}
    \eta = \frac{W}{Q_{in}}
\end{equation*}
the ratio of the total work done by the system and the heat put into it. From the first law of thermodynamics, we know that $\Delta E = Q - W$, where $W$ is the work done by the system. Since this process ends in the same state it started, we can infer that $\Delta E = 0$, so $Q = W$. Therefore, we get that $W = Q_{in} - Q_{out}$. So we have that
\begin{equation*}
    \eta = 1 - \frac{Q_{out}}{Q_{in}}
\end{equation*}
Heat only flows out of the system from states $c$ to $d$, and only flows into the system from states $a$ to $b$. Both processes are isothermal, so we get
\begin{equation*}
    \eta = 1 - \frac{-P_{3}V_{3}\ln{\Big(\frac{V_4}{V_3}\Big)}}{P_{1}V_{1}\ln{\Big(\frac{V_2}{V_1}\Big)}}
\end{equation*}
To be honest I am hand-waving the signs a bit here, so take some time to make sure you understand them. If you get stuck, remember that in this case positive heat flow is into the system and that the natural logarithm of a number less than one is negative. Anyways, since $T_c$ is the temperature of the low-temperature heat bath and $T_a$ is the temperature of the high-temperature heat bath, I can substitute temperature into the equation using ideal gas law to get
\begin{equation*}
    \eta = 1 - \frac{T_{C}\ln{\Big(\frac{V_3}{V_4}\Big)}}{T_{H}\ln{\Big(\frac{V_2}{V_1}\Big)}}
\end{equation*}
Now all that remains is to show that $\frac{V_2}{V_1} = \frac{V_3}{V_4}$. I'll leave that part to you in one of the questions below. Once you do get this result, you can use it to cancel the logarithms, leaving the equation as
\begin{equation*}
    \eta = 1 - \frac{T_C}{T_H}
\end{equation*}


\documentclass{beamer}

\mode<presentation>
{
  \usetheme{Hawke}
  \setbeamercovered{transparent}
}

\usepackage[english]{babel}
\usepackage[latin1]{inputenc}
\usepackage{times}
\usepackage[T1]{fontenc}
\usepackage{multimedia}

%%%%%%
% My Commands
%%%%%%

\newcommand{\ml}{{\sc matlab}}
\newcommand{\bb}{{\boldsymbol{b}}}
\newcommand{\bx}{{\boldsymbol{x}}}
\newcommand{\by}{{\boldsymbol{y}}}
\newcommand{\bfm}[1]{{\boldsymbol{#1}}}

%%%%

\title[Lecture 6] %
{Lecture 6 - Iterative Methods}

\author[I. Hawke]{I.~Hawke}

\institute[University of Southampton]%
{
  School of Mathematics, \\
  University of Southampton, UK
}

\date[Semester 1]{MATH3018/6141, Semester 1}

\subject{Numerical methods}

\pgfdeclareimage[height=0.5cm]{university-logo}{mathematics_7469}
\logo{\pgfuseimage{university-logo}}

\AtBeginSection[]
{
  \begin{frame}<beamer>
    \frametitle{Outline}
    \tableofcontents[currentsection]
  \end{frame}
}

\begin{document}

\begin{frame}
  \titlepage
\end{frame}

\section{Iterative Methods}

\subsection{Iterative Methods}

\begin{frame}
  \frametitle{Iterative Methods}

   We (still) want to solve the linear system
   \begin{equation*}
     A \bx = \bb.
   \end{equation*}
   Iterative methods build a \emph{sequence} of approximate
   solutions. Start from ``guess'' $\bx^{(0)}$ and compute
   approximations $\bx^{(1)}, \bx^{(2)}, \dots, \bx^{(N)}$ which
   converge to the exact solution. \pause

   \vspace{1ex}

   Standard ``notation'' is to split the matrix as
   \begin{equation*}
     A = N - P, \quad \implies \quad N \bx^{(i)} = P \bx^{(i - 1)} + \bb
   \end{equation*}
   and for Jacobi use $N=I$. \pause

   \vspace{1ex}

   Jacobi converges slowly: error drops by a constant amount each step.

\end{frame}

\subsection{Gauss-Seidel Method}

\begin{frame}
  \frametitle{Gauss-Seidel Method}

  In the Gauss-Seidel method split the coefficient matrix as
  \begin{equation*}
    A = I - (A_L + A_U)
  \end{equation*}
  as before. Choose
  \begin{align*}
    N & = I - A_L, \\
    P & = A_U.
  \end{align*}
  This makes the iteration scheme
  \begin{equation*}
    \bx^{(i)} =  A_L \bx^{(i)} + A_U \bx^{(i - 1)} + \bb, \quad i = 1,
    2, \dots
  \end{equation*} \pause

  Written in this form as $A_L$ \emph{strictly} lower
  triangular. Consider each component of the iteration scheme from the
  top down: each coefficient of $\bx^{(i)}$ is computed on the left
  before it is used on the right.

\end{frame}

\begin{frame}
  \frametitle{Example of Gauss-Seidel Method}

  As above we look at
  \begin{equation*}
    \begin{pmatrix}
      3 & 1 \\
      1 & 3
    \end{pmatrix}
    \begin{pmatrix}
      x_1 \\ x_2
    \end{pmatrix} =
    \begin{pmatrix}
      5 \\ 7
    \end{pmatrix} \rightarrow
    \begin{pmatrix}
      1 & \tfrac{1}{3} \\
      \tfrac{1}{3} & 1 \\
    \end{pmatrix}
    \begin{pmatrix}
      x_1 \\ x_2
    \end{pmatrix} =
    \begin{pmatrix}
      \tfrac{5}{3} \\ \tfrac{7}{3}
    \end{pmatrix}.
  \end{equation*}
  We have
  \begin{equation*}
    A_L =
    \begin{pmatrix}
      0 & 0 \\
      -1/3 & 0
    \end{pmatrix}, \quad
    A_U =
    \begin{pmatrix}
      0 & -1/3 \\
      0 & 0
    \end{pmatrix}
  \end{equation*}
  and hence the iteration algorithm becomes
  \begin{align*}
    \begin{pmatrix}
      x_1^{(i)} \\ x_2^{(i)}
    \end{pmatrix} & = \frac{1}{3} \left[
      \begin{pmatrix}
        -x_2^{(i-1)} \\ - x_1^{(i)}
      \end{pmatrix} +
      \begin{pmatrix}
        5 \\ 7
      \end{pmatrix} \right]
    = \frac{1}{3}
    \begin{pmatrix}
      5 - x_2^{(i-1)} \\ 7 - x_1^{(i)}
    \end{pmatrix}.
  \end{align*}
  If read row-by-row from the top we have always computed the
  $\bx^{(i)}$ entries before they are used.

\end{frame}

\begin{frame}
  \frametitle{Example of Gauss-Seidel Method 2}

  \begin{columns}
    \begin{column}{0.475\textwidth}
      The sequence has entries
      \begin{tabular}{|c|c c|}
        $i$ & $x_1$ & $x_2$ \\ \hline
        $0$ & $0$ & $0$ \\
        $1$ & $5 / 3$ & $16 / 9$ \\
        $2$ & $1.074074$ & $1.975309$ \\
        $5$ & $1.000102$ & $1.999966$ \\
        $10$ & $1.000000$ & $2.000000$ \\
        $100$ & $1.000000$ & $2.000000$
      \end{tabular}
    \end{column}
    \begin{column}{0.525\textwidth}
      \begin{center}
        \includegraphics[width=\textwidth]{figures/GaussSeidel1}
      \end{center}
    \end{column}
  \end{columns}

  \vspace{1ex}

  A starting ``guess'' of $\bx = \bfm{0}$ appears to converge to $\bx
  = (1, 2)^T$. The convergence is exponential with a slope $\sim 2$,
  much faster than Jacobi's Method.

\end{frame}

\subsection{Successive Over Relaxation}

\begin{frame}
  \frametitle{Successive Over Relaxation}

  View the sequence as repeated ``corrections'' to the previous term,
  \begin{equation*}
    \bx^{(i)} = \bx^{(i-1)} + \bfm{c},
  \end{equation*}
  where correction is based e.g.\ on Gauss-Seidel method,
  \begin{equation*}
    \bfm{c} = A_L \bx^{(i)} + (A_U - I) \bx^{(i-1)} + \bb.
  \end{equation*} \pause

  Modify correction by factor $\omega$:
  \begin{equation*}
    \bx^{(i)} = \bx^{(i-1)} + \omega \bfm{c}.
  \end{equation*} \pause

  \emph{A priori} best value of $\omega$ not clear.  Typically it is
  set close to one. However, poor choices may cause the algorithm to
  diverge!

\end{frame}

\begin{frame}
  \frametitle{Example of SOR Method 1}

  \begin{columns}
    \begin{column}{0.475\textwidth}
      Solving the previous example using $\omega = 1.035$, the
      sequence has entries
      \begin{tabular}{|c|c c|}
        $i$ & $x_1$ & $x_2$ \\ \hline
        $0$ & $0$ & $0$ \\
        $1$ & $1.725000$ & $1.819875$ \\
        $2$ & $1.036768$ & $1.993619$ \\
        $5$ & $0.999999$ & $2.000000$ \\
        $10$ & $1.000000$ & $2.000000$ \\
        $100$ & $1.000000$ & $2.000000$
      \end{tabular}
    \end{column}
    \begin{column}{0.525\textwidth}
      \begin{center}
        \includegraphics[width=\textwidth]{figures/SOR1}
      \end{center}
    \end{column}
  \end{columns}

  \vspace{1ex}

  A starting ``guess'' of $\bx = \bfm{0}$ appears to converge to $\bx
  = (1, 2)^T$. The convergence is exponential with a slope $\sim 3$,
  the fastest method so far.

\end{frame}

\begin{frame}
  \frametitle{Example of SOR Method 2}

  \begin{columns}
    \begin{column}{0.475\textwidth}
      Solving the previous example using $\omega = 1.2$, the
      sequence has entries
      \begin{tabular}{|c|c c|}
        $i$ & $x_1$ & $x_2$ \\ \hline
        $0$ & $0$ & $0$ \\
        $1$ & $2.000000$ & $2.000000$ \\
        $2$ & $0.800000$ & $2.080000$ \\
        $5$ & $0.998221$ & $2.000430$ \\
        $10$ & $1.000000$ & $2.000000$ \\
        $100$ & $1.000000$ & $2.000000$
      \end{tabular}
    \end{column}
    \begin{column}{0.525\textwidth}
      \begin{center}
        \includegraphics[width=\textwidth]{figures/SOR2}
      \end{center}
    \end{column}
  \end{columns}

  \vspace{1ex}

  A starting ``guess'' of $\bx = \bfm{0}$ appears to converge to $\bx
  = (1, 2)^T$. The convergence is exponential with a slope $\sim 1.6$,
  which is not as good as Gauss-Seidel.

\end{frame}

\begin{frame}
  \frametitle{The relaxation parameter}

  \begin{center}
    \includegraphics[width=0.9\textwidth]{figures/SOR}
  \end{center}
  The norm of the error obtained with SOR after four iterations.  The
  convergence behaviour depends in a highly non-trivial way on
  $\omega$.

  \vspace{1ex}

  SOR algorithms are much harder to analyze than Jacobi or
  Gauss-Seidel.

\end{frame}

\subsection{Convergence Analysis}

\begin{frame}
  \frametitle{Convergence analysis}

  Two theorems about \emph{when} these methods will work:

  \vspace{2ex}

  {\bf Theorem 1}: If the coefficient matrix $A$ is \emph{strictly}
  diagonally dominant then both the Jacobi method and the Gauss-Seidel
  method will converge.

  \vspace{2ex}

  {\bf Theorem 2}: If the coefficient matrix $A$ is symmetric and
  positive definite then the Gauss-Seidel method will converge.

\end{frame}

\section{Other matrix operations}

\subsection{Other matrix operations}

\begin{frame}
  \frametitle{Other matrix operations}

  We have so far only considered the linear system
  \begin{equation*}
    A \bx = \bb.
  \end{equation*}
  We have given methods to solve this system accurately without
  computing the matrix inverse or determinant. However, these methods
  can also be used to efficiently and accurately compute these
  quantities.
\end{frame}

\begin{frame}
  \frametitle{Determinants}

  Given the $LU$ decomposition we have
  \begin{equation*}
    \det (A) = \det(L) \times \det(U), \quad \text{and} \quad \det(U)
    = \prod_{i=1}^n u_{ii}.
  \end{equation*}
  Choose $\ell_{ii}=1$ giving $\det(L) = 1$: determinant
  follows. \pause

  \vspace{1ex}

  Similarly Gaussian elimination with partial pivoting gives
  \begin{equation*}
    \det(A) = \pm \prod_{i=1}^n u_{ii},
  \end{equation*}
  where the sign depends on the number of row swaps performed.\pause

  \vspace{1ex}

  {\bf Note:} Gaussian elimination, and hence finding the determinant,
  is an ${\cal O}(n^3)$ operation.  The expansion in minors is an
  ${\cal O}(n!)$ operation.

\end{frame}

\begin{frame}
  \frametitle{Matrix inversion}

  Solving for the matrix inverse can be written as a set of linear
  system problems. We write
  \begin{equation*}
    A A^{-1} = I
  \end{equation*}
  and consider each column of $A^{-1}$ as a separate problem. \pause
  Writing $\bfm{c}_i$ for the (unknown) column of $A^{-1}$ we have
  \begin{equation*}
    A \bfm{c}_i =
    \begin{pmatrix}
       \vdots \\ 0 \\ 1 \\ 0 \\ \vdots
    \end{pmatrix}
  \end{equation*}
  where the $1$ appears in the $i^{\text{th}}$ row. \pause

  \vspace{1ex}

  This finds the inverse by solving $n$ linear systems, which is much
  faster than evaluating the $n+1$ determinants required by Cramer's
  rule.

\end{frame}

\section{Summary}

\subsection{Summary}

\begin{frame}
  \frametitle{Summary}

  \begin{itemize}
  \item Iterative methods are often based on the split
    \begin{equation*}
      A = N - P.
    \end{equation*}
  \item Convergence depends on $M = N^{-1}P$.
  \item Jacobi chooses $N = I$ and converges slowly.
  \item Gauss-Seidel chooses $N = I - A_U$ and uses coefficients as
    soon as they become available. Slightly harder to code, it
    converges faster than Jacobi.
  \item SOR tries to ``accelerate'' the convergence of Gauss-Seidel
    depending on a parameter $\omega$. It can be faster, but the
    behaviour depends non-trivially on $\omega$ and it is very
    difficult to analyze.
  \item Strict diagonal dominance is sufficient for convergence of
    both Jacobi and Gauss-Seidel methods.
  \item Computing determinants and inverses from the solution of
    linear systems is usually much faster than ``standard''
    techniques.
  \end{itemize}

\end{frame}

\end{document}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: t
%%% End:


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% UMB-CS240-2016S: Programming in C
% Copyright 2016 Pejman Ghorbanzade <pejman@ghorbanzade.com>
% Creative Commons Attribution-ShareAlike 4.0 International License
% More info: https://github.com/ghorbanzade/UMB-CS240-2016S
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{Question 2}

Bellatrix Lestrange, a CS240 student, was asked to write a program that takes an arbitrary number of non-negative integers as command line arguments and prints their average.
A sample run of the program was expected to be as given below.

\begin{terminal}
$ gcc command-line.c -o command-line
$ ./command-line
error: missing command line arguments
$ ./command-line 2 5 8
5.00
$ ./command-line 1 2 3 4 5 6
3.50
\end{terminal}

To solve this problem, she wrote the following program.

\lstset{language=c,tabsize=4}
\begin{lstlisting}
// Bellatrix Lestrange
// <bellatrix.lestran001@umb.edu>

#include <stdio.h>
#include <string.h>

int str2num(char str[]);

int main(int argc, char *argv[])
{
	int i;
	int sum = 0;
	float avg;
	for (i = 0; i < argc; i++)
		sum += str2num(argv[i]);
	avg = sum / argc;
	printf("%.2f\n", avg);
}

int str2num(char str[])
{
	int i;
	int res = 0;
	for (i = 0; i < sizeof(str); i++)
		res = res * 10 + str[i] - '0';
	return res;
}
\end{lstlisting}

Her program produces the following output.

\begin{terminal}
$ gcc command-line.c -o command-line
$ ./command-line
-15202348.00
$ ./command-line 2 5 8
-1147083.00
$ ./command-line 1 2 3 4 5 6
-12761818.00
\end{terminal}

Clearly, something is wrong with Bellatrix's code.
This is why she is asking for your help to debug her program and fix all its bugs.
Rewrite the program \texttt{command-line.c} such that its output matches the expected output provided previously.


\documentclass[main.tex]{subfiles}
\begin{document}

\marginpar{Friday\\ 2020-3-20, \\ compiled \\ \today}

Today we will discuss another application of entanglement: Quantum Key Distribution.
For more details, see \cite[]{scaraniSecurityPracticalQuantum2009}.

\section{Quantum Key Distribution}

The idea is that it is possible to exchange two keys between two people, in an \emph{unconditional} way, as opposed to the current way of doing security, which is based on hard-to-solve problems.

If Alice and Bob have a shared key \(K\), they can use a \emph{one-time-pad} to communicate.

Say \(X\) is our message: then Alice constructs \(Y = X \oplus K\), which is completely random since \(K\) is.

When Bob receives the message, he does \(Y \oplus K = X \oplus K \oplus K = X\).
This is old classical cryptography, it was discovered by Shannon.

This works as long as \(K\) is indeed \emph{one-time}: if it is reused, an attacker can start reconstructing the message: otherwise, .

So, the QKD is about transmitting the key.
The most famous protocol to do this is BB84.
Most of the things which will be covered today are covered in
Rev Mod Phys 81, 1301 (2009).

The basis of the algorithm is a public quantum channel between Alice and Bob. The thing which is needed is for the channel to be \emph{verified}, so that Alice is sure to be talking to Bob.
Eve can be watching the passing qubits.

Alice can prepare four possible states, which she associates to two logical bits: \(\ket{0}\) and \(\ket{+}\) are associated with 0, while \(\ket{1}\) and \(\ket{-}\) are associated with 1.

Bob either measures \(\sigma_{z}\) or \(\sigma_{x}\).
It they use the same basis, Bob measures the same thing Alice sent.

If they use a different basis, then the result is random, since \(\abs{\braket{\pm}{0}}^2 =  \abs{\braket{\pm}{1}}= 1 / 2\).

So, there is a need to do \emph{sifting} later: after the transmission, Alice and Bob say publicly which bases they used.
With this information, they discard the qubits for which they used different bases.

Any measurement by the eavesdropper necessarily increases the Q-Bit Error Rate, which can be detected by Alice and Bob.

If Eve performs an ``Intercept and Resend'' attack, \(1/2\) of the time she will get the basis wrong, and those times she will create an error half of the time.
So, in this case we will have a QBER equal to \SI{25}{\percent}.

This can be measured by selecting, a posteriori, some qubits to be used as a check.

Alice could also use the states \(\ket{+i}\) and \(\ket{-i}\), which are eigenstates of \(\sigma_{y}\). Using these states we can increase the security.

The probabilities can be changed: we can have probabilities \(\epsilon \) for \(\sigma_{x}\) and \(\sigma_{y}\), while we use \(1-2\epsilon \) probability for \(\sigma_{z}\).

Then, we encode the key using only \(\sigma_{z}\), so we are communicating \(\approx1 -4 \epsilon  \) of the time.
We use the qubits encoded with \(\sigma_{x}\) and \(\sigma_{y}\) to measure how many times Eve has been measuring, since she will be always measuring \(\sigma_{z}\).

Eve cannot determine which basis is being used by Alice, since the mixed states corresponding to having \(\ket{0}\) or \(\ket{1}\) with equal probability and to having \(\ket{+}\) or \(\ket{-}\) with equal probability are the same, \(\rho = \frac{1}{2} \mathbb{1}\).

This works well if the devices are working as expected.
If the implementation is insecure, QKD can still be breached: so, we want to get \emph{device-independent} implementations.

A possible implementation is one in which we use an entangled source which gives us pairs with \(\ket{\Psi^{-}}\).

Then, Alice measures \(\sigma_{x}\) and \(\sigma_{z}\) and so does Bob.
Using the correlations between Alice and Bob we can determine whether the qubits are being generated entangled or not.
The state is
%
\begin{align}
\ket{\Psi^{-}} = \frac{1}{\sqrt{2}} \qty(\ket{01} - \ket{10})
= \frac{1}{\sqrt{2}} \qty(\ket{+-} - \ket{- +})
\,,
\end{align}
%
so the measurements are precisely anticorrelated for both bases 01 and +-.

So, we can do QKD with black boxes.
However, this is difficult to implement.

The receiver is more vulnerable to attacks, since they can receive any signal from the outside.
So, we want to do \emph{measurement device independent} coding: Alice and Bob each prepare the four states with equal probabilities, they send it to an untrusted station \(C\), in which there is a beamsplitter.
This performs a Bell measurement: if \(C\) sees a coincidence, then they communicate this to \(A\) and \(B\): this means that the measured state is \(\ket{\Psi^{-}}\). This means that the original qubits were opposite, but there is no information as to what the states originally were.

If the operator at \(C\) does anything else other than the Bell measurement, this can be detected by Alice and Bob by looking at the QBER.

By adding some more detectors we can also measure \(\ket{\Psi^{+}}\); then we do efficient BB84 by mostly sending in one basis.
This way, we can also get an efficiency of \(1- \epsilon \).

Now, how do we measure the secret Key Rate?
It is defined by
%
\begin{align}
r = I_{AB} - I_E
\,,
\end{align}
%
where \(I_{AB}\) is the mutual information of Alice and Bob, while \(I_E\) is the information of the Eavesdropper.

The definition of \(I_{AB}\) is:
%
\begin{align}
I_{AB} = H(A) - H(A|B)
\,,
\end{align}
%
where \(H(A)\) is the Shannon entropy of Alice:
%
\begin{align}
H(A) = - \sum_{x = 0,1} p_x \log_2  p_x
\,.
\end{align}

If the bits are random, then \(H(A )= 1\). On the other hand,
%
\begin{align}
H(A|B) = - \sum _{\substack{a=0,1 \\ b=0,1} } p_{ab} \log_2 p_{a|b}
\,,
\end{align}
%
which is zero if Alice and Bob's bits are perfectly correlated. It can be shown that
%
\begin{align}
H(A|B) = h_2  (Q) = -Q \log_2 Q - (1-Q) \log_2 (1-Q)
\,,
\end{align}
%
where \(Q\) is the error rate.

So, we write this as \(I_{AB} = 1 - h_2 (Q)\).
\(Q\) is the number of bits which are wrong, and the formula tells us that in order to correct these errors we need to reveal \(h_2 (Q)\).

It can be shown that the information of the eavesdropper in the BB84 case is
%
\begin{align}
I_E = h_2 (Q)
\,.
\end{align}

With the 6-state coding we have
%
\begin{align}
I_E = Q + (1- Q) h_2 \qty(\frac{1 - 3 Q / 2}{1 - Q})
\,.
\end{align}

This is purely quantum: classically there is no connection between \(I_E\) and \(Q\).
So, for BB84 the error rate must be \(Q \leq \SI{11}{\percent}\).

So, we need to do \emph{error correction} and then \emph{privacy amplification} so that the eavesdropper has no residual information.
Privacy amplification reduces the key length by a factor \(1/r\), so we get bits which are surely secure.

What we do, practically, is to use an attenuated laser. The state of the laser is a coherent one:
%
\begin{align}
\ket{\alpha } = e^{- \abs{\alpha }^2 / 2} \sum _{n} \frac{\alpha ^{n}}{\sqrt{n!}} \ket{k}
\,.
\end{align}

We can attenuate the laser so that we have \(\abs{\alpha } = \mu \), with \(\mu = 1\): usually we will have zero photons, often one, but sometimes \(2\): this is an issue, since the attacker can take one and leave the other: this is called Photon Number Splitting.

What can be done is to use a \emph{decoy state}: we use three values for the intensity, \(\mu = 1 \), but  also  \( \nu_1 = .1 \) and \(\nu_2 =  0\). Using this, we can see whether there is a PNS attack or not.

So, from the source point of view we have no issue in using a classical source like a laser.

\end{document}