\input{../header_overview}

%---------- start document ---------- %
\section{Introduction}
 NZMATH\cite{NZMATH} is a number theory oriented calculation system
mainly developed by the Nakamula laboratory at Tokyo Metropolitan University.
 NZMATH system provides you mathematical, especially number-theoretic computational power.
 It is freely available and distributed under the BSD license.
The most distinctive feature of NZMATH is that it is written entirely
using a scripting language called Python.

If you want to learn how to start using NZMATH,
see Installation (section \ref{installation}) and
Tutorial (section \ref{tutorial}).
%
 \subsection{Philosophy -- Advantages over Other Systems}
 In this section, we discuss philosophy of NZMATH, that is, the advantages of
NZMATH compared to other similar systems.
%
\subsubsection{Open Source Software}
%
Many computational algebra systems, such as Maple\cite{Maple},
Mathematica\cite{Mathematica}, and Magma\cite{Magma} are fare-paying systems.
These non-free systems are not distributed with source codes.
Then, users cannot modify such systems easily.
It narrows these system's potentials
for users not to take part in developing them.
NZMATH, on the other hand, is an open-source software
and the source codes are openly available.
Furthermore, NZMATH is distributed under the BSD license.
BSD license claims as-is and redistribution or commercial use are permitted
provided that these packages retain the copyright notice.
NZMATH users can develop it just as they like.
%
\subsubsection{Speed of Development}
%
We took over developing of SIMATH\cite{SIMATH},
which was developed under the leadership of Prof.Zimmer
at Saarlandes University in Germany.
However, it costs a lot of time and efforts to develop these system.
Almost all systems including SIMATH are
implemented in C or C++ for execution speed,
but we have to take the time to work memory management,
construction of an interactive interpreter,
preparation for multiple precision package and so on.
In this regard, we chose Python which is a modern programming language.
Python provides automatic memory management, a sophisticated interpreter
and many useful packages.
We can concentrate on development of mathematical matters by using Python.
%
\subsubsection{Bridging the Gap between Users And Developers}
%
KANT/KASH\cite{KANT} and PARI/GP\cite{PARI} are similar systems to NZMATH.
But programming languages for modifying these systems are
different between users and developers.
We think the gap makes evolution speed of these systems slow.
On the other hand, NZMATH has been developed with Python for bridging this gap.
Python grammar is easy to understand
and users can read easily codes written by Python.
And NZMATH, which is one of Python libraries, works on very wide platform
including UNIX/Linux, Macintosh, Windows, and so forth.
Users can modify the programs and feedback to developers with a light heart.
So developers can absorb their thinking.
Then NZMATH will progress to more flexible user-friendly system.
%
\subsubsection{Link with Other Softwares}
%
NZMATH distributed as a Python library
enables us to link other Python packages with it.
For example, NZMATH can be used with IPython\cite{IPython},
which is a comfortable interactive interpreter.
And it can be linked with matplotlib\cite{matplotlib},
which is a powerful graphic software.
Also mpmath\cite{mpmath}, which is a module for floating-point operation,
can improve efficiency of NZMATH.
In fact, the module ecpp\linkingzero{ecpp} improves performance with mpmath.
There are many softwares implemented in Python.
Many of these packages are freely available.
Users can use NZMATH with these packages
and create an unthinkable powerful system.
%
\subsection{Information}
  NZMATH has more than 25 modules.
These modules cover a lot of territory including
elementary number theoretic methods, combinatorial theoretic methods,
solving equations, primality, factorization,
multiplicative number theoretic functions,
matrix, vector, polynomial, rational field, finite field,
elliptic curve, and so on.
NZMATH manual for users is at:
\begin{verbatim}
http://tnt.math.se.tmu.ac.jp/nzmath/manual/
\end{verbatim}
If you are interested in NZMATH, please visit the official website
below to obtain more information about it.
\begin{verbatim}
http://tnt.math.se.tmu.ac.jp/nzmath/
\end{verbatim}
Note that NZMATH can be used even if users do not have any experience of
writing programs in Python.
%
\subsection{Installation}\label{installation}
In this section, we explain how to install NZMATH.
If you use Windows (Windows XP, Windows Vista, Windows 7 etc.)
as an operating system (OS),
then see \ref{windows install} ``Install for Windows Users''.
%
\subsubsection{Basic Installation}
There are three steps for installation of NZMATH.

%First, make preparation of computer with the Python installed.
First, check whether Python is installed in the computer.
Python 2.5 or a higher version is needed for NZMATH.
If you do not have a copy of Python, please install it first.
Python is available from \verb+http://www.python.org/+.

Second, download a NZMATH package and expand it.
It is distributed at official web site:
\begin{verbatim}
http://tnt.math.se.tmu.ac.jp/nzmath/download
\end{verbatim}
or at sourceforge.net:
\begin{verbatim}
http://sourceforge.net/project/showfiles.php?group_id=171032
\end{verbatim}
The package can be easily extracted, depending on the operating system.
For systems with recent GNU tar, type a single command below:
\begin{verbatim}
    % tar xf NZMATH-*.*.*.tar.gz
\end{verbatim}
where, \verb|%| is a command line prompt.
With standard tar, type
\begin{verbatim}
    % gzip -cd NZMATH-*.*.*.tar.gz | tar xf -
\end{verbatim}.
Please read *.*.* as the version number of which you downloaded the package.
For example, if the latest version is 1.0.0, then type the following command.
\begin{verbatim}
    % tar xf NZMATH-1.0.0.tar.gz
\end{verbatim}
Then, a subdirectory named NZMATH-*.*.* is created.

Finally, install NZMATH to the standard python path.
Usually, this can be translated into writing files somewhere
under \verb+/usr/lib+ or \verb+/usr/local/lib+.
So the appropriate write permission may be required at this step.
Typically, type commands below:
\begin{verbatim}
    % cd NZMATH-*.*.*
    % su
    # python setup.py install
\end{verbatim}

\subsubsection{Installation for Windows Users}\label{windows install}
We also distribute installation packages for specific platforms.
Especially, we started distributing the installer for Windows in 2007.

Please download the installer (NZMATH-*.*.*.win32Install.exe) from
\begin{verbatim}
http://tnt.math.se.tmu.ac.jp/nzmath/download
\end{verbatim}
or at sourceforge.net:
\begin{verbatim}
http://sourceforge.net/project/showfiles.php?group_id=171032
\end{verbatim}

Here, we explain a way of installing NZMATH with the installer.
First please open the installer.
If you use Windows Vista or higher version,
UAC (User Account Control) may ask if you run the program. click "Allow".
Then the setup window will open.
Following the steps in the setup wizard,
you can install NZMATH with only three clicks.
%
\subsection{Tutorial}\label{tutorial}
  In this section, we describe how to use NZMATH.
\subsubsection{Sample Session}
  Start your Python interpreter.
  That is, open your command interpreter such as Terminal for MacOS or bash/csh for linux, type the strings ``python'' and press the key Enter.
\begin{ex}
% python
Python 2.6.1 (r261:67515, Jan 14 2009, 10:59:13)
[GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
\end{ex}
  For windows users, it normally means opening IDLE (Python GUI), which is a Python software.
\begin{ex}
Python 2.6.1 (r261:67517, Dec  4 2008, 16:51:00) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.

    ****************************************************************
    Personal firewall software may warn about the connection IDLE
    makes to its subprocess using this computer's internal loopback
    interface.  This connection is not visible on any external
    interface and no data is sent to or received from the Internet.
    ****************************************************************

IDLE 2.6.1
>>>
\end{ex}
Here, '\verb+>>>+' is a Python prompt, which means that the system waits you to input commands.

Then, type:
\begin{ex}
>>> from nzmath import *
>>>
\end{ex}
This command enables you to use all NZMATH features.
If you use only a specific module (the term ``module'' is explained later), for example, prime, type as the following:
\begin{ex}
>>> from nzmath import prime
>>>
\end{ex}
You are ready to use NZMATH.
For example, type the string ``prime.nextPrime(1000)'',
then you obtain `1009''
as the smallest prime among numbers greater than $1000$.
\begin{ex}
>>> prime.nextPrime(1000)
1009
>>>
\end{ex}
``prime'' is a name of a module, which is a NZMATH file including Python codes.
``nextPrime'' is a name of a function, which outputs values after the system executes some processes for inputs.
NZMATH has various functions for mathematical or algorithmic computations.
See \ref{function} Functions.

Also, we can create some mathematical objects.
For example, you may use the module ``matrix''.
If you want to define the matrix
\begin{equation*}
\left(
\begin{array}{rl}
1 & 2\\
5 & 6\\
\end{array}
\right)
\end{equation*}
and compute the square, then type as the following:
\begin{ex}
>>> A = matrix.Matrix(2, 2, [1, 2]+[5, 6])
>>> print A
1 2
5 6
>>> print A ** 2
11 14
35 46
>>>
\end{ex}
``Matrix'' is a name of a class, which is a template of mathematical objects.
See \ref{class} Classes for using NZMATH classes.

The command ``print'' enables us to represent outputs with good-looking forms.
The data structure such as ``[a, b, c, $\cdots$]'' is called list.
Also, we use various Python data structures like
tuple ``(a, b, c, $\cdots$)'',
dictionary ``$\{x_1:y_1, x_2:y_2, x_3:y_3, \cdots\}$'' etc.
%These follow syntax of Python programming language.
Note that we do not explain Python's syntax in detail
because it is not absolutely necessary to use NZMATH.
However, we recommend that you learn Python for developing your potential.
Python grammar are easy to study.
For information on how to use Python,
see \verb+http://docs.python.org+ or many other documents about Python.

 % \subsection{Python $\&$ NZMATH}
 %  \subsubsection{Save to a file}
 \subsection{Note on the Document}
   \paragraph{$\dagger$}
   Some beginnings of lines or blocks such as sections or sentences
    may be marked $\dagger$.
   This means these lines or blocks is for advanced users.
   For example, the class \textit{FiniteFieldElement} %
   (See \linkingone{finitefield}{FinitePrimeFieldElement}) is %
   one of abstract classes in NZMATH, %
   which can be inherited to new classes similar to the finite field.
   \paragraph{[$\cdots$]}
   For example, we may sometimes write as \textit{function(a,b[,c,d])}.
   It means the argument ``c, d'' or only ``d'' can be discarded.
   Such functions use ``default argument values'',
   which is one of the feature of Python.\\
   (See
   \verb+http://docs.python.org/tutorial/controlflow.html#default-argument-values+)

   \textbf{Warning}:\ Python also have the feature ``keyword arguments''.
   We have tried to keep the feature in NZMATH too.
   However, some functions cannot be used with this feature
   because these functions are written
   expecting that arguments are given in order.

%---------- end document ---------- %

\input{../footer}


% Document Type: LaTeX
\input ../6001mac

\def\fbox#1{%
 \vtop{\vbox{\hrule%
    \hbox{\vrule\kern3pt%
\vtop{\vbox{\kern3pt#1}\kern3pt}%
\kern3pt\vrule}}%
\hrule}}

\begin{document}

\psetheader{SAMPLE}{Simple Problem Set}

\medskip

\begin{flushleft}
Reading:
\begin{itemize}

\item Text (SICP 2nd Edition by Abelson \& Sussman): section 1.1

\end{itemize}
\end{flushleft}

\smallskip
The purpose of the exercises below is to familiarize you with the
basics of Scheme language and interpreter.  Spending a little time on
simple mechanics now will save you a great deal of time over the rest
of the semester.

\section{1. Before Turning on your Computer}

Read the course notes through section 1.1.  Then, predict what the
interpreter will print in response to evaluation of each of the following
expressions.  Assume that the sequence is evaluated in the order in which
it is presented here.

\beginlisp
(- 8 9)
\null
(> 3.7 4.4)
\null
(- (if (> 3 4) 7 10) (/ 16 10))
\null
(define b 13)
\null
13
\null
b
\null
>
\null
(define square (lambda (x) (* x x)))
\null
square
\null
(square 13)
\null
(square b)
\null
(square (square (/ b 1.3)))
\null
(define multiply-by-itself square)
\null
(multiply-by-itself b)
\null
(define a b)
\null
(= a b)
\null
(if (= (* b a) (square 13))
    (< a b)
    (- a b))
\null
(cond ((>= a 2) b)
      ((< (square b) (multiply-by-itself a)) (/ 1 0))
      (else (abs (- (square a) b))))
\endlisp

\section{2. Getting started with the Scheme Interpreter}

In the Scheme interpreter, you will use a text-editing system called Edwin,
which is an editor closely resembling the widely used editor Emacs.  Even
if you are already familiar with Emacs, you should take some time {\it now}
to run the Emacs/Edwin tutorial.  This can be invoked by typing {\tt C-h}
followed by {\tt t}.  You will probably get bored before you finish the
tutorial. (It's too long, anyway.)  But at least skim all the topics so
you know what's there.  You'll need to gain reasonable facility with the
editor in order to complete the exercises below.

\subsection{Evaluating expressions}

After you have learned something about Edwin, go to the Scheme buffer
(i.e., the buffer named {\tt *scheme*}).\footnote{If you don't know
how to do this, go back and learn more about Edwin.}  As with any
buffer, you can type Scheme expressions, or any other text into this
buffer.  What distinguishes the Scheme buffer from other buffers is
the fact that underlying this buffer is a Scheme evaluator, which you
can ask to evaluate expressions, as explained in the section of the
tutorial entitled ``Evaluating Scheme expressions.''

Type in and evaluate (one by one) the expressions from section 1 of
this assignment to see how well you predicted what the system would
print.  If the system gives a response that you do not understand, ask
for help from a lab tutor or from the person sitting next to you.

Observe that some of the examples printed above in section 1 are
indented and displayed over several lines for readability.  An
expression may be typed on a single line or on several lines; the
Scheme interpreter ignores redundant spaces and carriage returns.  It
is to your advantage to format your work so that you (and others) can
read it easily. It is also helpful in detecting errors introduced by
incorrectly placed parentheses.  For example the two expressions

\beginlisp
(* 5 (- 2 (/ 4 2) (/ 8 3)))
\null
(* 5 (- 2 (/ 4 2)) (/ 8 3))
\endlisp

\noindent
look deceptively similar but have different values. Properly indented,
however, the difference is obvious.

\beginlisp
(* 5
   (- 2
      (/ 4 2)
      (/ 8 3)))
\null
(* 5
   (- 2
      (/ 4 2))
   (/ 8 3))
\endlisp

Edwin provides several commands that ``pretty-print'' your code, e.g.,
indents lines to reflect the inherent structure of the Scheme
expressions.\footnote{Make a
habit of typing {\tt ctrl-j} at the end of a line, instead of {\tt
enter}, when you enter Scheme expressions, so that the cursor will
automatically indent to the right place.}

\subsection{Creating a file}

Since the Scheme buffer will chronologically list all the expressions
you evaluate, and since you will generally have to try more than one
version of the same procedure as part of the coding and debugging
process, it is usually better to keep your procedure definitions in a
separate buffer, rather than to work only in the Scheme buffer.  You
can save this other buffer in a file on your disk so you can split
your lab work over more than one session.

The basic idea is to create another buffer, into which you can type your
code, and later save that buffer in a disk file.  You do this by
typing {\tt C-x C-f} {\it filename}.  If you already have a buffer
open for that file Edwin simply switches you into this buffer.
Otherwise, Edwin will create a new buffer for the file.  If you give
the system a name that has not yet been used, with an extension of
{\tt .scm}, Edwin will automatically create a new buffer with that
file name, in {\tt scheme} mode.  In a Scheme-mode buffer some editing
commands treat text as code, for example, typing {\tt C-j} at the end
of a line will move you to the next line with appropriate indentation.

Once you are ready to transfer your procedures to Scheme, you can use any
of several commands: {\tt M-z} to evaluate the current definition, {\tt
ctrl-x ctrl-e} to evaluate the expression preceding the cursor, {\tt M-o}
to evaluate the entire buffer, or using {\tt M-x eval-region} to evaluate
a ``marked'' region in the buffer.  Each of these commands will cause the
Scheme evaluator to evaluate the appropriate set of expressions (usually
definitions).  By returning to the {\tt *scheme*} buffer, you can now use
these expressions.

To practice these ideas, create a new buffer, called {\tt
ps1-ans.scm}.  In this buffer, create a definition for a simple
procedure, by typing in the following (verbatim):

\beginlisp
(define square (lambda (x) (*x x)))
\endlisp

\noindent
Now evaluate this definition by using either {\tt M-z} or {\tt M-o}.
Go back to the Scheme buffer, and try evaluating {\tt (square 4)}.

If you actually typed in the definition {\em exactly} as printed above,
you should have hit an error at this point.  The system will now be
offering you a chance to enter the debugger.  For now, type {\tt Q} to
quit the evaluation (we'll see how to use the debugger below).  Go back to
the {\tt ps1-ans.scm} buffer and edit the definition to insert a space
between {\tt *} and {\tt x}.  Re-evaluate the definition, and return to
Scheme and try evaluating {\tt (square 4)} again.

As a second method of evaluating expressions, try the following.  Go
to the {\tt *scheme*} buffer, and again type in:

\beginlisp
(define square (lambda (x) (*x x)))
\endlisp

Place the cursor at the end of the line, and type {\tt C-x C-e} to
evaluate this expression.  Again try {\tt (square 4)}.  When it fails,
again use {\tt Q} to quit out of the debugger.  This time, you should
type {\tt M-p} several times, until the definition of {\tt square}
that you typed in appears on the screen.  Edit this definition to
insert a space between {\tt *} and {\tt x}, evaluate the new
expression, and use {\tt M-p} to get back the expression {\tt (square
4)}.  Make a habit of using {\tt M-p}, rather than going back and
editing previous expressions in the Scheme buffer in place.  That
way, the buffer will contain an intact record of your work.

\section{3. The Scheme Debugger}

While you work, you will often need to debug programs.  This
section contains an exercise to acquaint you with some of the features
of Scheme to aid in debugging.  Learning to use the debugging features
will save you much grief on later problem sets.  Additional
information about the debugger can be found by typing {\tt ?} in the debugger.

Load the code for this problem set using the Edwin {\tt C-x C-f}
command. Evaluate the code using {\tt M-x eval-buffer}.  This will
load definitions of the following three procedures {\tt p1}, {\tt p2}
and {\tt p3}:

\beginlisp
(define p1
  (lambda (x y)
    (+ (p2 x y)
       (p3 x y))))
\null
(define p2
  (lambda (z w)
    (* z w)))
\null
(define p3
  (lambda (a b)
    (+ (p2 a)
       (p2 b))))
\endlisp

In the Scheme buffer, evaluate the expression {\tt (p1 1 2)}.  This
should signal an error, with the message:

{\small
\begin{verbatim}
;The procedure #[compound-procedure P2] has been called with 1 argument
;it requires exactly 2 arguments.
;Type D to debug error, Q to quit back to REP loop:
\end{verbatim}
}

Don't panic.  Beginners have a tendency, when they hit an error, to
quickly type {\tt Q}, often without even reading the error message.
Then they stare at their code in the editor trying to see what the bug
is.  Indeed, the example here is simple enough so that you probably
can find the bug by just reading the code.  Instead, however, let's
see how Scheme can be coaxed into producing some helpful information
about the error.

First of all, there is the error message itself.  It tells you that
the error was caused by a procedure being called with one argument,
which is the wrong number of arguments for that procedure.
Unfortunately, the error message alone doesn't say where in the code
the error occurred.  In order to find out more, you need to use the
debugger.  To do this type {\tt D} to start the debugger.

\subsection{Using the debugger}

The debugger allows you to grovel around examining pieces of the
execution in progress, in order to learn more about what may have
caused the error.  When you start the debugger, it will create a new
window showing two buffers.  The top buffer should look like this.

{\small
\begin{verbatim}
     COMMANDS:   ? - Help   q - Quit Debugger   e - Environment browser
This is a debugger buffer:
Lines identify stack frames, most recent first.
   Sx means frame is in subproblem number x
   Ry means frame is reduction number y
The buffer below describes the current subproblem or reduction.
-----------
The *ERROR* that started the debugger is:
  The procedure #[compound-procedure 119 p2] has been called with 1 argument;
  it requires exactly 2 arguments.

>S0  (#[compound-procedure 119 p2] 2)
    R0  (p2 b)
 S1  (+ (p2 a) #(p2 b)#)
    R0  (+ (p2 a) (p2 b))
    R1  (p3 x y)
 S2  (+ (p2 x y) #(p3 x y)#)
    R0  (+ (p2 x y) (p3 x y))
    R1  (p1 1 2)
 --more--
\end{verbatim}
}

You can select a ``frame'' by clicking on its line with the mouse or by
using the ordinary cursor line-motion commands to move from line to line.
Notice that the bottom, information, buffer changes as the selected line
changes.

The frames in the list in the top buffer represent the steps in the
evaluation of the expression.  There are two kinds of steps---subproblems
and reductions.  For now, you should think of a reduction step as
transforming an expression into ``more elementary'' form, and think of a
subproblem as picking out a piece of a compound expression to work on.

So, starting at the bottom of the list and working upwards, we see
{\tt (p1 1 2)}, which is the expression we tried to evaluate.  The
next line up indicates that {\tt (p1 1 2)} reduces to {\tt (+
(p2 x y) (p3 x y))}.  Above that, we see that in order to evaluate
this expression the interpreter chose to work on the subproblem {\tt
(p3 x y)}, and so on, moving upwards until we reach the error: the
call to {\tt (p2 b)} from within the procedure {\tt p3} has only one
argument, and {\tt p2} requires two arguments.\footnote{Notice that
the call that produced the error was {\tt (p2 b)}, and that {\tt (p2
a)} would have also given an error.  This indicates that in this case
Scheme was evaluating the arguments to {\tt +} in right-to-left order,
which is something you may not have expected.  You should never write
code that depends for its correct execution on the order of evaluation
of the arguments in a combination.  The Scheme system does not
guarantee that any particular order will be followed, nor even
that this order will be the same each time a combination is
evaluated.}

Take a moment to examine the other debugger information (which will
come in handy as your programs become more complex).  Specifically, in
the top buffer, select the line
{\small
\begin{verbatim}
>S2  (+ (p2 x y) #(p3 x y)#)
\end{verbatim}
}

The bottom buffer should now look like this:
{\small
\begin{verbatim}
                         SUBPROBLEM LEVEL: 2
Expression (from stack):
 Subproblem being executed highlighted.
    (+ (p2 x y) (p3 x y))
---------------------------------------------------------------------
ENVIRONMENT named: (user)
    p1 = #[compound-procedure 31 p1]
    p3 = #[compound-procedure 32 p3]
    p2 = #[compound-procedure 27 p2]
==> ENVIRONMENT created by the procedure: P1
      x = 1
      y = 2
---------------------------------------------------------------------
;EVALUATION may occur below in the environment of the selected frame.
\end{verbatim}
}

The information here is in three parts.  The first shows the
expression again, with the subproblem being worked on.  The next major
part of the display shows information about the {\it environments}.
We'll have a lot more to say about environments later in the course,
but, for now, notice the line
{\small
\begin{verbatim}
==> ENVIRONMENT created by the procedure: P1
\end{verbatim}
}
This indicates that the evaluation of the current expression is within
procedure {\tt p1}.  Also we find the environment has two {\it bindings}
that specify the particular values of {\tt x} and {\tt y} referred
to in the expression, namely {\tt x = 1} and {\tt y = 2}.  At the bottom
of the description buffer is an area where you can evaluate
expressions in this environment (which is often useful in debugging).

Before quitting the debugger try one final experiment (you may have
already done this).  Continue to scroll down through the stack past
the line: {\tt R1 (p1 1 2)} (you can also click the mouse on the line
{\tt --more--} to show the next subproblem).  You will then see
additional frames that various complicated expressions.  What you
are looking at is some of the guts of the Scheme system---the part
shown here is a piece of the interpreter's read-eval-print program.
In general, backing up from any error will eventually land you in the
guts of the system. (Yes: almost all of the system is itself a Scheme
program.)

You can type {\tt q} to return to the Scheme top level interpreter.

\section{4. Exploring the system}

The following exercises are meant to help you practice editing and
debugging, and using on-line documentation.

\paragraph{Exercise 1:  More debugging}

The code you loaded for problem set 1 also defined three other procedures,
called {\tt fold}, {\tt spindle}, and {\tt mutilate}.  One of these
procedures contains an error.  Evaluate the expression {\tt (fold 1 2)}.
What is the error?  How should the procedure be defined?  (Notice that you
can examine the code for a procedure by using the {\tt pp} (``pretty
print'') command.  For example, evaluating the expression {\tt (pp fold)}
will print the definition of {\tt fold}.

\paragraph{Exercise 2: Still more debugging}

The code you loaded also contains a buggy definition of a procedure meant
to compute the factorials of positive integers:
$n!=n\cdot(n-1)\cdot(n-2)\cdots3\cdot2\cdot1$.  Evaluate the expression
{\tt (fact 5)} (which is supposed to return 120).  Use the debugger to
find the bug and correct the definition.  Compute the factorial of 243.
Copy the corrected definition and the value of {\tt (fact 243)} into your
{\tt ps1-ans.scm} buffer.

\paragraph{Exercise 3: Defining a simple procedure}

The number of combinations of $n$ things taken $k$ at a time is given
by $n!/k!(n-k)!$.  Define a procedure {\tt (comb n k)} that computes
the number of combinations, and find the number of combinations of 243
things taken 90 at a time.  Include your procedure definition and your
answer in {\tt ps1-ans.scm}.

\paragraph{Exercise 4: Practice with the editor}

Find the Free Software Foundation copyright notice at the end of the Edwin
tutorial.  Copy it into {\tt ps1-ans.scm}.

\paragraph{Exercise 5: Learning to use {\tt Info}}

Start up the {\tt info} program with the Edwin command {\tt M-x info}.
{\tt Info} is a directory of useful information.  You select a topic
from the menu, by typing {\tt m} followed by the name of the topic.
(You can type a few characters that begin the topic name, and then type
a space, and Edwin will complete the name.  One of the {\tt info}
options (type {\tt h}) gives you a brief tutorial in how to use the
program.  Use {\tt info} to find the cost of a 12-inch cheese pizza
from Pizza Ring, and copy this to the answer buffer.

\paragraph{Exercise 6: Hacker Jargon}

The info {\tt Jargon} entry is a collection of hacker terms that was
eventually published in 1983 as the book {\it The Hacker's
Dictionary}.  The {\tt New Jargon} info entry is an expanded version
that was published in 1991 by MIT Press as {\it The New Hacker's
Dictionary}.  The old Jargon file is structured as an {\tt info}
file, with submenus; the new version is a single text file, which you
can search.  Find the definition of ``Unix conspiracy'' from the new
jargon file, and find the definition of ``Phase of the Moon'' from
either jargon file.  Include these in the answer buffer.

\paragraph{Exercise 7: Scheme documentation}

The {\tt Scheme} info entry is an on-line copy of the Scheme
reference manual that was distributed with the notes.  Find the
description of ``identifiers'' in the documentation.  What is the
longest identifier in the list of example identifiers?

\paragraph{Exercise 8: More documentation}

There is an Edwin command that you can use to automatically re-indent
expressions.  For example, if you have a procedure typed as

\beginlisp
(define test-procedure
(lambda (a b)
(cond ((>= a 2) b)
   ((< (square b)
 (multiply-by-itself a))
         (/ 1 0))
(else (abs (- (square a) b))))))
\endlisp

\noindent
you can move the cursor to the beginning of the line with the {\tt
define}, type this Edwin command, and the expression will
automatically be re-indented as

\beginlisp
(define test-procedure
  (lambda (a b)
    (cond ((>= a 2) b)
          ((< (square b)
              (multiply-by-itself a))
           (/ 1 0))
          (else (abs (- (square a) b))))))
\endlisp

\noindent
Find the description of this command.  What is the command?  Where did
you find it?

\paragraph{Exercise 9: Running Shell Commands}

If you run the Scheme interpreter within a *nix environment, the Edwin command
{\tt M-x shell} will create a shell buffer, which you can use to run
various *nix programs.  Some of the more interesting ones are {\tt ls}, {\tt
date}, and {\tt echo}.
For example, typing

\beginlisp
date
\endlisp

\noindent
will show you the current date and time (e.g. {\tt Tue Jul  2 15:39:45 EDT
1996}). See if you can figure out what the other commands do, and copy your
results into your answer buffer.

\paragraph{Exercise 10: Getting information from around the world}

You can also use the finger program to query computers on the
Internet, all over the world.  You can finger a particular name at
some location, or just finger the location (e.g., {\tt finger
@mit.edu}) to get general information.  Try fingering some of the
following places:

\begin{itemize}
\item {\tt cs.berkeley.edu}---a computer run by the computer science
department at UC Berkeley

\item {\tt idt.unit.no}---a computer at the Norwegian Institute of
Technology, a part of the The University of Trondheim, Norway.

\item {\tt whitehouse.gov}---the White House
\end{itemize}

See what else you can find.  Include some piece of this in your answer
file.

\paragraph{Exercise 11:}
An {\em application} of an expression $E$ is an expression of the form
$(E\, E_1 \ldots E_n)$.  This includes the case $n = 0$, corresponding to
an expression $(E)$.  A {\em Curried application} of $E$ is either an
application of $E$ or an application of a Curried application of E.  For
each of the procedures defined below, give a Curried application of the
procedure which evaluates to 3.

\beginlisp
(define foo1
  (lambda (x)
    (* x x)))
\endlisp

\beginlisp
(define foo2
  (lambda (x y)
      (/ x y)))
\endlisp

\beginlisp
(define foo3
  (lambda (x)
    (lambda (y)
      (/ x y))))
\endlisp

\beginlisp
(define foo4
  (lambda (x)
    (x 3)))
\endlisp

\beginlisp
(define foo5
  (lambda (x)
    (cond ((= x 2)
           (lambda () x))
          (else
           (lambda () (* x 3))))))
\endlisp

\beginlisp
(define foo6
  (lambda (x)
    (x (lambda (y) (y y)))))
\endlisp

Type in these definitions, and include a demonstration of your answers
in the answer buffer.

\end{document}


\documentclass[12pt]{article}
\author{David Alves}

\usepackage{amsfonts}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{dirtytalk}
\usepackage[a4paper, total={6.5in, 8.75in}]{geometry}
\usepackage{forest}
\usepackage{skak}
\usepackage{tikz}
\usepackage{titling}
\usepackage{wrapfig}
\usepackage{xcolor}

\def\multichoose#1#2{\ensuremath{\left(\kern-.3em\left(\genfrac{}{}{0pt}{}{#1}{#2}\right)\kern-.3em\right)}}

\title{Math 142 Problem Set 6}
\author{David Alves}
\date{2016-10-06}

\begin{document}
\pagenumbering{gobble}

\begin{center}
\large \thetitle \\
\theauthor \\
\thedate
\end{center}

\subsection*{Sources}

    \begin{itemize}
    \item http://tex.stackexchange.com and https://www.sharelatex.com for help with \LaTeX
    \end{itemize}

\section{Some Chairs are Dry}

\subsection*{Problem Statement}
Find the exact problem with the following proof: \say{We prove that all sets of $n$ chairs are wet, $n \geq 0$ an integer, by induction. For base case, it is (vacuously) true that for all chairs in an empty set of 0 chairs, they are wet. Suppose our statement is true for $n$. Consider a set of $(n+1)$ chairs. Sit on a chair $C$. The chairs not counting $C$ form a set of $n$ chairs. By assumption, they are all wet. Now get off your chair $C$ and sit on a different chair. Now the other chairs, including $C$, form a set of $n$ chairs. By assumption, these chairs are all wet, so $C$ is wet. Since $C$ is wet and all other chairs are wet, all $(n+1)$ chairs are wet and the inductive step is complete.}
\subsection*{Solution}

The inductive step requires at least one wet chair in the set of $n$ wet chairs (the one which gets swapped with $C$) in order to prove $n+1$, but the base case is 0 chairs. Therefore the inductive step is not valid for $n=0$ and the proof fails. (Note that if the base case was $n=1$, \say{All sets of 1 chair are wet} then the proof would work.)

\section{Rooks on a Chessboard}

\subsection*{Problem Statement}
Find the number of ways to place three rooks on an $8\times8$ chessboard such that no rooks attack each other (meaning no two rooks are in the same row or column).
\subsection*{Solution}

There are 18816 ways to place three indistinguishable rooks onto an $8\times8$ chessboard so that no two rooks can attack each other.

\begin{proof}
In order for rooks not to attack each other, they need to all be on different ranks and all be different files. Label the rooks $R_1, R_2, R_3$. There are $8 \text{ ranks} \times 8 \text{ files} = 64$ ways to place $R_1$ on an empty board. There are 7 empty ranks and 7 empty files left for $R_2$, giving 49 choices. Finally there are 6 empty ranks and 6 empty files left for $R_3$. This gives $64\times49\times36=112896$ ways to place rooks if they were distinguishable. We then divide this result by $3!$ permutations of the labelled rooks due to the quotient principle (since rooks are indistinguishable) to get $\frac{112896}{3!} = 18816$ placements.
\end{proof}

\section{Digit Lists}

\subsection*{Problem Statement}
How many $n$-digit lists of digits 0 through 9 are there such that the digits 1,2,3, and 9 all appear at least once?
\subsection*{Solution}

\newcommand{\dlist}[1]{D_{#1}}
There are $10^n - 4(9^n) + 6(8^n) - 4(7^n) + 6^n$ $n$-digit lists which contain each of the digits 1,2,3 and 9 one or more times.

\begin{proof}
Let $D_k$ be the set of length-$n$ lists which do not contain the digit $k$. The set of length-$n$ lists which is missing at least one of the digits 1, 2, 3, and 9 is therefore $D_1 \cup D_2 \cup D_3 \cup D_9$. By the inclusion-exclusion principle, we have

\begin{multline}\label{eq:ps6_3_1}
    |\dlist{1} \cup \dlist{2} \cup \dlist{3} \cup \dlist{9}| = |\dlist{1}| + |\dlist{2}| + |\dlist{3}| + |\dlist{9}|\\
    -|\dlist{1} \cap \dlist{2}| - |\dlist{1} \cap \dlist{3}| - |\dlist{1} \cap \dlist{9}| - |\dlist{2} \cap \dlist{3}| - |\dlist{2} \cap \dlist{9}| - |\dlist{3} \cap \dlist{9}|\\
    +|\dlist{1} \cap \dlist{2} \cap \dlist{3}| + |\dlist{1} \cap \dlist{2} \cap \dlist{9}| + |\dlist{1} \cap \dlist{3} \cap \dlist{9}| + |\dlist{2} \cap \dlist{3} \cap \dlist{9}|\\
    -|\dlist{1} \cap \dlist{2} \cap \dlist{3} \cap \dlist{9}|
\end{multline}

The size of $D_k$ for any digit $k$ is $9^n$ since there are 9 possibilities for each digit (anything other than $k$). The size of $D_a \cap D_b : a \neq b$ is $8^n$ since each digit can be anything except $a$ and $b$. Similarly, the size of $D_a \cap D_b \cap D_c: (a,b,c \text{ all distinct})$ is $7^n$ and the size of $D_a \cap D_b \cap D_c \cap D_d: (a,b,c,d \text{ all distinct})$ is $6^n$. Substituting these values into equation \ref{eq:ps6_3_1} gives $
    |\dlist{1} \cup \dlist{2} \cup \dlist{3} \cup \dlist{9}| = 4(9^n) - 6(8^n) + 4(7^n) - 6^n$.

By the sum principle, the number of digit lists of size $n$ is equal to the number of size-$n$ lists where all of the digits $1,2,3,9$ appear plus the number of size-$n$ lists where one or more of the digits $1,2,3,9$ does not appear, which is $|\dlist{1} \cup \dlist{2} \cup \dlist{3} \cup \dlist{9}|$. The total number of lists of length $n$ is $10^n$ since there are 10 possibilities for each digit, therefore the number of lists of size $n$ where each digit $1,2,3,9$ appears one or more times is $10^n - 4(9^n) + 6(8^n) - 4(7^n) + 6^n$
\end{proof}

\section{Black \& White Graphs}
\subsection*{Problem Statement}
Consider a complete graph on 6 vertices. Color some of the edges white and some of the edges black. Prove that there must be a triangle with all white edges or all black edges somewhere in the graph. (A triangle with all white edges would be some 3 vertices $a,b,c$ such that the edges $(a,b), (a,c), (b,c)$ are all white)
\subsection*{Solution}
\begin{proof}
Assume that an edge coloring exists on $K_6$ such that each edge is either black or white and there are no all-black or all-white triangles. Consider a vertex $v$. $v$ is connected to the other five vertices by five edges. At least three of these edges must share some color $c$. (If more than three share a color, consider only three of them). Let the three vertices connected to $v$ by $c$-colored edges be $x$, $y$, and $z$. Since $(v,x)$ and $(v,y)$ are colored $c$, $(x,y)$ must be non-$c$-colored otherwise $(v,x), (v,y), (x,y)$ would form a $c$-colored triangle which would contradict our assumption. Similarly since $(v,x)$ and $(v,z)$ are $c$-colored, $(x,z)$ must be non-$c$-colored, and since $(v,y)$ and $(v,z)$ are $c$-colored, $(y,z)$ must be a non-$c$-colored. Since $(x,y)$, $(x,z)$, and $(y,z)$ are all non-$c$-colored, it is an all-black or all-white triangle, which contradicts our assumption. Therefore the assumption is incorrect, and there are no edge colorings on $K_6$ such that there are no all-black or all-white triangles.\\
\end{proof}

\colorlet{verylightgray}{black!45}
\begin{center}
\begin{tikzpicture}
\node[shape=circle,draw=black] (A) at (0,0) {$v$};
    \node[shape=circle,draw=black] (B) at (4,0) {$z$};
    \node[shape=circle,draw=black] (C) at (1.24,3.8) {$y$};
    \node[shape=circle,draw=black] (D) at (-3.24, 2.35) {$x$};
    \node[shape=circle,draw=black] (E) at (-3.24, -2.35) { };
    \node[shape=circle,draw=black] (F) at (1.24,-3.8) { } ;

    \path [-,line width=1pt] (A) edge node[left] {} (B);
    \path [-,line width=1pt] (A) edge node[left] {} (C);
    \path [-,line width=1pt] (A) edge node[left] {} (D);
    \path [-,color=verylightgray] (A) edge node[left] {} (E);
    \path [-,color=verylightgray] (A) edge node[left] {} (F);
    \path [-,line width=1pt,style=dashed] (B) edge node[left] {} (C);
    \path [-,line width=1pt,style=dashed] (B) edge node[left] {} (D);
    \path [-,color=verylightgray] (B) edge node[left] {} (E);
    \path [-,color=verylightgray] (B) edge node[left] {} (F);
    \path [-,line width=1pt,style=dashed] (C) edge node[left] {} (D);
    \path [-,color=verylightgray] (C) edge node[left] {} (E);
    \path [-,color=verylightgray] (C) edge node[left] {} (F);
    \path [-,color=verylightgray] (D) edge node[left] {} (E);
    \path [-,color=verylightgray] (D) edge node[left] {} (F);
    \path [-,color=verylightgray] (E) edge node[left] {} (F);
\end{tikzpicture}
\end{center}

\section{Friends at a Party}
\subsection*{Problem Statement}
Thirteen people meet at a party. Then none or some of the people decide to send Facebook friend requests to some of the other people at the party. How many outcomes are there? (Two outcomes are equivalent if the same set of friend requests were made, where a request only depends on the identity of the person asking and the person asked.)
\subsection*{Solution}

There are $2^{156}$ possible outcomes.

\begin{proof}
Let $S$ be the set of people at the party. A friend request can be represented as a two-element list $(a, b)$ where $a$ is the sender and $b$ is the recipient and $a \neq b$. Thus for each of the 13 senders, there are 12 possible recipients making a set of 156 possible friend requests by the product principle. An outcome is a subset of this set, thus there are $2^{156}$ possible outcomes.
\end{proof}

\section{Time Spent \& Thoughts}

This problem set was about average difficulty. I spent about six hours, mostly on the Black \& White problem. Typing up the Digit Lists problem also took a while, although the problem itself wasn't hard.

 The black and white graphs problem had me stuck for a while before I figured out the trick of starting with a single vertex and saying that there must be at least 3 outgoing edges with the same color.  I spent a while trying to prove that a set of 8 edges on $K_6$ guarantees a triangle (since you are partitioning 15 edges into two groups, one must have size 8 or more) but that was a more difficult approach. There doesn't seem to be any clear connection between the black and white problem and what we've been doing in this class, so I'm not sure what the idea behind including it was --- just more practice with proofs?
\end{document}


\begin{center}
{{\Large \sc High-Performance Computing}}
\end{center}
\rule{\textwidth}{1pt}
\begin{description}
\item[Student name and id:] Marie Mørk \{s112770\} Anja Liljedahl Christensen\{s162876\}
 Andreas Vedel Jantzen \{s162858\}
 Anders Launer Bæk \{s160159\}
\item[Hold name:] matmult\_MAAA
\item[Hand-in:] Matrix Multiplication
\end{description}
\rule{\textwidth}{1pt}

\section{Summary}
Different functions, performing matrix-matrix multiplication are implemented in C in order to compare performance. Cases considered are ordering of the nested loops, optimizers in the compiler and blocking. The implemented functions are compared with the CBLAS subroutine DGEMM. As memory in C is stored row-wise, it has been found that the most optimal ordering of nested loops is to have the column index in the most inner loop. The optimizers are found to enhance performance significantly. The blocked version is found to be less efficient than the optimized version of the most efficient ordering of the loops. This might be due to the implementation of the blocking in which as many columns as rows are considered at a time.

\section{Statement of the problem}

The goal of the assignment is to create a library of functions that perform matrix-matrix multiplications. Performance of the implementations are to be compared with the DGEMM matrix-matrix multiplication routine from the BLAS library. As input, each of the functions should take the dimensions of the matrices (M, N, K) as well as the matrices (A, B, C), according to figure \ref{fig:matmult}. The problem is divided into four sub-problems;
\begin{itemize}
    \item Wrap the call DGEMM into a function called \texttt{matmult\_lib()}, that takes the same inputs as
    specified above.
    \item Implement a function that performs double precision matrix-matrix multiplication. The function should be named \texttt{matmult\_nat()}.
    \item Functions for each of the permutations of the three nested loops needed to perform matrix multiplications. The performance of the permutations should be compared. Furthermore, the timings should be compared with and without different compiler optimizations. The functions should be named as \texttt{matmult\_per()}, where \texttt{per} is the ordering of the nested loops for the given implementation, e.g. \texttt{nmk,knm}, ect.
    \item A blocked version of the matrix-matrix multiplication function named \texttt{matmult\_blk()}. This function should take an extra input, \texttt{bs}, which specifies the blocking size.
\end{itemize}

\begin{figure}[h]
    \centering
    \includegraphics[]{contents/pics_1/matmult.PNG}
    \caption{Visualisation of the matrix-matrix multiplications to be performed. The picture is taken from the problem description on DTU Inside.}
    \label{fig:matmult}
\end{figure}

\section{Hardware and software}\label{sec:hard}

Specifications of the test environment are listed below:
\begin{itemize}
\item CPU information
\begin{itemize}
\item Vendor ID:             GenuineIntel
\item CPU family:            6
\item Model:                 79
\item Model name:            Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
\item CPU MHz:               1297.656
\item L1 (d / i) cache:             32K
\item L2 cache:              256K
\item L3 cache:              30720K
\end{itemize}
\item Compiler and Matrix implementations
\begin{itemize}
\item The GCC compiler have been applied with and without compiler optimization using the compiler flag \texttt{-Ofast}.
\item The library function uses DGEMM from the CBLAS interface
\item Matrices are indexed in the following way: [m] [n]
\item The following line has been specified in the job script: "\texttt{\#BSUB -n 1}" to make sure the batch job only uses one core.
\end{itemize}
\end{itemize}

\section{Theory}
Matrix multiplications are central operations in many numerical algorithms. Due to this, it is essential to make the operation fast to keep the computation time low. In order to achieve an optimal matrix-matrix multiplication algorithm, it is therefore important to take the ordering of nested loops and memory storage in the computer into account.\\

In C matrices are stored row-wise. Hence, when an element is fetched from memory the next couple of elements in the same row are collected as well, since they are stored in the same cache line. This is useful if the next couple of elements from the same row are needed in the near future, since they have already been loaded into memory. If on the other hand the next element is extracted from another row, e.g. in the case of looping through a column, there will be a cache miss and a new cache line is loaded into memory. Thus, in the worst case there will be a cache miss each time a new element is needed. In the case of matrix multiplication it is necessary to loop through both rows and columns in multiple nested loops and hence there is a great chance of cache miss when the matrices are sufficiently large. If the matrices are small enough such that all three will fit in memory there should be no cache misses. For large scale matrices it is essential to optimize the implementation of the matrix-matrix multiplication in order to avoid too many cache misses.\\

Double precision matrix multiplication of square matrices requires space for $3\cdot d^3$ doubles in memory, if the dimensions of the matrices is given by $d$. As storage of a double takes up 8 bits, the maximum dimensions for the matrices to fit in the memory of a given cache is found theoretically by calculating
\begin{align*}
    d = \sqrt{\frac{L/8}{3}}
\end{align*}
where L is the memory in the given cache. The maximum dimensions of square matrices fitting into each of the caches used are listed in table \ref{tab:cache_max_dim}. In reality, the maximum allowed dimensions might be higher, as the program performs hardware prefetch.
\begin{table}[h]
    \centering
    \begin{tabular}{c|c|c}
         Cache & Size [bytes] & Maximum dimension of Matrix  \\ \hline
         $L_1$ & 32K & 36 \\
         $L_2$ & 256K & 103 \\
         $L_3$ & 30720K & 1131
    \end{tabular}
    \caption{Maximum dimensions of square matrices fitting into each of the caches.}
    \label{tab:cache_max_dim}
\end{table}

In the last sub problem of the report, blocking is implemented in order to optimize the memory usage in the algorithm. The idea behind blocking is to divide the iterations into smaller blocks, so that operations can be performed for parts of the matrices at a time, instead of considering the full size of the problem. This is especially useful when handling large problems, for which the memory needed to compute the full operation exceeds the bounds of the memory. When using an algorithm that relies on blocking it is important to use one of the respective compiler flags: \texttt{-O3} or \texttt{-Ofast}. By doing so the compiler analyses the code several lines ahead and is hereby able to optimize the code.
%Notet med optimizers i compileren

%Noget med blocking (evt. sammen med hukommelse)

% hardware prefetch

%Hit/miss og mflops/s - memory

\section{Algorithms}
\subsection{\texttt{matmult\_lib}} %Wrapping of GDEMM
The subroutine DGEMM from the library BLAS will be used to evaluate the performance of and compare with the different implementations of the matrix-matrix multiplication. For this purpose the CBLAS interface is used and wrapped into the function \texttt{matmult\_lib} such that the library implementation of the matrix-matrix multiplication takes the same arguments as the other functions.  The call to DGEMM using the CBLAS interface is \texttt{cblas\_dgemm} which takes 14 arguments\footnote{http://www.netlib.org/lapack/explore-html/dc/d18/cblas} as shown in algorithm \ref{alg:lib}. From the inputs, the matrix multiplication is calculated as
\begin{align*}
    C = \alpha \cdot A \cdot B + \beta \cdot C.
\end{align*}
Here, $\alpha$ and $\beta$ are arguments to be specified in the input. In this case $\alpha$ is set to 1 while $\beta$ is set to 0 in order to get the wanted output. The other inputs in DGEMM (aside from M, N, K, A, B, and C) are the layout (set to rowmajor), whether the A and B matrices should be transposed (TRANSA and TRANSB) and the leading dimensions of A, B and C. As seen in the algorithm \ref{alg:lib}, the A and B matrices are not transposed, and the dimensions are set to K, N and N for A, B and C.
\lstinputlisting[label={alg:lib},caption={\texttt{matmult\_lib}},style=CStyle]{code/matmult_lib.c}

\subsection{\texttt{matmult\_nat}} %nat
The implementation of the \texttt{matmult\_nat} function is listed in algorithm \ref{alg:nat}. For this function, it has been chosen to use the a MNK ordering of the nested loops. That is that the outer loop iterates over the dimension M, the second over N and the inner over K.
\lstinputlisting[label={alg:nat},caption={\texttt{matmult\_nat}},style=CStyle]{code/matmult_nat.c}

\subsection{\texttt{matmult\_per}} % permutations
The \texttt{matmult\_per} permutations are implemented as presented in algorithm \ref{alg:per}, where the ordering of the nested loops are interchanged for each permutation of M, N and K. There exist six different permutations of the \texttt{matmult\_nat} function, namely MNK, NKM, MKN, KMN, MNK, and NMK. The \texttt{matmult\_per} algorithm has been optimized in relation to the \texttt{matmult\_nat} function. This has been done using stripmining, and setting all entries of C to zero using 1 for loop instead of 2 enables the compiler to perform vectorization.
\lstinputlisting[label={alg:per},caption={\texttt{matmult\_per} for the \texttt{mnk} ordering of the loops.},style=CStyle]{code/permutationcodepiece.c}

\subsection{\texttt{matmult\_blk}} % Blocking
In order to implement blocking in the matrix-matrix multiplications, additional nested loops are needed. The algorithm used for blocking is based on one of the best performing versions of the \texttt{matmult\_per} functions – in this case \texttt{matmult\_mkn}. In algorithm \ref{alg:blk} the function is presented. As mentioned in the problem description \texttt{matmult\_blk} takes the additional input \texttt{bs}, which is the blocking size. In blocking, the matrices A, B and C are portioned into smaller matrices for which the multiplications are performed and then added to each other to get the full C matrix. In this implementation the blocks are chosen to be square matrices. This can be seen from the first three nested loops in which blocks of the matrices of size bs$\times$bs are chosen. In the inner three nested loops the extracted matrices are multiplied and added to C. When using \texttt{matmult\_blk} the compiler needs an optimizer flag, in order for the blocking to set in.
\lstinputlisting[label={alg:blk},caption={\texttt{matmult\_blk}},style=CStyle]{code/matmult_blk.c}

\newpage
\section{Results}
The implementation of the different functions were checked in a test-script to see if the C matrix was calculated as expected. Each of the functions \texttt{matmult\_nat}, \texttt{matmult\_per}, \texttt{matmult\_blk}, and \texttt{matmult\_lib} were tested using the square matrices:
\begin{align*}
    A_s =
    \begin{pmatrix}
        1 & 2\\
        2 & 4
    \end{pmatrix} \qquad
    B_s =
    \begin{pmatrix}
        2 & 2\\
        2 & 2
    \end{pmatrix}
\end{align*}
and the non-square matrices:
\begin{align*}
    A_{ns} =
    \begin{pmatrix}
        1 & 2\\
        2 & 4\\
        3 & 6
    \end{pmatrix} \qquad
    B_{ns} =
    \begin{pmatrix}
        2 & 2 & 2\\
        2 & 2 & 2
    \end{pmatrix}
\end{align*}
which all returned
\begin{align*}
    C_{s} =
    \begin{pmatrix}
        6 & 6\\
        12 & 12
    \end{pmatrix} \qquad
    C_{ns} =
    \begin{pmatrix}
        6 & 6 & 6\\
        12 & 12 & 12\\
        18 & 18 & 18
    \end{pmatrix}
\end{align*}
% The calculations are checked against the DGEMM subroutine to ensure that the matrix-matrix multiplication is computed correctly for each of the implemented functions.
The calculation check sum provided by the driver has also been studied during the performance evaluation of the implemented functions, and in each of the cases the return value was 0 as expected.

Only square matrices are considered, e.g. N = M = K when the performance of the functions is evaluated.

Furthermore the performances of the above functions have been evaluated with and without the compiler flag \texttt{-Ofast} which enable compiler optimization, indicated as "opt".
It has been decided to capture the hit and miss counters for the L1 cache and L2 cache in the first project.

\subsection{Performance of \texttt{matmult\_nat}}

Figure \ref{fig:nat_perf} illustrates two realizations of the naive implementation of the \texttt{matmult\_nat} function. The blue realization is the naive implementation without compiler optimization enabled and the black realization has compiler optimization enabled. By considering the differences between the realizations in figure \ref{fig:nat_perf} huge increases in performance can be achieved by using compiler optimization.\\

\begin{figure}[!th]
\centering
\begin{tikzpicture}[scale = 1]
\begin{axis}[
width=17cm, height=8cm,     % size of the image
grid = major,
grid style={dashed, gray!30},
xmode=log,log basis x=10,
xmin=0,    % start the diagram at this
ymin=50,    % start the diagram at this
ymax=2000, % end   the diagram at this
axis background/.style={fill=white},
ylabel=Mflops/s,
xlabel=Memory/kbytes]

\addplot[mark=*, blue] table [x=mem, y=mflops, col sep=comma]{data/matmult_nat.txt};
\addlegendentry{matmult\_nat}
\addplot[mark=*, black] table [x=mem, y=mflops, col sep=comma]{data/matmult_nat_opt.txt};
\addlegendentry{matmult\_nat\_opt}

\addplot[red] table [x=x, y=y, col sep=comma]{data/L1.txt};
\addplot[red] table [x=x, y=y, col sep=comma]{data/L2.txt};
\addplot[red] table [x=x, y=y, col sep=comma]{data/L3.txt};
\addlegendentry{L1, L2 and L3 cache}
\end{axis}
\end{tikzpicture}
\caption{Performance of the \texttt{matmult\_nat} function measured in Mflops/s as a function of kbytes.}
\label{fig:nat_perf}
\end{figure}

Table \ref{tab:L1L2hitmis_nat} reports the cache hits and cache miss for the \texttt{matmult\_nat} function. It is observed that there are fewer hits in the optimized version for the L1 cache, while the misses are of equal size. The hypothesis would be that the number of hits increases for the optimized version, but it might be that the optimizer reduces the memory to be fetched, why the number of hits drops. Even though the number of hits reduces, we can still conclude that the optimized function performs significantly better than the non-optimized.

\begin{table}[!th]
\centering
\begin{tabular}{l|rr|rr|rr|rr}
\multirow{3}{*}{} & \multicolumn{4}{|c}{Non-optimized}& \multicolumn{4}{|c}{Optimized}\\
& \multicolumn{2}{|c}{L1 in $\%$} & \multicolumn{2}{|c}{L2 in $\%$}& \multicolumn{2}{|c}{L1 in $\%$} & \multicolumn{2}{|c}{L2 in $\%$} \\
& Hit& Miss& Hit& Miss& Hit& Miss& Hit& Miss\\ \hline
\texttt{matmult\_nat(50x50)}& 10 & 0 & 0 & 0 &0 & 0 & 0 & 0\\
\texttt{matmult\_nat(850x850)} & 129129 & 6072 & 5368 & 704 & 11651 & 6069 & 5524 & 544\\
\texttt{matmult\_nat(1600x1600)}& 861503 & 34583 & 134 & 34445& 73553 & 31510 & 92 & 31416\\
\end{tabular}
\caption{Hits and miss of the L1 and L2 cache for nat, presented in millions}
\label{tab:L1L2hitmis_nat}
\end{table}
\newpage

The naive implementation of the matrix multiplication routine, \texttt{matmult\_nat} is compared to the library subroutine DGEMM, which has been wrapped in \texttt{matmult\lib}. In figure \ref{fig:nat_perf}, the performance of DGEMM is shown with and without optimization in the compiler. Please note, that the axes is different from the one in \ref{fig:nat_perf}.\\
It is seen from the figure that the performance of DGEMM is nearly the same whether or not an optimizer is used in the compiler. This is not surprising, as we expect the library function to be optimized in itself, and the wrapper funciton \texttt{matmult\_lib} only calls the DGEMM subroutine and assigns values to some of the inputs. The performance of the library sub routine is significantly better than that of \texttt{matmult\_nat}. In table \ref{tab:L1L2hitmis_lib}, the hits and miss of the function are shown. All hits and misses are seen to be zero. This might be a consequence of the way the analyzer function handles the wrapper function. We would have expected the library function to have a very high number of hits and only few misses.

\begin{figure}[!th]
\centering
\begin{tikzpicture}[scale = 1]
\begin{axis}[
width=17cm, height=8cm,     % size of the image
grid = major,
grid style={dashed, gray!30},
xmode=log,log basis x=10,
%xmin=0,    % start the diagram at this
%ymin=50,    % start the diagram at this
ymax=10000, % end   the diagram at this
axis background/.style={fill=white},
ylabel=Mflops/s,
xlabel=Memory/kbytes,
legend pos=south east]

%\addplot[mark=*, blue] table [x=mem, y=mflops, col sep=comma]{data/matmult_nat.txt};
%\addlegendentry{matmult\_nat}
\addplot[mark=*, black] table [x=mem, y=mflops, col sep=comma]{data/matmult_lib_opt.txt};
\addlegendentry{matmult\_lib\_opt}

\addplot[mark=*, blue] table [x=mem, y=mflops, col sep=comma]{data/matmult_lib.txt};
\addlegendentry{matmult\_lib}

\addplot[red] table [x=x, y=y, col sep=comma]{data/L1.txt};
\addplot[red] table [x=x, y=y, col sep=comma]{data/L2.txt};
\addplot[red] table [x=x, y=y, col sep=comma]{data/L3.txt};
\addlegendentry{L1, L2 and L3 cache}
\end{axis}
\end{tikzpicture}
\caption{Performance of the \texttt{matmult\_lib} function measured in Mflops/s as a function of kbytes.}
\label{fig:nat_perf}
\end{figure}

\begin{table}[!th]
\centering
\begin{tabular}{l|rr|rr|rr|rr}
\multirow{3}{*}{} & \multicolumn{4}{|c}{Non-optimized}& \multicolumn{4}{|c}{Optimized}\\
& \multicolumn{2}{|c}{L1 in $\%$} & \multicolumn{2}{|c}{L2 in $\%$}& \multicolumn{2}{|c}{L1 in $\%$} & \multicolumn{2}{|c}{L2 in $\%$} \\
& Hit& Miss& Hit& Miss& Hit& Miss& Hit& Miss\\ \hline
\texttt{matmult\_lib(50x50)}& 0.0 & 0.0 & 0.0 & 0.0&0.0 & 0.0 & 0.0 & 0.0\\
\texttt{matmult\_lib(850x850)}& 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0\\
\texttt{matmult\_lib(1600x1600)}& 0.0 & 0.0 & 0.0 & 0.0& 0.0 & 0.0 & 0.0 & 0.0\\
\end{tabular}
\caption{Hits and miss of the L1 and L2 cache for lib, presented in millions}
\label{tab:L1L2hitmis_lib}
\end{table}

\newpage
\subsection{Optimal permutations of \texttt{matmult\_nat}}

As mentioned earlier it is possible to create several permutations of the naive \texttt{matmult\_nat} function. The six possible (the naive is included) permutations are presented in figure \ref{fig:opt_nmk_1_non} and the compiler optimized permutations in figure \ref{fig:opt_nmk_1}.

\begin{figure}[!th]
\centering
\begin{tikzpicture}[scale = 1]
\begin{axis}[
width=17cm, height=8cm,     % size of the image
grid = major,
grid style={dashed, gray!30},
xmode=log,log basis x=10,
xmin=0,    % start the diagram at this
ymin=50,    % start the diagram at this
ymax=400, % end   the diagram at this
axis background/.style={fill=white},
ylabel=Mflops/s,
xlabel=Memory/kbytes,
legend pos=south west]

\addplot[mark=*, black] table [x=mem, y=mflops, col sep=comma]{data/matmult_nmk.txt};
\addlegendentry{nmk}

\addplot[mark=*, red] table [x=mem, y=mflops, col sep=comma]{data/matmult_nkm.txt};
\addlegendentry{nkm}

\addplot[mark=*, blue] table [x=mem, y=mflops, col sep=comma]{data/matmult_mnk.txt};
\addlegendentry{mnk}

\addplot[mark=*, orange] table [x=mem, y=mflops, col sep=comma]{data/matmult_mkn.txt};
\addlegendentry{mkn}

\addplot[mark=*, green] table [x=mem, y=mflops, col sep=comma]{data/matmult_knm.txt};
\addlegendentry{knm}

\addplot[mark=*, magenta] table [x=mem, y=mflops, col sep=comma]{data/matmult_kmn.txt};
\addlegendentry{kmn}

\addplot[red] table [x=x, y=y, col sep=comma]{data/L1.txt};
\addplot[red] table [x=x, y=y, col sep=comma]{data/L2.txt};
\addplot[red] table [x=x, y=y, col sep=comma]{data/L3.txt};
%\addlegendentry{L1, L2 and L3 cache}
\end{axis}
\end{tikzpicture}
\caption{Performance of all permutations of the \texttt{matmult\_per} functions measured in Mflops/s as a function of kbytes.}
\label{fig:opt_nmk_1_non}
\end{figure}

\begin{figure}[!th]
\centering
\begin{tikzpicture}[scale = 1]
\begin{axis}[
width=17cm, height=8cm,     % size of the image
grid = major,
grid style={dashed, gray!30},
xmode=log,log basis x=10,
xmin=0,    % start the diagram at this
ymin=0,    % start the diagram at this
ymax=5000, % end   the diagram at this
axis background/.style={fill=white},
ylabel=Mflops/s,
xlabel=Memory/kbytes,
legend pos=north west]

\addplot[mark=*, black] table [x=mem, y=mflops, col sep=comma]{data/matmult_nmk_opt.txt};
\addlegendentry{nmk\_opt}

\addplot[mark=*, red] table [x=mem, y=mflops, col sep=comma]{data/matmult_nkm_opt.txt};
\addlegendentry{nkm\_opt}

\addplot[mark=*, blue] table [x=mem, y=mflops, col sep=comma]{data/matmult_mnk_opt.txt};
\addlegendentry{mnk\_opt}

\addplot[mark=*, orange] table [x=mem, y=mflops, col sep=comma]{data/matmult_mkn_opt.txt};
\addlegendentry{mkn\_opt}

\addplot[mark=*, green] table [x=mem, y=mflops, col sep=comma]{data/matmult_knm_opt.txt};
\addlegendentry{knm\_opt}

\addplot[mark=*, magenta] table [x=mem, y=mflops, col sep=comma]{data/matmult_kmn_opt.txt};
\addlegendentry{kmn\_opt}

\addplot[red] table [x=x, y=y, col sep=comma]{data/L1.txt};
\addplot[red] table [x=x, y=y, col sep=comma]{data/L2.txt};
\addplot[red] table [x=x, y=y, col sep=comma]{data/L3.txt};
%\addlegendentry{L1, L2 and L3 cache}
\end{axis}
\end{tikzpicture}
\caption{Performance of all permutations of the the compiler optimized \texttt{matmult\_***} functions measured in Mflops/s as a function of kbytes.}
\label{fig:opt_nmk_1}
\end{figure}

% \begin{figure}[!th]
% \centering
% \begin{tikzpicture}[scale = 1]
% \begin{axis}[
% width=17cm, height=8cm,     % size of the image
% grid = major,
% grid style={dashed, gray!30},
% xmode=log,log basis x=10,
% xmin=0,    % start the diagram at this
% ymin=0,    % start the diagram at this
% ymax=5000, % end   the diagram at this
% axis background/.style={fill=white},
% ylabel=Mflops/s,
% xlabel=Memory/kbytes,
% legend pos=north west]

% \addplot[mark=*, black] table [x=mem, y=mflops, col sep=comma]{data/matmult_mkn_opt.txt};
% \addlegendentry{mkn\_opt}

% \addplot[mark=*, red] table [x=mem, y=mflops, col sep=comma]{data/matmult_knm_opt.txt};
% \addlegendentry{knm\_opt}

% \addplot[mark=*, blue] table [x=mem, y=mflops, col sep=comma]{data/matmult_kmn_opt.txt};
% \addlegendentry{kmn\_opt}

% \addplot[red] table [x=x, y=y, col sep=comma]{data/L1.txt};
% \addplot[red] table [x=x, y=y, col sep=comma]{data/L2.txt};
% \addplot[red] table [x=x, y=y, col sep=comma]{data/L3.txt};
% %\addlegendentry{L1, L2 and L3 cache}
% \end{axis}
% \end{tikzpicture}
% \caption{Performance of the compiler optimized \texttt{matmult\_mkn}, \texttt{matmult\_knm} and \texttt{matmult\_kmn} functions measured in Mflops/s as a function of kbytes.}
% \label{fig:opt_nmk_2}
% \end{figure}

Figure \ref{fig:opt_nmk_1_non} tells that \texttt{matmult\_mkn} and \texttt{matmult\_kmn} are the best performing permutations in the L3 cache. Hence having N as the last index, makes the matrix-matrix multiplication the fastest. This was also expected, because M and K are the row indexes of the input matrices and fits to the way memory is stored the most efficient way. \\
The permutations has calculated using an optimizer. This result is presented in \ref{fig:opt_nmk_1}. This confirms that \texttt{matmult\_mkn} and \texttt{matmult\_kmn} are the fastest permutations, having \texttt{matmult\_mkn} a bit faster. When the optimizer is used the performance difference already occours in L1 and definately in L2. Hence, the combination of optimizer and having the right indexing makes \texttt{matmult\_mkn} the fastest permutation. \\
Table \ref{tab:L1L2hitmis_1600}, \ref{tab:L1L2hitmis_850} and \ref{tab:L1L2hitmis_50} presents the cache hits and misses for L1 and L2.
The functions have been evaluated for 50, 850 and 1600 and the global variable \texttt{} has been set to perform 10 iterations in each loop in order to make comparable tests. Table \ref{tab:L1L2hitmis_850} and \ref{tab:L1L2hitmis_1600} does again confirm that \texttt{matmult\_mkn} and \texttt{matmult\_kmn} are the most optimal matrix-matrix multiplication indexing. This can be concluded because the ratio of their respective hits and misses are big, both for L1 and L2.

\begin{table}[!th]
\centering
\begin{tabular}{l|rr|rr|rr|rr}
\multirow{3}{*}{50x50} & \multicolumn{4}{|c}{Non-optimized}& \multicolumn{4}{|c}{Optimized}\\
& \multicolumn{2}{|c}{L1} & \multicolumn{2}{|c}{L2}& \multicolumn{2}{|c}{L1} & \multicolumn{2}{|c}{L2} \\
& Hit& Miss& Hit& Miss& Hit& Miss& Hit& Miss\\ \hline
\texttt{matmult\_nmk}& 10 & 0.0 & 0.0 & 0.0 &0 & 0 & 0 & 0\\
\texttt{matmult\_nkm}& 10 & 0.0 & 0.0 & 0.0&0 & 0 & 0 & 0\\
\texttt{matmult\_mnk}& 10 & 0.0 & 0.0 & 0.0 &0 & 0 & 0 & 0 \\
\texttt{matmult\_mkn}& 10 & 0.0 & 0.0 & 0.0&0 & 0 & 0 & 0\\
\texttt{matmult\_knm}& 10 & 0.0 & 0.0 & 0.0&0 & 0 & 0 & 0\\
\texttt{matmult\_kmn}& 10 & 0.0 & 0.0 & 0.0 & 0 & 0 & 0 & 0
\end{tabular}
\caption{Hits and miss of the L1 and L2 cache for 50x50}
\label{tab:L1L2hitmis_50}
\end{table}

\begin{table}[!th]
\centering
\begin{tabular}{l|rr|rr|rr|rr}
\multirow{3}{*}{850x850} & \multicolumn{4}{|c}{Non-optimized}& \multicolumn{4}{|c}{Optimized}\\
& \multicolumn{2}{|c}{L1} & \multicolumn{2}{|c}{L2}& \multicolumn{2}{|c}{L1} & \multicolumn{2}{|c}{L2} \\
& Hit& Miss& Hit& Miss& Hit& Miss& Hit& Miss\\ \hline
\texttt{matmult\_nmk}& 129209 & 6094 & 6081 & 10& 11291 & 6350 & 6338 & 7 \\
\texttt{matmult\_nkm}& 123043 & 12138 & 11430 & 707 &13814 & 13310 & 12714 & 593\\
\texttt{matmult\_mnk}& 129169 & 6072 & 5364 & 703 &11851 & 6024 & 5448 & 573 \\
\texttt{matmult\_mkn}& 135215 & 192 & 16 & 2& 356 & 531 & 528 & 2\\
\texttt{matmult\_knm} & 123043 & 12144 & 11446 & 696& 13693 & 13341 & 12765 & 576 \\
\texttt{matmult\_kmn}& 135215 & 35 & 25 & 90& & 3524 & 525 & 509 & 18
\end{tabular}
\caption{Hits and miss of the L1 and L2 cache for 850x850}
\label{tab:L1L2hitmis_850}
\end{table}

\begin{table}[!th]
\centering
\begin{tabular}{l|rr|rr|rr|rr}
\multirow{3}{*}{1600x1600} & \multicolumn{4}{|c}{Non-optimized}& \multicolumn{4}{|c}{Optimized}\\
& \multicolumn{2}{|c}{L1} & \multicolumn{2}{|c}{L2}& \multicolumn{2}{|c}{L1} & \multicolumn{2}{|c}{L2} \\
& Hit& Miss& Hit& Miss& Hit& Miss& Hit& Miss\\ \hline
\texttt{matmult\_nmk}& 861423 & 34484 & 86 & 34397 & 51572 & 33418 & 26 & 33390 \\
\texttt{matmult\_nkm}& 819981 & 75809 & 121 & 75685 &11379 & 64094 & 134 & 63958\\
\texttt{matmult\_mnk}& 861623 & 34644 & 124 & 34518 &73593 & 31507 & 92 & 31412\\
\texttt{matmult\_mkn}& 901463 & 262 & 2464 & 16 & 22983 & 3790 & 3774 & 17\\
\texttt{matmult\_knm}& 820021 & 75806 & 89 & 75715 & 114034 & 64055 & 118 & 63937\\
\texttt{matmult\_kmn}& 901383 & 211 & 172 & 39 & 22943 & 3627 & 3521 & 105
\end{tabular}
\caption{Hits and miss of the L1 and L2 cache for 1600x1600}
\label{tab:L1L2hitmis_1600}
\end{table}

\newpage

% \todo[inline]{Hvad viser disse nedenstående plots??}
% From the Figure \ref{fig:permatrix}, \ref{fig:perhits} and \ref{fig:permisses} we can tell that the permutations that has N as the last index, is the ones that run the fastest. This makes sense because index K and M is the rows index's of the input matrices. The memory is build the way that it is faster to read rows than columns, which one can make a benefit of when indexing loops the right way.

% \begin{figure}[!th]
% \centering
% \begin{tikzpicture}[scale = 1]
% \begin{axis}[
% width=17cm, height=8cm,     % size of the image
% grid = major,
% grid style={dashed, gray!30},
% %xmode=log,log basis x=10,
% ymode=log,log basis y=10,
% %xmin=0,    % start the diagram at this x-coordinate
% %xmax=10, % end   the diagram at this x-coordinate
% %ymin=0,    % start the diagram at this y-coordinate
% %ymax=11000, % end   the diagram at this y-coordinate
% %/pgfplots/xtick={0,1,...,10}, % make steps of length 5
% %/pgfplots/ytick={0,1,...,10}, % make steps of length 5
% axis background/.style={fill=white},
% xlabel=CPU time [s],
% ylabel=Matrix size,
% legend style={at={(0.03,0.5)},anchor=center}]

% \addplot[mark=*,red] table [x=CPUsec, y=L2Misses, col sep=comma]{data/plotmnk.txt};
% \addlegendentry{matmult\_mnk()}

% \addplot[mark=*,blue] table [x=CPUsec, y=L2Misses, col sep=comma]{data/plotmkn.txt};
% \addlegendentry{matmult\_mkn()}

% \addplot[mark=*,green] table [x=CPUsec, y=L2Misses, col sep=comma]{data/plotnmk.txt};
% \addlegendentry{matmult\_nmk()}

% \addplot[mark=*,black] table [x=CPUsec, y=L2Misses, col sep=comma]{data/plotnkm.txt};
% \addlegendentry{matmult\_nkm()}

% \addplot[mark=*,magenta] table [x=CPUsec, y=L2Misses, col sep=comma]{data/plotkmn.txt};
% \addlegendentry{matmult\_kmn()}

% \addplot[mark=*,yellow] table [x=CPUsec, y=L2Misses, col sep=comma]{data/plotknm.txt};
% \addlegendentry{matmult\_knm()}

% \end{axis}
% \end{tikzpicture}
% \caption{Misses on permutations}
% \label{fig:permisses}
% \end{figure}

\newpage
\subsection{Blocking version}
As mentioned in the section on algorithms, the blocked version is based on \texttt{matmult\_mkn} because it is the best performing permutation of the \texttt{matmult\_nat} function, see figure \ref{fig:opt_nmk_1_non}.

Figure \ref{fig:blk_opti} illustrates a realization of the performance of \texttt{matmult\_blk} as a function of \texttt{bs} for $M=N=K=1500$. This problem size has been chosen for the experiment because three matrices of this size only fits in the L3 cache. Seeing as the blocking version performs operations on smaller blocks of the problem at a time, even though the full problem does not fit in the first two caches, the subproblems might fit and hence improve computation time. The red vertical lines in the figure illustrates the theoretical maximum blocking size for the matrices to fit in the memory of cache L1, L2, and L3 respectively. The maximum sizes are calculated in the theory section and are equal to those presented in table \ref{tab:cache_max_dim}.

Close to the boarder of L1, Mflops/s fluctuates a lot which can be caused by the hardware prefetching. Even though it is not visible on the logarithmic scale used for the blocking size, other experiments have shown that the drop in Mflops/s occurs just after the limit of L1. This is due to hardware prefetching and optimization of the compiler. The same behaviour is seen for L2. It is surprising that Mflops/s drops again well before the theoretical L3 limit, as we would have expected the drop to occur closer to the limit. The premature drop might be due to the implementation of blocking.

\begin{figure}[!th]
\centering
\begin{tikzpicture}[scale = 1]
\begin{axis}[
width=17cm, height=8cm,     % size of the image
grid = major,
grid style={dashed, gray!30},
xmode=log,log basis x=10,
%xmin=0,    % start the diagram at this
ymin=1300,    % start the diagram at this
ymax=1550, % end   the diagram at this
axis background/.style={fill=white},
ylabel=Mflops/s,
xlabel=block size,
legend pos=south east]

\addplot[mark=*, blue] table [x=mnk, y=mflops, col sep=comma]{data/matmult_blk_1500_bs.txt};
\addlegendentry{bs}

\addplot[red] table [x=x, y=y, col sep=comma]{data/L1_blk.txt};
\addplot[red] table [x=x, y=y, col sep=comma]{data/L2_blk.txt};
\addplot[red] table [x=x, y=y, col sep=comma]{data/L3_blk.txt};
\addlegendentry{L1, L2 and L3 cache}
\end{axis}
\end{tikzpicture}
\caption{Performance of the \texttt{matmult\_blk} function measured in Mflops/s as a function of block size.}
\label{fig:blk_opti}
\end{figure}

\begin{figure}[!th]
\centering
\begin{tikzpicture}[scale = 1]
\begin{axis}[
width=17cm, height=8cm,     % size of the image
grid = major,
grid style={dashed, gray!30},
xmode=log,log basis x=10,
xmin=0,    % start the diagram at this
ymin=800,    % start the diagram at this
ymax=1700, % end   the diagram at this
axis background/.style={fill=white},
ylabel=Mflops/s,
xlabel=Memory/kbytes,
legend pos=south east]

\addplot[mark=*, black] table [x=mem, y=mflops, col sep=comma]{data/matmult_blk_36_opt.txt};
\addlegendentry{blk\_36\_opt}
\addplot[mark=*, red] table [x=mem, y=mflops, col sep=comma]{data/matmult_blk_103_opt.txt};
\addlegendentry{blk\_103\_opt}
\addplot[mark=*, blue] table [x=mem, y=mflops, col sep=comma]{data/matmult_blk_1103_opt.txt};
\addlegendentry{blk\_1103\_opt}

\addplot[red] table [x=x, y=y, col sep=comma]{data/L1.txt};
\addplot[red] table [x=x, y=y, col sep=comma]{data/L2.txt};
\addplot[red] table [x=x, y=y, col sep=comma]{data/L3.txt};
%\addlegendentry{L1, L2 and L3 cache}
\end{axis}
\end{tikzpicture}
\caption{Performance of the \texttt{matmult\_blk\_opt} for different block sizes with compiler optimization.}
\label{fig:opt_blk_1}
\end{figure}

Figure \ref{fig:opt_blk_1} illustrates the three realizations of the  \texttt{matmult\_blk\_opt}. The three realizations illustrates that the choice of block size is depends on the problem size, e.g. the size of the matrix.

\begin{table}[!th]
\centering
\begin{tabular}{l|rr|rr|rr|rr|rr|rr}
\multirow{3}{*}{} & \multicolumn{4}{|c}{bs = 36}& \multicolumn{4}{|c}{bs = 103} & \multicolumn{4}{|c}{bs = 1103}\\
& \multicolumn{2}{|c}{L1 in $\%$} & \multicolumn{2}{|c}{L2 in $\%$}& \multicolumn{2}{|c}{L1 in $\%$} & \multicolumn{2}{|c}{L2 in $\%$}& \multicolumn{2}{|c}{L1 in $\%$} & \multicolumn{2}{|c}{L2 in $\%$} \\
& Hit& Miss& Hit& Miss& Hit& Miss& Hit& Miss & Hit & Miss & Hit & Miss\\ \hline
(50x50)&0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0\\
(850x850)& 18458 & 26 & 26 & 1 & 17978 & 131 & 128 & 3 & 18418 & 10 & 10 & 0\\
(1600x1600)& 123003 & 176 & 170 & 6 & 119800 & 931 & 912 & 20 & 121882 & 125 & 42 & 81\\
\end{tabular}
\caption{Hits and misses of the L1 and L2 cache for blk, presented in millions}
\label{tab:L1L2hitmis_lib}
\end{table}

\newpage
\subsection{Comparison}
As can be seen in figure \ref{fig:compareperfor} below non of the implemented functions beat the CBLAS library function \texttt{cblas\_dgemm}. The optimized permutations MKN and KMN of \texttt{matmult\_nat} are the ones who come closest to \texttt{matmult\_lib}. The blocked version of MKN performs worse than expected since using a blocking size of $bs = 103$ it should be almost similar to \texttt{matmult\_mkn} for small problems, though of course 3 additional unnecessary outer loops are used (since the step size is the entire width of the matrices). It is possible that a different implementation of \texttt{matmult\_blk} would result in better performance, e.g. if the extracted blocks were not quadratic but rectangular instead – I.e. if the number of columns were less than the number of rows. The library function increases in performance as the problem size increases as it should, since the library function is optimized for large scale problems. As discussed earlier the non-library functions experience drops in Mflops/s later than expected as a result of prefetching which is most apparent for the optimized versions of \texttt{matmult\_mkn} and \texttt{matmult\_kmn}.

\begin{figure}[!th]
\centering
\begin{tikzpicture}[scale = 1]
\begin{axis}[
width=17cm, height=8cm,     % size of the image
grid = major,
grid style={dashed, gray!30},
xmode=log,log basis x=10,
%ymode=log,log basis y=10,
xmin=0,    % start the diagram at this x-coordinate
%xmax=10, % end   the diagram at this x-coordinate
ymin=0,    % start the diagram at this y-coordinate
ymax=10000, % end   the diagram at this y-coordinate
%/pgfplots/xtick={0,1,...,10}, % make steps of length 5
%/pgfplots/ytick={0,1,...,10}, % make steps of length 5
axis background/.style={fill=white},
ylabel=Mflops/s,
xlabel=Memory/kbytes,
legend pos=north west]

\addplot[mark=*, blue] table [x=mem, y=mflops, col sep=comma]{data/matmult_lib_opt.txt};
\addlegendentry{lib\_opt}

\addplot[mark=*, red] table [x=mem, y=mflops, col sep=comma]{data/matmult_nat_opt.txt};
\addlegendentry{nat\_opt}

\addplot[mark=*, black] table [x=mem, y=mflops, col sep=comma]{data/matmult_blk_103_opt.txt};
\addlegendentry{blk\_103\_opt}

\addplot[mark=*, orange] table [x=mem, y=mflops, col sep=comma]{data/matmult_mkn_opt.txt};
\addlegendentry{mkn\_opt}

\addplot[mark=*, magenta] table [x=mem, y=mflops, col sep=comma]{data/matmult_kmn_opt.txt};
\addlegendentry{kmn\_opt}

\addplot[red] table [x=x, y=y, col sep=comma]{data/L1.txt};
\addplot[red] table [x=x, y=y, col sep=comma]{data/L2.txt};
\addplot[red] table [x=x, y=y, col sep=comma]{data/L3.txt};
%\addlegendentry{L1, L2 and L3 cache}
\end{axis}
\end{tikzpicture}
\caption{Performance of the compiler optimized \texttt{matmult\_lib}, \texttt{matmult\_nat}, \texttt{matmult\_blk\_1103} and \texttt{matmult\_mkn} functions measured in Mflops/s as a function of kbytes.}
\label{fig:compareperfor}
\end{figure}

Though the advantage of the blocked version was not visible in figure \ref{fig:compareperfor} it seems to pay off when considering the number of cache hits in table \ref{tab:L1L2hitmis_compare}. I.e. the blocked version is the implementation with the highest number of cache hits in the L1 cache, while the naive implementation has the overall lowest number of hits and highest number of misses. The \texttt{analyzer} tool never managed to collect the number of cache misses and cache hits for any of the caches optimized or not for the library function, which is most likely caused by the implementation.

\begin{table}[!th]
\centering
\begin{tabular}{l|rr|rr|rr|rr}
\multirow{3}{*}{1600x1600} & \multicolumn{4}{|c}{Non-optimized}& \multicolumn{4}{|c}{Optimized}\\
& \multicolumn{2}{|c}{L1} & \multicolumn{2}{|c}{L2}& \multicolumn{2}{|c}{L1} & \multicolumn{2}{|c}{L2} \\
& Hit& Miss& Hit& Miss& Hit& Miss& Hit& Miss\\ \hline
\texttt{matmult\_lib\_opt}&0& 0& 0& 0 &0& 0& 0& 0\\
\texttt{matmult\_nat\_opt}&861503 & 34584 & 134 & 34445 & 73554 & 31511 & 93 & 3142\\
\texttt{matmult\_blk\_103\_opt}&1062545 & 1012 & 970 & 40 & 119800 & 931 & 912 & 20\\
\texttt{matmult\_mkn\_opt}&901464 & 262 & 246 & 16 & 22983 & 3790 & 3774 & 17\\
\texttt{matmult\_kmn\_opt}&901383 & 211 & 173 & 39 & 22943 & 3627 & 3521 & 105\\
\end{tabular}
\caption{Hits and miss of the L1 and L2 cache presented in millions}
\label{tab:L1L2hitmis_compare}
\end{table}

\newpage
\section{Conclusion}

The naive implementation of the \texttt{matmult\_nat} has successfully been implemented. It is possible to create six permutations of the naive structure. The permutations \texttt{matmult\_mkn} and \texttt{matmult\_kmn} has the best performance, compared to the four other permutations.

The performances of the implementation are lower than the performance of the \texttt{matmult\_lib} function which uses the DGEMM subroutine from the CBLAS library.

It is possible to achieve a huge increases in performance of the functions when using compiler optimization particularly for the \texttt{matmult\_mkn} function and the \texttt{matmult\_kmn} function.