\chapter{State of the Art of Trustless Timestamping}
\label{chpr:state-of-art}
We analysed the reasons why Bitcoin chain is a valid tool to produce timestamps of arbitrary data and we showed two procedures to create such proofs.
However each one may follow his own set of rules to create and formalize proofs.
One can use a different set of commitment operations and time attestations or the same set but formalized in a different way, or even a mixture ot the preceding cases.
Although various set of rules may be valid and completely reasonable, such situation would be a nightmare for users, both creators and verifiers; during the creation procedure one may ask himself if challenger will retain his proof correctly formalized, verifiers should equip themselves with several tools increasing the cost of verifying proofs.
In addition this setting has huge security issues: an higher number of accepted formalizations increases the surface of attack through which proofs may be corrupted.

Moreover Bitcoin chain is not the only place where to bind attestation in a trustless manner.
There are other similar technologies that can be used as a notary with analogous techniques, for instance Litecoin, Ethereum or MimbleWimble.
Although timestamping with other chains is possible, it is fundamental to realize that it changes the security of the time attestations.
Each chain has its own rules and its own community behind, in some settings some assumptions may be weaker or false, while in others they may be an improvement.
For each specific case one should realize which is the best notary to use in order to properly address the problem.

For these reasons is important to have a common an shared standard to agree on the format used to create timestamps.
Such standard should be open source to let everyone analyse its security and contribute to it with improvement proposals.

In 2012 Peter Todd started working on \textit{OpenTimestamps} \cite{OTSWeb, OpenTimestampsGithub, OTSannouncment}, a project that provides a solution to the above issues.

\begin{quotation}
	OpenTimestamps aims to be a standard format for blockchain timestamping. The format is flexible enough to be vendor and blockchain independent.
\end{quotation}

In the following years other developers contributed to the source code improving the standard with more libraries, features and implementations.

\section{OpenTimestamps as a Standard}
OpenTimestamps defines a standard for creating a proof that can be verified in an easy way and that is not prone to inconsistent behaviours. The definition comes from the implementation, precisely from the python library which is currently taken as a reference by the other libraries.
Proofs consist in a sequence of commitment operations heading to at least one time attestation.

Commitment operations take one (unary) or two (binary) inputs to produce a single output. Note that a binary operation can be turned in a unary operation by fixing one of the inputs. The available operations are:
\begin{itemize}
	\item
	\verb|OpAppend| binary
	\item \verb|OpPrepend| binary
	\item \verb|OpReverse| unary, may get removed\footnote{https://github.com/opentimestamps/python-opentimestamps/issues/5}
	\item \verb|OpHexlify| unary
	\item \verb|OpSHA256| unary, cryptographic
	\item \verb|OpRIPEMD160| unary, cryptographic
	\item \verb|OpSHA1| unary, cryptographic
	\item \verb|OpKECCAK256| unary, cryptographic
\end{itemize}

Time attestations are time-attesting signature, they link a commitment to an event which has a time associated to. As of writing, the available ones are:
\begin{itemize}
	\item \verb|UnknownAttestation| Placeholder for attestations that aren't support
	\item \verb|PendingAttestation| Commitment has been recorded in a remote server for future attestation
	\item \verb|BitcoinBlockHeaderAttestation| Signed by the Bitcoin blockchain: the commitment digest will be the merkleroot of the blockheader
	\item \verb|EthereumBlockHeaderAttestation| Signed by the Ethereum blockchain: the commitment digest will be the merkleroot of the blockheader\footnote{Ethereum attestations were developed for a PoC proving the flexibility of the protocol, however they are in the \textit{dubious} module of the OpenTimestamps repository, the reason given is:
		\begin{quotation}
			... Ethereum has changed repeatedly in the past due to consensus failures and forks; as of writing the Ethereum developers plan to radically change Ethereum's consensus model to proof-of-stake, whose security model is at best dubious.
	\end{quotation}}
	\item
	\verb|LitecoinBlockHeaderAttestation| Signed by the Litecoin blockchain: the commitment digest will be the merkleroot of the blockheader
\end{itemize}
In the case of a single attestation, a proof is an ordered list of unary operations ending with a time attestation.
However timestamps may have more than one attestation, in fact they are not limited to be linear lists of operations, instead they can be structured as a \textit{tree} with $d$ as the root, commitment operations as edges and attestations as leaves.
This enables the possibility to attach different attestations to a proof, with possibly completely different meanings.
A proof is conveniently serialized in a receipt, which is conventionally stored in a file whose name ends with \verb|.ots|;
let's examine an example of a receipt for a file \verb|test.txt| containing \verb|b'Hello World!\n'|.
Its receipts \verb|test.txt.ots| is:
\begin{Verbatim}[frame=single]
File sha256: 03ba204e50d126e4674c005e04d82e84c21366780af1f43bd
             54a37816b6ab340
Timestamp:
ripemd160
prepend 0100000001e482f9d32ecc3ba657b69d898010857b54457a904979
        82ff56f97c4ec58e6f98010000006b483045022100b253add1d1cf
        90844338a475a04ff13fc9e7bd242b07762dea07f5608b2de36702
        2000b268ca9c3342b3769cdd062891317cdcef87aac310b6855e9d
        93898ebbe8ec0121020d8e4d107d2b339b0050efdd4b4a09245aa0
        56048f125396374ea6a2ab0709c6ffffffff026533e60500000000
        1976a9140bf057d40fbba6744862515f5b55a2310de5772f88aca0
        860100000000001976a914
append 88ac00000000
# Bitcoin transaction id
7e9f0f7d9daa2d9e51b2e22f4abe814c3f90539afa778a9bef88dc64627cb2
ec
sha256
sha256
prepend a987f716c533913c314c78e35d35884cac943fa42cac49d2b2c69f
        4003f85f88
sha256
sha256
prepend dec55b3487e1e3f722a49b55a7783215862785f4a3acb392846019
        f71dc64a9d
sha256
sha256
prepend b2ca18f485e080478e025dab3d464b416c0e1ecb6629c9aefce8c8
        214d042432
sha256
sha256
append 11b0e90661196ff4b0813c3eda141bab5e91604837bdf7a0c9df37d
       b0e3a1198
sha256
sha256
append c34bc1a4a1093ffd148c016b1e664742914e939efabe4d3d3565159
       14b26d9e2
sha256
sha256
append c3e6e7c38c69f6af24c2be34ebac48257ede61ec0a21b9535e44432
       77be30646
sha256
sha256
prepend 0798bf8606e00024e5d5d54bf0c960f629dfb9dad69157455b6f26
        52c0e8de81
sha256
sha256
append 3f9ada6d60baa244006bb0aad51448ad2fafb9d4b6487a0999cff26
       b91f0f536
sha256
sha256
prepend c703019e959a8dd3faef7489bb328ba485574758e7091f01464eb6
        5872c975c8
sha256
sha256
append cbfefff513ff84b915e3fed6f9d799676630f8364ea2a6c7557fad9
       4a5b5d788
sha256
sha256
prepend 0be23709859913babd4460bbddf8ed213e7c8773a4b1face30f8ac
        fdf093b705
sha256
sha256
verify BitcoinBlockHeaderAttestation(358391)
# Bitcoin block merkle root
8a1b66ecb7cbd07d8139a7e7d7f2c41aab1f5009b8364aaf61d03ad245e47e
00
\end{Verbatim}
The recepit is attesting that the file whose hash is \verb|03ba204e...| is committed into the block header number 358391, which has the \verb|nTime| field set to \verb|"May 28, 2015, 17:41:18 +0200"|. This means that the file existed prior to that time, still this is not completely safe: we are assuming that the miner has not lied \cite{TimeInaccuracy}. However he cannot put an extremely different date and he does not have a great incentive to put a false timestamp, nevertheless, we stay conservative and say that the data contained in the file existed prior to May 28, 2015.

This timestamp was created with an address commitment, to clarify how it is committed in the chain in Table \ref{tab:ots-ex} it is shown the receipt execution.
At the end, the value remaining on the stack is tested against the transaction Merkle root of block 358391.
If they are equal the proof is correct.

\begin{table}
\begin{center}
\begin{tabular}{|p{3cm}|p{4cm}|}
	\hline
	\textbf{Operation} & \textbf{Object on the stack} \\ \hline
 & file \\
	\verb|sha256| & hash of the file \\
	\verb|ripemd160| & address commitment \\
	\verb|prepend 0100...| &  \\
	\verb|append 88ac...| & raw transaction \\
	\verb|sha256| &  \\
	\verb|sha256| & TXID in little endian \\
	\verb|...| &  \\
	\verb|sha256| &  \\
	\verb|sha256| & transactions merkle root \\ \hline
\end{tabular}
\end{center}
\caption[OpenTimestamp receipt execution.]{OpenTimestamp receipt execution.}
\label{tab:ots-ex}
\end{table}
The raw transaction of the example is:
\begin{Verbatim}[commandchars=+\[\], frame=single]
0100000001e482f9d32ecc3ba657b69d898010857b54457a90497982ff56f9
7c4ec58e6f98010000006b483045022100b253add1d1cf90844338a475a04f
f13fc9e7bd242b07762dea07f5608b2de367022000b268ca9c3342b3769cdd
062891317cdcef87aac310b6855e9d93898ebbe8ec0121020d8e4d107d2b33
9b0050efdd4b4a09245aa056048f125396374ea6a2ab0709c6ffffffff0265
33e605000000001976a9140bf057d40fbba6744862515f5b55a2310de5772f
88aca0860100000000001976a914+underline[1df8859e60bc679503d16dcb870e6ce91a]
+underline[57e9df]88ac00000000
\end{Verbatim}
It can be decoded with the Bitcoin Core client with:
\begin{verbatim}
$ bitcoin-cli decoderawtransction <raw-transaction>
\end{verbatim}
Decoding the transaction one can see the commitment address\footnote{It is possible to verify that it is still unspent.}:
\begin{verbatim}
13jUKAuPDfgVPVgsVbeKVNWBv6wAh31vkN
\end{verbatim}
which is the base58 encoding of:
\begin{verbatim}
1df8859e60bc679503d16dcb870e6ce91a57e9df
\end{verbatim}
which can be spotted inside the transaction.

Moving forward from the example it is possible to define procedures to include the timestamp in the Bitcoin chain, create and verify the corresponding proof as outlined in Algorithm \ref{alg:create-timestamp}.
\begin{algorithm}
\caption{Create timestamp on the Bitcoin chain}
\label{alg:create-timestamp}
\begin{algorithmic}[1]
\Procedure{IncludeTimestamp}{$d$}
\Comment{$d$ data to timestamp}
\State choose sequence of commitment operations $S_d^C=[C_i]_{i=1}^n$
\State apply $S_d^C$ to $d$, obtain $C$
\Comment{$d$ to $C$}
\State include $C$ in a transaction $TX$
\State broadcast the transaction to the network
\State wait until the transaction is confirmed
\State \textbf{return} $d$, $S_d^C$, $C$, $TX$
\EndProcedure
\Statex
\Procedure{CreateTimestamp}{$d$, $S_d^C$, $C$, $TX$}
\State decompose the transaction, $TX = TX_p||C||TX_a$
\State $S_C^{TXID} = [\verb|prepend | TX_p,\verb| append | TX_a,\verb| sha256|,\verb| sha256|]$
\Statex
\Comment{$C$ to TXID}
\State find the block $B$ including $TX$
\State store $B$ transactions Merkle root $MR$ and height $H$
\State retrieve the Merkle path $S_{TXID}^{MR} \gets [C^{TXID}_i]_{i=1}^{n_{TXID}}$
\Comment{TXID to $MR$}
\State $proof \gets S_d^C + S_C^{TXID} + S_{TXID}^{MR}$
\Comment{join the commitment operations}
\State attach at the end of $proof$ \verb|BitcoinBlockHeaderAttestation(H)|
\State \textbf{return} $proof$
\EndProcedure
\Statex
\Procedure{VerifyTimestamp}{$d$, $proof$, $MR$}
\Statex
\Comment{$MR$ is retrieved from a \textit{trustworthy} source}
\State apply $proof$ to $d$, obtain $MR_V$
\Comment{execute commitment operations}
\If{$MR = MR_V$}{\textbf{ return }\verb|TRUE|}
\Comment{verify attestation}
\Else{\textbf{ return }\verb|FALSE|}
\EndIf
\EndProcedure
\end{algorithmic}
\end{algorithm}
The procedures could be easily generalized to more general settings, for instance to timestamp on another chain with slightly different rules.
It is important to highlight that the creator of the proof should always verify its correctness right after the creation. If the verification is successful he can store the file containing the proof in several insecure places, since it does not reveal anything more than the hash of the timestamped data, and store the file containing the data in a secure (if he needs) location; when challenged to prove the existence of that data prior to a certain time he will provide data and the corresponding timestamp to who he needs to convince.

\section{OpenTimestamps as a Scalability Solution}
As we have seen in the previous section, to create a timestamp is necessary to do a transaction.
To timestamp multiple files it is possible to commit those in a single hash value using a Merkle tree. As a result, the initial part of each proof will have different operations, going through different branches of the tree, while, from the transaction on, the operations will be identical.

However, each time one has to timestamp, he has to do a bitcoin transaction, which is not sustainable if too many people have timestamp needs: it implies an effort for the network and higher fees for all the users.

The solution proposed by the OpenTimestamps developers includes a \textit{centralized trust-minimized} system to aggregate timestamps, it involves an \textit{aggregation server} and a \textit{public calendar server} which actually timestamp.
Their operation is descibed in Algorithm \ref{alg:calendar-working}.

\begin{algorithm}
	\caption{Client - aggregator - calendar simplified working}
	\label{alg:calendar-working}
	\begin{algorithmic}[1]
		\State clients send data to timestamp to the aggregator\Comment{timestamp requests}
		\State aggregator \Call{merklelize}{} requests received each second\Comment{aggregation}
		\State aggregator sends to calendar the Merkle tip to timestamp
		\State calendar promise he will timestamp the tip\Comment{pending attestation}
		\State aggregator sends back to clients incomplete proof until the tip
		\State calendar aggregates pending tips in a merkle tree\Comment{aggregation}
		\State calendar sends a transaction including a merkle tip\Comment{timestamp}
		\State the transaction gets confirmed\footnotemark \Comment{attestation complete}
		\State clients ask to the calendar to upgrade the timestamp\Comment{upgrade}
		\State calendar sends back clients the complete proofs
		\State clients verify their proofs\Comment{verification}
	\end{algorithmic}
\end{algorithm}
\footnotetext{As pointed out in Section \ref{bitcoin}, a transaction is considered confirmed when it is 6 or more blocks deep.}

Clients do not send their data directly, instead they send salted hash values of the data, so the aggregator does not dig into their privacy.
At the first aggregation phase the aggregator append to each value received by clients a random nonce, so that a client sending malicious data instead of an hash do not pollute the proof of the leaf adjacent to his.
This system is trust-minimized beacause the clients trust that the calendar will create timestamp for them. Both calendar and aggregator can censor them, sending back incorrect proofs or malicious data. However once the calendar has sent them a correct proof they are fine, the proof will be valid forever regardless of any future bad behaviour of the aggregator or calendar.

The OpenTimestamp standard supports multiple branch for the commitment operations to include multiple attestations inside a single proof. With this feature one request can be forwarded to multiple public calendars, giving the redundancy that mitigates the problem when a calendar server is down and lowering the trust put in each single calendar.

\begin{figure}
	\begin{center}
		\includegraphics[width=\linewidth]{Images/bitcoin-chain-calendar.png}
		\caption[Calendar timestamping]{Calendar timestamping. $TX_3$ is the transaction made by the calendar, it includes a commitment to $\{data_i\}_{i=1}^n$ obtained with a Merkle tree. Note that transaction must follow the consensus rules to be included in the chain, instead data are completely arbitrary.}
		\label{fig:chain-calendar}
	\end{center}
\end{figure}

A calendar spends its own bitcoin to do a transaction, this implies that he won't be able to perform an extremely high number of transactions, leading to less frequent timestamps. As of writing, the calendars use the \textit{replace by fee} (\textit{RBF}) mechanism to spend a fix amount of bitcoin each day. In Algorithm \ref{alg:calendar-rbf} is described a simplified scheme of its working.
This procedure makes the calendar expenditures fixed: each day has approximately 144 block, so the amount spent in fee per day is $144 \cdot a$, where $a$ is a fixed value set by who run the calendar.

Thus if the mean fee to get into the next block is high, then the calendar may wait several blocks before timestamping and clients will have to wait for a long time to have their receipt upgraded and their data timestamped. This issue can be addressed if someone is timestamping in place of the calendar, which could be facilitated by the technique we are showing in Chapter \ref{chpr:ec-commitments}.

Despite all these considerations, the existence of public calendars is an extremely remarkable achievement for users: they enable clients to timestamp completely for free their own data.
The solution involves some trust\footnote{Actually a client may implement a trustless system involving the calendar to timestamp his data. He would first ask the calendar to timestamp and then wait for the completed proofs; after a reasonable amount of time, if the calendar delivered what promised, then the client is fine, otherwise he timestamps by himself.}, but for a limited interval of time and with minimal possible downsides; once the proof is completed, it is not important if it was created through a central hub, it has the exact same respectability of any other proof.
Furthermore the calendar learned nothing about the data timestamped since it received only an hash value.
Ultimately the efficient aggregation reduces the burden caused to the network, making timestamping sustainable even on large scale.

\begin{algorithm}
	\caption{Calendar replace by fee}
	\label{alg:calendar-rbf}
	\begin{algorithmic}[1]
		\Procedure{calendarRBF}{$a,LT$}\Comment{step $a$, last timestamp $LT$}
		\State $fee \gets 0$
		\Repeat
		\State $fee \gets fee + a$
		\State $S \gets \{h_i\}_{i=1}^n$ \Comment{collect clients timestamp requests}
		\State $MR \gets$ \Call{merklelize}{$S$}
		\State $TX \gets$ \Call{maketx}{$fee, MR$} \Comment{create transaction including $MR$}
		\State broacast $TX$
		\Until{$TX$ is mined}
		\State \textbf{return} $LT$ \Comment{$LT$ to restart the procedure}
		\EndProcedure
	\end{algorithmic}
\end{algorithm}


\documentclass[12pt]{article}

\oddsidemargin 0pt
\evensidemargin 0pt
\textwidth 16.2cm

\newcommand{\vect}[1]{{\bf #1}}
\newcommand{\tens}[1]{{\bf #1}}
\newcommand{\gvect}[1]{\mathbf{#1}}
\newcommand{\gtens}[1]{\mathbf{#1}}
\newcommand{\ti}[2]{\tens{#1}^{#2}}
\newcommand{\gti}[2]{\gtens{#1}^{#2}}
\newcommand{\vr}{\vect{r}}
\newcommand{\vR}{\vect{R}}
\newcommand{\vE}{\vect{E}}
\newcommand{\vv}{\vect{v}}
\newcommand{\vF}{\vect{F}}
\newcommand{\vU}{\vect{U}}
\newcommand{\tzeta}{\gtens{\zeta}}
\newcommand{\tmu}{\gtens{\mu}}
\newcommand{\prog}[1]{{\tt #1}}
\newcommand{\name}[1]{{\sc #1}}
\newcommand{\btitle}[1]{{#1,}}
\newcommand{\vol}[1]{{\bf #1}}

\newcommand{\wsp}{\hspace*{0.5cm}}

\begin{document}

\title{HYDROLIB: installation and usage guide}
\author{K. Hinsen\\
        Institut f\"ur Theoretische Physik A\\
        RWTH Aachen}
\date{}
\maketitle
\tableofcontents
\newpage

\section{Overview}

The Hydrodynamic Interactions Library (\prog{HYDROLIB}) consists of a
set of Fortran subroutines that calculate hydrodynamic interactions in
an arbitrary configuration of equal hard spheres with stick boundary
conditions, optionally with arbitrary rigid links between the
spheres. The quantities that can be calculated are:
\begin{itemize}
\item the friction matrix for a given particle configuration
\item the mobility matrix for a given particle configuration
\item the forces on all particles of a given configuration moving
with given velocities
\item particle velocities for a given particle configuration and a given set
of forces
\end{itemize}
The main features of the library are:
\begin{itemize}
\item controllable and adjustable accuracy
\item efficient calculation
\item suitability for finite and periodic geometries
\end{itemize}

The program has been checked intensively for various configurations,
ranging from two spheres (for which it produces exact results) to more
than a hundred. In addition, many internal consistency checks have
been done and the code has been checked with
\prog{ftnchek}.\footnote{\prog{ftnchek} checks Fortran programs for
semantic errors that compilers normally don't detect. It is available
by \prog{ftp} from \prog{netlib.att.com}.} Nevertheless, I cannot
guarantee for anything; if you use \prog{HYDROLIB}, you do so at your
own risk.

\section{Friction and mobility matrices}

\subsection{Physical background}

When spheres immersed into a liquid move with prescribed linear
velocities $\vect{U}$ and angular velocities $\gvect{\Omega}$, the
forces $\vect{F}$ and torques $\vect{T}$ acting on them are described
by the {\em friction matrix} $\tzeta$:
\[
\left(\begin{array}{c} \vect{F}\\ \vect{T} \end{array}\right) =
\left(\begin{array}{cc}
\tzeta^{tt} & \tzeta^{tr} \\
\tzeta^{rt} & \tzeta^{rr}
\end{array}\right) \left(\begin{array}{c}
\vv_0-\vU\\ \frac{1}{2}\nabla\times\vv_0-\gvect{\Omega}
\end{array}\right)
\]
The velocity field $\vv_0$ is the incident velocity field, to be
evaluated at the particle centers.

Force and torque vectors are described by three components for each
particle, therefore the friction matrix has the size $6N\times 6N$,
where $N$ is the number of particles. The friction matrix can be shown
to be symmetric.

When the forces and torques are given, the velocities can be obtained
from the
{\em mobility matrix} $\tmu$:
\[
\left(\begin{array}{c}
\vv_0-\vU\\ \frac{1}{2}\nabla\times\vv_0-\gvect{\Omega}
\end{array}\right) =
\left(\begin{array}{cc}
\tmu^{tt} & \tmu^{tr} \\
\tmu^{rt} & \tmu^{rr}
\end{array}\right) \left(\begin{array}{c}
\vect{F}\\ \vect{T}
\end{array}\right)
\]
The mobility matrix is the inverse of the friction matrix.

A more detailed description of hydrodynamic interactions and an explanation
of the theory on which the calculations are based can be found in
\cite{cifehiwabl}. A description of this implementation has been
published in \cite{hi}.

\subsection{Units}

Dimensionless quantities are used for all numerical calculations.
They are obtained as follows:
\begin{itemize}
\item[-] the radius $a$ of the spheres is set to 1.
\item[-] the viscosity $\eta$ of the liquid is set to $1/4\pi$.
\end{itemize}
This leaves one unit undetermined (e.g. velocity or force), which can
be chosen arbitrarily since it affects only the velocities and forces,
but not their ratios, i.e.\ the friction and mobility matrices are
independent of this choice.

\subsection{Numerical calculation}
\label{sec:numcalc}

The numerical calculation of hydrodynamic interactions is based on a
multipole expansion to take into account long-range interactions, and
on results from lubrication theory to correctly include short-range
interactions. The accuracy of the results can be controlled with the
integer parameter $l_{\rm max}$, which is related to the highest
multipole order included in the calculation. The parameter $l_{\rm
max}$ must be in the range $0\ldots 3$. Increasing this parameter
gives better accuracy, but increases the amount of memory and CPU time
needed by the program. The parameter $l_{\rm max}$ is related to
the multipole order $L$ used in \cite{cifehiwabl} and \cite{hi}
by $l_{\rm max} = L-1$.

The following table indicates how much memory is needed for a given
value of $l_{\rm max}$, expressed in terms of the number of unknowns
per particle in the system of equations that must be solved, and a
rough estimate of the accuracy of the friction matrix elements:

\begin{center}
\begin{tabular}{|l|r|l|}
\hline
$l_{\rm max}$ & Unknowns & Accuracy\\
\hline
0 & 9 & 10--20\% \\
1 & 24 & 1--5\% \\
2 & 45 & $< 1\%$ \\
3 & 84 & $<< 1\%$ \\
\hline
\end{tabular}
\end{center}

It is recommended to use $l_{\rm max} > 1$ only, since otherwise not
all long-range terms are taken into account (see \cite{cifehiwabl} for
details).

\section{Installation and usage}

\subsection{System requirements}

\prog{HYDROLIB} can be used with most standard Unix systems.
In addition to standard Unix tools, it needs a C preprocessor and a
Fortran compiler. Unfortunately, Fortran compilers vary a lot in their
abilities and command line syntax; some configuration is therefore
unavoidable.

You also need a linear algebra library. Currently only \prog{LAPACK}
is supported.\footnote{The source code contains optional calls to
\prog{IMSL}, dating back to early versions of this library.  However,
they have not been updated to the current versions of
\prog{HYDROLIB} and \prog{IMSL}, since \prog{IMSL} was not
available for testing. The calls have been left in to facilitate later
work on full \prog{IMSL} support. In their current state, they are
definitely incomplete and probably not functional.} \prog{LAPACK}
routines are free and available from several sources, e.g.\ by
\prog{ftp} from \prog{netlib.att.com}\footnote{Login as
\prog{anonymous} and use your e-mail address as password.} or by e-mail
from \prog{netlib@ornl.gov}\footnote{Send a mail with the single word
``help'' to that address to get detailed information.}. The \prog{HYDROLIB}
archive file contains all the necessary files from \prog{LAPACK} and
\prog{BLAS} (a set of low-level routines used by \prog{LAPACK}).
However, you might wish to install a newer version or a version optimized
for your machine.

Linear algebra functions are needed for the calculation of eigenvalues
(which is done only during initialization and for small matrices, so
efficiency is not important) and for solving systems of linear
equations. For some types of calculation, a matrix inversion routine
is also called. Users who have experience with linear algebra packages
and with the C preprocessor should be able to add support for other
libraries. To do this, check all occurrences of
\prog{\#ifdef LAPACK} in the source files. Add an appropriate
section for whatever library functions you want to use, and delimit it
by \prog{\#ifdef} and \prog{\#endif} with a suitable label. Replace
all \prog{\#define LAPACK} lines in the file \prog{config.h} and in
your application programs with a definition of the new label you have
chosen. Add an appropriate section in the script file \prog{install\_lib}
by copying and modifying the one for \prog{LAPACK}. Then reinstall the
library.

Porting \prog{HYDROLIB} to other operating systems is certainly
possible, but some knowledge of Unix and the target system is
required. However, porting the library is not necessary for running
working programs on other machines, since it is possible to create
``clean'' Fortran-77 code on a Unix system that can then be used
wherever a Fortran compiler is available.

\subsection{Installation}

\begin{enumerate}

\item
Edit the file \prog{local.def}. The supplied version already contains
the correct configurations for the most common Unix systems. If you are
using one of these systems, simply uncomment the respective lines. For
other machines, you must find the correct values from the manual for
the Fortran compiler. The values of the variables have the following
meaning:
\begin{description}
\item[\prog{FC}] The name of the command that invokes the Fortran compiler.
\item[\prog{RANLIB}] The name of the command that creates a table of
contents for archives. If your system has a command called \prog{ranlib} (check
with \prog{which ranlib}), use it. Otherwise, try \prog{touch} (which actually
does nothing). If this leads to error messages during linking, check the
man page for the \prog{ar} command.
\item[\prog{FCFLAGS1}] The options for compilation of a single file without
linking. These options are used to compile the invariable parts of the
library during installation.
\item[\prog{FCFLAGS2}] The options for compilation and linking of one
or several Fortran files. These options are used when compiling
a program that uses \prog{HYDROLIB}.
\item[\prog{cpp}] This variable should have the value \prog{1} if
the Fortran compiler understands the option \prog{-cpp}, which
causes it to run programs through the C preprocessor before compiling
them, and if your Fortran compiler accepts the non-standard statement
``implicit none''. Otherwise it should have the value \prog{0}.
\item[\prog{dpoption}] If your Fortran compiler has an option to
convert single precision programs to double precision, this option
should become the value of this variable. Otherwise the value must
be \prog{NONE}.
\end{description}

At the end of \prog{local.def}, there is a line starting with
\prog{set lalib =}. You need to change this line only if you
use another linear algebra library than \prog{LAPACK} or another
version of \prog{LAPACK} than the one supplied with \prog{HYDROLIB}.
Change the contents of the following string to
whatever compiler options are necessary on your system to make the
compiler find the linear algebra libraries.

\item
Type\\
\wsp\prog{install\_lib}\\
This will initiate the installation procedure. Installation might take several
minutes, and in the course of it a series of Fortran compiler calls will
be displayed.

\item
To check the installation, type\\
\wsp\prog{fc example.f}\\
This will compile a short test program. After
compilation is finished, type\\
\wsp\prog{a.out}\\
to run the test program. The ouptut should be\\
\wsp\prog{N = 6}\\
\wsp\prog{3.288792419691321   4.296910545080898}\\
\wsp\prog{9.912899454371797   94.71505251108327}\\
The accuracy of the printed results will be system dependent.

\end{enumerate}

\subsection{Preparing application programs}

To use the library in an application program, insert the following two lines
before the first subroutine, function, or program statement:\\
\wsp\prog{\#include "}{\em program}\prog{.h"}\\\
\wsp\prog{\#include <subr.f>}\\
{\em program} should be the name of the program file without the
extension \prog{.f}. The file {\em program}\prog{.h} should be a
copy of \prog{config.h} in which the parameters have been modified
appropriately (see section~\ref{sec:config} for details).
Note that the character \prog{\#} in the lines above may not be
preceeded by white space.

The available library calls are explained in section~\ref{sec:imp}.

\subsection{Compilation and Fortran source code generation}

Two commands are supplied to handle compilation and generation of
portable Fortran source code.  The command \prog{fc} will compile an
application program using all necessary compiler options and link it
with the library. File arguments and additional compiler options
should be specified as for the standard \prog{f77} command. The
name of the configuration file can be specified using the option
\prog{-h}~{\em config-file}\/. If this option is missing, \prog{fc}
will use the name of the first source code file given and
replace its extension \prog{.f} by \prog{.h}. If no file with
this name exists, \prog{fc} will issue a warning and assume
the name \prog{config.h}.

The command \prog{fprog} will produce a complete Fortran source
program that contains all necessary code (except for routines from the
linear algebra package) and can be ported to any machine with a
Fortran 77 compiler. It accepts the name of a single file with
extension \prog{.f} and produces a file with the same name and
extension \prog{.fsrc}. To run this program on another machine,
transfer the generated program file {\em and} the files \prog{z2cl*},
\prog{ocoeff} and \prog{hcoeff} from the folder \prog{hydrolib}
to this machine.

\section{Configuration files}
\label{sec:config}

The Hydrodynamic Interactions Library requires some numerical parameters
and some options that determine what is to be calculated. A special
configuration file contains all this information. An example of such
a file is the file \prog{config.h} supplied with the library. For each
application program, create a copy of this file and modify it according
to your needs.

\subsection{Numerical parameters}

These are specified by lines of the type\\
\wsp\prog{\#define }{\em parameter\_name value}

\begin{description}

\item[]
\prog{\_NP\_}\\
This parameter must be set to the number of particles.

\item[]
\prog{\_LM\_}\\
This parameter must be set to the value of $l_{\rm max}$, described in
section~\ref{sec:numcalc}. It must be a number between 0 and 3.

\end{description}

\subsection{Compile-time options}

Compile-time options are used to adapt the program code to the
requirements of the application and the programming environment. They
all result in different sections of code being included in the final
program. An option is activated by a line of the type\\
\wsp\prog{\#define }{\em option\_name}\\
and switched off by\\
\wsp\prog{\#undef }{\em option\_name}\\
(In principle lines starting with \prog{\#undef} could be completely
removed, but for ease of modification this is not recommended.)

The first set of compile-time options is related to the type of
problem being solved:

\begin{description}

\item[]
\prog{PERIODIC}\\
If defined, hydrodynamic interactions are calculated for a
system with periodic boundary conditions. Only cubic elementary cells
are supported. A common block named \prog{pbc} contains a variable
\prog{box} of type \prog{real} which must be set to the edge length of
the cubic cell. The coordinates that specify the particle
configuration must be positive and less than the edge length. The
particle configuration must be consistent with periodic boundary
conditions, i.e.\ the particles may not overlap with each other or
with each other's periodic images. This is {\em not} checked by the
program.\\
If \prog{PERIODIC} is not defined, a finite assembly of
spheres in an infinite fluid is assumed. The particles may not overlap
with each other; again this is {\em not} checked by the program. The
particle coordinates can otherwise be arbitrary.

\item[]
\prog{RIGID}\\
If defined, \prog{HYDROLIB} is able to handle
rigid connections between the particles. A ``rigid connection'' between
two particles means that no relative motion, neither translational nor
rotational, is possible between these particles. Other than that, the
``connections'' have no influence on the calculations; they are assumed
to have no masses and no influence on the fluid flow. From this definition it
follows that connectivity is transitive: if particle~1 is connected with
particle~2 and particle~3, there is also an implicit connection between
particles~2 and 3, since no relative motion between them is possible.
The specification of connections is explained in section~\ref{sec:imp}.\\
The presence of connections means that the number of degrees of freedom
of the system is reduced, since any cluster of interconnected particles
has only 6 degrees of freedom in total. Consequently, the friction and
mobility matrices have fewer elements, namely 6 times the number of
rigid clusters in the system. A particle cluster is described by the
translational velocity of its ``center of mass'' (or rather what would
be its center of mass if all particles had identical mass; in fact masses
never enter in any calculation) and by an angular velocity about this
point\footnote{In principle any point could be chosen as reference
point for describing clusters, but this choice makes the interpretation
of results easier if the cluster happens to have some symmetry.}.
To make use of the results, the application program needs some information
about the rigid clusters, which are available in a common block named
\prog{rb}, which has the following form:\\
\wsp\prog{real vrb(6*\_NP\_,6*\_NP\_)}\\
\wsp\prog{integer nrb,irb(\_NP\_)}\\
\wsp\prog{common /rb/ vrb,nrb,irb}\\
The variable \prog{nrb} contains the number of rigid clusters. The
array \prog{irb} contains for each particle the number of the cluster
it belongs to (i.e.\ a number between 1 and \prog{nrb}). All particles
with the same number form a cluster. The array \prog{vrb} makes it
possible to convert forces and velocities between the rigid-cluster
description (\prog{6*nrb} degrees of freedom) and the single-sphere
description (6$N$ variables). Its first index runs from 1 to $6N$, its
second index from 1 to \prog{6*nrb}. Multiplying a \prog{6*nrb}-dimensional
velocity vector from the right gives the individual particle velocities,
whereas multiplying by a 6$N$-dimensional force vector from the left gives
the total forces and torques acting on the rigid clusters.\\

\item[]
\prog{FIXED}\\
If defined, mobility calculations can be performed on systems
in which some spheres are fixed while others can freely move.
An additional common block named \prog{fix} is defined,
which has the following form:\\
\wsp\prog{integer nfixed}\\
\wsp\prog{logical full}\\
\wsp\prog{common /fix/ nfixed,full}\\
The variable \prog{nfixed} contains the number of fixed particles.
The fixed particles are always the {\em last} \prog{nfixed}
particles in the configuration. If the variable \prog{full} has
the value \prog{.true.}, everything will be recalculated.
If it has the value \prog{.false.}, the program will assume that
the fixed particles have not moved since the evaluation for the
previous configuration. The variable \prog{full} must always be
set to \prog{.true.} initially.

\item[]
\prog{FIXED\_FAST}\\
Identical to \prog{FIXED}, except that the calculations are
performed faster at the expense of a higher memory requirement.

\item[]
\prog{LUBRICATION}\\
If defined, the program adds short-range corrections calculated from
lubrication theory to the results of the multipole
expansion. This should always be done unless it is known that these
corrections are negligible.\\
A potential problem is caused by the fact that the short-range
corrections diverge for touching spheres. This will cause loss of
precision for short distances and a program crash for actually
touching spheres. The latter can be avoided by enabling the option
\prog{RIGID} and defining a rigid connection between the touching
particles (this does not affect the results since with stick
boundary conditions, there can never be any relative motion between
touching spheres). Specifying rigid connections also removes the
loss of precision at small non-zero distances, but in that case the
connection will also lead to different results.

\item[]
\prog{FRICTION}\\
If defined, the program calculates the friction matrix. The friction
matrix is returned in a common block named \prog{frict} of the form\\
\wsp\prog{real fr(6*\_NP\_,6*\_NP\_)}\\
\wsp\prog{common /frict/ fr}\\
\begin{sloppy}
The friction matrix consists of $6\times 6$-submatrices for each particle
pair. It is always symmetric. The order of the 6 coordinates is
$(x,y,z)_{\rm Translation},$ $(x,y,z)_{\rm Rotation}$.\\
\end{sloppy}
If the option \prog{RIGID} is used (see description above), the friction
matrix has dimensions \prog{6*nrb}$\times$\prog{6*nrb}. Since
\prog{nrb} is calculated at runtime, the array containing the friction
matrix is still dimensioned as $6N\times 6N$, but not all elements
are used.

\item[]
\prog{MOBILITY}\\
If defined, the program calculates the mobility matrix. The mobility
matrix is returned in a common block named \prog{mobil} of the form\\
\wsp\prog{real mo(6*\_NP\_,6*\_NP\_)}\\
\wsp\prog{common /mobil/ mo}\\
\begin{sloppy}
The mobility matrix consists of $6\times 6$-submatrices for each particle
pair. It is always symmetric. The order of the 6 coordinates is
$(x,y,z)_{\rm Translation},$ $(x,y,z)_{\rm Rotation}$.\\
\end{sloppy}
If the option \prog{RIGID} is used (see description above), the mobility
matrix has dimensions \prog{6*nrb}$\times$\prog{6*nrb}. Since
\prog{nrb} is calculated at runtime, the array containing the mobility
matrix is still dimensioned as $6N\times 6N$, but not all elements
are used.\\
If the option \prog{FIXED} is used, only the mobility matrix elements
for the freely moving particles have meaningful values.

\item[]
\prog{VELOCITIES}\\
If defined, the program calculates particle velocities for a given set of
forces and torques. The forces (input) and velocities (output) are
contained in a common block named \prog{fv} of the form\\
\wsp\prog{real f(6*\_NP\_),v(6*\_NP\_)}\\
\wsp\prog{common /fv/ f,v}\\
\begin{sloppy}
The arrangement of the components of \prog{f} is
$F_x^{(1)},F_y^{(1)},F_z^{(1)},$ $T_x^{(1)},T_y^{(1)},T_z^{(1)},
\ldots,$ $T_z^{(N)}$. Similarly, \prog{v} is arranged as
$U_x^{(1)},U_y^{(1)},U_z^{(1)},$ $\Omega_x^{(1)},\Omega_y^{(1)},
\Omega_z^{(1)},\ldots,$ $\Omega_z^{(N)}$.\\
\end{sloppy}
If the option \prog{RIGID} is used, the forces and velocities are still
single-particle quantities, but of course their calculation procedes
in a different way. If the rigid-body velocities of the particle
clusters are needed, the option \prog{RB\_VELOCITIES} (explained below)
should be used.\\
If the option \prog{FIXED} is used, only the velocity components
referring to the freely moving particles have meaningful values.

\item[]
\prog{RB\_VELOCITIES}\\
This option must be combined with \prog{RIGID} (see above).
It is similar to
\prog{VELOCITIES}, except that input forces and output velocities
describe the rigid-body motion of particle clusters. Consequently
only the first \prog{6*nrb} elements are used.

\item[]
\prog{NORMALIZE}\\
If defined, the friction and mobility matrices are normalized by their
one-particle values. Since for a single particle there is no coupling
between translation and rotation, normalization is possible only for
the pure translation and rotation submatrices $\tzeta^{tt}$,
$\tzeta^{rr}$, $\tmu^{tt}$ and $\tmu^{rr}$. Consequently it should
{\em not} be done if combinations of translational and rotational
motion are to be used.\\
Particle velocities calculated using the option \prog{VELOCITIES} are
never normalized.

\item[]
\prog{POSDEF}\\
The system of linear equations that is solved to obtain the friction
or mobility matrix can be shown to be positive definite. However, for
large systems the coefficient matrix may not be positive definite from
a numerical point of view. Linear algebra packages might therefore
issue error messages when special algorithms for positive definite
matrices are used.  Therefore the decision whether or not to use these
algorithms is left to the user.\\ If \prog{POSDEF} is defined,
routines from \prog{LAPACK} are called that are optimized for positive
definite matrices. Otherwise, routines for general symmetric matrices
are used.  It is recommended to leave \prog{POSDEF} defined until an
actual warning about indefinite matrices occurs.\\

\end{description}

The second set of compile-time options is related to the programming
environment:

\begin{description}

\item[]
\prog{PACKED}\\
If defined, the coefficient matrix for the system of linear equations
to be solved is stored in a packed format suitable for symmetric
matrices. This saves almost 50\% of memory space, but can lead to
slightly higher execution times. In most applications the memory
saving will outweigh the increased execution time, therefore it is
recommended to leave this option enabled. In some cases, a
small {\em decrease} of execution time was even observed with packed
storage.

\item[]
\prog{DP}\\
If defined, double precision code is used everywhere, and all external
library functions are called in their version for double-precision
calculations. If your Fortran compiler has an option for treating
single-precision code as double precision (most compilers have), this
will be applied to your application program.  In this case, defining
and undefining \prog{DP} is sufficient to switch between single and
double precision versions. If your compiler does not have such an
option, you must prepare your code manually for double-precision
calculations. However, there is no need to ever modify \prog{HYDROLIB}
code.

\item[]
\prog{LAPACK}\\
If defined, routines from the linear algebra package \prog{LAPACK} are
used for the calculation of eigenvalues, for the solution of a system
of linear equations, and for matrix inversion.\\
Note that currently no other linear algebra libraries are
supported. If you add support for another library, you can
define the label you have chosen for that library instead
of \prog{LAPACK}.

\end{description}

\section{Fortran subroutines in the library}
\label{sec:imp}

To calculate hydrodynamic interactions for a certain configuration of
particles, you must write a Fortran program that calls subroutines
from \prog{HYDROLIB}. For illustrational
purposes, an example program (\prog{example.f}) is included that
calculates the four independent friction coefficients for a linear
chain of touching particles.

During the initialization of the program, the subroutine \prog{init}
must be called. If periodic boundary conditions are used, the
variable \prog{box} in the common block \prog{pbc} must be set to the
edge length of the cubic box, as explained in section~\ref{sec:config}.

After that, the subroutine \prog{eval} can be called any number of times.
This subroutine has no arguments, all input and output data are
passed via common blocks. Which quantities are calculated by
\prog{eval} depends on the compile-time options used, see
section~\ref{sec:config} for details.

Before calling \prog{eval}, the particle configuration must be
stored in a two-dimensional array in the
common block \prog{conf}. The first dimension of this array is 3, for
the $x$-, $y$-, and $z$-coordinates, the second dimension is the
number of particles. An example of a definition for this common block
is\\
\wsp\prog{real c(3,\_NP\_)}\\
\wsp\prog{common /conf/ c}\\
If the option \prog{RIGID} is activated, the particle configuration
must be extended by a list of rigid connections between the particles.
This takes the form of a $N\times N$-array of \prog{character*1}. Any
character other than a space stands for a rigid connection between
two particles. Only the upper-diagonal part of this array, i.e.\ elements
for which the second index is larger than the first one, are used.
With this addition, the common block \prog{conf} takes the form\\
\wsp\prog{real c(3,\_NP\_)}\\
\wsp\prog{character*1 cnct(\_NP\_,\_NP\_)}\\
\wsp\prog{common /conf/ c,cnct}\\

When particle velocities are to be calculated (option
\prog{VELOCITIES}), the forces must be defined before calling
\prog{eval}. This is done in the common block
\prog{fv} as explained in section~\ref{sec:config}.

After calling \prog{eval}, the results can be found in various common
blocks as explained in section~\ref{sec:config}. The input data
are not changed, with the possible exception of the character array
specifying rigid connections. The contents of this array may have been
changed to another equivalent specification of the same set of
connections.

\begin{thebibliography}{9}

\bibitem{cifehiwabl}
  \name{B. Cichocki, B.U. Felderhof, K. Hinsen,
        E. Wajnryb, J. B{\l}awzdziewicz},
  Friction and mobility of many spheres in Stokes flow,
  J. Chem. Phys. \vol{100}, 3780~(1994)

\bibitem{hi}
  \name{K. Hinsen},
  \prog{HYDROLIB}: a library for the evaluation of hydrodynamic
       interactions in colloidal suspensions,
  Computer Physics Communications (1995)

\end{thebibliography}

\end{document}


\section{Measurement of fiducial cross section}
\label{sec:xsec}

The fiducial cross section for inclusive $ZZjj$ production, including both EW and QCD components, is then measured.
The definition of fiducial volume, which is used for cross section measurement, follows closely to the detector-level selection
but use physics objects at particle-level, which are reconstructed in simulation from stable final-state particles,
prior to their interactions with the detector.

For electrons and muons, QED final-state radiation is for the most part recovered
by adding the four-momenta of surrounding photons that are not originating from hadrons and within an angular distance $\Delta R < 0.1$
to the lepton four-momentum, called lepton ``dressing" at truth level.
Particle-level jets are built with anti-$k_{T}$ algorithm with radius parameter $R = 0.4$ using all final-state particles except leptons and neutrinos as inputs.
Comparing to the events selection at detector-level in section~\ref{sec:vbszz_selection},
at particle-level, the selected dilepton pair mass required is relaxed to be within 60 to 120~\gev~ for the reasons of reducing the migration effect,
as well as being more compatibility with CMS publication~\cite{2017682}.
All other kinematic selections are the same as the definition at detector-level.

\subsection{Calculation of C-factor}
\label{sec:cf}

C-factor is defined as the ratio between the number of selected events at detector-level and the number of particle-level events in fiducial volume (FV):
\begin{equation}
	\mathcal{C} = \frac{N_{detector-level}}{N_{FV.}}
\end{equation}
The value of C-factor for each $ZZjj$ process are firstly calculated from each individual simulation samples together with their systematics.
%\begin{table}[H]
%\begin{center}
%   \begin{tabular}{|c|c|c|c|c|}
%   \hline
%   Process          & $\mathcal{C}$ & $\Delta$C (stats.) & $\Delta$C (sys.)        & $\Delta$C (theo.)       \\
%   \hline
%   EWK $ZZjj$         & 0.663         & $\pm$0.002       & $\pm^{0.032}_{0.031}$ & NA                    \\
%   \hline
%   QCD \qqZZ        & 0.702         & $\pm$0.003       & $\pm^{0.061}_{0.051}$ & $\pm^{0.015}_{0.018}$ \\
%   \hline
%   QCD \ggZZ        & 0.741         & $\pm$0.021       & $\pm^{0.143}_{0.072}$ & $\pm{0.002}$          \\
%   \hline
%\end{tabular}
%\end{center}
%\caption{C Factor of different $ZZjj$ processes.}
%\label{tab:xs_cf}
%\end{table}
Then the $\mathcal{C}$ from different processes are combined together according to their fraction of events in fiducial volume,
and used as inputs for cross section calculation:
\begin{equation}
	\mathcal{C} = \Sigma_{i} \frac{N_{FV.}^{i}}{\Sigma_{j} N_{FV.}^{j}} \times \mathcal{C}_{i} = 0.699\pm0.003(stats.)\pm^{0.011}_{0.013}(theo.)\pm0.028(exp.)
\end{equation}
The ``stats." refers to the statistical uncertainty from MC simulation statistics.
The ``theo." and ``exp." denote the theoretical and experimental uncertainties described in section~\ref{sec:systematics}.

\subsection{Result of fiducial cross section}

The cross section in fiducial volume is computed as:
\begin{equation}\label{eq:xs}
	\sigma^{FV.} = \frac{N_{data} - N_{bkg}}{\mathcal{C} \times Lumi}
\end{equation}
where $N_{data}$ and $N_{bkg}$ denote the number of events selected from detector-level selection from data and sum of backgrounds,
and $\mathcal{C}$ is the C-factor calculated above, Lumi represents the integrated luminosity of data from 2015 to 2018 of 139~\ifb.
%As shown in table~\ref{tab:yield_prefit}, in inclusive measurement, only the ``Others" represents background,
%processes of EW-$ZZjj$, QCD-\qqZZ and QCD-\ggZZ are signals.
Table~\ref{tab:xs} shows the fiducial cross section for \llll final state measured from equation~\ref{eq:xs},
as well as the predicted cross section measured from MC simulation directly.

\begin{table}[!htbp]
\begin{center}
\scalebox{0.90}{
\begin{tabular}{ c | c}
\hline
\hline\noalign{\smallskip}
Measured fiducial $\sigma$ [fb] & Predicted fiducial $\sigma$ [fb] \\
\noalign{\smallskip}\hline\noalign{\smallskip}
$1.27 \pm 0.12(\mathrm{stat}) \pm 0.02(\mathrm{theo}) \pm 0.07(\mathrm{exp}) \pm 0.01(\mathrm{bkg}) \pm 0.02(\mathrm{lumi})$ & $1.14 \pm 0.04(\mathrm{stat}) \pm 0.20(\mathrm{theo})$ \\
\noalign{\smallskip}\hline
\hline
\end{tabular}}
\end{center}
\caption{
Measured and predicted fiducial cross-sections in \lllljj final-state.
Uncertainties due to different sources are presented.
}
\label{tab:xs}
\end{table}
The measured cross section has a total uncertainty of 11\%, and is found to be compatible with SM prediction.
This measurement is still dominant by data statistic.


\chapter{Discussion}

In this chapter, we review results of our experiments and discuss applications of the analyzed models in other systems. We also give tips on how to perhaps further improve performance of our models and which aspects of human memory could be considered in future research.

\section{Results}

In chapter~\ref{analysis}, we analyzed several models based on timing information (taking into account either ages of past trials of items or past response times of students). Most promising results in the context of learning geography facts shows the PFA/E/T model with a staircase time effect function, although the power function proves to be almost as equally a good choice. Main advantage of the staircase function is that the parameters are easy to learn for the purpose of online learning, i.e. there is no need to periodically run parameter optimization procedure since the parameters are slightly adjusted with each new answer.

The PFA/G/T model suffers from the inability to account for multiple-choice questions. It might be interesting to analyze the models on data from adaptive systems where questions with multiple-choice options are not used---this is not possible using the data set described in section~\ref{data-set} since the system adaptively selects number of options in multiple choice tests, i.e. filtering out all multiple-choice questions creates data set with skewed difficulties and abilities of students.

The PFA/E/RT model that considers past response times of students shows in most cases only a very small improvement. However, very interesting results provides the PFA/G model with values of the decay factor lower then 0.5, this may indicate that the past trials of items do not actually influence model's performance in the context of learning geography facts as much as we thought. When we compare the PFA/G model and the PFA/G/T model, it seems that penalizing old trials based on order of answers leads in most cases to a better performance than the penalty of trials based on timing information.

\section{Applications and Recommendations}

Currently, models used in the adaptive system Outline Maps do not take into account ages of past trials. We have demonstrated that the timing information improves model's performance and thus, the extensions of presented models should be considered in production. As we already discussed in previous section, the best choice is the PFA/E/T model with a time effect function fitted by gradient descent described in chapter~\ref{approx-gradient}.

Another application of our models could be considered in the adaptive system for practicing medical anatomy (\url{practiceanatomy.com}) targeted mostly at medical students. The system allows the practice of various organ systems, body parts, bones, muscles, etc. The evaluation of our models on data from students who practice anatomy could also open interesting questions about difficulty of items and the effect of forgetting where most students are collage students.

\section{Possible Extensions and Alternative Models}

The examined models approximate the effect of human memory decay with a time effect function which ignores some very important aspects of memory and forgetting. For example, in our models the spacing effect is completely disregarded. One extension of the PFA/G/T model could reuse some properties of the memory activation equation~\ref{eq-pavlik-activation} described in chapter~\ref{spacing-effect}. Next, an extension of the PFA/G/T model which considers multiple-choice questions might also be possible and could significantly improve its performance.

Another extension of the models could consider the fact that students mix up the places that are, for instance, geographically very close, have similar names, or seem somehow similar for entirely different reasons. Students typically know the approximate geographical location of the Nordic countries, however, they might struggle when determining exact location of each country, often they confuse Norway with Sweden as can be seen in Figure~\ref{fig:confusion-network}. If a students chooses Finland when the highlighted place is actually Norway, it usually does not mean that they have no knowledge of the country's location. In such cases it might be more accurate to slightly increase student's memory activation of the place.

\begin{figure}[htbp]
  \centering
  \includegraphics[width=\textwidth]{img/confusion-network}
  \caption{Confusion network of European countries. The edges between countries represent confusion of students. Dark blue edge means that the two connected countries are confused by students quite often while light blue indicates that the countries are still sometimes confused but at least less often.}
  \label{fig:confusion-network}
\end{figure}

A completely different approach on student modeling is the usage of Recurrent Neural Networks (RNNs), the family of RNN model have important advantage over models analyzed in our thesis---they do not require explicit encoding of the essential aspects of human memory and can capture more complex patterns of learning. The usage of RNNs in education, especially the Long Short Term Memory (LSTM) model, has been already explored and shows promising results~\cite{piech2015deep}. It might be interesting to analyze the RNN model in the context of the system for learning geography and also compare the performance with PFA models.


% $Id: a36f51514b47d1c52a9dbe2f8e29a2addeafd1ac $

\section{Introduction}

These days, Boolean satisfiability (SAT) solvers are undeniably quite reliable
tools despite the worst-case exponential run time of all known
algorithms. Modern SAT solvers provide a ``black-box'' procedure that can often
solve hard structured problems with over a million variables and several million
constraints. These tools are therefore used in many applications outside the
scope of knowledge representation and reasoning, which nonetheless was their
initial domain of application. The underlying representational formalism of SAT
solvers is propositional logic, and even if a given problem is not a pure
propositional problem, it is not unusual to encode it so that it can be viewed
as a propositional reasoning task. This encoding may be complex and possibly
costly, but the effectiveness of SAT solvers generally compensates for the cost
of this encoding.

This growing interest for SAT solvers has steered them toward a more first order
approach, with the possibility to deal with first order literals and reason
modulo theories. This resulted in the design of Satisfiability Modulo Theories
(SMT) solvers, which are built on top of SAT solvers. Over the last past few
years, SMT solvers have appeared as very efficient tools to reason over some
well identified theories (equality, uninterpreted functions, linear arithmetic,
arrays, etc.), and have allowed us to drag SAT solving toward first order logic.
Although modern SMT solvers support first order logic, most of them use
heuristic quantifier instantiation for incorporating quantifier reasoning with
ground decision procedures. This mechanism is relatively effective in some cases
in practice, but it is not refutationally complete for first order logic. Hints
(triggers) are usually required, and it is sensitive to the syntactic structure
of the formula, so that it fails to prove formulas that can be easily discharged
by provers based on more traditional first order proof search methods (tableaux,
resolution, etc.).

In this paper, we propose to overcome this problem of completeness for first
order logic in SMT solving by combining a SAT solver core with tableau
calculus. The tableau calculus is introduced as a regular theory of SMT solving,
and the rules are used to unfold propositional content into clauses while atomic
formulas are handled using satisfiability decision procedures as in
SMT solvers. To deal with quantified first order formulas, we use
metavariables and perform rigid unification modulo equalities and rewriting, for
which we introduce an algorithm based on superposition, but where all clauses
contain a single atomic formula.

We also propose to improve proof search by introducing rewriting into the SAT
solver as a regular SMT theory and along the lines of deduction modulo
theory. Deduction modulo theory~\cite{DA03} focuses on the computational part of
a theory, where axioms are transformed into rewrite rules, which induces a
congruence over propositions, and where reasoning is performed modulo this
congruence. In deduction modulo theory, this congruence is then induced by a set
of rewrite rules over both terms and propositions.

Our approach provides several advantages compared to usual SMT solving and first
order proof search methods. First, we benefit from the efficiency of a SAT
solver core together with a complete method of instantiation (when a
propositional model is found, we try to find a conflict between two literals by
unification). Second, it should be noted that our approach requires no change in
the architecture of the SMT solver, since the tableau calculus and rewriting are
seen as regular theories. Finally, no preliminary Skolemization and Conjunctive
Normal Form (CNF) transformation is required. This transformation is performed
lazily by applying the tableau rules progressively when a literal is propagated
or decided. This makes the production of genuine output proofs easier, contrary
to the usual approach, where the Skolemization/CNF translation is realized at
the beginning and externalized with respect to the proof search.

Our proposal combining SAT solving with tableau calculus and rewriting has been
implemented and the corresponding tool is called \archsat{}. This tool is able
to deal with first order logic extended to polymorphic types à la ML, through a
type system in the spirit of~\cite{BP13}. To test this tool, we propose a
benchmark in the framework of the set theory of the \bmth{}
method~\cite{B-Book}. This theory~\cite{BA15} has been expressed using first
order logic extended to polymorphic types and turned into a theory that is
compatible with deduction modulo theory, i.e. where a large part of axioms has
been turned into rewrite rules. The benchmark itself gathers 319~lemmas coming
from Chap.~2 of the \bbook{}~\cite{B-Book}.

The paper is organized as follows: in Sec.~\ref{sec:smt}, we recall the classic
architecture of SMT solvers, and we introduce the tableau and rewriting
theories; we then describe, in Sec.~\ref{sec:super}, our mechanism of equational
reasoning by means of rigid unit superposition; finally, in
Sec.~\ref{sec:bench}, we present some experimental results obtained by running
our implementation over a benchmark of problems in the \bmth{} set theory.


\def\pathToRoot{../../}
\input{\pathToRoot/headers/uebungsheader.tex}

\def\issolution{}

\begin{document}

% {Sheet number}{headline}{deadline}
\exercisehead{5}{}{15.12.2020, 23:59}

\section*{Instructions}
Submit the jupyter notebook with the solution for exercise 5.2 b) in an archive along with the latex file.

\section*{Exercises}

\begin{exercise}[Computing Jacobian and Hessian][1 + 1 = 2]

Let $f(x,y) = 3x^2 y + 4x^3 y^4 - 7x^9 y^4$. Compute Jacobian and Hessian matrices of $f$.
\end{exercise}

\begin{solution}
   % write the solution here
\end{solution}

\begin{exercise}[Taylor Series and Newton's Method][1 + 2 + 1 + 1 = 5]

    \begin{enumerate}
    \item Derive the first 5 terms of the Taylor series about $x_0 = 0$ for $f(x) = cos(x)$, and write the series in sigma notation (e.g. as an infinite sum).

    \item In python, apply Newton's method to find the nearest critical point of \\
    \begin{center} $f(x, y) = x^2 - y^2 + 4 - 3xy$\\
    from the initial point $x_0 = -0.3, y_0 = 0.3$.\\ \end{center}
    After each iteration, check the value of the first derivative, i.e. Jacobian: if Jacobian is 0, then we reached the critical point.\\
    Plot the original function for x and y in range from -0.5 to 0.5 with step size of 0.01, along with the initial point and the points computed after each iteration.
    Use method \href{https://matplotlib.org/3.1.0/gallery/mplot3d/surface3d.html}{.surface\_plot()} with parameter $alpha=0.3$ for plotting the function and \href{https://matplotlib.org/mpl_toolkits/mplot3d/tutorial.html}{.scatter()} for plotting the points. \\
	What kind of problem of function minimization task is illustrated with this example?

    \item How is Newton’s Method related to gradient descent?

	\item In which case is it impossible to apply Newton's method? Hint: look at the multidimensional generalization of the formula.

  \end{enumerate}

\end{exercise}

\begin{solution}
   % write the solution here
\end{solution}

\begin{exercise}[Activation Functions][1.5 + 1 + 0.5 = 3]

    \begin{enumerate}
    \item Three of the most commonly-used activation functions are the sigmoid function, hyperbolic tangent, and ReLU.  The equations for these are given below.  Compute the first derivative of each function.  Note that your final derivative for tanh should not be written in terms of other hyperbolic functions, though you may use these in your calculation. Hint: ReLU is not differentiable at x = 0.  For the purposes of your derivative, you may define its derivative piecewise, ignoring this point.
    \[\sigma(x) = \frac{1}{1+e^{-x}}\,\,\,\,\,\tanh(x) = \frac{e^{2x} - 1}{e^{2x} + 1}\]
    \[ReLU(x) =   \left\{
\begin{array}{ll}
      0 & x < 0 \\
      x & x > 0 \\
\end{array}
\right. \]

    \item Using an online resource like Wolfram Alpha or Desmos, graph each function along with its derivative.  Discuss the differences you observe.  What are the advantages and disadvantages of each?  In particular, think about how the range of the function and the amplitude of the derivative would affect a network.

    \item Which activation function would be most appropriate for a classification problem when there are only two classes?  Would adding more classes change your choice?  Why or why not?
  \end{enumerate}

\end{exercise}

\begin{solution}
   % write the solution here
\end{solution}

\section*{Submission instructions}

\framebox{
	\begin{minipage}{\linewidth}
		The following instructions are mandatory. If you are not following them, tutors can
		decide to not correct your exercise.
	\end{minipage}
}

\begin{itemize}
    \item You have to submit the solutions of this assignment sheet as a team of 2-3 students.
    \item  Hand in a \textbf{single} PDF file with your solutions.
    \item Make sure to write the student teams ID and the name of each
    member of your team on your submission.
    \item Your assignment solution must be uploaded by only \textbf{one} of your team members to the course website.
    \item If you have any trouble with the submission, contact your tutor \textbf{before} the deadline.
\end{itemize}

\end{document}


% !TEX root = atlas_iros_16.tex
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{\large Collision Detection}
\label{sec:CollDet}
%
In the following experiments (Sec.~\ref{sec:CollDet_EE} and \ref{sec:CollDet_Link}), we used the collision handling (\ref{eqn:CollDet}) and (\ref{eqn:CollReact}) to detect collisions with the Atlas robot with a disturbance joint torque threshold  $\bm{\zeta}$ of $11$~Nm for each joint.
A lower threshold frequently leads to false alarms due to remaining modeling errors (see Fig.~\ref{fig:ident_torque_compare}) and the large friction effects (see Fig.~\ref{fig:ident_friction_char}).
Our model is not able to reliably remove the static friction torque from the observer ($\kappa_{\mathrm{f}}=1$ in (\ref{eqn:observer_settings})) due to the indeterminate friction state for low velocities \cite{BidardLibArhMea2005}.
The friction compensation in (\ref{eqn:controller}) currently leads to unwanted oscillations indicating closed-loop stability problems originating in the velocity feedback, as described in \cite{ConnerKohRomStu2015}.
Therefore, this feature is disabled in the experiments with $\kappa_{\mathrm{f}}=0$.
However, it is possible to reliably detect soft collisions at moderate velocity (100~mm/s) in adequate time with the Atlas system.
For the collision detection and reaction experiments, the disturbance compensation in (\ref{eqn:controller}) was disabled ($\kappa_\varepsilon=0$).
The first joint was selected for evaluation of the experiments, since the collision is detected at this joint first. The other axes show qualitatively the same behavior.

\subsection{End-effector Collision}
\label{sec:CollDet_EE}
%
\begin{figure}
\begin{subfigure}{.24\textwidth}
  \centering
  \includegraphics[trim=550 250 350 100,clip,width=.98\linewidth]{figures/CollDetCinderblock/ImpCtrlv5_E057_R04_impCtrl_K300_wall_colldet_move_away_021.jpg}
  \caption{Enabled (``Coll.Det'' in Fig.~\ref{fig:CinderCollisionSummary})}
  \label{fig:CinderColl_zerog}
\end{subfigure}%
\begin{subfigure}{.24\textwidth}
  \centering
  \includegraphics[trim=550 250 350 100,clip,width=.98\linewidth]{figures/CollDetCinderblock/ImpCtrlv5_E057_R05_impCtrl_K300_wall_nocolldet_collision_022.jpg}
  \caption{Disabled (``Stiff'' in Fig.~\ref{fig:CinderCollisionSummary})}
  \label{fig:CinderColl_push}
\end{subfigure}
\caption{Result after the end-effector collision with and without detection and reaction (snapshot at $t\approx1.5$~s of Fig.~\ref{fig:CinderCollisionSummary})}
\label{fig:CinderCollPictures}
\SkipBeforeText
\end{figure}
%
\begin{figure}
\centering
\parbox{\columnwidth}{
\includegraphics{figures/CollDetCinderblock/Collision_Summary_Cinderblock.pdf}
}
\vspace*{-1ex}
\caption{Absolute external force $|\bm{f}_\mathrm{ext,EE}|$, observed disturbance torque $\hat{\tau}_{\varepsilon,1}$ and joint position $q_1$ are depicted for the cinderblock collision experiment (see also Fig.~\ref{fig:CinderCollPictures}).
For the ``Stiff'' and ``Coll.Det'' mode, $k_i=300$~Nm/rad and for the ``Compliant'' mode $k_i=100$~Nm/rad were chosen.}
\label{fig:CinderCollisionSummary}
\SkipBeforeText
\end{figure}
%
The first experiment lets the end-effector collide with a styrofoam-protected cinderblock, see Fig.~\ref{fig:CinderCollPictures}.
The protection aimed to protect the robot from damage and has no major influence on the collision detection.
The measured data during the three runs with different settings is depicted in Fig.~\ref{fig:CinderCollisionSummary}.
Using a rather stiff joint impedance controller with joint-wise $k_i=300$~Nm/rad  (``Stiff'') and deactivated collision handling leads to pushing the cinderblock away and tipping it over till reaching the final goal position, see Fig.~\ref{fig:CinderCollPictures}~(b) and Fig.~\ref{fig:CinderCollisionSummary}.
Reducing the joint stiffness to \mbox{$k_i=100$~Nm/rad} (``Compliant'') leads to smaller resulting quasi-static contact forces, while the positioning error increases, see Fig.~\ref{fig:CinderCollisionSummary}.

Using the collision detection and reaction (``Col. Det.''), the arm drifts away after the collision has been detected and the contact force decays to zero, see Fig.~\ref{fig:CinderCollPictures}~(a) and Fig.~\ref{fig:CinderCollisionSummary}.
The collision in Fig.~\ref{fig:CinderCollisionSummary} is detected at $t=1.1$~s, when the disturbance torque line crosses the detection threshold $\bm{\zeta}$.
At this point in time, the maximum contact force of about 69~N is already reached, which is caused by the relatively slow dynamics of the observer (\mbox{$k_{\mathrm{o},i}=5~\mathrm{s}^{-1}$}).
A larger gain would allow an earlier detection with less reaction force.
Using retract reflexes would allow further reduction of contact forces.
In turn, larger observer gain causes vibrations when being used as disturbance compensation.
To overcome this issue, two observers could be used in parallel: a slower one for compensation and a faster one for collision detection.

\subsection{Link Collision}
\label{sec:CollDet_Link}
%
\begin{figure}
\centering
\parbox{\columnwidth}{
% \includegraphics[trim=550 50 350 340,clip,width=\linewidth]{figures/CollDetStick/GOPR2201_ImpCtrlv5_E055_part4_t036.jpg}
\input{./figures/CollDetStick/GOPR2201_ImpCtrlv5_E055_part4_t036_bearb.pdf_tex}
}
\caption{Scene during the collision from Fig.~\ref{fig:StickCollisionSummary} at $t\approx5.5$~s.}
\label{fig:StickCollisionPhoto}
\SkipBeforePicture
\end{figure}
%
\begin{figure}
\centering
\parbox{\columnwidth}{
\includegraphics{figures/CollDetStick/CollDetStickIROS.pdf}
}
\vspace*{-1ex}
\caption{Joint position $q_1$, model torque $\tau_{\mathrm{mdl},1}$, estimated disturbance torque $\hat{\tau}_{\varepsilon,1}$ and motor-side joint torque $\tau_{\mathrm{m},1}$ during link collision, see Fig.~\ref{fig:StickCollisionPhoto}.}
\label{fig:StickCollisionSummary}
\SkipBeforeText
\end{figure}
%
The second experiment shows that collisions at the proximal links and not only at the end-effector are detected as well.
The arm was pushed at the elbow with a cardboard stick against the moving direction during a joint trajectory, see Fig.~\ref{fig:StickCollisionPhoto} (in the depicted situation, the elbow was moving to the right).
The movement speed was $\approx50~\mathrm{mm/s}$ at the collision location.
Figure~\ref{fig:StickCollisionSummary} depicts the measurement data. The collision at link $4$ starts at $t\approx5.5$~s, which one can see from the high motor torque in joint 1.
The detection threshold is reached at $t\approx5.6$~s, leading to the halt in gravity compensation mode.


%\section{Radiation}
{\bf \Large
\begin{tabular}{ccc}
\hline
  Corresponding author & : & Hisashi Yashiro\\
\hline
\end{tabular}
}
\\

\subsection{mstrnX}
SCALE implements a broadband atmospheric radiative transfer model named ``Model Simulation radiation TRaNsfer code version X (mstrn-X)'' developed by \citet{nakajima_2000} and \citet{sekiguchi_2008}. \verb|mstrn| is based on the discrete ordinate method with a delta two-stream approximation and the correlated k-distribution method. The model calculates long- and short-wave radiation fluxes using the atmospheric states, the three dimensional distribution of clouds/gases/aerosols, and the property of land/ocean surface.
The spectrum between 0.2 and \SI{200}{\micro m} is divided into 29 spectral bands and 111 integration points.

The calculation of solar insolation is based on the parameterization of \citet{berger_1978}. The expected top of atmosphere in the radiation model is 100 km above sea level. However, it is sometimes higher than the top of the domain used in the limited-area simulations.
Thus, SCALE can add the climatological profile to the upper part of the model domain, as needed. The COSPAR International Reference Atmosphere (CIRA-86)  \citep{CSR_2006} is used for the climatological profile of temperature and pressure. The standard profiles of trace gases such as oxygen, carbon dioxide, ozone, water vapor, methane, etc., are referred to the MIPAS reference atmospheres (\citet{Remedios_2007}).


\section{Methods}
The computational methodology described herein has been implemented in the open-source C\_MPI program\footnote{MPI is the abbreviation of message passing interface.} \texttt{fitoog}.\autocite{mccluskey_arm61/fitoog_2019}

\subsection{Simulation Methodology}
%
\begin{marginfigure}
    \centering
    \includegraphics[width=\linewidth]{smallangle/director}
    \caption{A graphical description of the severe coarse-graining applied to the MARTINI description of the \emph{n}-decyltrimethylammonium surfactant molecule for the use of the particle swarm algorithm.}
    \label{fig:director}
\end{marginfigure}
%
The \texttt{fitoog} software takes a series of input files that define the molecules, intramolecular interactions, and scattering lengths for the constituent particles.
The molecule input file is a space-separated file consisting of an index, particle name, \emph{x}-coordinate, \emph{y}-coordinate, \emph{z}-coordinate, and scattering length.
When \texttt{fitoog} reads in a molecular input file, a \texttt{differences} object is created, this stores the differential between the atomistic, or near-atomistic\footnote{Such as the MARTINI-style beading use in this work.} description and the more severe coarse-grained description that is used in the PSO.

Inspired by the coarse-graining methodology applied to directional colloid self-assembly by Law \emph{et al.},\autocite{law_coarse-grained_2016} a severe coarse-graining methodology was developed for use on surfactant molecules in \texttt{fitoog}.
This allowed for a significantly reduced parameter dimensionality to which the PSO\footnote{Which is described in Section \ref{sec:partswarm}.} could be applied.
The severe coarse-graining reduced the surfactant to a ``director'' description; where each surfactant molecule is defined by a position and a direction, shown pictorially in Figure~\ref{fig:director}.
This reduced the parameter dimensionality to just six variables per molecule; three of which described the centre-of-mass position of the molecule and three that describe the angular orientation of the surfactant in space.\footnote{Given the symbols, $a$, $b$, $c$ for the positions and $\phi$, $\omega$, and $\kappa$ for the angles.}
Additionally, if the molecule was a single particle in length, as is the case for the \ce{NO3-} anion in the Section~\ref{sec:real_data}, then the dimensionality is just three, as the direction is arbitrary.

The methodology for the particle swarm in this work followed the implementation discussed in Section~\ref{sec:partswarm}.
Where the parameters to be optimised where the position and orientation of the input molecules.\footnote{The $a$, $b$, $c$, $\phi$, $\omega$, and $\kappa$ mentioned above.}
Following each PSO iteration, the molecule director is expanded from the position variable\footnote{Using the \texttt{differences} object mentioned above.}.
This representation is then rotated based on a rotation matrix, in this work the rotation matrix was constructed by first rotating the rotation axis by $-\phi$ and $-\omega$, then rotating by $\kappa$ in around the $z$-axis, before rotating the axis back to the original position by $\omega$ and $\phi$.\sidecite[Figure~\ref{fig:rot} defines these angles]{evans_rotations_2001}
Following the expansion and reorientation, the scattering profile is then calculated using the Debye equation,\autocite{debye_zerstreuung_1915} this was used over the more efficient Golden Vectors\autocite{watson_rapid_2013} or Fibonacci sequence\autocite{svergun_solution_1994} methods as the aim of this work was to assess the application of the PSO method and efficiency was not the initial goal.
The agreement between the calculated scattering profile and the experimental input scattering was used as a figure of merit, $\zeta$, that was to be optimised by the particle swarm algorithm.
This $\zeta$ was a simple $\chi^2$ value found as follows,
%
\begin{equation}
\zeta = \chi^2 = \sum\frac{(I_{\text{exp}}(q) - I_{\text{calc}}(q))^2}{\text{d}I_{\text{exp}}(q)},
\end{equation}
%
where $I_{\text{exp}}(q)$ is the experimental scattering intensity, $I_{\text{calc}}(q)$ is the calculated scattering intensity, and $\text{d}I_{\text{exp}}(q)$ is the uncertainty in the experimental scattering intensity, all at a given $q$-vector.
%
\begin{marginfigure}
    \centering
\includegraphics[width=\linewidth]{smallangle/rotdia}
    \caption{The definitation of the polar angles used in the coarse grained representation of the surfactant molecule.}
    \label{fig:rot}
\end{marginfigure}
%

Throughout this work, an interia weight for the PSO of 0.4 was used.
Generally, values of between 0 and 2 are used for the global and personal acceleration coefficients\footnote{These are standard values for the PSO method following \cite{sun_study_2010}.}.
For the test case discussed below a value of 2 was used for both acceleration coefficients, however, in the real case discussed later, the global acceleration coefficient was reduced to 1.
This was chosen to reduce the acceleration toward the global best and improve the ability for the PSO to search the parameter space available in this much larger problem.

\subsection{Parallelisation}
\label{sec:para}
The use of a population-based optimisation method, such as the PSO, allowed for easy access to highly parallel simulation.
Parallelisation was achieved by spreading the population evenly across the cores that were available to the simulation.
Inter-core messaging was performed using the MPI libraries, and to ensure efficiency only the figures of merit and the best possible structure were shared across the cores.
This means that during a given \texttt{fitoog} run, the only serial component was the determination of the lowest figure of merit.
The efficiency of the parallelisation was defined by considering the strong and weak scaling of the software.
It was possible to determine the percentage of serial, $s$, and parallel, $p$, components of the software by fitting the speedup\footnote{The time taken for a job to run on a single core divided by the time taken on multiple cores.} with Amdahl's law,\autocite{amdahl_validity_1967}
%
\begin{equation}
\text{speedup} = \frac{1}{s + \sfrac{p}{N}},
\end{equation}
%
where, $N$ is the number of cores in the parallel job, and $s + p = 1$.
While more sophisticated methodologies could be used to further reduce the serial component, such as having a core-level best population that is only occasionally communicated with the entire swarm, this implementation was shown to be highly parallelised and useful for assessing the utility of the PSO method.


% !TEX root = ../zeth-protocol-specification.tex

\section{\zeth~statement after primitive instantiation}\label{instantiation:statement}

After instantiating the various primitives and providing security proofs to justify that they comply with the security requirements listed in previous sections, $\RELCIRC$ now becomes:

\begin{itemize}
    \item For each $i \in [\jsin]$:
    \begin{enumerate}
        \item $ \auxinputs.\jsins{i}.\znote.\apk = \blake{2s}{\taggedaddr \concat \pad{0}{\blakeCompLen}}$ \\ with $\taggedaddr$ defined in~\cref{instantiation:prf-comm-crh:prf}
        \item $\auxinputs.\jsins{i}.\nf{} = \blake{2s}{\taggednf \concat \auxinputs.\jsins{i}.\znote.\rho}$ \\ with $\taggednf$ defined in~\cref{instantiation:prf-comm-crh:prf}
        \item $\auxinputs.\jsins{i}.\cm{} = \blake{2s}{\auxinputs.\jsins{i}.\znote.\noter{} \concat \msg}$ \\ with $\msg = \auxinputs.\jsins{i}.\znote.\apk \concat \auxinputs.\jsins{i}.\znote.\rrho \concat \auxinputs.\jsins{i}.\znote.\notev$
        \item $\auxinputs.\htags{i} = \blake{2s}{ \taggedpk \concat \priminputs.\hsig}$ (malleability fix, see~\cref{appendix:trnm}) with $\taggedpk$ defined in~\cref{instantiation:prf-comm-crh:prf}
        \item $(\auxinputs.\jsins{i}.\znote.\notev) \cdot (1 - e) = 0$ is satisfied for the boolean value $e$ set such that if $\auxinputs.\jsins{i}.\znote.\notev > 0$ then $e = 1$.
        \item The Merkle root $\mkroot'$ used to check the Merkle authentication path $\auxinputs.\jsins{i}.\mkpath$ of commitment $\auxinputs.\jsins{i}.\cm{}$, with $\mimcMPPrime{\rCURVE}$, equals $\priminputs.\mkroot$ if $e = 1$.
        \item $\priminputs.\nfs{i}$ \\ $= \indexedset{\pack{\slice{\auxinputs.\jsins{i}.\nf{}}{k \cdot \fieldBitCap}{(k+1) \cdot \fieldBitCap}}{\FFx{\rCURVE}}}{k \in [\floor{\prfNfOutLen/\fieldBitCap}]}$
        \item $\priminputs.\htags{i}$ \\ $= \indexedset{\pack{\slice{\auxinputs.\htags{i}}{k \cdot \fieldBitCap}{(k+1) \cdot \fieldBitCap}}{\FFx{\rCURVE}}}{k \in [\floor{\prfPkOutLen/\fieldBitCap}]}$
    \end{enumerate}
    \item For each $j \in [\jsout]$:
    \begin{enumerate}
        \item $\auxinputs.\znotes{j}.\rrho = \blake{2s}{ \taggedrho \concat \priminputs.\hsig}$ (malleability fix, see~\cref{appendix:trnm}) with $\taggedrho$ defined in~\cref{instantiation:prf-comm-crh:prf}
        \item $\priminputs.\cms{j} = \blake{2s}{\auxinputs.\znotes{j}.\noter{} \concat \msg }$ \\ with $\msg = \auxinputs.\znotes{j}.\apk \concat \auxinputs.\znotes{j}.\rrho \concat \auxinputs.\znotes{j}.\notev$
    \end{enumerate}
    \item $\priminputs.\hsig = \indexedset{\pack{\slice{\auxinputs.\hsig}{k \cdot \fieldBitCap}{(k+1) \cdot \fieldBitCap}}{\FFx{\rCURVE}}}{k \in [\floor{\crhhsigOutLen/\fieldBitCap}]}$
    \item $\priminputs.\resbits = \packResBits{\indexedset{\auxinputs.\jsins{i}.\nf{}}{i \in [\jsin]}, \auxinputs.\vin, \auxinputs.\vout, \auxinputs.\hsig, \indexedset{\auxinputs.\htags{i}}{i \in [\jsin]}}$
    \item Check that the ``\gls{joinsplit} is balanced'', i.e.~check that the \gls{joinsplit-eq} holds:
    \begin{align*}
        &\pack{\auxinputs.\vin}{\FFx{\rCURVE}} + \sum_{i \in [\jsin]} \pack{\auxinputs.\jsins{i}.\znote.\notev}{\FFx{\rCURVE}} \\
        & = \sum_{j \in [\jsout]} \pack{\auxinputs.\znotes{j}.\notev}{\FFx{\rCURVE}} + \pack{\auxinputs.\vout}{\FFx{\rCURVE}}
    \end{align*}
\end{itemize}

\begin{remark}
    For higher security, we could use \blake{2b}{} with 32-byte output instead of \sha{256}. In fact, since a precompiled contract computing the \blake{2}{}~compression function~\cite{blakecompietf} has been added to the Istanbul release of \ethereum~(EIP 152~\cite{blake-eip}), it could be possible to write a small wrapper on the smart contracts, in order to hash with \blake{2b}{} with any parameter.
\end{remark}

\subsection{Instantiating the packing functions}\label{instantiation:statement:pack}

As we consider SNARKs based on arithmetic circuits defined over a prime field, all variables in the constraint system are interpreted as field elements. Nevertheless, as illustrated in~\cref{zeth-protocol:statement}, part of the statement consists of functions whose co-domains are sets of binary strings (which may be longer than the bit representation of elements of the finite field). While a bit (i.e.~$\{0, 1\}$) is an element of $\FF_p$ ($p$ prime), it is important to minimize the number of gates in the arithmetic circuit (for proof generation efficiency), and to minimize the number of input wires (to improve verification time). This can be done by representing fragments of binary strings as the base 2 decomposition of field elements, thereby ``packing'' binary strings into multiple elements. Converting binary strings into field elements requires the addition of some arithmetic gates (extending the statement to be proven), but reduces the number of primary inputs (reducing the complexity of the SNARK verification carried out on-chain).
The cost of $\groth$ zk-SNARK~\cite{groth2016size} proof verification is linear in the number of primary inputs, since each input acts as a scalar in a costly scalar multiplication of a curve point in $\gset_1$. Hence, while packing slightly increases the prover cost -- by adding constraints to the circuit -- it simplifies the verifier's work.

In this section, we detail the method by which we encode (resp.~decode) a set of binary strings to (resp.~from) sets of field elements. In the rest of this section, the notion of \emph{packing policy} refers to the set of \emph{packing} and \emph{unpacking} functions.

The set of primary inputs is composed of the input nullifiers, the output commitments, the public values (see~\cite[Section 3.4.3]{zethpaper}) along with the signature hash and the authentication tags for security (malleability fix, see~\cref{appendix:trnm}). The complete description of the public inputs is represented in~\cref{instantiation:eq:primary-inputs}.
%
\begin{equation}\label{instantiation:eq:primary-inputs}
    (\indexedset{\priminputs.\nf{i}}{i \in [\jsin]}, \indexedset{\priminputs.\cms{j}}{j \in [\jsout]}, \vin, \vout, \hsig, \indexedset{\priminputs.\htags{i}}{i \in [\jsin]})
\end{equation}

The primary inputs that consist of binary strings are: the nullifiers $\nfs{}$, the public values $\vin$ and $\vout$, the signature hash $\hsig$ and the authentication tags $\htags{}$.

For a binary string $x$, let $\alpha_x = \ceil{\len{x} / \fieldBitCap}$ be the number of field elements required to completely encode $x$ and let $\beta_x = \floor{\len{x} / \fieldBitCap}$ be the number of field elements whose capacity is fully used. Let $\gamma_x = \len{x} \pmod{\fieldBitCap}$ be the number of ``residual'' bits remaining after fully using $\beta_x$ field elements.

\begin{example}\label{instantiation:eg:packing}
    Consider binary strings $A \in \bin^7$ of length 7, to be encoded over the field $\FFx{41}$. This field has a capacity of $5$ bits, and therefore $\alpha_A = 2$, $\beta_A = 1$, and $\gamma_A = 2$. That is, $A$ can be represented as 2 field elements, or as 1 field element with 2 ``residual'' bits.

    Consider $A = (1111011)$. \cref{instantiation:fig:packingA} illustrates how $A$ can be packed as field elements. Note that the 2 residual bits are taken from the ``beginning'' of the bit string, that is, the highest order bits.
\end{example}

\begin{figure}[ht]
    \centering
    \includegraphics[width=1\textwidth]{images/bit-packing-stringA.png}
    \caption{Packing of string $A$ (see~\cref{instantiation:eg:packing})}\label{instantiation:fig:packingA}
\end{figure}

We now consider strategies to pack all primary inputs that are binary strings. A naive approach is to encode each binary string $x$ as $\alpha_x$ field elements. In general, this results in significant waste (and consequently more field elements than necessary), especially when the number of residual bits is small compared to $\fieldBitCap$ (see~\cref{instantiation:fig:packing-multi-strings}). An alternative strategy could be to concatenate all binary strings into a single string $y$ and pack this string into $\alpha_y$ field elements. While this approach minimizes the set of unused bits, each unpack operation would require different shift and mask operations over 2 or 3 field elements. This significantly increases the complexity of the unpacking operation that must be performed on-chain, resulting in a higher gas cost (due to extra logic) or more contract code (if each unpack operation is hard-coded).

The \zeth~protocol requires that each binary string variable $x$ is packed into $\beta_x$ field elements, and the residual bits from all binary strings, along with the public values $\vin$ and $\vout$, are aggregated into a variable $\resbits$. Let $\resBitsBLen$ be the total number of residual bits, and $\resBitsFLen$ be the number of field elements required to represent $\resbits$. We assume that $\zvalueLen < \fieldBitCap$, and define the notation $\gamma_\notev = \zvalueLen$ for the bit lengths of public values $\vin$ and $\vout$. Thus $\resBitsBLen$ is given by
\[
    \resBitsBLen = \gamma_\hsig + 2 \cdot \gamma_\notev + \jsin \cdot (\gamma_{\nf{}} + \gamma_{\htag{}} )
\]
and the lengths, in field elements, of each of the correponding public inputs are
\begin{align*}
    \nfFLen &= \beta_{\nf{}} \\
    \hsigFLen &= \beta_{\hsig} \\
    \htagFLen &= \beta_{\htag{}} \\
    \resBitsFLen &= \ceil{\resBitsBLen / \fieldBitCap}
\end{align*}

\begin{figure}[ht]
    \centering
    \includegraphics[width=1\textwidth]{images/bit-packing-multi-strings.png}
    \caption{Packing of multiple strings. Observe that, by carefully arranging the bits of the input strings, it is possible to output fewer field elements}\label{instantiation:fig:packing-multi-strings}
\end{figure}

The residual bits $\resbits$ are formatted as follows:
\[
    \widetilde{\hsig} \concat \widetilde{\nfs{}} \concat \widetilde{\htags{}} \concat \vin \concat \vout
\]
where $\widetilde{\hsig}, \widetilde{\nfs{}}, \widetilde{\htags{}}$ are, respectively, the $\gamma_\hsig, \gamma_\nf{}, \gamma_\htag{}$ bits.

Note that the public values are packed into the ``last'', or lowest order, $2 \cdot \gamma_\notev$ bits of the resulting field element(s). In this way, their unpack functions are independent of the values $\jsin$ and $\jsout$ and of the number of residual bits required for each bit string (and consequently, independent of the finite field used).

To format the unpacked primary inputs into field elements, we define the following functions.
Given a bit string of length less than $\fieldBitCap$, the algorithm \pack{}{} (see~\cref{zeth-protocol:fig:packing-alg}) returns a field element. Given the nullifiers, public values and authentication tags, the algorithm \packResBits{}{} (see~\cref{zeth-protocol:fig:packing-resbits-alg}) outputs the residual bits. Given a set of packed field elements and the residual bits, the algorithm \unpack{}{} returns the variables reassembled as binary strings. In particular, we have that $\unpack{\priminputs.\nfs{}, \resbits}{\nf{}} = \indexedset{\auxinputs.\jsins{i}.\nf{}}{i \in [\jsin]}$.
\begin{align*}
    &\pack{}{\FFx{\rCURVE}} : \BB^{\leq \fieldBitCap} \to \FFx{\rCURVE} \\
    &\packResBits{} : {(\BB^\prfNfOutLen)}^{\jsin} \times {(\BB^\zvalueLen)}^2 \times \BB^\crhhsigOutLen \times {(\BB^\prfPkOutLen)}^{\jsin} \to (\FFx{\rCURVE})^\resBitsFLen \\
    &\unpack{}{} : \FFx{\rCURVE}^* \times (\FFx{\rCURVE})^{\resBitsFLen} \to \BB^* \\
\end{align*}

The $\unpack{}{}$ functions for nullifiers, public values and signature hash are represented as follows.
\begin{align*}
    &\unpack{}{}_{\hsig} : (\FFx{\rCURVE})^\hsigFLen \times (\FFx{\rCURVE})^{\resBitsFLen} \to \BB^\crhhsigOutLen \\
    &\unpack{}{}_{\nf{}} : (\FFx{\rCURVE})^\nfFLen \times (\FFx{\rCURVE})^{\resBitsFLen} \to \BB^\prfNfOutLen \\
    &\unpack{}{}_{\vin} : \FFx{\rCURVE}^0 \times (\FFx{\rCURVE})^{\resBitsFLen} \to \BB^\zvalueLen \\
    &\unpack{}{}_{\vout} : \FFx{\rCURVE}^0 \times (\FFx{\rCURVE})^{\resBitsFLen} \to \BB^\zvalueLen
\end{align*}

\begin{figure*}
    \begin{minipage}[t]{.4\textwidth}
        \centering
        \procedure{\pack{x}{\FFx{\rCURVE}}}{%
            out \gets 0_{\FFx{\rCURVE}}; \\
            \pcfor i \in [\len{x}] \pcdo: \\
            \t \pcif x[i] = 1 \pcdo: \\
            \t \t out \gets out +_{\FFx{\rCURVE}} 2^{\len{x}-1-i} \\
            \pcreturn out;
        }
        \caption{Algorithm to pack bits into a field element.}\label{zeth-protocol:fig:packing-alg}
    \end{minipage}%
    \begin{minipage}[t]{.6\textwidth}
        \centering
        \procedure{\packResBits{\nfs{}, \vin, \vout, \hsig, \htags{}}}{%
            out \gets []; r \gets \epsilon; \\
            r \gets \vout; \\
            r \gets \vin \concat r; \\
            %
            \pcfor i \in [\jsin] \pcdo:\\
            \t r \gets \slice{\htags{i}}{\beta_{\htags{i}} \cdot \fieldBitCap}{} \concat r; \\
            %
            \pcfor i \in [\jsin] \pcdo:\\
            \t r \gets \slice{\nfs{i}}{\beta_{\nfs{i}} \cdot \fieldBitCap}{} \concat r; \\
            %
            r \gets \slice{\hsig}{\beta_{\hsig} \cdot \fieldBitCap}{} \concat r; \\
            %
            \pcfor i \in [ \ceil{\len{r}/ \fieldBitCap} ] \pcdo: \\
            \t out[i] \gets \pack{\slice{r}{i \cdot \fieldBitCap}{ (i+1) \cdot \fieldBitCap}}{\FFx{\rBN}}; \\
            \pcreturn out;
        }
        \caption{Algorithm to pack residual bits.}\label{zeth-protocol:fig:packing-resbits-alg}
    \end{minipage}
\end{figure*}

\subsubsection{Packing Policy Security}
\begin{proposition}[Packing security]
    For a binary string $x$, it holds that $\unpack{\pack{x}{}}{} = x$ and $\unpack{\packResBits{x}}{} = x$.
\end{proposition}

\subsubsection{Packing Policy Example}

In the case where $\jsin=\jsout=2$, the \BNCurve~is being used (in which field elements hold $\bnFieldBitCap$ bits) and all PRFs and $\crhhsig{}$ output bit-strings of length 256, the unpacked primary inputs are 2167-bit long.
The packing parameters are therefore:
\begin{align*}
    & \resBitsBLen = 5 \times 3 + 64 + 64 = 143 \\
    & \nfFLen = \hsigFLen = \htagFLen = \resBitsFLen = 1
\end{align*}
The packed primary inputs are 2277 bits long, corresponding to a small space overhead of $\approx 5\%$ unused bits. Moreover, the 143-bit residual bits can be packed into a single field element. As such, the primary inputs are encoded as 9 field elements.
Finally, the residual bits are formatted as follows,
\[
    \underbrace{padding}_{\text{113 bits}}
    \concat \underbrace{\hsig}_{3\ bits}
    \concat \underbrace{\nf{1}}_{3\ bits}
    \concat \underbrace{\nf{0}}_{3\ bits}
    \concat \underbrace{\htag{1}}_{3\ bits}
    \concat \underbrace{\htag{0}}_{3\ bits}
    \concat \underbrace{\vin\vphantom{g}}_{64\ bits}
    \concat \underbrace{\vout\vphantom{g}}_{64\ bits}
\]

For the analogous case using \BLSCurve (in which field elements hold $\blsFieldBitCap$ bits), the packing parameters are:
\begin{align*}
    & \resBitsBLen = 5 \times 4 + 64 + 64 = 148 \\
    & \nfFLen = \hsigFLen = \htagFLen = \resBitsFLen = 1
\end{align*}
The residual bits can be packed into a single field element of the form
\[
    \underbrace{padding}_{\text{108 bits}} \concat \underbrace{\hsig}_{4\ bits} \concat \underbrace{\nf{0}}_{4\ bits} \concat \underbrace{\nf{1}}_{4\ bits} \concat \underbrace{\htag{0}}_{4\ bits} \concat \underbrace{\htag{1}}_{4\ bits} \concat \underbrace{\vin\vphantom{g}}_{64\ bits} \concat \underbrace{\vout\vphantom{g}}_{64\ bits}
\]
and the primary inputs are again encoded as 9 field elements.