\nonstopmode
\documentclass[submission,copyright,creativecommons]{eptcs}
\providecommand{\event}{LFMTP'11} % Name of the event you are submitting to
% \usepackage{breakurl}             % Not needed if you use pdflatex only.
\usepackage{hyperref}
\usepackage{proof}
\usepackage[all]{xy}

\input{mymacros}
\newcommand{\define}[1]{\mbox{\textbf{\textit{#1}}}}

\title{A Lambda Term Representation \\ Inspired by Linear Ordered Logic}
\author{Andreas Abel
\institute{
Theoretical Computer Science\\
Institut f\"ur Informatik\\
Ludwig-Maximilians-Universit\"at\\
M\"unchen, Germany}
\email{andreas.abel@ifi.lmu.de}
\and
Nicolai Kraus
\institute{
Functional Programming Laboratory\\
School of Computer Science\\
University of Nottingham\\
Nottingham, United Kingdom}
\email{ngk@cs.nott.ac.uk}
}
\def\titlerunning{A Lambda Term Representation Inspired by Linear Ordered Logic}
\def\authorrunning{Andreas Abel and Nicolai Kraus}
\begin{document}
\maketitle

\begin{abstract}
We introduce a new nameless representation of lambda terms inspired by
ordered logic.  At a lambda abstraction, number and relative
position of all occurrences of the bound variable are stored, and
application carries the additional information where to cut the
variable context into function and argument part.  This way,
complete information about free variable occurrence is available at each
subterm without requiring a traversal, and environments can
be kept exact such that they only assign values to variables that
actually occur in the associated term.
Our approach avoids space leaks in interpreters that build
function closures.

In this article, we prove correctness of the new representation and
present an experimental evaluation of its performance in a proof
checker for the Edinburgh Logical Framework.

Keywords:
representation of binders,
explicit substitutions,
ordered contexts,
space leaks,
Logical Framework.

% We introduce a nameless representation of lambda terms inspired by ordered logic.
% Information about number and places of variables bound by a lambda is available without examining the whole term, thus making it possible to drop unneeded substitutions early to avoid memory leaks.
% % We describe an implementation of this and other representations as well as suggested evaluation algorithms. These implementations were tested in Haskell by using them for typechecking large dependently typed terms of the logical framework (LF). The different needs of time and space are documented and compared.\\
% We also describe our implementation experiments and present some results.
\end{abstract}

\section{Introduction}
\label{sec:intro}

Type checking dependent types in languages like Agda~\cite{norell:PhD}
and Coq~\cite{inria:coq83} or logical frameworks like Twelf~\cite{carsten:twelf}
requires a large amount of evaluation, since types may depend on
values.  Such type checkers incorporate
an interpreter for purely functional programs with free variables---at least, the
$\lambda$-calculus---which is used to compute weak head normal forms
of types.  Efficiency of type checking is mostly identical with
efficiency of evaluation\footnote{Evaluation is necessary to reduce
  types to weak head normal form and compare types for equality.
  Subtracting these operations, type checking has linear complexity.
}
(and, in case of type reconstruction,
efficiency of unification), and remains a challenge as of today.
In seminal work, Gregoire and Leroy \cite{gregoireLeroy:icfp02} have
sped up Coq type checking by compiling to byte-code instead of
using an interpreter.  Boespflug \cite{boespflug:padl10} has obtained
further speed-ups by producing native code using stock-compilers.

While compilation approaches are successful on batch type \emph{checking}
fully explicit programs, they have not been attempted on type
\emph{reconstruction} using higher-order unification or on interactive
program construction such as in Agda and Epigram \cite{mcBrideMcKinna:view}.
These languages are involved and constantly evolving,
and their implementations are
prototypes and frequently modified and extended.  Implementing a
full compiler just to get type reconstruction going is deterring;
furthermore, compilation has not (yet) proven its feasibility in minor
evaluation tasks (like weak head evaluation)
that dominate higher-order unification.
At least for language prototyping, smart
interpreters are, and may remain, competitive with compilation.

For instance, Twelf's interpreter is sufficiently fast; it is
inspired by a term representation with de Bruijn indices
\cite{deBruijn:nameless} and explicit substitutions
\cite{abadiCardelliCurienLevy:jfp91}.  In the context of functional
programming, explicit substitutions are known as \emph{closures},
consisting of the code of a function plus an environment, assigning
values to the free variables appearing in the code.
In typical implementations of interpreters \cite{coquand:type},
these environments are not
precise; they assign values to all variables that are statically in
scope rather than only to those that are actually referred to in the
code.  This bears potential for space-leaks: the environment of a
closure might refer to a large value that is never used, but cannot be
garbage collected.  An obvious remedy to this threat is, when forming
a closure, to restrict the environment to the actual free variables;
however, this requires a traversal of the code.  We explore a
different direction: we are looking for a code representation that
maintains information about the free variables at each node of the
abstract syntax tree.

A principal candidate is \emph{linear typing} in Curry-Howard
correspondence with Girard's linear logic \cite{girard:linear}; there,
each variable in scope is actually referenced (more precisely,
referenced exactly once).  In other words, the
free variables are exactly the variables in the typing context.
Dropping types, we may talk of \emph{linear scoping}.
Yet we do not want to represent linear terms, but arbitrary
$\lambda$-terms.   Kesner and Lengrand \cite{kesnerLengrand:infcomp07}
achieve this by introducing explicit term constructs for weakening and
contraction.  We pursue a different path: we incorporate information
about variable use and multiplicity directly into abstraction and
application.

In the context of linear $\lambda$-calculus, the free
variables of a function application are the disjoint union of the free
variables of the function and the free variables of the argument.  If
we want to maintain the set of free variables during a term traversal,
at an application node we need to decide which variables go into the
function part and which into the argument part.  Thus, we would store
at each application a set of variables that go into the, say, function
part, all others would go to the argument part.  Less information is
needed if we switch to an \emph{ordered} representation.

\emph{Ordered logic}, also called \emph{non-commutative linear logic}
\cite{polakowPfenning:tlca99}, refines linear logic by removing the
structural rule \emph{exchange} which restricts hypotheses to be used
\emph{in the order they have been declared}.  Transferring this
principle to ordered scoping this means that the scoping context lists
the free variables in the order they occur in the term, from left to
right.  This allows pushing the context into an application with very
little information: we just need to know \emph{how many} variables
appear in the function part so we can cut the context in two at the
right position, splitting it into function context and argument
context.  This constitutes the central idea of our representation: at
each application node of the syntax tree, we store a number denoting
the number of free variable occurrences in the function part.  During
evaluation of an application in an environment, we can cut the
environment into two, the environment needed for the evaluation of the
function and the environment needed for the evaluation of the
argument.  Thus, our environments are precise and space leaks are
avoided.  In particular, a variable is always evaluated in a singleton
environment assigning only a value to this variable.  Following this
observation, variables do not need a name, they are identified by
their position; and environments are simply sequences of values.

Since we are not interested in proof terms of ordered logic per se,
but only borrow the \emph{ordered context} idea for our
representation of untyped $\lambda$-calculus,
we need to allow multiple occurrences of the same
variable.  In fact, the context shall list the variable
\emph{occurrences} in order.  At a lambda abstraction, we bind all
occurrences of the same variable.  Thus, at an abstraction node we
specify at which positions the bound variable should be inserted in
the scoping context.  This concludes the presentation of our idea.

In the rest of the paper, after an introductory example
(Section~\ref{sec:example}) we formally define our term representation
in Section~\ref{sec:syntax}.   Interpreter and handling of
environments are described in Section~\ref{sec:values}, followed by
the translation between ordinary lambda terms and ordered terms
(Section~\ref{sec:parsing}).  Soundness of the interpreter is formally
proven in Section~\ref{sec:sound}, before we conclude with an
experimental evaluation in Section~\ref{sec:experiments}.

This article summarizes the B.Sc.\ thesis of the second author
\cite{kraus:bachelor}.

\section{An Example}
\label{sec:example}

% Beta reduction is an important part of evaluation in functional
% programming languages, proof assistants and other formal
% systems. However, implementations often involve the risk of memory
% leaks and inefficiency. Here, we assume that all of our terms are
% well-typed and beta reduction is therefore strongly normalizing, so
% all we care about is efficiency.

%To demonstrate the problem we want to deal with, we apply the term we just mentioned on the free variables $g$ and $f$, consecutively.
%To get rid of the beta redexes, we could reduce it in the way shown below. By writing $t\multisubs {s_1} {x_1} {s_n} {x_n}$, we want to express that in the term $t$, each occurrence of the variable $x_1$ ($x_2, \ldots, x_n$) has to be replaced by the term $s_1$ (resp. $s_2, \ldots, s_n$) simultaneously. Such a substitution list always applies only to the directly preceding term:

To demonstrate the discussed risk of space-leaks during evaluation, we apply the term $\LaA x {\LaA y {\ApA{\ApA a b} y}}$ in basic syntax consecutively to the free variables $g$ and $f$. A possible (and, if the mentioned closures are used, typical) sequence of reduction steps is given below.
By writing $t\multisubs {s_1} {x_1} {s_n} {x_n}$, we want to express that in the term $t$, each occurrence of the variable $x_1$ ($x_2, \ldots, x_n$) has to be replaced by the term $s_1$ (resp. $s_2, \ldots, s_n$) simultaneously. Such a substitution list always applies only to the directly preceding term:
\[
\begin{array}{lr@{}l}
\shspace &(\LaA x {\LaA y {\ApA{\ApA a b} y}}) & \sspace g \sspace f \\
\shspace\re &(\LaA y {\ApA{\ApA a b} y}) & \sub g x \lspace f \\
\shspace\re &      (\ApA{\ApA a b} y) & \subs g x f y \\
\shspace\re &      \multicolumn 2 l {(a\sspace b)\subs g x f y \lspace y\subs g x f y} \\
\shspace\re &      \multicolumn 2 l {a\subs g x f y \lspace b\subs g x f y \lspace f} \\
\shspace\re &      \multicolumn 2 {c} {a \sspace b \sspace f}  \\
\end{array}
\]
Here, the substitution $\sub g x$ could be dropped instantly and there is no need to apply the other substitution $\sub f y$ to the term $\ApA a b$.
However, the term representation used above comes along with the problem that such an evaluation algorithm does not have the required information in time.
This is due to the fact that the binding information is always split between the $\la$ and the actual variable occurrence, as they both carry the variable name.
In contrast, using de Bruijn indices would make it possible to remove the piece of information from the $\la$.
Our goal is to do it the other way round: We want the whole information to be available at the $\la$, thus making it possible to know the number and places of the bound variable occurrences without looking at the whole term.

%A term representation always has to have a way to represent that a variable is bound by a lambda.
%For example, in the term $\LaA x {\LaA y {\ApA{\ApA a b} y}}$ in basic syntax, the information about the connection between the second $\la$ and the last variable is split: We have to look at both the $\la y$ and the $y$ in order to know what is going on. This may lead to some unnecessary difficulties. In contrast, the usage of de Bruijn indices makes it possible to remove the information from the $\la$ and concentrate them on the variable. Our goal is to do it the other way round: We want the whole information to be available at the $\la$.

% However, the algorithm used above does not have this information in time and this is a problem that many representations come along with. Motivated by this observation, we want to describe an alternative.

\section{Syntax}
\label{sec:syntax}

% The idea is that the information which variable is bound by which lambda should neither be split (like in the standard representation) nor carried by the variable (de Bruijn indices). Instead, it should be available as soon as the lambda becomes visible.
In this article, we only cover the core constructs of the lambda calculus as they are enough to make the approach clear. However, we do not see any limitations for common extensions. We first define \define{ordered preterms}:
\[
\begin{array}{lllrll@{\qquad}}
\mathsf{ordered \ preterms}       & \ni & t,u & ::= & x & \mbox{free variable (named $x$)} \\
                                           &&& \mid & \ovar & \mbox{bound variable (nameless)} \\
                                           &&& \mid & \ApO t m u & \mbox{application} \\
                                           &&& \mid & \LaO {\vec k} t & \mbox{abstraction} \\
\end{array}
\]
\emph{Free variables} are denoted by their name like in the standard syntax. \emph{Bound variables}, however, are just denoted by a dot $\ovar$, which does not carry any information beside the fact that it is a bound variable.
% This, for example, means that if $\left(\ApO \ovar 1 \ovar \right)$ is a subterm, it is not possible to tell whether we have two different variables or two times the same one without looking at the whole term.\footnote{However, the latter possibility could not be well-typed.}
In the case of an \emph{application}, there is a first term (the function part) and a second term (the argument) as usual.
Furthermore, the application carries an integer $m$ as an additional piece of information that will be important for the evaluation process and is explained
in a moment.
%the number of dots $\ovar$ in the first term which are not bound by lambdas in the first term itself should be denoted by the integer $k$. % COMMENT - there is no reason anymore to prefer the second to the first term. It might be a good idea to use the first term instead!? I have changed it now.
%This will be important for the evaluation process.
The most interesting part is the \emph{abstraction} $\LaO {\vec k} t$. The vector $\vec k = \ve{k_1, k_2, \ldots, k_n}$ is nothing else but a list of non-negative  integers of length $n$. It determines which dots $\ovar$ are bound by the $\la$ in the following way: Consider all $\ovar$ in the term $t$ which are not bound in $t$ itself. Now, the first $k_1$ of these are not bound by the $\la$, the next one is, the following $k_2$ are again not bound and so on (see examples below).

We denote the number of unbound $\ovar$ in an ordered preterm $t$ by $\freevars t$. Consequently, $\freevars \cdot$ is simply defined by:
\[
\begin{array}{cccc}
\freevars x = 0, & \freevars \ovar = 1, & \freevars {\ApO t m u} = \freevars {t} + \freevars{u}, & \freevars {\LaO {\ve {k_1, \ldots, k_n}} t} = \freevars t - n.
\end{array}
\]
%Note that in the case of $\ApO t k u$, we have $k = \freevars u$ by definition of $k$. \\
% new:
We call an ordered preterm $u$ an \define{ordered term} iff each sub-preterm $w$ of $u$ fulfils the following condition: If $w$ is an application $\ApO t m u$, the equation $m = \freevars t$ holds and if it is an abstraction $\LaO {\ve {k_1, \ldots, k_n}} t$, then $n + \sum_i k_i \leq \freevars t$ is satisfied. The latter condition states that if a $\la$ in $u$ binds a variable, this variable must actually exist while the first one just gives a meaning to the integer carried by an application. Clearly, any sub-preterm of an ordered term is again an ordered term.
%
%Any ordered preterm $u$ is an \define{ordered term} if each lambda abstraction $\LaO {\ve {k_1, \ldots, k_n}} t$ which is a sub-preterm of $u$ satisfies the condition $n + \sum_i k_i \leq \freevars t$. In other words, if a $\la$ in $u$ binds a variable, this variable must actually exist. In addition, each sub-preterm $\ApO t m u$ has to fulfil the condition $m = \freevars t$.
$u$ is called \define{closed} if $\freevars t = 0$.
% From now on, we only handle terms and forget about invalid preterms.

Here are some examples of closed terms.
The $S$ combinator
\[
\LaA x {\LaA y {\LaA z {\ApA x z \sspace \pa {\ApA y z}}}}
\]
would be written as
\[\LaO {\ve 0} {\LaO {\ve 1} {\LaO {\ve{1,1}} {\ApO {\ApO \ovar 1 \ovar} 2 {\pa{\ApO \ovar 1 \ovar}}}}}
\]
Moreover, the term
\[
(\LaA x {\LaA y {\ApA{\ApA a b} y}}) \sspace g \sspace f
\]
from Section~\ref{sec:example} would be represented as
\[
\ApO{\left(\ApO {\LaO{\emptyVec}{\LaO{\ve 0}{\ApO{\ApO{a}{0}{b}}{0}{\ovar}}}\right)}  0 f } 0 g
\]
(note that applications are still left-associative). We can see that the first $\la$ does not bind anything as it is annotated with the empty vector $\emptyVec$, while this is less obvious when it is written as $\la x$.

At this point, we hope to have clarified the intended meaning of our syntax. A formal definition will be given in Section~\ref{sec:parsing}.

\section{Values and Evaluation}
\label{sec:values}

Before specifying values and evaluation formally, we want to give an example to demonstrate how the information carried by a lambda should be used and why we always have exactly the needed information. Suppose we have the term
\[
\left(\LaO {\ve 0} {\LaO {\ve 1} {\LaO {\ve{1,1}} {\ovar \oapp 1 \ovar \oapp 2 {\pa{\ovar \oapp 1 \ovar}}}}}\right) \sspace g \sspace f \sspace n
%\left(\LaO {\ve 0} {\LaO {\ve 1} {\LaO {\ve{1,1}} {\ovar \oapp 1 \ovar \oapp 2 {\pa{\ovar \oapp 1 \ovar}}}}}\right) \oapp 0 g \oapp 0 f \oapp 0 n, \\
\]
that is, the $S$ combinator applied to three free variables
% i.e. the $S$ combinator which is first applied to a free variable $g$, then to $f$ and finally to $n$
(we suppress the application indices $0$ for better readability). We want to get rid of the beta redexes, so we start by eliminating the first one. The outermost $\la$ is decorated with the vector $\ve 0$ of length one. Now, the single variable bound by this $\la$ should be replaced by $g$, so we start a substitution list and insert a single $g$:
\[
\left(\LaO {\ve 1} {\LaO {\ve{1,1}} {\ovar \oapp 1 \ovar \oapp 2 {\pa{\ovar \oapp 1 \ovar}}}}\right) \left[ g \right] \lspace f \sspace n
\]
The first remaining $\la$ is $\la^{\ve 1}$, so it does not bind the first variable (thus $g$ remains first in the substitution list), but the second one. Consequently, we add an $f$ after the $g$:
\[
\left(\LaO {\ve{1,1}} {\ovar \oapp 1 \ovar \oapp 2 {\pa{\ovar \oapp 1 \ovar}}}\right)  \left[ g, f \right] \lspace n
\]
Now the situation becomes more interesting. The only remaining $\la$ is now decorated with the vector $\ve{1,1}$, so one $n$ has to be inserted after the first entry ($g$) and another one must be placed after the subsequent entry ($f$):
\[
\left(\ovar \oapp 1 \ovar \oapp 2 {\pa{\ovar \oapp 1 \ovar}}\right)  \left[ g, n, f, n \right]
\]
All $\la$ have now been eliminated. The applications' indices tell us how the substitution list should be divided between the terms:
\[
\left(\ovar \oapp 1 \ovar\right)\left[ g, n \right] \lspace \left(\ovar \oapp 1 \ovar\right) \left[ f, n \right]
% \left(\ovar \oapp 1 \ovar\right)\left[ g, n \right] \lspace \oapp 0 \lspace \left(\ovar \oapp 1 \ovar\right) \left[ f, n \right]
\]
We do the same step once more:
\[
\ovar[g]  \lspace \ovar[n]  \lspace \left(\ovar[f]  \lspace \ovar[n]\right)
\]
The only thing left to be done is to apply the substitutions in the obvious way:
\[
g \sspace n  \sspace (f \sspace n)
\]
Here, evaluation naturally leads to a term in beta normal form. This is not always the case: as an example, if we had tried to evaluate the above term without the $n$ (i.e. $S \oapp 0 f \oapp 0 g$), we would have got stuck at $\left(\LaO {\ve{1,1}} {\ovar \oapp 1 \ovar \oapp 2 {\pa{\ovar \oapp 1 \ovar}}}\right)  \left[ g, f \right]$. However, this would have been satisfactory as it would have shown that the term's normal form is an abstraction. In other words, our evaluation results in weak head normal forms.

Consequently, we define \define{values} in the following way:
\[
\begin{array}{lllrll@{\qquad}}
\mathsf{values}       & \ni & v,w & ::= & x \sspace \vec v & \mbox{large application} \\
% \noalign{\medskip}
                                           &&& \mid & \Val {\vec k} t {\vec v} & \mbox{closure} \\
\end{array}
\]
% \[
% \mathsf{values} \ni v, w ::= x \, \vec v \ \mid \ \ClosO{\vec k}{t}{}{\vec v}
% \]
The \emph{large application}\footnote{
The argument vector $\vec v$ of a large application is sometimes
called a \emph{spine} \cite{cervesatoPfenning:spineCalculus}.  Large
applications $x\,\vec v$ also appear in the formulation of B\"ohm
trees \cite{barendregt:lambdacalculus}.}
 consists of a variable $x$ which is applied to a vector $\ve{v_1, v_2, \ldots, v_m}$ of values. It is to be read as a left-associative application, i.e. as $\pa{\pa{x \, v_1} v_2 \ldots } v_m$.
Note that it is not necessarily ``large''. Quite the contrary, it
often only consists of the head (and the vector of values is empty).

A \emph{closure} $\Val {\vec k} t {\vec v}$ is the result of the evaluation process if the corresponding beta normal form of the term does not start with a free variable. The main part, $\LaO {\vec k} t$, is nothing other than a lambda abstraction in the syntax of ordered terms.
In addition, we need the substitution list $\vec v$ (which is simply a list of values) that satisfies $\length {\vec v} = \freevars{\LaO {\vec k} t}$. % as there might be unbound $\ovar$ in $\LaO {\vec k} t$.
The idea is that the $i^{th}$ unbound $\ovar$ is to be replaced by $v_i$.  These substitution lists have already been used in the example above.

At this point, we want to introduce a notation for inserting a single item multiple times into a list. More precisely, if $\vec v = [v_1, v_2, \ldots, v_m]$ is a list,
%(e.g. of values), % TODO neil suggests to delete this - I am not yet sure about it
$\vec k = [k_1, k_2, \ldots, k_n]$ is a vector (i.e. also a list) of  non-negative integers satisfying $\sum_{i=1}^n k_i \leq m$ and $w$ is a single item, we write $\multiinsert {\vec v} {\vec k} {w}$  for the list that is constructed by inserting $w$ at each of the positions $k_1, k_1 + k_2, \ldots, \sum_{i=1}^n k_i$ into $\vec v$, i.e. for the list $[v_1, v_2, \ldots, v_{k_1}, w, v_{k_1 + 1}, \ldots, v_{k_1 + k_2}, w, v_{k_1 + k_2 + 1}, \ldots, v_m]$ (of course, it is possible that $\multiinsert {\vec v} {\vec k} {w}$ starts or ends with $w$).

We are now able to define the evaluation function $\ev {\cdot} {\cdot}$ which takes an ordered term $t$ as well as an ordered substitution list $\vec v$ and returns a value. The tuple must always satisfy the condition $\freevars t = \length {\vec v}$. In other words, the list carries neither too little nor redundant information.
At the start of the evaluation, the ordered substitution list is empty.
Additionally, we specify the application $\cdot \ap \cdot$ of two values, which also returns a value and does not need anything else.
Our evaluation procedure uses a call-by-value strategy:

% this is not how it should be done using latex...
\[
\begin{array}{lcl@{\qquad} r}
 \ev x {\emptyVec} & = & x & \lab 1 \\ \noalign{\medskip}
 \ev \ovar {\ve{v_1}} & = & v_1 & \lab 2\\  \noalign{\medskip}
 \ev {t \oapp k u} {\ve{v_1, \ldots, v_n}} & = & {\ev t {\ve{v_1, \ldots, v_k}}} \ap {\ev u {\ve{v_{k+1}, \ldots, v_n}}}  & \lab 3 \\  \noalign{\medskip}
 \ev {\LaO {\vec k} t}  {\vec v} & = & \Val {\vec k} t {\vec v} & \lab 4 \\
\\
 \pa{x \sspace \vec v} \ap w & = & x \sspace \ve{\vec v, w}  & \lab 5
\\ \noalign{\medskip}
 \Val{\vec k} t {\vec v}   \ap w & = & \ev t {\multiinsert{\vec v}{\vec k}{w}} & \lab 6
\end{array}
\]
%\beginalign}
% \ev x {\emptyVec} &=  x \\ %\noalign{\medskip}
% \ev \ovar {\ve{v_1}} &=  v_1 \\  %\noalign{\medskip}
% \ev {t \oapp k u} {\ve{v_1, \ldots, v_n}} &= {\ev t {\ve{v_1, \ldots, v_k}}} \ap {\ev u {\ve{v_{k+1}, \ldots, v_n}}} \\  %\noalign{\medskip}
% \ev {\LaO {\vec k} t}  {\vec v} &=  \Val {\vec k} t {\vec v} \\
%\nonumber \\
% \pa{x \sspace \vec v} \ap w &=  x \sspace \ve{\vec v, w} \\
%%\\ \noalign{\medskip}
% \Val{\vec k} t {\vec v}   \ap w &=  \ev t {\multiinsert{\vec v}{\vec k}{w}}
%\end{align}
First, if we want to evaluate a free variable \lab 1, the substitution list must be empty because of the invariant mentioned above.
Second, in the case of a $\ovar$ \lab 2, the ordered list must have exactly one entry. This entry is the result of the evaluation.
If we evaluate an application \lab 3, we evaluate the left and the right term. The application's index enables us to split the substitution list at the right position. Then, we have to apply the first result to the second.
Evaluating an abstraction \lab 4 is easy. We just need to keep the substitutions to build a closure.

If we want to apply a large application to a value $w$ \lab 5 , we just append $w$ to the vector of values (we write $\ve{\vec v, w}$ for $\ve{v_1, \ldots, v_n, w}$).
The case of a closure $\Val {\vec k} t {\vec v}$ \lab 6 is less simple, but it is still quite clear what to do: $\vec k$ determines at which positions $w$ should be inserted in the ordered substitution list, so we just construct the list $\multiinsert {\vec v} {\vec k} w$. Then, $t$ is evaluated.

Concerning substitution lists, we talk about ``lists of values'' for
simplicity. More specifically, we want them to be lists of pointers to
avoid the duplication of ``real'' values during constructing lists
like $\multiinsert {\vec v} {\vec k} {w}$.
Instead of simple linked lists,
we have also implemented these lists as dynamic functional arrays
represented as binary trees.  This reduces the asymptotic costs of
list splitting---$[v_1,\dots,v_n]$ to
$([v_1,\dots,v_k],[v_{k+1},\dots,v_n])$---
and multi-insertion $\multiinsert {\vec v} {\vec k} w$
from linear to logarithmic time (in terms of the length of the list
$\vec v$).  For an
experimental comparison of the two implementations
see Section~\ref{sec:experiments}.

\section{Parsing and Printing}
\label{sec:parsing}

In this section, we define how terms in normal syntax are translated into our ordered syntax (Parsing) and vice versa (Printing).
To specify this, we need some notation.
First of all, we write $\x$ for the set of variable names we want to use and $\te$ for the set of lambda terms in basic standard syntax (i.e. $\x \subset \te$, furthermore, $x \in \x$ together with $t, u \in \te$ implies $\ApA t u \in \te$ and $\LaA x t \in \te$). Additionally, $\ot$ is the set of terms in our ordered syntax defined above, $\otc$ the subset of closed ordered terms ($\freevars t = 0$) and $\va$ the set of values (defined in the previous section).
Moreover, we write $\xl$ for the set of lists of variable names and $\vl$ for the set of lists of values.
For each set $\Gamma$ of variable names ($\Gamma \subseteq \x$), we denote the set of lists of elements of $\Gamma$ by $\gl$ and the ordered terms that do not contain any variable of $\Gamma$ (as a free variable) by $\otw$.
% We will often allow $\Gamma$ to be a finite multiset, i.e. a finite set that can contain each element more than once. The according notion of union $\uplus$ satisfies $\Gamma \uplus \{z\} \not= \Gamma$.

By writing $\ot \otimes \xl$ (resp. $\ot \otimes \vl$, $\otw \otimes \gl$, $\ldots$), we mean the subset $\{(t, \vec x) \ | \ \freevars t = \length {\vec x} \}$ of $\ot \times \xl$ (and analogous for the other cases).

% \begin{defin}
For a finite %multiset
set $\Gamma$ of variable names, we define the \define{correspondence relation}
% $\parse \cdot \Gamma \cdot \ \subset \  \te \times (\ot \times \xl)$.
${\parseb \cdot \Gamma \cdot \cdot ~ \subset ~ \te \times \ot \times
  \xl}$
(pronounce: ``corresponds in context $\Gamma$ to'').
The intuition is that ${\parseb M \Gamma u {\vec x}}$ means:
% The intuition is that $\parse M \Gamma (u, \vec x)$ (or simply $\parse
% M \Gamma {u \, \vec x}$) means:
$M$ is a term that corresponds to the ordered term $u$, where unbound
$\ovar$ are replaced by the (\emph{not} necessarily pairwise distinct)
variables in the list $\vec x$.
% This list $\vec x$ of variable names may contain duplicates!
The set $\Gamma$ can be seen as a filter that tells us which free variables do not occur in $u$ but in $\vec x$ instead.
\[
\begin{array}{c@{\qquad\qquad} c}
\infnamed{\quad (A)}{x \in \Gamma}{\parseb x \Gamma \ovar x}
&
\infnamed{\quad (B)}{ {\parseb M \Gamma t {\vec x}} \qquad
        {\parseb N \Gamma u {\vec y}} \qquad {\length{\vec x} = m} }
      {\parseb {M\,N} {\Gamma} {(\ApO t m u)}  {\vec x, \vec y} }
\\ \\
\infnamed{\quad (C)}{x \not\in \Gamma}{\parseb x \Gamma x {}}
&
\infnamed{\quad (D)}{
         {\parseb M { (\Gamma \cup \{z\}) }  t {\multiinsert{\vec x}{\vec k}{z}   }}
          \qquad
          z \not\in \vec x  }
          {\parseb {\LaA z M} {\Gamma} {(\LaO {\vec{k}} t)} {\vec x}}
\end{array}
\]
%\end{defin}
It is important to note that $\parseb M \Gamma u {\vec x}$ implies that each variable occurring in $\vec x$ is contained in $\Gamma$ and each free variable occurring in $u$ is not contained in $\Gamma$. This can be shown by induction on $M$ (simultaneously for all sets $\Gamma$).

By the same argument, one can see that (for each $M$ and $\Gamma$) there exists a unique tuple $(u, \vec x)$ satisfying $\parseb M \Gamma u  {\vec x}$, so we can consider $\parseFun \Gamma$ a function $\te \rightarrow \otw \times \gl$. Furthermore, we note that $\parseb M \Gamma u {\vec x}$ always implies $\freevars u = \length {\vec x}$.

This also works the other way round. For each $\Gamma$ and each tuple
$(u, \vec x) \in \otw \otimes \gl$, there is (by induction on $u$) a
term $M \in \te$ satisfying $\parseb M \Gamma u {\vec x}$. Moreover,
this term $M$ is unique up to $\alpha$ equivalence. So, $\parseFun
\Gamma$ is actually a bijection between $\te / \alpha$ (the set of
$\alpha$ equivalence classes of terms) and $\otw \otimes \gl$. The
inference rules above show how to apply this bijection or its inverse
to a term or a tuple (in the last rule, any variable satisfying the
condition can be chosen for $z$), so we have
% (computable) functions $\parseFun \Gamma : \te / \alpha \rightarrow \otw \otimes \gl$ and $\parseFun \Gamma ^{-1}$.
a computable bijection $\te / \alpha \leftrightarrow \otw \otimes \gl$
determined by the rules for $\parseFun \Gamma$.

Choosing $\Gamma = \emptySet$, we get a bijection $\te/\alpha
\leftrightarrow \ot \otimes {\vec \emptySet}$.  As $\vec \emptySet$ is
only inhabited by the empty vector $\emptyVec$, we naturally get the
\define{parse} function $\parseFunE$ which maps $\te / \alpha$
bijectively on $\otc$.

%Choosing $\Gamma = \emptySet$, we get the special cases $\parseFun \emptySet$ and $\parseFun \emptySet ^{-1}$, which are bijections between $\te$ and $\otc$. This also explaines how one representation can be transformed into the other.

The above construction also gives us a function $\otc \rightarrow
\te$, but this is not enough. We want to transform closures
(elements of $\ot \otimes \vl$) and values (elements of $\va$)
into basic terms $\te$.
Therefore, we
% \begin{defin}
% We
define the two \define{print} functions $\prE : \ot \otimes \vl \rightarrow \te / \alpha$ and $\prValE : \va \rightarrow \te / \alpha$ simultaneously by recursion on the structure:
\[
\begin{array}{lclc}
\medskip
\pr{x}{\emptyVec} & = & x & \lab I \\
\medskip
\pr{\ovar}{[v]} & = & \prVal v & \lab {II} \\
\pr{t \oapp m u}{\vec v} & = & \ApA {\pr {t}{\vstart}  }  {\quad \pr
  {u}{\vrest}}  & \lab {III} \\
\medskip
&& \mbox{(split $\vec v$ at position $m$ to get $\vstart$ and $\vrest$)} \\
% \pr{\LaO {k_1, \ldots, k_m} t}{[v_1, v_2, \ldots, v_n]} & = & \LaA z {\pr {t} {v_1, \ldots, v_{k_1}, z, v_{k_1+1}, \ldots, }}  & \lab {IV} \\
\pr{\LaO {\vec k} t}{\vec v} & = & \LaA z {\pr {t} {  \multiinsert{\vec v}{\vec k}{z}   }}   & \lab {IV}\\
\medskip
&& \mbox{where $z$ is any variable that does not occur freely in $t$ or $\vec v$ } \\
\prVal{x \sspace v_1 \sspace v_2 \sspace \ldots \sspace v_n} & = & x \ \prVal{v_1} \ \prVal{v_2} \ \ldots \ \prVal{v_n} & \lab V\\
\medskip && \mbox{(a large application simply becomes an application of terms)} \\
\prVal{\Val {\vec k} t {\vec v}} & = & \pr{\LaO {\vec k} t}{\vec v}  & \lab {VI}\\
\end{array}
\]
%\end{defin}
First, note that the printing functions are well-defined (i.e., they always terminate). This is because during evaluation of $\pr{u}{\vec v}$, we may safely assume that $\pr{t}{\vec w}$ is well-defined as long as $t$ is a strict subterm of $u$ and each value $w'$ in $\vec w$ is either only a variable (so termination of $\prVal {w'}$ is clear) or also in $\vec v$. Similar, during evaluation of $\prVal {x \sspace v_1 \sspace \ldots v_n}$, we may assume that $\prVal {v_i}$ is defined for each $i$.

For all $t \in \otc, M \in \te$, we have $\pr t \emptyVec = M$ if and only if $\parseb M \emptySet t { }$ (which just means $\parseFunE t = M$) as both judgements are defined identically in the case of closed ordered terms. This essentially (with implicit use of the bijection $\otc \leftrightarrow \ot \otimes \vec \emptySet$) means $\prE \circ \parseFunE  = id_{\te}$, i.e. the composition of parsing and printing is the identity.

\section{Correctness and Termination properties}
\label{sec:sound}

We still have not shown that our evaluation algorithm given in Section~\ref{sec:values} does not change the meaning of terms. The combination of parsing, evaluating and printing should never result in a term that is not beta equivalent to the original term. We also want to show a limited termination property.
To keep our argument simple, we just sketch the proofs and hope that the ideas become clear.

First, we attend to the correctness question.
We need to convince ourselves that \emph{rewriting} according to the rules of the functions $\ev \cdot \cdot$ and $\ap$ does not cause an error.
By \emph{rewriting}, we mean one step of the \emph{normal} or \emph{leftmost outermost} evaluation. We have demonstrated this in the example at the beginning of Section~\ref{sec:values}.
%If we have, e.g., the tupel $(t, \vec v)$, printing it should result in a term that is $\beta$ equivalent to the term we get if we rewrite it once before printing.
Printing should result in a term that is $\beta$ equivalent to the term we get if we rewrite before printing.
This basically means that, for each evaluation rule on the left hand
side of the following table, we have to check that the equality on the
right hand side holds:

\begin{center}
\begin{tabular}{|   rcl  |  rcl    | c  }
\cline{1-6}
&&&&&& \\
$\ev x {\emptyVec} $&$=$&$ x    $ & $ \pr{x}{\emptyVec} $&$\, =_\beta \, $&$\prVal{x}$ & \qquad \lab 1 \\
&&&&&& \\
$\ev \ovar {\ve{v}} $&$=$&$ v   $ & $ \pr{\ovar}{v} $&$\, =_\beta \,$&$ \prVal v$  & \qquad \lab 2 \\
&&&&&& \\
$\ev {t \oapp k u} {[\vec v , \vec w]} $&$=$&$ {\ev t {\vec v}} \ap {\ev u {\vec w}}   $ & $ \pr {t \oapp k u} {[\vec v , \vec w]} $&$\, =_\beta \,$&$  \ApA {\pr{t}{\vec v}} {\pr{u}{\vec w}} $  & \qquad \lab 3 \\
&&&&&& \\
$\ev {\LaO {\vec k} t}  {\vec v} $&$=$&$ \Val {\vec k} t {\vec v} $ & $  \pr{\LaO {\vec k} t}{\vec v}$&$ \, =_\beta \,$&$  \prVal {\Val {\vec k} t {\vec v}} $  & \qquad \lab 4 \\
&&&&&& \\
$\pa{x \sspace \vec v} \ap w $&$=$&$ x \sspace \ve{\vec v, w}   $ & $\ApA {\prVal {x \sspace \vec v} } {\prVal w}   $&$\, =_\beta \, $&$ \prVal{x \sspace \ve{\vec v, w}}$  & \qquad \lab 5\\
&&&&&& \\
$\Val{\vec k} t {\vec v}   \ap w $&$=$&$ \ev t {\multiinsert{\vec v}{\vec k}{w}}   $ & $ \ApA{\prVal{\Val{\vec k} t {\vec v}}}{\prVal w}  $&$ \, =_\beta \, $&$  \pr{t}{\multiinsert{\vec v}{\vec k}{w}}  $  & \qquad \lab 6\\
&&&&&& \\
\cline{1-6}
\end{tabular}
\end{center}

Note that ``rewriting using the evaluation rules'' results in expressions which are, more or less, a mixture of elements of $\ot \otimes \vl$ and $\va$. To be precise, such an expression is either in $\ot \otimes \vl$ or in $\va$ or a tuple (to read as simple application) of two expressions. The $1^{st}$, $2^{nd}$ and $4^{th}$ rule turn an element of $\ot \otimes \va$ into a value, the $3^{rd}$ turns it in a tuple of two such elements, and the last two rules turn a tuple of two values into one value or element.
Although we do not define it formally, it should be clear how those expressions can be printed by using the printing functions for ordered terms and values (if an expression is a tuple, just print the function part and the argument part separately). In fact, while the function $\ev \cdot \cdot$ is formulated as a big step evaluation, the rewriting process can be understood as the corresponding small step (or one step) evaluation.

In the first five rows of the table, we only have to look at the definitions of the printing functions to see that the printed terms are not only $\beta$ equivalent but also equal. The very last rule $\lab 6$ requires closer examination:
By rule $\lab {IV}$, the term $\prVal{\Val{\vec k} t {\vec v}}$ is equal to $\LaA z {\pr {t} {  \multiinsert{\vec v}{\vec k}{z}   }} $ for a (sufficiently) fresh variable $z$. Now, the definitions of the printing functions are ``context free'' in a way that guarantees  that $z$ occurs exactly $\length{\vec k}$ times (free) in $\pr {t} {  \multiinsert{\vec v}{\vec k}{z}   }$. Furthermore, replacing those occurrences by $\prVal{w}$ results in the term $\pr{t}{\multiinsert{\vec v}{\vec k}{w}}$. This means that, starting with
$ \ApA{\prVal{\Val{\vec k} t {\vec v}}}{\prVal w}  $,
we have to use exactly one $\beta$ reduction step to get the term $  \pr{t}{\multiinsert{\vec v}{\vec k}{w}}  $.

As we have already seen that the composition of parsing a term and printing it afterwards does not change anything (up to $\alpha$ equivalence), we can now conclude that parsing, evaluating (a finite number of rewriting steps) and printing is equivalent to a number of $\beta$ reduction steps.

Now we discuss termination. Obviously, our evaluation function $\ev \cdot \cdot$ does not always terminate as some terms do not have a weak head normal form. However, $\ev \cdot \cdot$ terminates whenever it is applied to ($\parseFunE t$) if the usual $\beta$ reduction is strongly normalizing on $t$.
The main consequence of this is that evaluation terminates for all well-typed terms. % TODO "all" !
To prove this statement, assume that there is such a term $s_0 \in \te$ so that the evaluation of $t_0 := \parseFunE s_0$ does not terminate.
%
Then, we get an infinite sequence $t_0, t_1, t_2, \ldots$ where $t_{i+1}$ is the result of rewriting (a subexpression of) $t_i$ using one of the evaluation rules. If we print $t_0, t_1, t_2, \ldots$, we get a sequence $s_0, s_1, s_2, \ldots$ of terms in $\te$, where $s_{i+1}$ is either ($\alpha$) equal to $s_i$ or arises from $s_i$ in exactly one $\beta$ reduction step. If $\beta$ reduction is strongly normalizing on $s_0$, the sequence has to become constant at some point, i.e. $s_N = s_{N+1} = s_{N+2} = \ldots$ for some $N$. This implies that, after the first $N$ rewriting steps, rule \lab 6 is not used anymore. Define the \emph{weight} $w(t)$ of an expression $t$ to be $1$, if the expression is just an element of $\ot \otimes \va$, to be $2$, if it is a value of the \emph{closure} type, to be %TODO
$1 + 2^n + w(v_1) + w(v_2) + \ldots + w(v_n)$, if it is a value of the form $x \, v_1 v_2 \ldots v_n$ (i.e. a \emph{large application}) and, if it is a tuple of two expressions, as the sum of both weights. Then, each of the rewritings that are induced by the first five lines in the table increase the weight of the expression, so we get $w(t_N) < w(t_{N+1}) < w(t_{N+2}) < \ldots$;  however, as the total number of values (and tuples in $\ot \otimes \va$) is bound by the length of the term we get after printing $t_N$ (or any $t_{N+i}$), the sequence is bounded, resulting in the required contradiction.

% -----------------------------------
%
% THIS IS AN OUTDATED VERSION OF THE SECTION. IT IS MORE "FORMALLY CORRECT" BUT CONTAINS TOO MUCH OVERHEAD AND IS THEREFORE DIFFICULT TO READ.
%
%We still have to prove two important facts about the evaluation algorithm given in section \ref{sec:values} - first, its termination properties and second, its correctness, i.e. it corresponds to beta reduction.
%
%Concerning correctness, we would obviously want to show that taking a term $t$, translating it to an ordered term, evaluation and printing it gives us a term which is beta equal to $t$, which is represented by the diagram
%\[
%\begin{xy}
%\xymatrix{
% t \in \te \ar@{-}[rrr]^{=_\beta} \ar@/_/[dd]_{\parseFunE} & & & t' \in \te \\
% \\
%% u \in \otc \ar@/_/[uu]_{\pr{\cdot}{\emptyVec}} \ar[rrr]^{\ev{\cdot}{\emptyVec}} & & & \ev{u}{\emptyVec} \in \va \ar[uu]_{\prValE}
% u \in \otc \ar@/_/[uu]_{\prE                  } \ar[rrr]^{\ev{\cdot}{\emptyVec}} & & & \ev{u}{\emptyVec} \in \va \ar[uu]_{\prValE}
%}
%\end{xy}
%\]
%However, the diagram does not make much sense as we do not know yet that $\ev \cdot \emptyVec$ is a well-defined function.
%What we need to do is to define a one-step-evaluation, thereby making it possible to check if each single evaluation step preserves beta equality. Unfortunately, this becomes even more complicated as we get binary trees during evaluation:
%\[
%\begin{array}{lllrll@{\qquad}}
%\evTr       & \ni & \mathsf{tree}_1, \mathsf{tree}_2 & ::= & (t, \vec v) & \mbox{a tupel in $\ot \otimes \vl$} \\
%                                          &&& \mid & v & \mbox{just a value} \\
%                                          &&& \mid & \tree {\mathsf{tree}_1} {\mathsf{tree}_2}  & \mbox{node with two subtrees}
%\end{array}
%\]
%Our one-step reduction $\osr$ is a relation on $\evTr \times \evTr$:
%
%\begin{gather*}
%\infnamed{^{\ev \cdot \cdot \mbox{\small -1} }}{}{(x , \emptyVec) \osr x}
%\qquad
%\qquad
%\infnamed{^{\ev \cdot \cdot \mbox{\small -2}}}{}{(\ovar , \ve v) \osr v}
%\qquad
%\qquad
%\infnamed{^{\ev \cdot \cdot \mbox{\small -3}}}{}{(t \oapp k u , \vec v \vec w) \osr \tree {(t, \vec v)}{(u, \vec w)}}
%\\ \\
%\infnamed{^{\ev \cdot \cdot \mbox{\small -4}}}{}{(\LaO {\vec k} t , \vec v) \osr \Val {\vec k} t {\vec v}}
%\qquad
%\qquad
%\infnamed{^{\mbox{\small (struct-1)}}}{a \osr a'}{\tree a b \osr \tree {a'} b}
%\qquad
%\qquad
%\infnamed{^{\mbox{\small (struct-2)}}}{b \osr b'}{\tree a b \osr \tree a {b'}}
%\\ \\
%\infnamed{^{@\mbox{\small -1}}}{}{\tree {x \sspace v_1 \ldots v_n} w \osr  x \sspace v_1 \ldots v_n \sspace w}
%\qquad
%\qquad
%\infnamed{^{@\mbox{\small -2}}}{}{\tree {\Val {\vec k} t {\vec v}}{w}   \osr   (t, \multiinsert {\vec v} {\vec k} w)}
%\end{gather*}
%Now it is easy to define a printing function $\prTreeE$ to transform a tree into a term:
%\[
%\begin{array}{lclll}
%\prTree{(t, \vec v)} & = & \pr {t} {\vec v} &&\\
%\prTree{v} & = & \prVal v &&\\
%\prTree{\tree a b} & = & \ApA {\prTree a}{\prTree b}&& \mbox{(simply the application in $\te$)}
%\end{array}
%\]
%\begin{prop}
% For $a, b \in \evTr$ satisfying $a \osr b$, we have $\prTree a =_\beta \prTree b$. More precisely, if one of the rules ${\ev \cdot \cdot \mbox{-1} }, {\ev \cdot \cdot \mbox{-2} }, {\ev \cdot \cdot \mbox{-3} }, {\ev \cdot \cdot \mbox{-4} }$ or ${@\mbox{-1}}$ was used, even $\prTree a = \prTree b$ (up to $\alpha$-equivalence) holds, if ${@\mbox{-1}}$ was used,  $\prTree a$ can be reduced to $\prTree b$ in exactly one $\beta$-reduction step.
%\end{prop}
%\begin{proof}
% For all of the $\ev \cdot \cdot$-rules and $@\mbox{-1}$, the equality $\prTree a = \prTree b$ is easy to check.
%% In the case of the structure rules, we may assume that the statement holds for the premises, so it is also clear for the conclusions. The first of the $@$-rules is also simple.
% We have to have a closer look at the second $@$-rule:\\
% By definition, we have
% \[
%\begin{array}{lclr}
% \prTree {\tree {\Val {\vec k} t {\vec v}}{w}} & = & \ApA {  \prVal {\Val {\vec k} t {\vec v}}   } \ {\prVal w}  & \\
%& = &\ApA {  \pr{\LaO {\vec k} t}{\vec v}   }  \ {\prVal w} \\
%& = & \ApA {\left( \LaA z {\pr {t} {\multiinsert {\vec v}{\vec k}{z}}} \right)} \ {\prVal w} &  \qquad \mbox{$z$ not free in $t$, $\vec v$}\\
%\mbox{and} &&& \\
% \prTree {    (t , \multiinsert{\vec v}{\vec k}{w} )    } & = & \pr{t}{\multiinsert{\vec v}{\vec k}{w}} &
%\end{array}
%\]
%Now, note that (again by induction on $t$) the term $\pr {t} {\multiinsert {\vec v}{\vec k}{z}}$ contains  $z$ exactly $\length {\vec k}$ times as a free variable and, by replacing each of those free occurrences  by $\prVal w$, we get the term $\pr{t}{\multiinsert{\vec v}{\vec k}{w}}$.
%Consequently, the term $\prTree {\tree {\Val {\vec k} t {\vec v}}{w}}$ can be reduced to $\prTree {    (t , \multiinsert{\vec v}{\vec k}{w} )    }$ in exactly one beta reduction step. \\
%Note that these statements are also true if one or more of the structure rules are used in addition.
%
%
%\end{proof}
%
%\begin{prop}
%For any well-typed term $t \in \te$, the One-Step-Reduction terminates for $(\parseFunE t)$.
%\end{prop}
%\begin{proof}
%Suppose there is an infinite sequence $a_0 \rightarrow a_1 \rightarrow a_2 \rightarrow \ldots$ with $a_0 = \parseFunE t$. According to the proposition above, we can conclude that for each $i$, either $\prTree {a_i} = \prTree {a_{i+1}}$ or $\prTree {a_i} \rightarrow_\beta \prTree {a_{i+1}}$ (in exactly one step). Assume the latter possibility occurs infinitely often. This contradicts the fact that $\beta$-reduction is strongly normalizing on well-types terms. Assume it occurs only finitely often. Then there is an $N$ such that $\prTree {a_N} = \prTree {a_{N+1}} = \prTree {a_{N+2}} = \ldots$. For $a \in \evTr$, we define the weight $w(a)$ as the number of leafs of $a$ containing an element of $\ot \otimes \vl$ plus twice the number of leafs that contain a value. % TODO: THIS HAS TO BE MODIFIED. (see newer version)
%Each of the one-step-rules except the last one (and the structure rules) obviously increases the weight of a tree, thus we get $w(a_N) < w(a_{N+1}) < w(a_{N+2}) < \ldots$. However, the printing function is defined in a way that forces the weight of any tree $a$ to be less or equal than twice the length of $a$ and consequently, the sequence above is bounded by $2 \cdot \length{ \prTree{a_N}}$, thus giving us a contradiction again.
%\end{proof}
%% TODO rename PARSINGfunction and make clear that sometimes parse: ... -> trees.
%
%Note that if the reduction terminates, the result is a tree that consists of exactly one leaf containing a value (as there is a reduction step for any other case). We identify this tree with the value.
%
%\begin{cor}
%For all well-typed terms, the evaluation function $\ev \cdot \cdot$ is well-defined (i.e. terminates) and correct (i.e. parsing a term, evaluating and printing it results in a $\beta$-equivalent term).
%\end{cor}
%\begin{proof}
%It is easy to check that the evaluation $\ev \cdot \cdot$ can be simulated by using a sequence of $\rightarrow$-steps. First, this shows that it terminates as we would get an infinite sequence of steps otherwise. Second, as each small step conserves $\beta$-equality, the whole sequence does the same. As we know that for $t \in \te$ the equality $\prTree {\parseFunE t , \emptyVec} = t$ holds, this implies $\prTree {\ev{\parseFunE t} {\emptyVec}} = t$.
%\end{proof}
%
% -----------------------------------

\section{Experiments and Results}
\label{sec:experiments}

The specified term representation and evaluation have been implemented
in Haskell. They have been used by a type checker to check large files
of dependently
typed terms of the Edinburgh Logical Framework which were kindly
provided by Andrew W. Appel (Princeton University). To make this
possible, an extended syntax has been used that includes $\Pi$-types% and technical elements like
, constants and definitions. It is straightforward to expand our
evaluation algorithm to the extended syntax---for details consult the
Bachelor's thesis of the second author \cite{kraus:bachelor}.
The substitution lists
have been implemented as simple Haskell lists, and also as balanced
binary trees (following Adams \cite{adams:jfp93})
for better asymptotic complexity.  Both variants were
evaluated for performance
[referred to as \emph{Ordered (trees)} and \emph{Ordered (lists)}].

For comparison, the completely analogous algorithm for terms in
extended basic syntax (i.e. $\te$) has been used [\emph{Simple
  Closures}]. Furthermore, we have tested a strategy that always
evaluates completely (i.e. produces $\beta$ normal forms) using
Hereditary Substitution [\emph{Beta Normal Values}]
\cite{watkins:concurrentlftr}.

Our main test file \textsf{w32\_sig\_semant.elf} with a size of
approximately 21 megabytes contains a proof described in
\cite{appel:toplas10}. We also tested smaller parts of this file, more
precisely, the first $6000$, $10,000$ and $12,000$ lines without the
rest (named \textsf{6000.elf} and so on). Later terms tend to be
larger, so the tests with fewer lines needed much less time.

All tests were executed on the same server
\mbox{\texttt{baerentatze.cip.ifi.lmu.de}}
% ("baerentatze",
% Ludwig-Maximilians-Universit\"at M\"unchen, Department of Computer
% Science)
working with a CPU of type \emph{AMD Phenom II X4 B95} (only
one core used, 3 GHz) and 8 GB system memory. The measurements
of space and time consumption are given in the following tables (rounded
average values). More specifically, \emph{time} refers to the total time, including parsing the input file and transforming (if necessary) the representation to our ordered representation or to one that uses De Bruijn terms. However, in our tests, the transformation process only took a negligible amount of time. The time value does, however, not include any printing of the terms or the values (with printing, the total time increased significantly). \emph{Space} refers to the peak space usage of the whole process.

\begin{center}
\begin{tabular}{| l || c | c |}
\multicolumn{3}{c}{\textsf{6000.elf} (file size: 3.8 MB)}\\
\hline
& time (sec) & space (MB) \\
\hline
\hline
Ordered (trees) & 18.9 & 1111 \\
\hline
Ordered (lists) & 18.6 & 1114\\
\hline
Simple Closures & 18.5 & 1152\\
\hline
Beta Normal Values & 27.6 & 2034\\
\hline
\end{tabular}

\vspace{5pt}

\begin{tabular}{| l || c | c |}
\multicolumn{3}{c}{\textsf{10000.elf} (file size: 12.9 MB)}\\
\hline
& time (sec) & space (MB) \\
\hline
\hline
Ordered (trees) & 61.0 & 3230\\
\hline
Ordered (lists) & 60.6 & 3237\\
\hline
Simple Closures & 60.0 & 3302\\
\hline
Beta Normal Values & 98.7 & 5878\\
\hline
\end{tabular}

\vspace{5pt}

\begin{tabular}{| l || c | c |}
\multicolumn{3}{c}{\textsf{12000.elf} (file size: 17.8 MB)}\\
\hline
& time (sec) & space (MB) \\
\hline
\hline
Ordered (trees) & 84.3 & 5096 \\
\hline
Ordered (lists) & 83.8 & 5103 \\
\hline
Simple Closures & 83.6 & 5226 \\
\hline
Beta Normal Values & 137.7 & 8513 \\
\hline
\end{tabular}
\end{center}

Unsurprisingly, beta normal values perform significantly worse than each of the other possibilities. However, the difference is smaller than it could have been expected. This might be due to the fact that during type checking, total evaluation of a term is often necessary anyway, thereby reducing the hereditary substitution's disadvantage.

Although none of the other strategies exhibited any shortcomings in the comparisons above, the following results for the complete file are remarkable. Here, implementing ordered substitutions as normal Haskell lists seems to be much more efficient than using tree structures:

\begin{center}
 \begin{tabular}{| l || c | c |}
\multicolumn{3}{c}{\textsf{w32\_sig\_semant.elf} (file size: 20.9 MB)}\\
\hline
& time (sec) & space (MB) \\
\hline
\hline
Ordered (trees) & 108.4 & 8877\\
\hline
Ordered (lists) & 94.8 & 4948\\ % average, 4 values.
\hline
Simple Closures & 94.3 & 5068 \\ % some more tests: 92.42 , 4994 (?!); 94.17, 5068; 94.97, 5068; 93.75, 4994
\hline
Beta Normal Values & 169.8 & 9044  \\
\hline
\end{tabular}
\end{center}

Our Simple Closures are still on the same level as Ordered Representation with lists, but the trees are far behind.
In comparison, the type checker of the Twelf project, \emph{Twelf
  r1697} (written in \emph{Standard ML} and compiled with
\emph{MLton}'s whole program optimizations \cite{fluetWeeks:icfp01})
does the job nearly five
times faster while using only 2720 megabytes of
memory. % TODO only due to an optimized compiler?

%Unfortunately, the idea of ordered substitution lists does not really pay off within the context of type checking in our tests.
%On the other hand, it can compete with the simple standard implementation. This offers hope that there might be some applications that benefit more from our suggested strategy.

\section{Related Work and Conclusions}
\label{sec:concl}

Our term representation is inspired by intuitionistic implicational
linear logic in natural deduction style which has explicit operations
for weakening and contraction
\cite{bentonBiermanDePaivaHyland:tlca93}.  With explicit weakening and
contraction, one easily maintains complete information about the free
variables of a term at each node \cite{kesnerLengrand:infcomp07}.  Our
term representation incorporates weakening and contraction into lambda
abstraction.  By using inspiration from ordered logic, we reduce the
stored information at application nodes to a minimum, namely an
integer; further, our variable nodes need to carry no information at all.

Another means to maintain information about free variables are
\emph{director strings} by Sinot \cite{sinot:jlc05}.  Application
nodes come with a map that tell for each variable whether it appears
in the left or the right subterm or in both.  Our term representation
can be seen as an optimized version of director strings, however, we
have no experimental comparison.
Sinot \etal~\cite{fernandezMackieSinot:aaecc05} present some
performance results of director strings; however, it is restricted
to evaluation of some specific big lambda-terms.  There is no study on
their relative performance in a realistic application---yet that is
our concern.

An alternative to explicit substitutions is Nadathur's suspension
calculus \cite{nadathur:jflp99}, which, in a refinement, also
maintains information about closedness of subterms.  In this
refinement, the suspension calculus maintains at least partial
information about the free variables of a subterm.
As the basis of an implementation of
$\lambda$-Prolog \cite{nadathur:flops01},
Nadathur has proven the efficiency of his term
representation not only for normalization and equality checking, but
also for higher-order unification.

% Building on Nadathur's alternative to explicit substitutions, the
% \emph{suspension calculus} \cite{nadathur:jflp99},
Building on the suspension calculus,
Liang, Nadathur, and Qi \cite{liangNadathurQi:jar05} have evaluated
different term representations in the context of $\lambda$Prolog, a
study that compares to our study of term representations for the
Edinburgh Logical Framework.  They have tested different combinations
of features, confirming our result that lazy substitution is
preferable to eager substitution {[Beta Normal Forms]},
even more so when several substitutions are gathered into one
traversal {[Closures, Ordered]}.  They also test a variant where
each term is equipped with an \emph{annotation}, a flag telling whether
this term is open, \ie, has free variables, or closed, \ie, has no
free variables.  In their experimental evaluation, these annotations
pay off greatly for the poorly behaving eager substitution, yet give
negligible advantage for explicit substitutions.  It is
hypothesized that in a combined substitution, each subterm will
mention at least one variable with a high probability, so the
traversal has to run over most of the whole term---this is certainly
different in the substitution for a single variable.

To summarize, we have presented a new term representation for the
lambda-calculus inspired by ordered linear logic, and experimentally
compared it with well-known representations (closures, normal forms)
in a prototypical implementation of a type checker for the Edinburgh
Logical Framework.  The experiments were carried out on large realistic
proof terms, constructed manually and mechanically.

The results were not significantly in favor of our new representation.
This might be due to the application domain, LF signature checking.
For one, LF-definitions are closed, which means that substitutions
never need to traverse a definition body when the definition is
expanded, and this optimization is shared by all the term
representations we compared.  Secondly, we only tested type checking,
not type reconstruction via unification.  During type checking, where
equality tests are expected to succeed, full normal forms are always
computed, and closures are very short-lived in memory.  More space
leaks are to be expected in applications such as logic programming or
type reconstruction, where unification is needed, which is not
expected to always succeed.   In constraint-based unification,
unsolvable constraints might be postponed, keeping closures alive for
longer.  In such situations, the benefits of our representation might
be more noticeable, more experiments are required.

In the future, we plan to investigate further term representations
such as term graphs, and perform more experiments.  The literature on
experimentally successful term representations is sparse, our work
contributes to close this gap.  Our long term goal is to find a
term representation which speeds up Agda's type reconstruction.

\paragraph*{Acknowledgements}
The idea for the here presented ordered term representation was
planted in a discussion with Christophe Raffalli who invited the first
author to Chambery in February 2010.  He mentioned to me that in his
library \texttt{bindlib} formation of closures restricts the
environment to the variables actually occurring free in the code.  I
also benefited from discussions with Brigitte Pientka and Stefan Monnier.

Thanks to Gabriel Scherer for comments on a draft version of this
paper and pointers to related work.
The final version of this paper was stylistically improved with the highly appreciated help of Neil Sculthorpe.

\bibliographystyle{eptcs}
\bibliography{nonauto-lfmtp11}

\end{document}


\section{Siamese Multi-Object Tracking and Embedding}
\label{sec:SiamMOTandFeatureEmb}

% ##############################################################################
\subsection{Motivation}

One of our experiments involved an end-to-end training of the \gls{siammot} together with a custom head aimed at embeddings based on \gls{roi}-pooled features for the object \gls{bbox}. The goal was to force the training process into extracting features that are not only satisfactory for detection and tracking but also do contain the necessary information to create embeddings for \gls{reid} purposes during the inference. We strived for simplicity by extending the processing pipeline without altering the existing infrastructure.

During research related to our Siamese tracking survey~\cite{ondrasovic2021siamese}, we noticed one work where the exemplar features were projected using \gls{gap} into an embedding space consisting of fewer dimensions~\cite{li2020figsiam}. The embedding vector was produced using the feature tensor representing the kernel for the cross-correlation operation. More concretely, let the extracted features be represented by an $8 \times 8 \times 256$ tensor. Then, the \gls{gap} along the channel dimension would produce a $1 \times 1 \times 256$ tensor, which could then be further flattened into a single $256$D vector. In the end, the obtained vector was $l_2$-normalized and thus projected onto a unit hypersphere. In the work of Li~\etal{}~\cite{li2020figsiam}, these embedding vectors were exploited for template updating and for combining multiple templates within a pool of size $n$ in an exponential fashion.

This observation led us to the following hypothesis. Given the fact the Siamese exemplar features do contain some, although probably not sufficient information for pure object \gls{reid}, would it be possible to map them further using a non-linear function to produce embedding vectors that could serve for \gls{reid}? Such features are just a learned template, therefore, some notion of similarity needs to be already built into it.

% ##############################################################################
\subsection{Feature Embedding Head Architecture}

% ------------------------------------------------------------------------------
\begin{figure}[!t]
    \centering
    \includegraphics[width=\linewidth]{figures/siamese_tracking/siammot_feature_emb_training.pdf}
    \caption[Embedding-enhanced \gls{siammot} architecture]{Our extension (shown in red) to the underlying \gls{siammot} architecture that incorporates vector embeddings to the end-to-end training. This diagram shows the pipeline that is used during the training, not inference.}
    \label{fig:SiamMOTWithEmbeddings}
\end{figure}
% ------------------------------------------------------------------------------

As for the vector embedding computation, we attached an \gls{femb} head (\tabletext{}~\ref{tab:FeatureEmbeddingHead}) after the \gls{roi}-pooling phase of the backbone features (\figtext{}~\ref{fig:SiamMOTWithEmbeddings}). This ensured fixed tensor shapes and allowed us to process the very same features that the object detector and Siamese tracker utilized, too. Simply put, for every proposal made for a particular frame, we looked at the delineated \gls{bbox} through the lens of \gls{roi}-pooling to extract its features. We simply reused the extracted exemplar features. Later on, we processed these features using our newly devised \gls{femb} head to produce \gls{femb}. The resulting embeddings were subjected to the triplet loss computation with all the necessary operations such as various types of hard negative mining.

\begin{table}[!t]
    \centering
    \begin{tabular}{lll}
        \toprule
        \textbf{layer}    & \textbf{tensor shape}        & \textbf{parameters no.} \\
        \midrule
        input             & $\sbrackets{B, 128, 15, 15}$ & $0$                     \\
        \midrule
        conv $3 \times 3$ & $\sbrackets{B, 128, 13, 13}$ & $147\ 456$              \\
        ReLU              & $\sbrackets{B, 128, 13, 13}$ & $0$                     \\
        \midrule
        conv $3 \times 3$ & $\sbrackets{B, 256, 11, 11}$ & $294\ 912$              \\
        ReLU              & $\sbrackets{B, 256, 11, 11}$ & $0$                     \\
        \midrule
        flatten           & $\sbrackets{B, 30976}$       & $0$                     \\
        linear            & $\sbrackets{B, 1024}$        & $31\ 720\ 448$          \\
        \midrule
        $l_2$-normalize   & $\sbrackets{B, 1024}$        & $0$                     \\
        \bottomrule
                          & \textbf{total}               & $32\ 162\ 816$          \\
        \cline{2-3}
    \end{tabular}
    \caption[\gls{femb} head]{Our custom \gls{femb} head that we used to process backbone-extracted features to produce embedding vectors. It is built from two convolutional layers separated by a \gls{relu} nonlinearity followed by a fully connected layer that produces a $1024$ dimensional feature embedding. The batch size dimension is given by $B$ in the tensor shape. Since each embedding vector is normalized to unit length, we avoided learning biases throughout the whole network.}
    \label{tab:FeatureEmbeddingHead}
\end{table}

% ##############################################################################
\subsection{Training Phase}

The training phase was altered by adding another loss function to the sum of already existing three losses from the original model. In particular, the general \gls{siammot} loss function defined in \eqtext{}~\ref{eq:SiamMOTGeneralLoss} was reformulated as
\begin{equation}
    \label{eq:SiamMOTFeatureEmbLoss}
    \lossf = l_{rpn} + l_{detect} + l_{motion} + l_{emb}.
\end{equation}
The $l_{emb}$ loss incorporated a triplet loss (\eqtext{}~\ref{eq:TripletLoss},~p.~\pageref{eq:TripletLoss}). We also experimented with a contrastive loss (\eqtext{}~\ref{eq:ContrastiveLoss},~p.~\pageref{eq:ContrastiveLoss}), but the effect was detrimental in every aspect as expected, so we will not discuss it any further.

As we remarked in \sectiontext{}~\ref{sec:LatentSpacesAndEmbeddings},~p.~\pageref{sec:LatentSpacesAndEmbeddings}, aimed at latent spaces and embeddings, it is crucial to adopt appropriate sample mining strategies when using the triplet loss. The rationale is that for the training to keep progressing, the model needs to encounter harder and harder triplets to generate sufficient learning signals. To this end, we went for the semi-hard triplet mining strategy (\eqtext{}~\ref{eq:BatchHardMining},~p.~\pageref{eq:BatchHardMining}). However, we struggled with collapsing embeddings~\cite{levi2021rethinking}. This phenomenon happens when the embedding training forces the model to project all the features onto a single point in the embedding space, thus incurring the loss equal to the used margin. We claim that the use of semi-hard negative mining produced triplets that were too difficult. Since we used all the \gls{rpn} proposals to generate triplets, one may imagine that there would always be proposals covering only some small part of the object, making it problematic for the network to learn the concept of ``similarity'' and ``difference'' if it only processes very hard images. Nevertheless, these situations are very common in margin-based losses~\cite{levi2021rethinking}. The computed loss is so high that it is more suitable for the model to map all the features onto a single point. To remedy this, we implemented a batch-all online mining strategy (\eqtext{}~\ref{eq:BatchAllLossFunction},~p.~\pageref{eq:BatchAllLossFunction}), which stabilized the training.

We recommend first utilizing the batch-all mining strategy during the training, and then proceeding to a batch-hard strategy after a certain point. However, this approach would be time-consuming to find the right hyperparameters. There are many open questions, such as how to mine the \gls{rpn} proposals in a better way or how to set the margin value. Loss functions aimed at object \gls{reid} are notoriously cumbersome to train. We implemented the entire mining algorithm followed by the loss computation in a \gls{gpu}-only fashion for fast execution and easy integration into the pipeline.

% ##############################################################################
\subsection{Inference Phase}

\subsubsection{Feature-based Non-Maximum Suppression}
\label{sssec:FeatureNonMaximumSuppression}

Salscheider~\cite{salscheider2020featurenms} proposed an extended \gls{nms} algorithm that incorporates a distance between feature embeddings dubbed as \featurenms{} (\algtext{}~\ref{alg:FeatureNMS}). Considering our idea introduced above, we had to encompass the vector embeddings into the solver reasoning. In the beginning, we came up with a solution that exactly copied the one the mentioned author proposed. That provided further justification for attempting to implement the algorithm and test it in practice. The advantage is that this approach is restricted to the inference phase, thus experimenting with it does not require model re-training.

\def\threshlower{\tau_{\text{lower}}}
\def\threshupper{\tau_{\text{upper}}}
\def\threshsim{\delta}

We assume the reader is acquainted with the \gls{nms} algorithm (\sectiontext{}~\ref{ssec:NonMaximumSuppression},~p.~\pageref{ssec:NonMaximumSuppression}). Here we repeat the same definitions for clarity. Let $\mset{B} = \cbrackets{\vect{b}_1, \vect{b}_2, \dots, \vect{b}_n}$ be a set of $n$ region proposals described by $n$ \glspl{bbox}. Scores for each detection are contained in a set $\mset{S} = \cbrackets{s_1, s_2, \dots, s_n}$, where $s_i$ denotes a detection score for the $i$-th box, $\vect{b_i}$. This time, we are also going to need the associated feature embedding vectors with each \gls{bbox}, represented by a set $\mset{E} = \cbrackets{\vect{e}_1, \vect{e}_2, \dots, \vect{e}_n}$. Let $\mset{B}_{fnms}$ be  the set of filtered proposal instances from the set $\mset{B}$ produced by the \featurenms{} algorithm. The distinction in parameters is the following. The original algorithm requires only one threshold for the maximum allowed portion of the overlap between regions. The \featurenms{} requires three parameters discussed below.
\begin{itemize}
    \item A minimum threshold $\threshlower$ denoting a boundary below which the two objects are deemed as different. This value should be low, for example, $0.2$, which means that if the \gls{iou} between the two objects is less than $0.2$, then the two instances should be treated as different objects.
    \item A maximum threshold $\threshupper$ denoting a boundary above which the two objects are considered identical. Unlike the $\threshlower$, this value should be high, \egtext{}, $0.8$, indicating that if the \gls{iou} of the two object instances surpasses this threshold, then it should be the same object. The \gls{bbox} with the lower confidence is discarded.
    \item A threshold $\threshsim$ is used as a decision boundary between the embedding vectors. This threshold should reflect a measure of similarity. If the adopted measure of similarity falls below $\threshsim$, then the two objects are different, otherwise, they are considered the same. This value of $\threshsim$ is used only if the two conditions above do not hold.
\end{itemize}

\begin{algorithm}[t]
    \caption[\featurenms{} algorithm]{\featurenms{} algorithm.}
    \label{alg:FeatureNMS}
    \begin{algorithmic}[1]
        \Function{Feature-NMS}{$\mset{B}$, $\mset{S}$, $\mset{E}$, $\threshlower$, $\threshupper$, $\threshsim$}

        \State $\mset{B}_{fnms}$ $\gets$ $\emptyset$
        \Comment{initialize the output (filtered) set of region proposals}

        \While {$\mset{B} \neq \emptyset$}
        \Comment{loop until all the proposals are processed}

        \State $m \gets \underset{i \in \cbrackets{1, 2, \dots, \msetsize{S}}}{\argmax{}} \mset{S}$
        \Comment{find an index of a proposal with the highest score}

        \State $\mset{B} \gets \mset{B} - \vect{b}_m$, $\mset{S} \gets \mset{S} - s_m$, $\mset{E} \gets \mset{E} - \vect{e}_m$
        \Comment{remove the proposal}

        \State $\mset{B}_{fnms} \gets \mset{B}_{fnms} \cup \vect{b}_m$
        \Comment{save the proposal with the highest score}

        \For{$i \gets 1$ to $\msetsize{B}$}
        \Comment{iterate through remaining proposals}

        \If{\Call{iou}{$\vect{b}_m$, $\vect{b}_i$} $\geq \threshlower$}
        \Comment{above the lower-bound threshold}

        \If{\Call{iou}{$\vect{b}_m$, $\vect{b}_i$} $\geq \threshupper$}
        \Comment{above the upper-bound threshold}

        \State $\mset{B} \gets \mset{B} - \vect{b}_i$, $\mset{S} \gets \mset{S} - s_i$, $\mset{E} \gets \mset{E} - \vect{e}_i$
        \Comment{remove the proposal}

        \Else

        \If{\Call{similarity}{$\vect{e}_m$, $\vect{e}_i$} $\geq \threshsim$}
        \Comment{similarity above threshold}
        \State $\mset{B} \gets \mset{B} - \vect{b}_i$, $\mset{S} \gets \mset{S} - s_i$, $\mset{E} \gets \mset{E} - \vect{e}_i$
        \Comment{remove the proposal}
        \EndIf

        \EndIf
        \EndIf
        \EndFor
        \EndWhile

        \State \Return $\mset{B}_{fnms}$
        \EndFunction
    \end{algorithmic}
\end{algorithm}

% ##############################################################################
\subsection{Experimental Evaluation and Discussion}

The proposed embedding-based enhancement was evaluated against the baseline model without the \gls{femb} head. For a fair comparison, we made sure that the hyperparameters were identical to the greatest possible extent. We only had to alter the batch size and the learning rate. Since the triplet loss requires the computation of a large number of triplets, especially the batch-all strategy, we had to decrease the batch size and employ \gls{ga} to avoid crashes due to not having enough \gls{gpu} \gls{vram} available.

We can tell that this experiment was also detrimental to the tracker's performance. We conjecture that it was solely caused by the introduction of the \gls{femb} head itself. Our ablation study also demonstrated that joint training with the \gls{femb} head harmed the tracker, even if the original solver was adopted during the evaluation. This raises the question of potential task/feature conflict between the heads. We venture to claim there is an inherent incompatibility between the detection, tracking, and \gls{reid} tasks, despite the existence of a recently published work Lu~\etal{}~\cite{lu2020retinatrack}, who developed their \retinatrack{} tracker. Their framework exploited the base visual object detector called a \retinanet{}~\cite{lin2018focal} and then added, in principle, a very similar head as we did to produce embeddings. However, there are obvious architectural differences between the two trackers in terms of how the inference phase is executed.

\begin{table}[!t]
    \centering
    \begin{tabular}{llllll}
        \toprule
        \textbf{\gls{femb} head} & \textbf{solver} & \textbf{MOTA} & \textbf{MOTP} & \textbf{precision} & \textbf{recall} \\
        \midrule
                                 & original        & $0.7416$      & $0.1478$      & $0.9454$           & $0.7896$        \\
        \checkmark               & \featurenms{}   & $0.6861$      & $0.1568$      & $0.9166$           & $0.7574$        \\
        \checkmark               & original        & $0.6882$      & $0.1568$      & $0.9184$           & $0.7574$        \\
        \bottomrule
    \end{tabular}
    \caption[The effect of \gls{femb} head inclusion]{Demonstration that introduction of the \gls{femb} head causes a feature conflict on the level of backbone before the \gls{roi}-pooling operation. The effect is clearly visible since the \featurenms{} algorithm is only part of the inference phase. Both the object detection and the tracking head are ``parallel'' to the \gls{femb} head, however, training with the \gls{femb} head and then avoiding it during the inference considerably reduces the tracker's accuracy. The last two rows are practically identical in terms of \gls{clear} evaluation.}
    \label{tab:OrigSolverVsFeatureNMS}
\end{table}

To see the aforementioned feature conflict, one of the key takeaways from this trial, \tabletext{}~\ref{tab:OrigSolverVsFeatureNMS} shows that training the model with the \gls{femb} head and then evaluating it without the \featurenms{} algorithm, \ietext{}, using the original solver algorithm (\figtext{}~\ref{fig:SiamMOTOnlineSolver}), resulted in a drastically reduced tracker's accuracy. The reason is that the \gls{roi}-pooled features were trained in a conflicting way, therefore no task was served satisfactorily. Having trained the model and then omitting the \gls{femb} head completely during the testing should not dramatically affect ``parallel heads''. However, once the entire end-to-end model tries to accommodate for the triplet, object detection, and tracking loss functions, the conflicting nature of these tasks manifests itself in a negative fashion. This table shows the best models given by the combination of \gls{mota} and \gls{motp} metrics. All models were trained with \gls{ga} using batch size of $32 = 16 \times 2$ with $256$ object proposals. The relatively big batch size was dictated by the unstable nature of the triplet loss training.

Although this extension is not appropriate for practical usage unless further modifications are devised, we still wanted to evaluate the impact on inference speed in terms of \gls{fps} for comparison with the next experiment. It can be seen that the reduction in the tracker's speed is noticable. Specifically, the original model inference runs on average at $26.49$ \gls{fps}, whilst the \fembmodel{} version achieves $18.57$ \gls{fps}, resulting in reduction speed of about $30$\% (more in \tabletext{}~\ref{tab:InferenceSpeedComparison},~p.~\pageref{tab:InferenceSpeedComparison}).

\subsubsection{``Unfairness'' between the detection and re-identification}

There is a recent publication~\cite{zhang2021fairmot} by Zhang~\etal{} that discusses the notion of ``fairness'' between detection and \gls{reid}. According to their empirical evidence, there are multiple levels of ``unfairness''. This term was used to describe a situation in which the importance of either \gls{reid} or the detection task is lessened. We would like to emphasize the recency of this work (the end of $2021$) since several of their remarks and conclusions coincide with ours. Had this paper been published before conducting our experiments, we would have taken a different path, or at least we would have striven to find another way to incorporate the embeddings into an end-to-end pipeline. The aforementioned work, among other things, introduces a joint tracker that effectively circumvents the obstacles related to conflicting tasks. This framework avoids the use of anchor proposals and is based on \gls{centernet}~\cite{zhou2019centernet} object detector.

They argue that the \gls{reid} task may become overlooked if the embedding vectors are produced solely from the \gls{roi}-pooled regions, as in our case. Consequently, the detection task becomes central in terms of influence upon the loss function since generating embedding vectors on incorrect \glspl{bbox} is meaningless. Thus, the bias to produce accurate object proposals is inescapable.

Next, a problem arises when one anchor corresponds to multiple identities, which is an obstacle we have already mentioned (\sectiontext{}~\ref{ssec:SiamMOTAndReIDDiscussion}). As a result, the extracted features are not optimal in terms of their accuracy as well as discriminative representativeness. To remedy this, features should be extracted at a single point, \ietext{}, at the estimated object centers (hence the use of \gls{centernet}), instead of region proposals.

An analogical problem is when multiple anchors correspond to one identity. A high overlap in terms of \gls{iou} directs the model to estimate the same identity for nearby anchors. But, even a small perturbation may result in falling below a specific threshold and the anchors become marked as belonging to different objects. This situation is very common as proposals are generated on the feature level (a coarse representation due to downsampling), and not on the pixel level.

Another inherent problem when merging object detection and \gls{reid} is feature conflict. We have highlighted numerous times the significance of multi-layer feature fusion. However, not all tasks benefit from this approach equally. In particular, object detection utilizes high-level features to estimate object classes and locations, whereas the \gls{reid} is more prone to utilizing low-level features owing to their discriminative power. For that reason, it is important to balance the loss optimization.

Last but not least, the feature dimension poses another source of imbalance. The authors remarked that learning lower-dimensional features for the \gls{reid} is better than higher-dimensional. In our research, we thought the opposite is true. High-dimensional embedding vectors notably harm the object detection accuracy due to the competition between the two tasks. The number of feature dimensions for detection is usually very low (compared to $1024$D or $2048$D embedding vectors). We adopted high-dimensional vector embeddings in conjunction with the usual low-dimensional detector features, thereby further exacerbating the rivalry. The rationale behind proposing to use, \egtext{}, $64$D embedding vectors, is that the \gls{mot} task executes just a few one-to-one matchings between two consecutive frames. However, there is a tacit assumption of having all objects sufficiently visible, as this work did not cover occlusion handling and their benchmark evaluations were not specifically targeted at full object occlusion, which is a path worth exploring. Nonetheless, the proposed framework achieved \gls{sota} performance among \gls{mot} approaches, surpassing the competitive frameworks by a large margin.


% This is an example for a `Introduction'.
% To generate the final document, run latex, build and quick build commands
% on the skeleton-thesis file not this one.

\chapter{Introduction}\label{chapters:Introduction}
%\vspace{-7mm}

While seemingly small objects, nuclei and their interactions can play an enormous role across a wide swath of energy and spatial scales.
From understanding the abundance of elements produced in astrophysical events to providing a general description of quantum many-body equilibration, the deceptively simple study of two colliding nuclei can be illuminating.
Even the more traditionally aligned investigations that are associated with nuclear reactions (such as superheavy element and neutron-rich nuclei formation) acts as a foundation for other areas of physics to piece together an understanding of the physical world.

Leaving aside the fact that nuclei are made of protons and neutrons, the dynamics of interacting quantum systems alone is of extreme interest to researchers from varied fields.
This is due to the striking fact that most quantum many-body systems (no matter the specific systems and particles that compose their structure) exhibit the same general features as atomic nuclei.
That is to say that studying the fusion, transfer, equilibration, vibrations, etc. of nuclear systems can provide vital insight into analogous studies in interactions between cold atoms or molecules -- systems which are several orders of magnitude larger than the few femtometers (fm) of interest to nuclear physicists.

\section{The nuclear many-body problem (in a nutshell)}

\begin{figure}[t]
	\includegraphics[width=\textwidth]{../Figures/intro_figs/ReidForce2.jpg}
	\caption{An example nucleon-nucleon potential from the Reid model~\citep{reid1968} of the nuclear force. Note the characteristic exponential decay in the attractive tail, as well as the strong repulsion that tends toward infinity at $R\approx0.7$. Figure from~\citep{bdushaw}.}
	\label{fig:reid}
\end{figure}

The specific subbranch of nuclear physics this thesis focuses on is the realm of low-energy nuclear physics.
At this level, relativistic effects are typically neglected and the nuclei are modeled as collections of protons and neutrons interacting with each other to form a bound system.
The force felt by the nucleons themselves is an artifact of the strong interaction, mediated primarily by the exchange of virtual pions with an effective range of about~$1$~fm.
Through further contributions by vector mesons (principally rho and omega), a complicated model of interacting nucleons can be built that depends on not only a nucleon's charge and distance, but also on its spin and angular momentum as well.
An example channel of the Reid potential~\citep{reid1968} is shown in Fig.~\ref{fig:reid}.
Extensive effort has been placed in modeling nuclei using such an interaction (see ~\citep{wiringa1995} for another popular choice of an N-N potential), though this \textit{ab initio} approach is not numerically feasible for large nuclei or for simulating dynamical interactions between nuclei.
It is for this reason that alternate approaches to modeling finite nuclei have been pursued throughout the field's history.

One such approach is that of the mean-field method which, simply stated, assumes that each nucleon moves freely through an average field made up of contributions from all the other nucleons.
This independent particle approximation is extremely powerful because it reduces a series of N-body self interactions to a simple sum over each individual particle.
%in the mean-field.
%A schematic image is shown in Fig.~\ref{fig:meanfield}.
While many takes on the mean-field idea have been pursued over time, the foundation of the current work was formulated in 1928 when Hartree developed the self-consistent field method~\citep{hartree1928}.
This was further extended for antisymmetric systems by Fock to form the more complete Hartree-Fock method that has been used since~\citep{fock1930}.
The term "self-consistent" used above refers to the fact that the effective Hamiltonian of the system depends on the single particle wave functions themselves, thus requiring the problem to be solved iteratively.
One common approach is to choose an initial guess of Gaussian wave packets for the first step and then a minimization procedure is followed to obtain the ground state solution for the nucleus.
Despite having a general purpose algorithm for self-consistently solving the static many-body quantum problem, the method wasn't widely adopted until computational abilities were readily available due to the large matrices involved in larger systems paired with the iterative nature of the problem.

While the Hartree-Fock method provides a general purpose, robust numerical algorithm, it is only useful for studying static properties of nuclei.
To obtain a time-dependent theory based on the mean-field approximation, one may proceed in several ways\footnote{For a more complete description of the various approaches to deriving TDHF (including a path integral formulation, many-body perturbation theory, the Balian-V\'en\'eroni variational principle, etc) see~\citep{simenel2018}.} though we will focus on the derivation by minimization of the Dirac action.
As it's vital to all chapters of this thesis and reveals much about the approximations and limitations of the theory, I will cover the derivation in detail.
First, consider the Dirac action from $t_0$ to $t_1$,
\begin{equation}
S\equiv S_{t_0,t_1}[\Psi] = \int_{t_0}^{t_1}dt \langle \Psi(t) | \left(i\hbar\frac{d}{dt}-\hat{H}\right)|\Psi(t)\rangle.
\end{equation}
For a generic many-body wave function, $|\Psi(t)\rangle$, the stationary solution of the action will simply be the Schr\"odinger equation, thus we impose that the many-body wave function is a Slater determinant of independent particle states $|\phi_i(t)\rangle$ to obtain
\begin{equation}
S_{t_0,t_1}[\Psi] = \int_{t_0}^{t_1}dt \left(\sum_{i=1}^{N} \langle\phi_i(t)|i\hbar\frac{d}{dt}|\phi_i(t)\rangle-\langle\phi_i(t)|\hat{H}|\phi_i(t)\rangle\right).
\end{equation}
This can be simplified further by defining the quantity $\sum_{i=1}^{N}\langle\phi_i(t)|\hat{H}|\phi_i(t)\rangle$ as an energy density functional (EDF), $E[\rho(t)]$.
Now, by varying with respect to single particle states we may impose the stationarity of the action
\begin{equation}
\frac{\delta S}{\delta\langle\phi_i(t)|}=i\hbar\frac{d}{dt}|\phi_i(t)\rangle - \int_{t_0}^{t_1}dt' \frac{\delta E[\rho(t)]}{\delta\langle\phi_i(t)|} = 0,
\end{equation}
with the final step being to utilize the chain rule in the EDF integral and define a final quantity, the single particle Hamiltonian, $h[\rho(t)]=\frac{\delta E[\rho(t)]}{\delta\rho(t)}$.
Collecting all of this, we are left with our final TDHF equation
\begin{equation}
i\hbar\frac{d}{dt}|\phi_i(t)\rangle = h[\rho(t)]|\phi_i(t)\rangle.
\label{eq:tdhf}
\end{equation}
By examining the form of the TDHF equations, we see that we have a description of the time evolution for each state, $i$, in the system.
Furthermore, the single particle Hamiltonian has a dependence on $\rho(t)$ and thus on the wave functions themselves.
This is in line with the self consistency feature of static Hartree-Fock mentioned above, and thus in the time propagation procedure densities from a half time step are typically used for stability.

\begin{figure}[t]
	\includegraphics[width=\textwidth]{../Figures/intro_figs/paths.png}
	\caption{A schematic depiction of the stationary path chosen for the TDHF evolution.\hfill\null}
	\label{fig:paths}
\end{figure}

An extremely important feature of the TDHF method comes about by restricting ourselves to the stationary path from $t_0$ to $t_1$, namely that we recover only a single path in time.
This is at odds with a fully quantum picture of a many-body system evolving in time where multiple paths have some probability of occurring.
Figure~\ref{fig:paths} shows a schematic explanation of this choice.
It is for this reason that TDHF is described as a semi-classical theory and only recovers the most probable event in a time evolution.
Beyond this, the derivation required us to consider fully independent particles, just as assumed in the Hartree-Fock method, meaning that many-body correlations will not be considered.
As will be seen later in this thesis, these (and other) limitations require extensions to the base theory to uncover features in nuclear reactions that are absent with the present formulation.
Despite the theoretical shortcomings of the approach, pure TDHF calculations have been extremely successful in reproducing experimental data over its history.

One final point of interest to note from the derivation above is the appearance of a density dependent single particle Hamiltonian in Eq.~\ref{eq:tdhf}.
As this Hamiltonian was derived by varying an EDF by the density, it allows one to skip a standard representation of the interaction between nucleons as a one or two-body potential and write down the EDF directly.
This is now the standard approach in mean-field, low-energy nuclear physics and there are various forms and fits of functionals that are in active use such as Skyrme and Gogny~\citep{skyrme1956,decharge1980}.
In practice, the nuclear EDF also depends on other quantities such as the probability current, spin density, spin-current density, and so on.
The interaction used in the results presented in the current work are of the Skyrme type and the specific EDFs are noted in the individual chapters' method sections.
%This approach is justified by the Hohenberg-Kohn theorems which prov~\citep{hohenberg1964}

The specific mean-field and beyond mean-field approaches to each project presented in this work are discussed in their respective chapters, though a general, brief word should be said about the validity of the mean-field method as it applies to atomic nuclei.
As the nucleus is traditionally thought of as a densely packed collection of nucleons, the primary assumption that individual protons and neutrons can travel freely in the nucleus is a bit counter intuitive.
It turns out, however, that the Pauli exclusion principle between nucleons ensures that the particles will remain at an average distance exceeding their radius at low energies~\citep{ring1980}.
Even at finite collision energies, the mean free path of nucleons in the nucleus is several times that of the nuclear radius, implying that a so-called hard-core collision is unlikely.
This means that the average potential felt by the nucleons in the nucleus will be around the minimum of a model potential like the one shown in Fig.~\ref{fig:reid}.
Should one go to higher collision energies, the mean-field approximation begins to break down and may misrepresent the outcome of such reactions.

\section{Nuclear dynamics}

\begin{figure}[t]
	\includegraphics[width=\textwidth]{../Figures/intro_figs/timescale.png}
	\caption{The general timescales of nuclear reactions from quasielastic scattering through fusion and subsequent decay.}
	\label{fig:timescale}
\end{figure}

Our focus is now turned to the dynamics of nuclear physics at low collision energies as predicted by TDHF.
Broadly speaking, for two incoming nuclei with their own distinct numbers of protons and neutrons, one could expect to see a large number of possible outcomes depending on factors such as the orientation of the incoming nuclei, the distance off the collision axis, and the energy between the two fragments.
What is seen experimentally will be some complex combination of initial configurations, however.
The typical technique employed to investigate a given system is then to perform a large number of calculations across the configuration space in an attempt to understand what is happening systematically as you change any initial quantity.
It is for this reason that one must take care to check the full parameter space in order to make authoritative statements on the physics at play in reactions, especially when comparing to experimental data.

The usual method of classifying nuclear reactions is by the length of time that the fragments stay in contact\footnote{The specific definition of contact varies from work to work, though it is typically stated as when the density in the neck region between the nuclei is around half the saturation density, $\rho=0.08$.} before coming apart again.
Of course, if the fragments stay together indefinitely this is interpreted as a fusion event in TDHF.
Though the timescales and definition of reaction channels are fluid, they may be roughly divided into categories as is shown in Fig.~\ref{fig:timescale}.
For reasons mentioned in the previous section TDHF is capable of investigating reaction mechanisms from the far left of the chart up through compound nucleus formation (around $10^{-20}-10^{-19}$s in Fig.~\ref{fig:timescale}).
Any subsequent events after a fused system is formed, like fission or the evaporation of a particle, will not occur in a TDHF calculation and must be dealt with using a separate technique.

\begin{figure}[t]
	\includegraphics[width=\textwidth]{../Figures/intro_figs/48Ca249BkEvolution_tr.png}
	\caption{A typical density evolution for a quasifission reaction. The system presented is $^{48}$Ca$+^{249}$Bk and the figure is taken from~\citep{godbey2020}.}
	\label{fig:qf}
\end{figure}

As an example case, a typical set of contours from a benchmark quasifission reaction may be seen in Fig.~\ref{fig:qf}.
In this reaction, as is common in quasifission, the initially mass asymmetric system collides and begins to exchange a substantial amount of energy and mass before rotating and forming two new nuclei as they separate.
As mentioned before, the nuclei produced in such a reaction will depend on the initial configuration, and substantial effort has been expended to understand the production rates of nuclei via these sorts of reactions.
From the image it is clear that the dynamics governing the outcome of a nuclear collisions depend on much more than the incoming number of particles, hence the sophisticated tools required to study these processes.

\section{Summary}

Through use of state-of-the-art many-body methods this thesis explores nuclear reactions through both theoretical development and extensions of the base theories, as well as through systematic, comprehensive studies of nuclei.
Each chapter either focuses on a specific aspect of nuclear reactions or attempts to exhaustively characterize a given system.
In Chapter~\ref{chapters:chapter_2}, I discuss the development of a new extension to explore the role of transfer in the initial stages of the collision on fusion.
The basis of the technique is described and then select applications are presented to demonstrate the applicability to generic nuclear systems.
Chapter~\ref{chapters:chapter_3} meanwhile continues to focus on fusion, though via a more basic aspect.
Specifically, a method is developed to directly show the impact of the Pauli exclusion principle on heavy ion collisions at low energies.

Chapters~\ref{chapters:chapter_4} and~\ref{chapters:chapter_5} turn away from the development of new extensions to TDHF and instead look into the effect of the Skyrme tensor interaction on fusion for multiple systems.
Similar to this is Chapter~\ref{chapters:chapter_6} which studies fusion cross sections of \textsuperscript{12}C+\textsuperscript{12}C both above and below the fusion barrier.
This study also represents an effort to make extremely precise fusion calculations be checking the effect of numerical changes on the results.

Finally, the last two chapters are largely devoted to transfer studies for two systems of nuclei.
Chapter~\ref{chapters:chapter_7} utilizes direct TDHF collisions to study fragment production in $^{48}$Ca+$^{249}$Bk reactions.
Due to the deformed nature of $^{249}$Bk, a complete sweep of angles was performed to paint the fullest picture yet of quasifission in TDHF.
While also a study of transfer, Chapter~\ref{chapters:chapter_8} goes beyond TDHF to look at fluctuations about the mean field to uncover the correlations that are missing in the base theory.
This system, as did the one before, requires a large amount of computation due to the deformed initial states and the added cost of the beyond TDHF method.

\clearpage