Multi-stage game

In game theory, a multi-stage game is a sequence of several simultaneous games played one after the other. This is a generalization of a repeated game: a repeated game is a special case of a multi-stage game, in which the stage games are identical.
Multi-Stage Game with Different Information Sets.
As an example, consider a two-stage game in which the stage game in "Figure 1" is played in each of two periods:
The payoff to each player is the simple sum of the payoffs of both games.
Players cannot observe the action of the other player within a round; however, at the beginning of Round 2, Player 2 finds out about Player 1's action in Round 1, while Player 1 does not find out about Player 2's action in Round 1.
For Player 1, there are formula_1 strategies.
For Player 2, there are formula_2 strategies.
The extensive form of this multi-stage game is shown in "Figure 2":
In this game, the only Nash Equilibrium in each stage is (B, b).
(BB, bb) will be the Nash Equilibrium for the entire game.
Multi-Stage Game with Changing Payoffs.
In this example, consider a two-stage game in which the stage game in "Figure 3" is played in the first period and the game in "Figure 4" is played in the second:
The payoff to each player is the simple sum of the payoffs of both games.
Players cannot observe the action of the other player within a round; however, at the beginning of Round 2, both players find out about the other's action in Round 1.
For Player 1, there are formula_2 strategies.
For Player 2, there are formula_2 strategies.
The extensive form of this multi-stage game is shown in "Figure 5":
Each of the two stages has two Nash Equilibria: which are (A, a), (B, b), (X, x), and (Y, y).
If the complete contingent strategy of Player 1 matches Player 2 (i.e. AXXXX, axxxx), it will be a Nash Equilibrium. There are 32 such combinations in this multi-stage game. Additionally, all of these equilibria are subgame-perfect.


Stochastic gradient descent

Stochastic gradient descent (often abbreviated SGD) is an iterative method for optimizing an objective function with suitable smoothness properties (e.g. differentiable or subdifferentiable). It can be regarded as a stochastic approximation of gradient descent optimization, since it replaces the actual gradient (calculated from the entire data set) by an estimate thereof (calculated from a randomly selected subset of the data). Especially in high-dimensional optimization problems this reduces the very high computational burden, achieving faster iterations in exchange for a lower convergence rate.
While the basic idea behind stochastic approximation can be traced back to the Robbins–Monro algorithm of the 1950s, stochastic gradient descent has become an important optimization method in machine learning.
Background.
Both statistical estimation and machine learning consider the problem of minimizing an objective function that has the form of a sum:
where the parameter formula_2 that minimizes formula_3 is to be estimated. Each summand function formula_4 is typically associated with the formula_5-th observation in the data set (used for training).
In classical statistics, sum-minimization problems arise in least squares and in maximum-likelihood estimation (for independent observations). The general class of estimators that arise as minimizers of sums are called M-estimators. However, in statistics, it has been long recognized that requiring even local minimization is too restrictive for some problems of maximum-likelihood estimation. Therefore, contemporary statistical theorists often consider stationary points of the likelihood function (or zeros of its derivative, the score function, and other estimating equations).
The sum-minimization problem also arises for empirical risk minimization. In this formula_6 is the value of the loss function at formula_5-th example, and formula_3 is the empirical risk.
When used to minimize the above function, a standard (or "batch") gradient descent method would perform the following iterations:
where formula_10 is a step size (sometimes called the "learning rate" in machine learning).
In many cases, the summand functions have a simple form that enables inexpensive evaluations of the sum-function and the sum gradient. For example, in statistics, one-parameter exponential families allow economical function-evaluations and gradient-evaluations.
However, in other cases, evaluating the sum-gradient may require expensive evaluations of the gradients from all summand functions. When the training set is enormous and no simple formulas exist, evaluating the sums of gradients becomes very expensive, because evaluating the gradient requires evaluating all the summand functions' gradients. To economize on the computational cost at every iteration, stochastic gradient descent samples a subset of summand functions at every step. This is very effective in the case of large-scale machine learning problems.
Iterative method.
In stochastic (or "on-line") gradient descent, the true gradient of formula_3 is approximated by a gradient at a single sample:
As the algorithm sweeps through the training set, it performs the above update for each training sample. Several passes can be made over the training set until the algorithm converges. If this is done, the data can be shuffled for each pass to prevent cycles. Typical implementations may use an adaptive learning rate so that the algorithm converges.
In pseudocode, stochastic gradient descent can be presented as :
A compromise between computing the true gradient and the gradient at a single sample is to compute the gradient against more than one training sample (called a "mini-batch") at each step. This can perform significantly better than "true" stochastic gradient descent described, because the code can make use of vectorization libraries rather than computing each step separately as was first shown in where it was called "the bunch-mode back-propagation algorithm". It may also result in smoother convergence, as the gradient computed at each step is averaged over more training samples.
The convergence of stochastic gradient descent has been analyzed using the theories of convex minimization and of stochastic approximation. Briefly, when the learning rates formula_10 decrease with an appropriate rate,
and subject to relatively mild assumptions, stochastic gradient descent converges almost surely to a global minimum
when the objective function is convex or pseudoconvex,
and otherwise converges almost surely to a local minimum.
This is in fact a consequence of the Robbins–Siegmund theorem.
Example.
Suppose we want to fit a straight line formula_18 to a training set with observations formula_19 and corresponding estimated responses formula_20 using least squares. The objective function to be minimized is:
The last line in the above pseudocode for this specific problem will become:
Note that in each iteration (also called update), the gradient is only evaluated at a single point formula_23 instead of at the set of all samples.
The key difference compared to standard (Batch) Gradient Descent is that only one piece of data from the dataset is used to calculate the step, and the piece of data is picked randomly at each step.
Notable applications.
Stochastic gradient descent is a popular algorithm for training a wide range of models in machine learning, including (linear) support vector machines, logistic regression (see, e.g., Vowpal Wabbit) and graphical models. When combined with the backpropagation algorithm, it is the "de facto" standard algorithm for training artificial neural networks. Its use has been also reported in the Geophysics community, specifically to applications of Full Waveform Inversion (FWI).
Stochastic gradient descent competes with the L-BFGS algorithm, which is also widely used. Stochastic gradient descent has been used since at least 1960 for training linear regression models, originally under the name ADALINE.
Another stochastic gradient descent algorithm is the least mean squares (LMS) adaptive filter.
Extensions and variants.
Many improvements on the basic stochastic gradient descent algorithm have been proposed and used. In particular, in machine learning, the need to set a learning rate (step size) has been recognized as problematic. Setting this parameter too high can cause the algorithm to diverge; setting it too low makes it slow to converge. A conceptually simple extension of stochastic gradient descent makes the learning rate a decreasing function of the iteration number , giving a "learning rate schedule", so that the first iterations cause large changes in the parameters, while the later ones do only fine-tuning. Such schedules have been known since the work of MacQueen on -means clustering. Practical guidance on choosing the step size in several variants of SGD is given by Spall.
Implicit updates (ISGD).
As mentioned earlier, classical stochastic gradient descent is generally sensitive to learning rate . Fast convergence requires large learning rates but this may induce numerical instability. The problem can be largely solved by considering "implicit updates" whereby the stochastic gradient is evaluated at the next iterate rather than the current one:
This equation is implicit since formula_25 appears on both sides of the equation. It is a stochastic form of the proximal gradient method since the update
can also be written as:
As an example,
consider least squares with features formula_27 and observations
formula_28. We wish to solve:
where formula_30 indicates the inner product.
Note that formula_31 could have "1" as the first element to include an intercept. Classical stochastic gradient descent proceeds as follows:
where formula_5 is uniformly sampled between 1 and formula_34. Although theoretical convergence of this procedure happens under relatively mild assumptions, in practice the procedure can be quite unstable. In particular, when formula_10 is misspecified so that formula_36 has large absolute eigenvalues with high probability, the procedure may diverge numerically within a few iterations. In contrast, "implicit stochastic gradient descent" (shortened as ISGD) can be solved in closed-form as:
This procedure will remain numerically stable virtually for all formula_10 as the learning rate is now normalized. Such comparison between classical and implicit stochastic gradient descent in the least squares problem is very similar to the comparison between least mean squares (LMS) and
normalized least mean squares filter (NLMS).
Even though a closed-form solution for ISGD is only possible in least squares, the procedure can be efficiently implemented in a wide range of models. Specifically, suppose that formula_6 depends on formula_2 only through a linear combination with features formula_41, so that we can write formula_42, where
formula_43 may depend on formula_44 as well but not on formula_2 except through formula_46. Least squares obeys this rule, and so does logistic regression, and most generalized linear models. For instance, in least squares, formula_47, and in logistic regression formula_48, where formula_49 is the logistic function. In Poisson regression, formula_50, and so on.
In such settings, ISGD is simply implemented as follows. Let formula_51, where formula_52 is scalar.
Then, ISGD is equivalent to:
The scaling factor formula_54 can be found through the bisection method since
in most regular models, such as the aforementioned generalized linear models, function formula_55 is decreasing,
and thus the search bounds for formula_56 are
formula_57.
Momentum.
Further proposals include the "momentum method" or the "heavy ball method", which in ML context appeared in Rumelhart, Hinton and Williams' paper on backpropagation learning and borrowed the idea from Soviet mathematician Boris Polyak's 1964 article on solving functional equations. Stochastic gradient descent with momentum remembers the update at each iteration, and determines the next update as a linear combination of the gradient and the previous update:
that leads to:
where the parameter formula_2 which minimizes formula_3 is to be estimated, formula_10 is a step size (sometimes called the "learning rate" in machine learning) and formula_64 is an exponential decay factor between 0 and 1 that determines the relative contribution of the current gradient and earlier gradients to the weight change.
The name momentum stems from an analogy to momentum in physics: the weight vector formula_2, thought of as a particle traveling through parameter space, incurs acceleration from the gradient of the loss ("force"). Unlike in classical stochastic gradient descent, it tends to keep traveling in the same direction, preventing oscillations. Momentum has been used successfully by computer scientists in the training of artificial neural networks for several decades.
The "momentum method" is closely related to underdamped Langevin dynamics, and may be combined with Simulated Annealing.
In mid-1980s the method was modified by Yurii Nesterov to use the gradient predicted at the next point, and the resulting so-called "Nesterov Accelerated Gradient" was sometimes used in ML in the 2010s.
Averaging.
"Averaged stochastic gradient descent", invented independently by Ruppert and Polyak in the late 1980s, is ordinary stochastic gradient descent that records an average of its parameter vector over time. That is, the update is the same as for ordinary stochastic gradient descent, but the algorithm also keeps track of
When optimization is done, this averaged parameter vector takes the place of .
AdaGrad.
"AdaGrad" (for adaptive gradient algorithm) is a modified stochastic gradient descent algorithm with per-parameter learning rate, first published in 2011. Informally, this increases the learning rate for sparser parameters and decreases the learning rate for ones that are less sparse. This strategy often improves convergence performance over standard stochastic gradient descent in settings where data is sparse and sparse parameters are more informative. Examples of such applications include natural language processing and image recognition.
It still has a base learning rate , but this is multiplied with the elements of a vector which is the diagonal of the outer product matrix
where formula_68, the gradient, at iteration . The diagonal is given by
This vector essentially stores a historical sum of gradient squares by dimension and is updated after every iteration. The formula for an update is now
or, written as per-parameter updates,
Each gives rise to a scaling factor for the learning rate that applies to a single parameter . Since the denominator in this factor, formula_72 is the "ℓ"2 norm of previous derivatives, extreme parameter updates get dampened, while parameters that get few or small updates receive higher learning rates.
While designed for convex problems, AdaGrad has been successfully applied to non-convex optimization.
RMSProp.
"RMSProp" (for Root Mean Square Propagation) is a method invented by Geoffrey Hinton in 2012 in which the learning rate is, like in Adagrad, adapted for each of the parameters. The idea is to divide the learning rate for a weight by a running average of the magnitudes of recent gradients for that weight. Unusually, it was not published in an article but merely described in a Coursera lecture.
So, first the running average is calculated in terms of means square,
where, formula_74 is the forgetting factor. The concept of storing the historical gradient as sum of squares is borrowed from Adagrad, but "forgetting" is introduced to solve Adagrad's diminishing learning rates in non-convex problems by gradually decreasing the influence of old data.
And the parameters are updated as,
RMSProp has shown good adaptation of learning rate in different applications. RMSProp can be seen as a generalization of Rprop and is capable to work with mini-batches as well opposed to only full-batches.
Adam.
"Adam" (short for Adaptive Moment Estimation) is a 2014 update to the "RMSProp" optimizer combining it with the main feature of the "Momentum method". In this optimization algorithm, running averages with exponential forgetting of both the gradients and the second moments of the gradients are used. Given parameters formula_76 and a loss function formula_77, where formula_78 indexes the current training iteration (indexed at formula_79), Adam's parameter update is given by:
where formula_85 is a small scalar (e.g. formula_86) used to prevent division by 0, and formula_87 (e.g. 0.9) and formula_88 (e.g. 0.999) are the forgetting factors for gradients and second moments of gradients, respectively. Squaring and square-rooting is done element-wise. The profound influence of this algorithm inspired multiple newer, less well-known momentum-based optimization schemes using Nesterov-enhanced gradients (eg: "NAdam" and "FASFA") and varying interpretations of second-order information (eg: "Powerpropagation" and "AdaSqrt"). However, the most commonly used variants are "AdaMax", which generalizes "Adam" using the infinity norm, and "AMSGrad", which addresses convergence problems from "Adam" by using maximum of past squared gradients instead of the exponential average.
"AdamW" is a later update which mitigates an unoptimal choice of the weight decay algorithm in "Adam".
Sign-based stochastic gradient descent.
Even though sign-based optimization goes back to the aforementioned Rprop, only in 2018 researchers tried to simplify Adam by removing the magnitude of the stochastic gradient from being taked into account and only considering its sign.
Backtracking line search.
Backtracking line search is another variant of gradient descent. All of the below are sourced from the mentioned link. It is based on a condition known as the Armijo–Goldstein condition. Both methods allow learning rates to change at each iteration; however, the manner of the change is different. Backtracking line search uses function evaluations to check Armijo's condition, and in principle the loop in the algorithm for determining the learning rates can be long and unknown in advance. Adaptive SGD does not need a loop in determining learning rates. On the other hand, adaptive SGD does not guarantee the "descent property" – which Backtracking line search enjoys – which is that formula_89 for all n. If the gradient of the cost function is globally Lipschitz continuous, with Lipschitz constant L, and learning rate is chosen of the order 1/L, then the standard version of SGD is a special case of backtracking line search.
Second-order methods.
A stochastic analogue of the standard (deterministic) Newton–Raphson algorithm (a "second-order" method) provides an asymptotically optimal or near-optimal form of iterative optimization in the setting of stochastic approximation. A method that uses direct measurements of the Hessian matrices of the summands in the empirical risk function was developed by Byrd, Hansen, Nocedal, and Singer. However, directly determining the required Hessian matrices for optimization may not be possible in practice. Practical and theoretically sound methods for second-order versions of SGD that do not require direct Hessian information are given by Spall and others. (A less efficient method based on finite differences, instead of simultaneous perturbations, is given by Ruppert.) Another approach to the approximation Hessian matrix is replacing it with the Fisher information matrix, which transforms usual gradient to natural. These methods not requiring direct Hessian information are based on either values of the summands in the above empirical risk function or values of the gradients of the summands (i.e., the SGD inputs). In particular, second-order optimality is asymptotically achievable without direct calculation of the Hessian matrices of the summands in the empirical risk function.
Approximations in continuous time.
For small learning rate formula_90 stochastic gradient descent formula_91 can be viewed as a discretization of the gradient flow ODE
formula_92
subject to additional stochastic noise. This approximation is only valid on a finite time-horizon in the following sense: assume that all the coefficients formula_93 are sufficiently smooth. Let formula_94 and formula_95 be a sufficiently smooth test function. Then, there exists a constant formula_96 such that for all formula_97
formula_98
where formula_99 denotes taking the expectation with respect to the random choice of indices in the stochastic gradient descent scheme.
Since this approximation does not capture the random fluctuations around the mean behavior of stochastic gradient descent solutions to stochastic differential equations (SDEs) have been proposed as limiting objects. More precisely, the solution to the SDE
formula_100
for formula_101 where formula_102 denotes the Ito-integral with respect to a Brownian motion is a more precise approximation in the sense that there exists a constant formula_96 such that
formula_104
However this SDE only approximates the one-point motion of stochastic gradient descent. For an approximation of the stochastic flow one has to consider SDEs with infinite-dimensional noise.
History.
SGD was gradually developed by several collectives during the 1950s.
The scaling behavior of SGD (i.e. how the performance evaluation metric of interest (e.g. test loss) varies as the number of training steps varies) was found to follow a broken neural scaling law functional form in 2022.


Trajectory optimization

Trajectory optimization is the process of designing a trajectory that minimizes (or maximizes) some measure of performance while satisfying a set of constraints. Generally speaking, trajectory optimization is a technique for computing an open-loop solution to an optimal control problem. It is often used for systems where computing the full closed-loop solution is not required, impractical or impossible. If a trajectory optimization problem can be solved at a rate given by the inverse of the Lipschitz constant, then it can be used iteratively to generate a closed-loop solution in the sense of Caratheodory. If only the first step of the trajectory is executed for an infinite-horizon problem, then this is known as Model Predictive Control (MPC).
Although the idea of trajectory optimization has been around for hundreds of years (calculus of variations, brachystochrone problem), it only became practical for real-world problems with the advent of the computer. Many of the original applications of trajectory optimization were in the aerospace industry, computing rocket and missile launch trajectories. More recently, trajectory optimization has also been used in a wide variety of industrial process and robotics applications.
History.
Trajectory optimization first showed up in 1697, with the introduction of the Brachystochrone problem: find the shape of a wire such that a bead sliding along it will move between two points in the minimum time. The interesting thing about this problem is that it is optimizing over a curve (the shape of the wire), rather than a single number. The most famous of the solutions was computed using calculus of variations.
In the 1950s, the digital computer started to make trajectory optimization practical for solving real-world problems. The first optimal control approaches grew out of the calculus of variations, based on the research of Gilbert Ames Bliss and Bryson in America, and Pontryagin in Russia. Pontryagin's maximum principle is of particular note. These early researchers created the foundation of what we now call indirect methods for trajectory optimization.
Much of the early work in trajectory optimization was focused on computing rocket thrust profiles, both in a vacuum and in the atmosphere. This early research discovered many basic principles that are still used today.
Another successful application was the climb to altitude trajectories for the early jet aircraft. Because of the high drag associated with the transonic drag region and the low thrust of early jet aircraft, trajectory optimization was the key to maximizing climb to altitude performance. Optimal control based trajectories were responsible for some of the world records. In these situations, the pilot followed a Mach versus altitude schedule based on optimal control solutions.
One of the important early problems in trajectory optimization was that of the singular arc, where Pontryagin's maximum principle fails to yield a complete solution. An example of a problem with singular control is the optimization of the thrust of a missile flying at a constant altitude and which is launched at low speed. Here the problem is one of a bang-bang control at maximum possible thrust until the singular arc is reached. Then the solution to the singular control provides a lower variable thrust until burnout. At that point bang-bang control provides that the control or thrust go to its minimum value of zero. This solution is the foundation of the boost-sustain rocket motor profile widely used today to maximize missile performance.
Applications.
There are a wide variety of applications for trajectory optimization, primarily in robotics: industry, manipulation, walking, path-planning, and aerospace. It can also be used for modeling and estimation.
Robotic manipulators.
Depending on the configuration, open-chain robotic manipulators require a degree of trajectory optimization. For instance, a robotic arm with 7 joints and 7 links (7-DOF) is a redundant system where one cartesian position of an end-effector can correspond to an infinite number of joint angle positions, thus this redundancy can be used to optimize a trajectory to, for example, avoid any obstacles in the workspace or minimize the torque in the joints.
Quadrotor helicopters.
Trajectory optimization is often used to compute trajectories for quadrotor helicopters. These applications typically used highly specialized algorithms.
One interesting application shown by the U.Penn GRASP Lab is computing a trajectory that allows a quadrotor to fly through a hoop as it is thrown. Another, this time by the ETH Zurich Flying Machine Arena, involves two quadrotors tossing a pole back and forth between them, with it balanced like an inverted pendulum. The problem of computing minimum-energy trajectories for a quadcopter, has also been recently studied.
Manufacturing.
Trajectory optimization is used in manufacturing, particularly for controlling chemical processes (such as in
) or computing the desired path for robotic manipulators (such as in
Walking robots.
There are a variety of different applications for trajectory optimization within the field of walking robotics. For example, one paper used trajectory optimization of bipedal gaits on a simple model to show that walking is energetically favorable for moving at a low speed and running is energetically favorable for moving at a high speed.
Like in many other applications, trajectory optimization can be used to compute a nominal trajectory, around which a stabilizing controller is built.
Trajectory optimization can be applied in detailed motion planning complex humanoid robots, such as Atlas.
Finally, trajectory optimization can be used for path-planning of robots with complicated dynamics constraints, using reduced complexity models.
Aerospace.
For tactical missiles, the flight profiles are determined by the thrust and lift histories. These histories can be controlled by a number of means including such techniques as using an angle of attack command history or an altitude/downrange schedule that the missile must follow. Each combination of missile design factors, desired missile performance, and system constraints results in a new set of optimal control parameters.
Trajectory optimization techniques.
The techniques to any optimization problems can be divided into two categories: indirect and direct. An indirect method works by analytically constructing the necessary and sufficient conditions for optimality, which are then solved numerically. A direct method attempts a direct numerical solution by constructing a sequence of continually improving approximations to the optimal solution.
The optimal control problem is an infinite-dimensional optimization problem, since the decision variables are functions, rather than real numbers. All solution techniques perform transcription, a process by which the trajectory optimization problem (optimizing over functions) is converted into a constrained parameter optimization problem (optimizing over real numbers). Generally, this constrained parameter optimization problem is a non-linear program, although in special cases it can be reduced to a quadratic program or linear program.
Single shooting.
Single shooting is the simplest type of trajectory optimization technique. The basic idea is similar to how you would aim a cannon: pick a set of parameters for the trajectory, simulate the entire thing, and then check to see if you hit the target. The entire trajectory is represented as a single segment, with a single constraint, known as a defect constraint, requiring that the final state of the simulation matches the desired final state of the system. Single shooting is effective for problems that are either simple or have an extremely good initialization. Both the indirect and direct formulation tend to have difficulties otherwise.
Multiple shooting.
Multiple shooting is a simple extension to single shooting that renders it far more effective. Rather than representing the entire trajectory as a single simulation (segment), the algorithm breaks the trajectory into many shorter segments, and a defect constraint is added between each. The result is large sparse non-linear program, which tends to be easier to solve than the small dense programs produced by single shooting.
Direct collocation.
Direct collocation methods work by approximating the state and control trajectories using polynomial splines. These methods are sometimes referred to as direct transcription. Trapezoidal collocation is a commonly used low-order direct collocation method. The dynamics, path objective, and control are all represented using linear splines, and the dynamics are satisfied using trapezoidal quadrature. Hermite-Simpson Collocation is a common medium-order direct collocation method. The state is represented by a cubic-Hermite spline, and the dynamics are satisfied using Simpson quadrature.
Orthogonal collocation.
Orthogonal collocation is technically a subset of direct collocation, but the implementation details are so different that it can reasonably be considered its own set of methods. Orthogonal collocation differs from direct collocation in that it typically uses high-order splines, and each segment of the trajectory might be represented by a spline of a different order. The name comes from the use of orthogonal polynomials in the state and control splines.
Pseudospectral discretization.
In pseudospectral discretization the entire trajectory is represented by a collection of basis functions in the time domain (independent variable). The basis functions need not be polynomials. Pseudospectral discretization is also known as spectral collocation. When used to solve a trajectory optimization problem whose solution is smooth, a pseudospectral method will achieve spectral (exponential) convergence. If the trajectory is not smooth, the convergence is still very fast, faster than Runge-Kutta methods.
Temporal Finite Elements.
In 1990 Dewey H. Hodges and Robert R. Bless proposed a weak Hamiltonian finite element method for optimal control problems. The idea was to derive a weak variational form of first order necessary conditions for optimality, discretise the time domain in finite intervals and use a simple zero order polynomial representation of states, controls and adjoints over each interval.
Differential dynamic programming.
Differential dynamic programming, is a bit different than the other techniques described here. In particular, it does not cleanly separate the transcription and the optimization. Instead, it does a sequence of iterative forward and backward passes along the trajectory. Each forward pass satisfies the system dynamics, and each backward pass satisfies the optimality conditions for control. Eventually, this iteration converges to a trajectory that is both feasible and optimal.
Comparison of techniques.
There are many techniques to choose from when solving a trajectory optimization problem. There is no best method, but some methods might do a better job on specific problems. This section provides a rough understanding of the trade-offs between methods.
Indirect vs. direct methods.
When solving a trajectory optimization problem with an indirect method, you must explicitly construct the adjoint equations and their gradients. This is often difficult to do, but it gives an excellent accuracy metric for the solution. Direct methods are much easier to set up and solve, but do not have a built-in accuracy metric. As a result, direct methods are more widely used, especially in non-critical applications. Indirect methods still have a place in specialized applications, particularly aerospace, where accuracy is critical.
One place where indirect methods have particular difficulty is on problems with path inequality constraints. These problems tend to have solutions for which the constraint is partially active. When constructing the adjoint equations for an indirect method, the user must explicitly write down when the constraint is active in the solution, which is difficult to know a priori. One solution is to use a direct method to compute an initial guess, which is then used to construct a multi-phase problem where the constraint is prescribed. The resulting problem can then be solved accurately using an indirect method.
Shooting vs. collocation.
Single shooting methods are best used for problems where the control is very simple (or there is an extremely good initial guess). For example, a satellite mission planning problem where the only control is the magnitude and direction of an initial impulse from the engines.
Multiple shooting tends to be good for problems with relatively simple control, but complicated dynamics. Although path constraints can be used, they make the resulting nonlinear program relatively difficult to solve.
Direct collocation methods are good for problems where the accuracy of the control and the state are similar. These methods tend to be less accurate than others (due to their low-order), but are particularly robust for problems with difficult path constraints.
Orthogonal collocation methods are best for obtaining high-accuracy solutions to problems where the accuracy of the control trajectory is important. Some implementations have trouble with path constraints. These methods are particularly good when the solution is smooth.


The International Conference on Autonomous Agents and Multiagent Systems or AAMAS is the leading scientific conference for research in the areas of artificial intelligence, autonomous agents, and multiagent systems. It is annually organized by a non-profit organization called the International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).
History.
The AAMAS conference is a merger of three major international conferences/workshops, namely International Conference on Autonomous Agents (AGENTS), International Conference on Multi-Agent Systems (ICMAS), and International Workshop on Agent Theories, Architectures, and Languages (ATAL). As such, this highly respected joint conference provides a quality forum for discussing research in this area.
Activities.
Besides the main program that consists of a main track, an industry and applications track, and a couple of special area tracks, AAMAS also hosts over 20 workshops (e.g., AOSE, COIN, DALT, ProMAS, to mention a few) and many tutorials. There is also a demonstration session and a doctoral symposium. Finally, each year AAMAS features a bunch of awards, most notably the IFAAMAS Influential Paper Award. It publishes proceedings which are available online.


Decentralized partially observable Markov decision process

The decentralized partially observable Markov decision process (Dec-POMDP) is a model for coordination and decision-making among multiple agents. It is a probabilistic model that can consider uncertainty in outcomes, sensors and communication (i.e., costly, delayed, noisy or nonexistent communication).
It is a generalization of a Markov decision process (MDP) and a partially observable Markov decision process (POMDP) to consider multiple decentralized agents.
Definition.
Formal definition.
A Dec-POMDP is a 7-tuple formula_1, where
At each time step, each agent takes an action formula_13, the state updates based on the transition function formula_14 (using the current state and the joint action), each agent observes an observation based on the observation function formula_15 (using the next state and the joint action) and a reward is generated for the whole team based on the reward function formula_16. The goal is to maximize expected cumulative reward over a finite or infinite number of steps. These time steps repeat until some given horizon (called finite horizon) or forever (called infinite horizon). The discount factor formula_17 maintains a finite sum in the infinite-horizon case (formula_18).


Robust principal component analysis

Robust Principal Component Analysis (RPCA) is a modification of the widely used statistical procedure of principal component analysis (PCA) which works well with respect to "grossly" corrupted observations. A number of different approaches exist for Robust PCA, including an idealized version of Robust PCA, which aims to recover a low-rank matrix L0 from highly corrupted measurements M = L0 +S0. This decomposition in low-rank and sparse matrices can be achieved by techniques such as Principal Component Pursuit method (PCP), Stable PCP, Quantized PCP, Block based PCP, and Local PCP. Then, optimization methods are used such as the Augmented Lagrange Multiplier Method (ALM), Alternating Direction Method (ADM), Fast Alternating Minimization (FAM), Iteratively Reweighted Least Squares (IRLS )
or alternating projections (AP).
Algorithms.
Non-convex method.
The 2014 guaranteed algorithm for the robust PCA problem (with the input matrix being formula_1) is an alternating minimization type algorithm. The computational complexity is formula_2 where the input is the superposition of a low-rank (of rank formula_3) and a sparse matrix of dimension formula_4 and formula_5 is the desired accuracy of the recovered solution, i.e., formula_6 where formula_7 is the true low-rank component and formula_8 is the estimated or recovered low-rank component. Intuitively, this algorithm performs projections of the residual onto the set of low-rank matrices (via the SVD operation) and sparse matrices (via entry-wise hard thresholding) in an alternating manner - that is, low-rank projection of the difference the input matrix and the sparse matrix obtained at a given iteration followed by sparse projection of the difference of the input matrix and the low-rank matrix obtained in the previous step, and iterating the two steps until convergence.
This alternating projections algorithm is later improved by an accelerated version, coined AccAltProj. The acceleration is achieved by applying a tangent space projection before project the residue onto the set of low-rank matrices. This trick improves the computational complexity to formula_9 with a much smaller constant in front while it maintains the theoretically guaranteed linear convergence.
Another fast version of accelerated alternating projections algorithm is IRCUR. It uses the structure of CUR decomposition in alternating projections framework to dramatically reduces the computational complexity of RPCA to formula_10
Convex relaxation.
This method consists of relaxing the rank constraint formula_11 in the optimization problem to the nuclear norm formula_12 and the sparsity constraint formula_13 to formula_14-norm formula_15. The resulting program can be solved using methods such as the method of Augmented Lagrange Multipliers.
Deep-learning augmented method.
Some recent works propose RPCA algorithms with learnable/training parameters. Such a learnable/trainable algorithm can be unfolded as a deep neural network whose parameters can be learned via machine learning techniques from a given dataset or problem distribution. The learned algorithm will have superior performance on the corresponding problem distribution.
Applications.
RPCA has many real life important applications particularly when the data under study can naturally be modeled as a low-rank plus a sparse contribution. Following examples are inspired by contemporary challenges in computer science, and depending on the applications, either the low-rank component or the sparse component could be the object of interest:
Video surveillance.
Given a sequence of surveillance video frames, it is often required to identify the activities that stand out from the background. If we stack the video frames as columns of a matrix M, then the low-rank component L0 naturally corresponds to the stationary background and the sparse component S0 captures the moving objects in the foreground.
Face recognition.
Images of a convex, Lambertian surface under varying illuminations span a low-dimensional subspace. This is one of the reasons for effectiveness of low-dimensional models for imagery data. In particular, it is easy to approximate images of a human's face by a low-dimensional subspace. To be able to correctly retrieve this subspace is crucial in many applications such as face recognition and alignment. It turns out that RPCA can be applied successfully to this problem to exactly recover the face.
Resources and libraries.
Libraries.
The LRS Library (developed by Andrews Sobral) provides a collection of low-rank and sparse decomposition algorithms in MATLAB. The library was designed for moving object detection in videos, but it can be also used for other computer vision / machine learning tasks. Currently the LRSLibrary offers more than 100 algorithms based on "matrix" and "tensor" methods.


Internet Gold Golden Lines

Internet Gold Golden Lines (Internet Gold) is a principal communication service group in Israel. The company was founded in 1992 and is headquartered in Israel. It is a subsidiary of Eurocom Communications Ltd., owned by Shaul Elovitch. It has subsidiaries such as B Communications (formerly 012 Smile Communications) and GoldMind Ltd. (formerly Smile.Media Ltd.). The company was formerly known as Euronet Golden Lines Ltd. and changed its name to Internet Gold - Golden Lines Ltd. in 1999.
Background.
The company operates in four areas including Bezeq Domestic Fixed-line Communications, Pelephone Communications ltd., Bezeq International Ltd. And D.B.S. Satellite Service Ltd.
As of April 4, 2013, the company has a market capitalization of $789.67 million with an enterprise value of $3.79 billion.
The company’s subsidiary, B Communications, is the controlling holder, holding 31.37% interest, of Bezeq, which is the largest communication service provider in Israel. The company owns 75.3% interest of its subsidiary, 012 Smile Communications Ltd, which is one of Israel’s major Internet and international telephony service providers and the largest providers of enterprise/IT integration services. In 2010, 012 Smile Communication completed the acquisition of all shares of Golden Lines Ltd. Smile Media Ltd. is 100% owned by Internet Gold and is engaged in Internet portals and e-Commerce business.


Soccer robot

A soccer robot is a specialized autonomous robot and mobile robot that is used to play variants of soccer.
The main organised competitions are RoboCup or FIRA tournaments played each year.
The RoboCup contest currently has a number of soccer leagues:
Additionally, there is a RoboCupJunior league for younger students.
qfix Soccer robot.
The qfix soccer robot "Terminator" is an omnidrive robot that can be used for RoboCup Junior. It includes a kicker and a dribbler as well as a controller board with Atmel controller.
The robot can be programmed using the GNU GCC compiler.
Graupner RC-SOCCERBOT.
The Graupner "RC-SOCCERBOT" is a mobile robot platform developed by qfix which can be used as a radio-controller toy playing soccer with ping-pong balls. Gaining more experience in robotics the user can also implement C++ programs on the robot.


Zakai equation

In filtering theory the Zakai equation is a linear stochastic partial differential equation for the un-normalized density of a hidden state. In contrast, the Kushner equation gives a non-linear stochastic partial differential equation for the normalized density of the hidden state. In principle either approach allows one to estimate a quantity function (the state of a dynamical system) from noisy measurements, even when the system is non-linear (thus generalizing the earlier results of Wiener and Kalman for linear systems and solving a central problem in estimation theory). The application of this approach to a specific engineering situation may be problematic however, as these equations are quite complex. The Zakai equation is a bilinear stochastic partial differential equation. It was named after Moshe Zakai.
Overview.
Assume the state of the system evolves according to
and a noisy measurement of the system state is available:
where formula_3 are independent Wiener processes. Then the unnormalized conditional probability density formula_4 of the state at time t is given by the Zakai equation:
where the operator formula_6
As previously mentioned, formula_7 is an unnormalized density and thus does not necessarily integrate to 1. After solving for formula_7, integration and normalization can be done if desired (an extra step not required in the Kushner approach).
Note that if the last term on the right hand side is omitted (by choosing h identically zero), the result is a nonstochastic PDE: the familiar Fokker–Planck equation, which describes the evolution of the state when no measurement information is available.


UNIO Satu Mare

UNIO, established in 1911, is a large company specialised in machine building, mechanical assembling works and hydraulic, pneumatic and electric equipment assembling commissioning and service for the mining, energy and assembling industries. It is one of the largest companies in Satu Mare.
In January 2005 UNIO signed a contract with the German industrial company KUKA for the manufacturing of several assembly lines for the giant carmaker Mercedes-Benz.


Evolutionary developmental robotics

Evolutionary developmental robotics (evo-devo-robo for short) refers to methodologies that systematically integrate evolutionary robotics, epigenetic robotics and morphogenetic robotics to study the evolution, physical and mental development and learning of natural intelligent systems in robotic systems. The field was formally suggested and fully discussed in a published paper and further discussed in a published dialogue.
The theoretical foundation of evo-devo-robo includes evolutionary developmental biology (evo-devo), evolutionary developmental psychology, developmental cognitive neuroscience etc. Further discussions on evolution, development and learning in robotics and design can be found in a number of papers, including papers on hardware systems and computing tissues.


Robotics middleware

Robotics middleware is middleware to be used in complex robot control software systems.
It can be described as "software glue" to make it easier for robot builders focus on their specific problem area.
Robotics middleware projects.
A wide variety of projects for robotics middleware exist, but no one of these dominates - and in fact many robotic systems do not use any middleware. Middleware products rely on a wide range of different standards, technologies, and approaches that make their use and interoperation difficult, and some developers may prefer to integrate their system themselves.
Player Project.
The Player Project (formerly the "Player/Stage Project") is a project to create free software for research into robotics and sensor systems. Its components include the "Player" network server and the "Stage" robot platform simulators. Although accurate statistics are hard to obtain, Player is one of the most popular open-source robot interfaces in research and post-secondary education. Most of the major intelligent robotics journals and conferences regularly publish papers featuring real and simulated robot experiments using Player and Stage.
RT-middleware.
RT-middleware is a common platform standards for Robots based on distributed object technology. RT-middleware supports the construction of various networked robotic systems by the integration of various network-enabled robotic elements called RT-Components. The specification standard of RT-components is discussed and defined by the Object Management Group (OMG).
Urbi.
Urbi is an open source cross-platform software platform in C++ used to develop applications for robotics and complex systems. It is based on the UObject distributed C++ component architecture. It also includes the urbiscript orchestration language which is a parallel and event-driven script language. UObject components can be plugged into urbiscript and appear as native objects that can be scripted to specify their interactions and data exchanges. UObjects can be linked to the urbiscript interpreter, or executed as autonomous processes in "remote" mode, either in another thread, another process, a machine on the local network, or a machine on a distant network.
MIRO.
Miro is a distributed object oriented framework for mobile robot control, based on CORBA (Common Object Request Broker Architecture) technology.
The Miro core components have been developed under the aid of ACE (Adaptive Communications Environment), an object oriented multi-platform framework for OS-independent interprocess, network and real time communication. They use TAO (The ACE ORB) as their ORB (Object Request Broker), a CORBA implementation designed for high performance and real time applications.
Currently supported platforms include Pioneers, the B21, some robot soccer robots and various robotic sensors.
Orca.
Orca describes its goals as:
They also state: "To be successful, we think that a framework with such objectives must be: general, flexible and extensible; sufficiently robust, high-performance and full-featured for use in commercial applications, yet sufficiently simple for experimentation in university research environments."
They describe their approach as:
Orca software is released under LGPL and GPL licenses.
OpenRDK.
OpenRDK is an open-source software framework for robotics for developing loosely coupled modules. It provides transparent concurrency management, inter-process (via sockets) and intra-process (via shared memory) blackboard-based communication and a linking technique that allows for input/output data ports conceptual system design. Modules for connecting to simulators and generic robot drivers are provided.
Rock.
Rock (Robot Construction Kit), is a software framework for the development of robotic systems. The underlying component model is based on the Orocos RTT (Real Time Toolkit). Rock provides all the tools required to set up and run high-performance and reliable robotic systems for wide variety of applications in research and industry. It contains a rich collection of ready to use drivers and modules for use in your own system, and can easily be extended by adding new components.
ISAAC SDK / Simulation.
ISAAC, The NVIDIA Isaac Software Development Kit (SDK) is a developer toolbox for accelerating the development and deployment of Artificial Intelligence-powered robots. The SDK includes the Isaac Robot Engine, packages with high-performance robotics algorithms (to perform perception and navigation), and hardware reference applications. Isaac Sim is a virtual robotics laboratory and a high-fidelity 3D world simulator. It accelerates research, design, and development in robotics by reducing cost and risk. Developers can quickly and easily train and test their robots in detailed, highly realistic scenarios. There is an open source community version available at GitHub with supported hardware platform includes BOM details, refer kaya-robot
ROS.
ROS (Robot Operating System) is a collection of software frameworks for robot software development on a heterogeneous computer cluster. ROS provides standard operating system services such as hardware abstraction, low-level device control, implementation of commonly used functionality, message-passing between processes, and package management.
YARP.
YARP is an open-source software package, written in C++ for interconnecting sensors, processors, and actuators in robots.
DDX.
DDX (Dynamic Data eXchange) is (Linux/BSD/Unix) middleware developed by CSIRO to provide a lightweight real-time publish/subscribe service to distributed robot controllers. DDX allows a coalition of programs to share data at run-time through an efficient shared memory mechanism. Multiple machines can be linked by means of a global naming service and, when needed, data is multi-cast across machines. DDX was developed to automate a number of large mining machines: including draglines, LHD trucks, excavators and rock-breakers.


Full state feedback

Full state feedback (FSF), or pole placement, is a method employed in feedback control system theory to place the closed-loop poles of a plant in pre-determined locations in the s-plane. Placing poles is desirable because the location of the poles corresponds directly to the eigenvalues of the system, which control the characteristics of the response of the system. The system must be considered controllable in order to implement this method.
Principle.
If the closed-loop dynamics can be represented by the state space equation (see State space (controls))
with output equation
then the poles of the system transfer function are the roots of the characteristic equation given by
Full state feedback is utilized by commanding the input vector formula_4. Consider an input proportional (in the matrix sense) to the state vector,
Substituting into the state space equations above, we have
The poles of the FSF system are given by the characteristic equation of the matrix formula_8, formula_9. Comparing the terms of this equation with those of the desired characteristic equation yields the values of the feedback matrix formula_10 which force the closed-loop eigenvalues to the pole locations specified by the desired characteristic equation.
Example of FSF.
Consider a system given by the following state space equations:
The uncontrolled system has open-loop poles at formula_12 and formula_13. These poles are the eigenvalues of the formula_14 matrix and they are the roots of formula_15. Suppose, for considerations of the response, we wish the controlled system eigenvalues to be located at formula_12 and formula_17, which are not the poles we currently have. The desired characteristic equation is then formula_18, from formula_19.
Following the procedure given above, the FSF controlled system characteristic equation is
where
Upon setting this characteristic equation equal to the desired characteristic equation, we find
Therefore, setting formula_5 forces the closed-loop poles to the desired locations, affecting the response as desired.
This only works for Single-Input systems. Multiple input systems will have a formula_10 matrix that is not unique. Choosing, therefore, the best formula_10 values is not trivial. A linear-quadratic regulator might be used for such applications.


Small-world routing

In network theory, small-world routing refers to routing methods for small-world networks. Networks of this type are peculiar in that relatively short paths exist between any two nodes. Determining these paths, however, can be a difficult problem from the perspective of an individual routing node in the network if no further information is known about the network as a whole.
Greedy routing.
Nearly every solution to the problem of routing in small world involves the application of greedy routing. This sort of routing depends on a relative reference point by which any node in the path can choose the next node it believes is closest to the destination. That is, there must be something to be greedy about. For example, this could be geographic location, IP address, etc. In the case of Milgram's original small-world experiment, participants knew the location and occupation of the final recipient and could therefore forward messages based on those parameters.
Constructing a reference base.
Greedy routing will not readily work when there is no obvious reference base. This can occur, for example, in overlay networks where information about the destination's location in the underlying network is not available. Friend-to-friend networks are a particular example of this problem. In such networks, trust is ensured by the fact that you only know underlying information about nodes with whom you are already a neighbor.
One solution in this case, is to impose some sort of artificial addressing on the nodes in such a way that this addressing can be effectively used by greedy routing methods. A 2005 paper by a developer of the Freenet Project discusses how this can be accomplished in friend to friend networks. Given the assumption that these networks exhibit small world properties, often as the result of real-world or acquaintance relationships, it should be possible to recover an embedded Kleinberg small-world graph. This is accomplished by selecting random pairs of nodes and potentially swapping them based on an objective function that minimizes the product of all the distances between any given node and its neighbors.
An important problem involved with this solution is the possibility of local minima. This can occur if nodes are in a situation that is optimal only considering a local neighborhood, while ignoring the possibility of a higher optimality resulting from swaps with distant nodes. In the above paper, the authors proposed a simulated annealing method where less-than-optimal swaps were made with a small probability. This probability was proportional to the value of making the switches. Another possible metaheuristic optimization method is a tabu search, which adds a memory to the swap decision. In its most simplistic form, a limited history of past swaps is remembered so that they will be excluded from the list of possible swapping nodes.
This method for constructing a reference base can also be adapted to distributed settings, where decisions can only be made at the level of individual nodes who have no knowledge of the overall network. It turns out that the only modification necessary is in the method for selecting pairs of random nodes. In a distributed setting, this is done by having each node periodically send out a random walker terminating at a node to be considered for swapping.
The Kleinberg model.
The Kleinberg model of a network is effective at demonstrating the effectiveness of greedy small world routing. The model uses an n x n grid of nodes to represent a network, where each node is connected with an undirected edge to its neighbors. To give it the "small world" effect, a number of long range edges are added to the network that tend to favor nodes closer in distance rather than farther. When adding edges, the probability of connecting some random vertex formula_1 to another random vertex w is proportional to formula_2, where formula_3 is the clustering exponent.
Greedy routing in the Kleinberg model.
It is easy to see that a greedy algorithm, without using the long range edges, can navigate from random vertices formula_4 on the grid in formula_5 time. By following the guaranteed connections to our neighbors, we can move one unit at a time in the direction of our destination. This is also the case when the clustering component formula_3 is large and the "long range" edges end up staying very close; we simply do not take advantage of the weaker ties in this model. When formula_7, the long range edges are uniformly connected at random which means the long range edges are "too random" to be used efficiently for decentralized search. Kleinberg has shown that the optimal clustering coefficient for this model is formula_8, or an inverse square distribution.
To reason why this is the case, if a circle of radius r is drawn around the initial node it will have nodal density formula_9 where n is the number of nodes in the circular area. As this circle gets expanded further out, the number of nodes in the given area increases proportional to formula_10 as the probability of having a random link with any node remains proportional formula_11, meaning the probability of the original node having a weak tie with any node a given distance away is effectively independent of distance. Therefore, it is concluded that with formula_8, long-range edges are evenly distributed over all distances, which is effective for letting us funnel to our final destination.
Some structured Peer-to-peer systems based on DHTs often are implementing variants of Kleinberg's Small-World topology to enable efficient routing within Peer-to-peer network with limited node degrees.


Pitch drop-back

Pitch drop-back is the phenomena by which an aircraft which is perturbed in flight path angle from its trim position by a step input exhibits an output which is indicative of a second order system.
A pilot who actuates an elevator input may find that the aircraft then "droops" or "drops back" to a position further toward the start position. The phenomenon is particularly marked in tilt-rotor aircraft. Pitch drop-back may be controlled using a Stability Augmentation System or Stability Control and Augmentation System.


Weighting pattern

A weighting pattern for a linear dynamical system describes the relationship between an input formula_1 and output formula_2. Given the time-variant system described by
then the output can be written as
where formula_6 is the weighting pattern for the system. For such a system, the weighting pattern is formula_7 such that formula_8 is the state transition matrix.
The weighting pattern will determine a system, but if there exists a realization for this weighting pattern then there exist many that do so.
Linear time invariant system.
In a LTI system then the weighting pattern is:
where formula_10 is the matrix exponential.


Tardiness (scheduling)

In scheduling, tardiness is a measure of a delay in executing certain operations and earliness is a measure of finishing operations before due time. The operations may depend on each other and on the availability of equipment to perform them.
Typical examples include job scheduling in manufacturing and data delivery scheduling in data processing networks.
In manufacturing environment, inventory management considers both tardiness and earliness undesirable. Tardiness involves backlog issues such as customer compensation for delays and loss of goodwill. Earliness incurs expenses for storage of the manufactured items and ties up capital.
Mathematical formulations.
In an environment with multiple jobs, let the deadline be formula_1 and the completion time be formula_2 of job formula_3. Then for job formula_3
In scheduling common objective functions are formula_8 or weighted version of these sums, formula_9, where every job comes with a weight formula_10. The weight is a representation of job cost, priority, etc.
In a large number of cases the problems of optimizing these functions are NP-hard.


Realization (systems)

In systems theory, a realization of a state space model is an implementation of a given input-output behavior. That is, given an input-output relationship, a realization is a quadruple of (time-varying) matrices formula_1 such that
with formula_4 describing the input and output of the system at time formula_5.
LTI System.
For a linear time-invariant system specified by a transfer matrix, formula_6, a realization is any quadruple of matrices formula_7 such that formula_8.
Canonical realizations.
Any given transfer function which is strictly proper can easily be transferred into state-space by the following approach (this example is for a 4-dimensional, single-input, single-output system)):
Given a transfer function, expand it to reveal all coefficients in both the numerator and denominator. This should result in the following form:
The coefficients can now be inserted directly into the state-space model by the following approach:
This state-space realization is called controllable canonical form (also known as phase variable canonical form) because the resulting model is guaranteed to be controllable (i.e., because the control enters a chain of integrators, it has the ability to move every state).
The transfer function coefficients can also be used to construct another type of canonical form
This state-space realization is called observable canonical form because the resulting model is guaranteed to be observable (i.e., because the output exits from a chain of integrators, every state has an effect on the output).
General System.
"D" = 0.
If we have an input formula_14, an output formula_15, and a weighting pattern formula_16 then a realization is any triple of matrices formula_17 such that formula_18 where formula_19 is the state-transition matrix associated with the realization.
System identification.
System identification techniques take the experimental data from a system and output a realization. Such techniques can utilize both input and output data (e.g. eigensystem realization algorithm) or can only include the output data (e.g. frequency domain decomposition). Typically an input-output technique would be more accurate, but the input data is not always available.


Intermittent control

Intermittent control is a feedback control method which not only explains some human control systems but also has applications to control engineering.
In the context of control theory, intermittent control provides a spectrum of possibilities between the two extremes of continuous-time and discrete-time control: the control signal consists of a sequence of (continuous-time) parameterised trajectories whose parameters are adjusted intermittently. It is different from discrete-time control in that the control is not constant between samples; it is different from continuous-time control in that the trajectories are reset intermittently. As a class of control theory, intermittent predictive control is more general than continuous control and provides a new paradigm incorporating continuous predictive and optimal control with intermittent, open loop (ballistic) control.
There are at least three areas where intermittent control is relevant. Firstly, continuous-time model-based predictive control where the intermittency is associated with on-line optimisation. Secondly, event-driven control systems where the intersample interval is time varying and determined by the event times. Thirdly, explanation of physiological control systems which, in some cases, have an intermittent character. This intermittency may be due to the “computation” in the central nervous system.
Conventional sampled-data control uses a zero-order hold, which produces a piecewise-constant control signal and can be used to give a
sampled-data implementation which approximates previously-designed continuous-time controller. In contrast to conventional sampled data control, intermittent control explicitly embeds the underlying continuous-time closed-loop system in a "system-matched" hold which generates an open-loop intersample control trajectory based on the underlying continuous-time closed-loop control system.
History.
Intermittent control initially evolved separately in the engineering and physiological literature.
Physiological literature.
The concept of "intermittent control" appeared in a posthumous paper by Kenneth Craik which states “The human operator behaves basically as an intermittent correction servo”. A colleague of Kenneth Craik, Margaret Vince, related the concept of intermittency to the Psychological refractory period and provided experimental verification of intermittency. Fernando Navas and James Stark showed experimentally that human hand movements were synchronised to input signals rather than to an internal clock: in other words the hand control system is event-driven not clock-driven. The first detailed mathematical model of intermittency was presented by Peter Neilson, Megan Neilson, and Nicholas O’Dwyer.
A more recent mathematical model of intermittency is given by PeterGawthrop, Ian Loram, Martin Lakie and Henrik Gollee.
Engineering literature.
In the context of Control Engineering, the term intermittent control was used by Eric Ronco, Taner Arsan and Peter Gawthrop.
They stated that “A conceptual, and practical difficulty with the continuous-time generalised predictive controller is solved by replacing the continuously moving horizon by an intermittently moving horizon. This allows slow optimisation to occur concurrently with a fast control action.” The concept of intermittent model predictive control was refined by Peter Gawthrop working with Liuping Wang, who also looked at event-driven intermittent control.
In a separate line of development Tomas Estrada, Hai Lin and Panos Antsaklis developed the concept of model-based control with intermittent feedback in the context of a networked control system.


Berkeley printing system

The Berkeley printing system is one of several standard architectures for printing on the Unix platform. It originated in 2.10BSD, and is used in BSD derivatives such as FreeBSD, NetBSD, OpenBSD, and DragonFly BSD. A system running this print architecture could traditionally be identified by the use of the user command "lpr" as the primary interface to the print system, as opposed to the System V printing system "lp" command.
Typical user commands available to the Berkeley print system are:
The "lpd" program is the daemon with which those programs communicate.
These programs support the line printer daemon protocol, so that other machines on a network can submit jobs to a print queue on a machine running the Berkeley printing system, and so that the Berkeley printing system user commands can submit jobs to machines that support that protocol.


CSIv2

In distributed computing, CSIv2 (Common Secure Interoperability Protocol Version 2) is a protocol implementing security features for inter-ORB communication. It intends, in part, to address limitations of SSLIOP.
CSIv2 also facilitates secure EJB-CORBA interoperability.


Information systems technician

An information systems technician is a technician whose responsibility is maintaining communications and computer systems.
Description.
Information systems technicians operate and maintain information systems, facilitating system utilization. In many companies, these technicians assemble data sets and other details needed to build databases. This includes data management, procedure writing, writing job setup instructions, and performing program librarian functions. Information systems technicians assist in designing and coordinating the development of integrated information system databases. Information systems technicians also help maintain Internet and Intranet websites. They decide how information is presented and create digital multimedia and presentation using software and related equipment.
Information systems technicians install and maintain multi-platform networking computer environments, a variety of data networks, and a diverse set of telecommunications infrastructures. Information systems technicians schedule information gathering for content in a multiple system environment. Information systems technicians are responsible for the operation, programming, and configuration of many pieces of electronics, hardware and software. ITs often are also tasked to investigate, troubleshoot, and resolve end-user problems. Information systems technicians conduct ongoing assessments of short and long-term hardware and software needs for companies, developing, testing, and implementing new and revised programs.
Information systems technicians cooperate with other staff to inventory, maintain and manage computer and communication systems. Information systems technicians provide communication links and connectivity to the department in an organization, serving to equipment modification and installation tasks. This includes:
Additionally, Information systems technicians can conduct training and provide technical support to end-users, providing this for a departments (sometimes across multiple organizations).


Mesoscale manufacturing

Mesoscale manufacturing is the process of creating components and products in a range of approximately from 0.1mm to 5mm with high accuracy and precision using a wide variety of engineering materials. Mesomanufacturing processes are filling the gap between macro- and micromanufacturing processes and overlaps both of them (see picture). Other manufacturing technologies are nanoscale ( 0.5 mm).
Applications.
Application of mesomanufacturing include electronics, biotechnology, optics, medicine, avionics, communications, and other areas. Specific applications include mechanical watches, and extremely small motors and bearings; lenses for cameras and other micro parts for mobile telephones; micro-batteries, mesoscale fuel cells, microscale pumps, valves, and mixing devices for microchemical reactors; biomedical implants, microholes for fiber optics; medical devices such as stents and valves; mini nozzles for high-temperature jets; mesoscale molds; desktop- or micro-factories, and many others.
Processes.
Manufacturing in the mesoscale can be accomplished by scaling down macroscale manufacturing processes or scaling up nanomanufacturing processes. Macroscale techniques like mill and lathe machining have been successful used to create features in the range of 25 µm. Meso Machine tools (mMTs), for example a miniaturized milling machine, is an expansion of using traditional macroscale techniques to manufacture mesoscale products. With the limitation of self-excited vibration of machine tools and fatigue, microassembly and micro- and mesoscale milling are created to improve the maximum stiffness and dynamic operation of the milling process, which improves the overall performance of manufacturing. The development of mMTs has revealed many challenges that are specific to machining at the small scales. These challenges stem from the large influence of grain size at small scales and the necessity of extremely small tolerances for both the machine tools and the measuring tools.
Laser machining is a traditional technique that uses nanosecond pulses of ultraviolet light to create mesoscale features like holes, fillets, etc. The removal of material during laser machining is proportional to exposure time and therefore this process can be used to create three dimensional features.
A less traditional technique is to use focused ion beam sputtering (FIB) to remove material. This process involves focusing a beam of ions, for example gallium, to the work piece, causing material to be removed. FIB sputtering has a relatively low rate of material removal and therefore has limited application.
Electrical discharge machining (EDM) is another subtractive manufacturing process used in the mesoscale. This process requires that electricity be transferred between the tool electrode and the work piece and therefore it can only be used to manufacture materials that conduct electricity. One advantage of EDM is that it can be used on hard materials that do not work well in traditional machining processes, such as titanium.


SCSI standalone enclosure services

SCSI standalone enclosure services is a computer protocol used mainly with disk storage enclosures. It allows a host computer to communicate with the enclosure to access its power, cooling, and other non-data characteristics.
The host computer communicates with one or more SCSI Enclosure Services (SES) controllers in the enclosure via a SCSI interface which may be Parallel SCSI, FC-AL, SAS, or SSA. Each SES controller has a SCSI identity (address) and so can accept direct SCSI commands.
Implemented commands.
The following SCSI commands are implemented by standalone enclosure services devices:
Note 1: The initiator needs to send a SCSI inquiry to interrogate the SCCS bit which says whether the SES controller has this command.


Electrohydraulic forming

Electrohydraulic forming is a type of process in which an electric arc discharge in liquid is used to convert electrical energy to mechanical energy and change the shape of the workpiece. A capacitor bank delivers a pulse of high current across two electrodes, which are positioned a short distance apart while submerged in a fluid (water or oil). The electric arc discharge rapidly vaporizes the surrounding fluid, creating a shock wave. The workpiece, which is kept in contact with the fluid, is deformed into an evacuated die.
The potential forming capabilities of submerged arc discharge processes were recognized as early as the mid-1940s (Yutkin L.A.). During the 1950s and early 1960s, the basic process was developed into production systems. This work principally was by and for the aerospace industries. By 1970, forming machines based on submerged arc discharge were available from machine tool builders. A few of the larger aerospace fabricators built machines of their own design to meet specific part fabrication requirements.
Electrohydraulic forming (EHF) is based on the ultra-high-speed deformation of metal using shockwaves in water. Using the discharge of current from a capacitor bank, an electric arc is generated in water between two electrodes. This electric arc vaporizes the surrounding water, converting electrical energy into an intense shockwave of mechanical energy.
The shockwave simultaneously transforms the metal workpiece into a visco-plastic state and accelerates it into a die, enabling forming of complex shapes at high speeds in cold conditions. All of which happens in a matter of milliseconds; total cycle time of seconds including charging time of the system. This process is not limited by size and allows forming of parts up to a few square meters in size. An array of electrodes can be placed over a large workpiece, enabling pressure distribution according to the product’s topology, still using a one-sided die to create complex shapes and fine details.
Very large capacitor banks are needed to produce the same amount of energy as a modest mass of high explosives - which is expensive for large parts. On the other hand, the electrohydraulic method was seen as better suited to automation because of the fine control of multiple, sequential energy discharges and the relative compactness of the electrode-media containment system.
Advantages of EHF


Liquid Impact Forming

Liquid Impact Forming is a metalworking process in which the combined use of a stamping press and a liquid medium forms the desired shape on the workpiece. This technique is a synthesis of two metalworking processes; stamping (metalworking) and hydroforming. It is especially suited for the cold forming of tubular structural parts in automotive, railroad and aerospace industries.
The process is based on a patent by Stanley Ash from the Greenville Tool & Die Company in Greenville, Michigan.
Process.
Liquid impact forming uses the principles of hydroforming process with conventional stamping equipment. Even though hydroforming offers great advantages over conventional tube stamping through the reduction of manufacturing steps and the reduction of variation in workpieces, it still requires expensive mechanical equipment such as dies to withstand extreme pressures and pressurizing equipment such as pumps and intensifiers. As an alternative to this, the liquid impact forming utilizes the increase in the internal pressure of the liquid inside of a tube during the stamping process, eliminating the need for the use of above mentioned equipment.
The process includes the following stages:
1. A metal tube is filled with a liquid, preferably water and placed between lower and upper die sections of stamping dies.
2. The ends of the liquid-filled tube are sealed to confine the liquid within the tube at approximately atmospheric pressure.
3. The liquid-filled sealed tube is stamped in a conventional die to form the tube into a desired configuration, such as a box-shaped structural member. The compressive forces produced as the die closes to form stamped tube also compress the liquid within the interior of the sealed tube as it changes shape. Thus, the pressure of the liquid increases as the die closes. As the liquid resists compression, it forces the tube walls outwardly toward the interior surface of the die cavity. Once die sections are fully closed around the sealed tube, the tube walls take the shape of the die cavity.
4. The remaining liquid is drained from the formed tube.
Applications and Variations.
The liquid impact forming process is especially advantageous for the cold forming of tubular structural parts in automotive, railroad and aerospace industries. It may be used in the cold forming of cylindrical or non-cylindrical parts. It is limited for the applications requiring extensive metal flow or bulging because of the absence of external pressure utilization as in hydroforming.
One further variation of the liquid impact forming comprises the use of a change-of-state material in liquid state in order to prevent the tube wall from buckling or wrinkling during piercing. The change-of-state material can be water or a metallic lead-bismuth alloy. The liquid can be frozen before the stamping phase. After stamping, the liquid can be melted and drained from the shaped tube. Other applications of the process would be the piercing or bulging of tubes, which could also include the use of the change-of-state material.


Precision Time Protocol Industry Profile

Industrial automation systems consisting of several distributed controllers need a precise synchronization for commands, events and process data.
For instance, motors for newspaper printing are synchronized within some 5 microseconds to ensure that the color pixels in the different cylinders come within 0.1 mm at a paper speed of some 20 m/s. Similar requirements exist in high-power semiconductors (e.g. for converting between AC and DC grids) and in drive-by-wire vehicles (e.g. cars with no mechanical steering wheel).
This synchronisation is provided by the communication network, in most cases Industrial Ethernet.
Many ad-hoc synchronization schemes exist, so IEEE published a standard Precision Time Protocol IEEE 1588 or "PTP", which allows sub-microsecond synchronization of clocks.
PTP is formulated generally, so concrete applications need a stricter profile. In particular, PTP does not specify how the clocks should operate when the network is duplicated for better resilience to failures.
The PTP Industrial Profile (PIP) is a standard of the IEC 62439-3 that specifies in its Annex C two Precision Time Protocol IEEE 1588 / IEC 61588 profiles, L3E2E and L2P2P, to synchronize network clocks with an accuracy of 1 μs and provide fault-tolerance against clock failures.
The IEC 62439-3 PTP profiles are applicable to most Industrial Ethernet networks, for synchronized drives, robotics, vehicular technology and other applications that require precise time distribution, not necessarily using redundant networks.
The IEC 62439-3 profile L2P2P has been adopted as IEC/IEEE 61850-9-3 by the power utility industry to support precise time stamping of voltage and current measurement for differential protection, wide area monitoring and protection, busbar protection and event recording.
The IEC 62439-3 PTP profiles can be used to ensure deterministic operation of critical functions in the automation system itself, for instance precise starting of tasks, resource reservation and deadline supervision.
The IEC 62439-3 Annexes belongs to the Parallel Redundancy Protocol and High-availability Seamless Redundancy standard suite for high availability automation networks. However, this specification also applies to networks that have no redundancy and do not use PRP or HSR.
Topology.
The PIP relies on the IEEE 1588 topology, consisting of grandmaster clocks (GC), ordinary clocks (OC), boundary clocks (BC), transparent clocks (TC) and hybrid clocks (HC = TC&OC).
For redundancy, a PIP network contains several clocks that are master-capable. Normally, the best master clock ensures that only one grandmaster broadcasts the time.
In redundant networks, and especially in PRP, several masters can be active at the same time, the slave then chooses its master.
Main features.
IEC 62439-3 Annex C uses the following IEEE Std 1588 options:
Performance.
IEC 62439-3 Annex C aims at an accuracy of better than 1 μs after crossing 15 bridges with transparent clocks.
It assumes that all network elements (bridges, routers, media converters, links) support PTP with a given performance:
By relying on these guaranteed values, the network engineer can calculate the time inaccuracy at different nodes of the network and place the clocks, especially the grandmaster clocks suitably.
IEC TR 61850-90-4 (Network engineering guidelines) gives advice on the use of IEC/IEEE 61850-9-3 in substation automation networks.
IEEE 1588 settings.
IEC 62439-3 Annex C restricts the parameters of IEEE Std 1588 to the following values:
Additions to IEEE Std 1588.
IEC 62439-3 Annex C specifies requirements in addition to IEEE 1588:
Standard owners.
This protocol has been developed by the IEC SC65C WG15 in the framework of IEC 62439, which applies to all IEC industrial networks.
To avoid parallel standards in IEC and IEEE in the field of grid automation, the L2PTP profile specific to grid automation previous IEC 62439-3 Annex B has been placed under the umbrella of the IEC&IEEE Joint Development 61850-9-3.
Technical responsibility rests with IEC SC65C WG15, which is committed to keep the IEC 62439-3 profile L2P2P and IEC/IEEE 61850-9-3 aligned.


Coefficient diagram method

In control theory, the coefficient diagram method (CDM) is an algebraic approach applied to a polynomial loop in the parameter space, where a special diagram called a ""coefficient diagram" is used as the vehicle to carry the necessary information, and as the criterion of good design. The performance of the closed loop system is monitored by the coefficient diagram.
The most considerable advantages of CDM can be listed as follows:
It is usually required that the controller for a given plant should be designed under some practical limitations.
The controller is desired to be of minimum degree, minimum phase (if possible) and stable. It must have enough bandwidth and power rating limitations. If the controller is designed without considering these limitations, the robustness property will be very poor, even though the stability and time response requirements are met. CDM controllers designed while considering all these problems is of the lowest degree, has a convenient bandwidth and results with a unit step time response without an overshoot. These properties guarantee the robustness, the sufficient damping of the disturbance effects and the low economic property.
Although the main principles of CDM have been known since the 1950s, the first systematic method was proposed by Shunji Manabe. He developed a new method that easily builds a target characteristic polynomial to meet the desired time response. CDM is an algebraic approach combining classical and modern control theories and uses polynomial representation in the mathematical expression. The advantages of the classical and modern control techniques are integrated with the basic principles of this method, which is derived by making use of the previous experience and knowledge of the controller design. Thus, an efficient and fertile control method has appeared as a tool with which control systems can be designed without needing much experience and without confronting many problems.
Many control systems have been designed successfully using CDM. It is very easy to design a controller under the conditions of stability, time domain performance and robustness. The close relations between these conditions and coefficients of the characteristic polynomial can be simply determined. This means that CDM is effective not only for control system design but also for controller parameters tuning.


Rule-based DFM analysis for direct metal laser sintering

Rule based DFM analysis for direct metal laser sintering. Direct metal laser sintering (DMLS) is one type of additive manufacturing process that allows layer by layer printing of metal parts having complex geometries directly from 3D CAD data. It uses a high-energy laser to sinter powdered metal under computer control, binding the material together to create a solid structure. DMLS is a net shape process and allows the creation of highly complex and customized parts with no extra cost incurred for its complexity.
DMLS is being used to fabricate complex metal parts that are difficult to do so using traditional manufacturing processes thus gives immense freedom to the designer while designing the component. However, there are certain Design for Manufacturability (DFM) considerations that should be taken care of while designing the parts to be printed. DFM provides guidance to the design team in making the product structure more compliant to the given manufacturing process. It removes the wall between the designing and manufacturing phases of product development thus enables designers to take advantages of all the inherent costs and other benefits available in the manufacturing process. The early considerations of DFM principles and guidelines can lead to significant cost and time cutting in the final development of the product. Some of the common guidelines for DMLS are:
Size.
The size of the part that can be printed depends upon the printer that is being used. With the current technology a maximum build size of 228 X 228 X 304 mm can be achieved. Hence, the size of the part to be printed should be within required dimensions. DMLS has a minimum sintering width (depends on laser diameter) varying from 0.6 mm to 0.9 mm. This defines the minimum external feature size of the part and thus the design with any external features having smaller dimensions must be avoided.
Accuracy.
The accuracy and surface roughness of the part depends on the powder grain size ranging between 50 μm to 100 μm. The layer thickness which lies between 0.02 mm and 0.05 mm determines the resolution in the vertical direction. Therefore, the regions of the parts which require high accuracy should be designed with planned allowance of 0.1 mm to 0.5 mm and secondary finishing and/or machining operations should then be used to achieve the required accuracy.
Overhangs.
In DMLS, powder bed supports the parts and keep them held in place. However, support structures are explicitly required for most of the downward facing surfaces that make an angle less than 45 degrees with the powder bed. This is because powder bed alone is not sufficient enough to hold the liquid phase of the metal that is created when laser is scanning the powder. Support structures are also required to restrict curling/warping of the melted powder due to high-temperature gradients. The overhangs having angles less than 45 degrees should be avoided if possible at the design stage. The main advantage of this is to reduce material usage and the post processing requirement of removing support structures from the designed components.
Height.
The total number of layers required to build the whole part is directly proportional to the height of the part measured along the build direction. Every layer of the part to be printed requires tightly laying compacted thin layer of powdered material using roller, tracing of laser according to the 3D data fed to the machine in the horizontal plane and incremental lowering of powder bed for the successive layer to be laid. These processes require a significant amount of time thus redesigning the product for smaller heights may save manufacturing time greatly. The build orientation should be such that the height of the part should be least along the build direction.
Anisotropy.
The main direction of heat flow which is generated by the laser at the top is along the build direction due to the fact that powder bed lying at the bottom is the major heat sink. The layered addition of material and the directional heat flow in DMLS lead to the growing of microstructural grains along the build direction leading to anisotropic properties. The structure printed through DMLS has weaker properties along the build direction. This anisotropy can be removed using heat treatment methods but they are highly energy intensive and costly processes. Hence, it is advisable to consider the anisotropy in the very beginning of designing such structural parts and the direction of largest stress in the structure should lie in the horizontal plane.
Complexity.
Being an additive manufacturing technique, DMLS doesn't incur any extra cost for the complexity of the part. The build volume along with the number of layers is what determines the production cost and time. DMLS eliminates the need for tool production however such technologies are impervious to economies of scale. Therefore, it is recommended to design parts with least amount of superfluous volumes, building only the relevant geometries. Furthermore, the parts should be designed to avoid assembly requirements because printing sub-assembly with intricate geometries is now possible.


Logical clock

A logical clock is a mechanism for capturing chronological and causal relationships in a distributed system. Often, distributed systems may have no physically synchronous global clock. In many applications (such as distributed GNU make), if two processes never interact, the lack of synchronization is unobservable and in these applications it is enough for the processes to agree on the event ordering (i.e., logical clock) rather than the wall-clock time. The first logical clock implementation, the Lamport timestamps, was proposed by Leslie Lamport in 1978 (Turing Award in 2013).
Local vs global time.
In logical clock systems each process has two data structures: "logical local time" and "logical global time". Logical local time is used by the process to mark its own events, and logical global time is the local information about global time. A special protocol is used to update logical local time after each local event, and logical global time when processes exchange data.
Applications.
Logical clocks are useful in computation analysis, distributed algorithm design, individual event tracking, and exploring computational progress.
Algorithms.
Some noteworthy logical clock algorithms are: