Ranking is a central task in machine learning and information retrieval.

In this task, it is especially important to present the user with a slate of items that is appealing as a whole.

This in turn requires taking into account interactions between items, since intuitively, placing an item on the slate affects the decision of which other items should be chosen alongside it.

In this work, we propose a sequence-to-sequence model for ranking called seq2slate.

At each step, the model predicts the next item to place on the slate given the items already chosen.

The recurrent nature of the model allows complex dependencies between items to be captured directly in a flexible and scalable way.

We show how to learn the model end-to-end from weak supervision in the form of easily obtained click-through data.

We further demonstrate the usefulness of our approach in experiments on standard ranking benchmarks as well as in a real-world recommendation system.

A pointer network architecture for re-ranking items, learned from click-through logs.

Seq2Slate: Re-ranking and Slate Optimization with RNNs


While momentum-based methods, in conjunction with the stochastic gradient descent, are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods.

In practice, the momentum parameter is often chosen in a heuristic fashion with little theoretical guidance.

In this work, we use the framework of algorithmic stability to provide an upper-bound on the generalization error for the class of strongly convex loss functions, under mild technical assumptions.

Our bound decays to zero inversely with the size of the training set, and increases as the momentum parameter is increased.

We also develop an upper-bound on the expected true risk,  in terms of the number of training steps, the size of the training set, and the momentum parameter.

Stochastic gradient method with momentum generalizes.

Stability of Stochastic Gradient Method with Momentum for Strongly Convex Loss Functions


Imitation learning from demonstrations usually relies on learning a policy from trajectories of optimal states and actions.

However, in real life expert demonstrations, often the action information is missing and only state trajectories are available.

We present a model-based imitation learning method that can learn environment-specific optimal actions only from expert state trajectories.

Our proposed method starts with a model-free reinforcement learning algorithm with a heuristic reward signal to sample environment dynamics, which is then used to train the state-transition probability.

Subsequently, we learn the optimal actions from expert state trajectories by supervised learning, while back-propagating the error gradients through the modeled environment dynamics.

Experimental evaluations show that our proposed method successfully achieves performance similar to (state, action) trajectory-based traditional imitation learning methods even in the absence of action information, with much fewer iterations compared to conventional model-free reinforcement learning methods.

We also demonstrate that our method can learn to act from only video demonstrations of expert agent for simple games and can learn to achieve desired performance in less number of iterations.

Learning to imitate an expert in the absence of optimal actions learning a dynamics model while exploring the environment.

Model-based imitation learning from state trajectories


Recent research has proposed the lottery ticket hypothesis, suggesting that for a deep neural network, there exist trainable sub-networks performing equally or better than the original model with commensurate training steps.

While this discovery is insightful, finding proper sub-networks requires iterative training and pruning.

The high cost incurred limits the applications of the lottery ticket hypothesis.

We show there exists a subset of the aforementioned sub-networks that converge significantly faster during the training process and thus can mitigate the cost issue.

We conduct extensive experiments to show such sub-networks consistently exist across various model structures for a restrictive setting of hyperparameters (e.g., carefully selected learning rate, pruning ratio, and model capacity).

As a practical application of our findings, we demonstrate that such sub-networks can help in cutting down the total time of adversarial training, a standard approach to improve robustness, by up to 49% on CIFAR-10 to achieve the state-of-the-art robustness.

We show the possibility of pruning to find a small sub-network with significantly higher convergence rate than the full model.

Boosting Ticket: Towards Practical Pruning for Adversarial Training with Lottery Ticket Hypothesis


Disentangled representations, where the higher level data generative factors are reflected in disjoint latent dimensions, offer several benefits such as ease of deriving invariant representations, transferability to other tasks, interpretability, etc.

We consider the problem of unsupervised learning of disentangled representations from large pool of unlabeled observations, and propose a variational inference based approach to infer disentangled latent factors.

We introduce a regularizer on the expectation of the approximate posterior over observed data that encourages the disentanglement.

We also propose a new disentanglement metric which is better aligned with the qualitative disentanglement observed in the decoder's output.

We empirically observe significant improvement over existing methods in terms of both disentanglement and data likelihood (reconstruction quality).

We propose a variational inference based approach for encouraging the inference of disentangled latents. We also propose a new metric for quantifying disentanglement.

Variational Inference of Disentangled Latent Concepts from Unlabeled Observations

inference disentangle latent
variational


In multiagent systems (MASs), each agent makes individual decisions but all of them contribute globally to the system evolution.

Learning in MASs is difficult since each agent's selection of actions must take place in the presence of other co-learning agents.

Moreover, the environmental stochasticity and uncertainties increase exponentially with the increase in the number of agents.

Previous works borrow various multiagent coordination mechanisms into deep learning architecture to facilitate multiagent coordination.

However, none of them explicitly consider action semantics between agents that different actions have different influences on other agents.

In this paper, we propose a novel network architecture, named Action Semantics Network (ASN), that explicitly represents such action semantics between agents.

ASN can be easily combined with existing deep reinforcement learning (DRL) algorithms to boost their performance.

Experimental results on StarCraft II micromanagement and Neural MMO show ASN significantly improves the performance of state-of-the-art DRL approaches compared with several network architectures.

Our proposed ASN characterizes different actions' influence on other agents using neural networks based on the action semantics between them.

Action Semantics Network: Considering the Effects of Actions in Multiagent Systems


Spiking neural networks are being investigated both as biologically plausible models of neural computation and also as a potentially more efficient type of neural network.

While convolutional spiking neural networks have been demonstrated to achieve near state-of-the-art performance, only one solution has been proposed to convert gated recurrent neural networks, so far.

Recurrent neural networks in the form of networks of gating memory cells have been central in state-of-the-art solutions in problem domains that involve sequence recognition or generation.

Here, we design an analog gated LSTM cell where its neurons can be substituted for efficient stochastic spiking neurons.

These adaptive spiking neurons implement an adaptive form of sigma-delta coding to convert internally computed analog activation values to spike-trains.

For such neurons, we approximate the effective activation function, which resembles a sigmoid.

We show how analog neurons with such activation functions can be used to create an analog LSTM cell; networks of these cells can then be trained with standard backpropagation.

We train these LSTM networks on a noisy and noiseless version of the original sequence prediction task from Hochreiter & Schmidhuber (1997), and also on a noisy and noiseless version of a classical working memory reinforcement learning task, the T-Maze.

Substituting the analog neurons for corresponding adaptive spiking neurons, we then show that almost all resulting spiking neural network equivalents correctly compute the original tasks.

We demonstrate a gated recurrent asynchronous spiking neural network that corresponds to an LSTM unit.

Gating out sensory noise in a spike-based Long Short-Term Memory network


CNNs are widely successful in recognizing human actions in videos, albeit with a great cost of computation.

This cost is significantly higher in the case of long-range actions, where a video can span up to a few minutes, on average.

The goal of this paper is to reduce the computational cost of these CNNs, without sacrificing their performance.

We propose VideoEpitoma,  a neural network architecture comprising two modules: a timestamp selector and a video classifier.

Given a long-range video of thousands of timesteps, the selector learns to choose only a few but most representative timesteps for the video.

This selector resides on top of a lightweight CNN such as MobileNet and uses a novel gating module to take a binary decision: consider or discard a video timestep.

This decision is conditioned on both the timestep-level feature and the video-level consensus.

A heavyweight CNN model such as I3D takes the selected frames as input and performs video classification.

Using off-the-shelf video classifiers, VideoEpitoma reduces the computation by up to 50\% without compromising the accuracy.

In addition, we show that if trained end-to-end, the selector learns to make better choices to the benefit of the classifier, despite the selector and the classifier residing on two different CNNs.

Finally, we report state-of-the-art results on two datasets for long-range action recognition: Charades and Breakfast Actions, with much-reduced computation.

In particular, we match the accuracy of I3D by using less than half of the computation.

Efficient video classification using frame-based conditional gating module for selecting most-dominant frames, followed by temporal modeling and classifier.

VideoEpitoma: Efficient Recognition of Long-range Actions


Human annotation for syntactic parsing is expensive, and large resources are available only for a  fraction of languages.

A question we ask is whether one can leverage abundant unlabeled texts to improve syntactic parsers, beyond just using the texts to obtain more generalisable lexical features (i.e. beyond word embeddings).

To this end, we propose a novel latent-variable generative model for semi-supervised syntactic dependency parsing.

As exact inference is intractable, we introduce a differentiable relaxation to obtain approximate samples and compute gradients with respect to the parser parameters.

Our method (Differentiable Perturb-and-Parse) relies on differentiable dynamic programming over stochastically perturbed edge scores.

We demonstrate effectiveness of our approach with experiments on English, French and Swedish.

Differentiable dynamic programming over perturbed input weights with application to semi-supervised VAE

Differentiable Perturb-and-Parse: Semi-Supervised Parsing with a Structured Variational Autoencoder


DeConvNet, Guided BackProp, LRP, were invented to better understand deep neural networks.

We show that these methods do not produce the theoretically correct explanation for a linear model.

Yet they are used on multi-layer networks with millions of parameters.

This is a cause for concern since linear models are simple neural networks.

We argue that explanation methods for neural nets should work reliably in the limit of  simplicity, the linear models.

Based on our analysis of linear models we propose a generalization that yields two explanation techniques (PatternNet and PatternAttribution) that are theoretically sound for linear models and produce improved explanations for deep networks.

Without learning, it is impossible to explain a machine learning model's decisions.

Learning how to explain neural networks: PatternNet and PatternAttribution


Graph neural networks have shown promising results on representing and analyzing diverse graph-structured data such as social, citation, and protein interaction networks.

Existing approaches commonly suffer from the oversmoothing issue, regardless of whether policies are edge-based or node-based for neighborhood aggregation.

Most methods also focus on transductive scenarios for fixed graphs, leading to poor generalization performance for unseen graphs.

To address these issues, we propose a new graph neural network model that considers both edge-based neighborhood relationships and node-based entity features, i.e. Graph Entities with Step Mixture via random walk (GESM).

GESM employs a mixture of various steps through random walk to alleviate the oversmoothing problem and attention to use node information explicitly.

These two mechanisms allow for a weighted neighborhood aggregation which considers the properties of entities and relations.

With intensive experiments, we show that the proposed GESM achieves state-of-the-art or comparable performances on four benchmark graph datasets comprising transductive and inductive learning tasks.

Furthermore, we empirically demonstrate the significance of considering global information.

The source code will be publicly available in the near future.

Simple and effective graph neural network with mixture of random walk steps and attention

GRAPHS, ENTITIES, AND STEP MIXTURE


Basis pursuit is a compressed sensing optimization in which the l1-norm is minimized subject to model error constraints.

Here we use a deep neural network prior instead of l1-regularization.

Using known noise statistics, we jointly learn the prior and reconstruct images without access to ground-truth data.

During training, we use alternating minimization across an unrolled iterative network and jointly solve for the neural network weights and training set image reconstructions.

At inference, we fix the weights and pass the measurements through the network.

We compare reconstruction performance between unsupervised and supervised (i.e. with ground-truth) methods.

We hypothesize this technique could be used to learn reconstruction when ground-truth data are unavailable, such as in high-resolution dynamic MRI.

We present an unsupervised deep learning reconstruction for imaging inverse problems that combines neural networks with model-based constraints.

Unsupervised Deep Basis Pursuit: Learning inverse problems without ground-truth data


Deep learning approaches usually require a large amount of labeled data to generalize.

However, humans can learn a new concept only by a few samples.

One of the high cogntition human capablities is to learn several concepts at the same time.

In this paper, we address the task of classifying multiple objects by seeing only a few samples from each category.

To the best of authors' knowledge, there is no dataset specially designed for few-shot multiclass classification.

We design a task of mutli-object few class classification and an environment for easy creating controllable datasets for this task.

We demonstrate that the proposed dataset is sound using a method which is an extension of prototypical networks.

We introduce a diagnostic task which is a variation of few-shot learning and introduce a dataset for it.

Multi-Class Few Shot Learning Task and Controllable Environment


We present a new approach to defining a sequence loss function to train a summarizer by using a secondary encoder-decoder as a loss function, alleviating a shortcoming of word level training for sequence outputs.

The technique is based on the intuition that if a summary is a good one, it should contain the most essential information from the original article, and therefore should itself be a good input sequence, in lieu of the original, from which a summary can be generated.

We present experimental results where we apply this additional loss function to a general abstractive summarizer on a news summarization dataset.

The result is an improvement in the ROUGE metric and an especially large improvement in human evaluations, suggesting enhanced performance that is competitive with specialized state-of-the-art models.

We present the use of a secondary encoder-decoder as a loss function to help train a summarizer.

Encoder-decoder Network as Loss Function for Summarization


Existing unsupervised video-to-video translation methods fail to produce translated videos which are frame-wise realistic, semantic information preserving and video-level consistent.

In this work, we propose a novel unsupervised video-to-video translation model.

Our model decomposes the style and the content, uses specialized encoder-decoder structure and propagates the inter-frame information through bidirectional recurrent neural network (RNN) units.

The style-content decomposition mechanism enables us to achieve long-term style-consistent video translation results as well as provides us with a good interface for modality flexible translation.

In addition, by changing the input frames and style codes incorporated in our translation, we propose a video interpolation loss, which captures temporal information within the sequence to train our building blocks in a self-supervised manner.

Our model can produce photo-realistic, spatio-temporal consistent translated videos in a multimodal way.

Subjective and objective experimental results validate the superiority of our model over the existing methods.

A temporally consistent and modality flexible unsupervised video-to-video translation framework trained in a self-supervised manner.

Unsupervised  Video-to-Video Translation via Self-Supervised Learning


Providing transparency of AI planning systems is crucial for their success in practical applications.

In order to create a transparent system, a user must be able to query it for explanations about its outputs.

We argue that a key underlying principle for this is the use of causality within a planning model, and that argumentation frameworks provide an intuitive representation of such causality.

In this paper, we discuss how argumentation can aid in extracting causalities in plans and models, and how they can create explanations from them.

Argumentation frameworks are used to represent causality of plans/models to be utilized for explanations.

Towards an argumentation-based approach to explainable planning


Point clouds, as a form of Lagrangian representation, allow for powerful and flexible applications in a large number of computational disciplines.

We propose a novel deep-learning method to learn stable and temporally coherent feature spaces for points clouds that change over time.

We identify a set of inherent problems with these approaches: without knowledge of the time dimension, the inferred solutions can exhibit strong flickering, and easy solutions to suppress this flickering can result in undesirable local minima that manifest themselves as halo structures.

We propose a novel temporal loss function that takes into account higher time derivatives of the point positions, and encourages mingling, i.e., to prevent the aforementioned halos.

We combine these techniques in a super-resolution method with a truncation approach to flexibly adapt the size of the generated positions.

We show that our method works for large, deforming point sets from different sources to demonstrate the flexibility of our approach.

We propose a generative neural network approach for temporally coherent point clouds.

Tranquil Clouds: Neural Networks for Learning Temporally Coherent Features in Point Clouds

point cloud
temporally coherent
neural network


We study the problem of generating adversarial examples in a black-box setting in which only loss-oracle access to a model is available.

We introduce a framework that conceptually unifies much of the existing work on black-box attacks, and demonstrate that the current state-of-the-art methods are optimal in a natural sense.

Despite this optimality, we show how to improve black-box attacks by bringing a new element into the problem: gradient priors.

We give a bandit optimization-based algorithm that allows us to seamlessly integrate any such priors, and we explicitly identify and incorporate two examples.

The resulting methods use two to four times fewer queries and fail two to five times less than the current state-of-the-art.

The code for reproducing our work is available at https://git.io/fAjOJ.

We present a unifying view on black-box adversarial attacks as a gradient estimation problem, and then present a framework (based on bandits optimization) to integrate priors into gradient estimation, leading to significantly increased performance.

Prior Convictions: Black-box Adversarial Attacks with Bandits and Priors


Collecting high-quality, large scale datasets typically requires significant resources.

The aim of the present work is to improve the label efficiency of large neural networks operating on audio data through multitask learning with self-supervised tasks on unlabeled data.

To this end, we trained an end-to-end audio feature extractor based on WaveNet that feeds into simple, yet versatile task-specific neural networks.

We describe three self-supervised learning tasks that can operate on any large, unlabeled audio corpus.

We demonstrate that, in a scenario with limited labeled training data, one can significantly improve the performance of a supervised classification task by simultaneously training it with these additional self-supervised tasks.

We show that one can improve performance on a diverse sound events classification task by nearly 6\% when jointly trained with up to three distinct self-supervised tasks.

This improvement scales with the number of additional auxiliary tasks as well as the amount of unsupervised data.

We also show that incorporating data augmentation into our multitask setting leads to even further gains in performance.

Improving label efficiency through multi-task learning on auditory data

improve label efficiency multi-task learning auditory datum


Information need of humans is essentially multimodal in nature, enabling maximum exploitation of situated context.

We introduce a dataset for sequential procedural (how-to) text generation from images in cooking domain.

The dataset consists of 16,441 cooking recipes with 160,479 photos associated with different steps.

We setup a baseline motivated by the best performing model in terms of human evaluation for the Visual Story Telling (ViST) task.

In addition, we introduce two models to incorporate high level structure learnt by a Finite State Machine (FSM) in neural sequential generation process by: (1) Scaffolding Structure in Decoder (SSiD) (2) Scaffolding Structure in Loss (SSiL).

These models show an improvement in empirical as well as human evaluation.

Our best performing model (SSiL) achieves a METEOR score of 0.31, which is an improvement of 0.6 over the baseline model.

We also conducted human evaluation of the generated grounded recipes, which reveal that 61% found that our proposed (SSiL) model is better than the baseline model in terms of overall recipes, and 72.5% preferred our model in terms of coherence and structure.

We also discuss analysis of the output highlighting key important NLP issues for prospective directions.

The paper presents two techniques to incorporate high level structure in generating procedural text from a sequence of images.

Storyboarding of Recipes: Grounded Contextual Generation


Implicit models, which allow for the generation of samples but not for point-wise evaluation of probabilities, are omnipresent in real-world problems tackled by machine learning and a hot topic of current research.

Some examples include data simulators that are widely used in engineering and scientific research, generative adversarial networks (GANs) for image synthesis, and hot-off-the-press approximate inference techniques relying on implicit distributions.

The majority of existing approaches to learning implicit models rely on approximating the intractable distribution or optimisation objective for gradient-based optimisation, which is liable to produce inaccurate updates and thus poor models.

This paper alleviates the need for such approximations by proposing the \emph{Stein gradient estimator}, which directly estimates the score function of the implicitly defined distribution.

The efficacy of the proposed estimator is empirically demonstrated by examples that include meta-learning for approximate inference and entropy regularised GANs that provide improved sample diversity.

We introduced a novel gradient estimator using Stein's method, and compared with other methods on learning implicit models for approximate inference and image generation.

Gradient Estimators for Implicit Models


Noise injection is a fundamental tool for data augmentation, and yet there is no widely accepted procedure to incorporate it with learning frameworks.

This study analyzes the effects of adding or applying different noise models of varying magnitudes to Convolutional Neural Network (CNN) architectures.

Noise models that are distributed with different density functions are given common magnitude levels via Structural Similarity (SSIM) metric in order to create an appropriate ground for comparison.

The basic results are conforming with the most of the common notions in machine learning, and also introduces some novel heuristics and recommendations on noise injection.

The new approaches will provide better understanding on optimal learning procedures for image classification.

Ideal methodology to inject noise to input data during CNN training

Data Augmentation in Training CNNs: Injecting Noise to Images


State-of-the-art Unsupervised Domain Adaptation (UDA) methods learn transferable features by minimizing the feature distribution discrepancy between the source and target domains.

Different from these methods which do not model the feature distributions explicitly, in this paper, we explore explicit feature distribution modeling for UDA.

In particular, we propose Distribution Matching Prototypical Network (DMPN) to model the deep features from each domain as Gaussian mixture distributions.

With explicit feature distribution modeling, we can easily measure the discrepancy between the two domains.

In DMPN, we propose two new domain discrepancy losses with probabilistic interpretations.

The first one minimizes the distances between the corresponding Gaussian component means of the source and target data.

The second one minimizes the pseudo negative log likelihood of generating the target features from source feature distribution.

To learn both discriminative and domain invariant features, DMPN is trained by minimizing the classification loss on the labeled source data and the domain discrepancy losses together.

Extensive experiments are conducted over two UDA tasks.

Our approach yields a large margin in the Digits Image transfer task over state-of-the-art approaches.

More remarkably, DMPN obtains a mean accuracy of 81.4% on VisDA 2017 dataset.

The hyper-parameter sensitivity analysis shows that our approach is robust w.r.t hyper-parameter changes.

We propose to explicitly model deep feature distributions of source and target data as Gaussian mixture distributions for Unsupervised Domain Adaptation (UDA) and achieve superior results in multiple UDA tasks than state-of-the-art methods.

Distribution Matching Prototypical Network for Unsupervised Domain Adaptation

unsupervised domain adaptation
distribution


Efficiently learning to solve tasks in complex environments is a key challenge for reinforcement learning (RL) agents.

We propose to decompose a complex environment using a task-agnostic world graphs, an abstraction that accelerates learning by enabling agents to focus exploration on a subspace of the environment.The nodes of a world graph are important waypoint states and edges represent feasible traversals between them

.  Our framework has two learning phases

: 1) identifying world graph nodes and edges by training a binary recurrent variational auto-encoder (VAE) on trajectory data and

2) a hierarchical RL framework that leverages structural and connectivity knowledge from the learned world graph to bias exploration towards task-relevant waypoints and regions.

We show that our approach significantly accelerates RL on a suite of challenging 2D grid world tasks: compared to baselines, world graph integration doubles achieved rewards on simpler tasks, e.g. MultiGoal, and manages to solve more challenging tasks, e.g. Door-Key, where baselines fail.

We learn a task-agnostic world graph abstraction of the environment and show how using it for structured exploration can significantly accelerate downstream task-specific RL.

Learning World Graph Decompositions To Accelerate Reinforcement Learning


We introduce the notion of property signatures, a representation for programs and

program specifications meant for consumption by machine learning algorithms.

Given a function with input type τ_in and output type τ_out, a property is a function

of type: (τ_in, τ_out) → Bool that (informally) describes some simple property

For instance, if τ_in and τ_out are both lists

of the same type, one property might ask ‘is the input list the same length as the

If we have a list of such properties, we can evaluate them all for our

function to get a list of outputs that we will call the property signature.

we can ‘guess’ the property signature for a function given only a set of input/output

We discuss several potential applications of

property signatures and show experimentally that they can be used to improve

over a baseline synthesizer so that it emits twice as many programs in less than

We represent a computer program using a set of simpler programs and use this representation to improve program synthesis techniques.

Learning to Represent Programs with Property Signatures


Social dilemmas are situations where individuals face a temptation to increase their payoffs at a cost to total welfare.

Building artificially intelligent agents that achieve good outcomes in these situations is important because many real world interactions include a tension between selfish interests and the welfare of others.

We show how to modify modern reinforcement learning methods to construct agents that act in ways that are simple to understand, nice (begin by cooperating), provokable (try to avoid being exploited), and forgiving (try to return to mutual cooperation).

We show both theoretically and experimentally that such agents can maintain cooperation in Markov social dilemmas.

Our construction does not require training methods beyond a modification of self-play, thus if an environment is such that good strategies can be constructed in the zero-sum case (eg. Atari) then we can construct agents that solve social dilemmas in this environment.

How can we build artificial agents that solve social dilemmas (situations where individuals face a temptation to increase their payoffs at a cost to total welfare)?

Maintaining cooperation in complex social dilemmas using deep reinforcement learning


The reparameterization trick has become one of the most useful tools in the field of variational inference.

However, the reparameterization trick is based on the standardization transformation which restricts the scope of application of this method to distributions that have tractable inverse cumulative distribution functions or are expressible as deterministic transformations of such distributions.

In this paper, we generalized the reparameterization trick by allowing a general transformation.

Unlike other similar works, we develop the generalized transformation-based gradient model formally and rigorously.

We discover that the proposed model is a special case of control variate indicating that the proposed model can combine the advantages of CV and generalized reparameterization.

Based on the proposed gradient model, we propose a new polynomial-based gradient estimator which has better theoretical performance than the reparameterization trick under certain condition and can be applied to a larger class of variational distributions.

In studies of synthetic and real data, we show that our proposed gradient estimator has a significantly lower gradient variance than other state-of-the-art methods thus enabling a faster inference procedure.

We propose a novel generalized transformation-based gradient model and propose a polynomial-based gradient estimator based upon the model.

Generalized Transformation-based Gradient


The fault diagnosis in a modern communication system is traditionally supposed to be difficult, or even impractical for a purely data-driven machine learning approach, for it is a humanmade system of intensive knowledge.

A few labeled raw packet streams extracted from fault archive can hardly be sufficient to deduce the intricate logic of underlying protocols.

In this paper, we supplement these limited samples with two inexhaustible data sources: the unlabeled records probed from a system in service, and the labeled data simulated in an emulation environment.

To transfer their inherent knowledge to the target domain, we construct a directed information flow graph, whose nodes are neural network components consisting of two generators, three discriminators and one classifier, and whose every forward path represents a pair of adversarial optimization goals, in accord with the semi-supervised and transfer learning demands.

The multi-headed network can be trained in an alternative approach, at each iteration of which we select one target to update the weights along the path upstream, and refresh the residual layer-wisely to all outputs downstream.

The actual results show that it can achieve comparable accuracy on classifying Transmission Control Protocol (TCP) streams without deliberate expert features.

The solution has relieved operation engineers from massive works of understanding and maintaining rules, and provided a quick solution independent of specific protocols.

semi-supervised and transfer learning on packet flow classification, via a system of cooperative or adversarial neural blocks

Detecting Anomalies in Communication Packet Streams based on  Generative Adversarial Networks


Our work addresses two important issues with recurrent neural networks: (1) they are over-parameterized, and (2) the recurrent weight matrix is ill-conditioned.

The former increases the sample complexity of learning and the training time.

The latter causes the vanishing and exploding gradient problem.

We present a flexible recurrent neural network model called Kronecker Recurrent Units (KRU).

KRU achieves parameter efficiency in RNNs through a Kronecker factored recurrent matrix.

It overcomes the ill-conditioning of the recurrent matrix by enforcing soft unitary constraints on the factors.

Thanks to the small dimensionality of the factors, maintaining these constraints is computationally efficient.

Our experimental results on seven standard data-sets reveal that KRU can reduce the number of parameters by three orders of magnitude in the recurrent weight matrix compared to the existing recurrent models, without trading the statistical performance.

These results in particular show that while there are advantages in having a high dimensional recurrent space, the capacity of the recurrent part of the model can be dramatically reduced.

Out work presents a Kronecker factorization of recurrent weight matrices for parameter efficient and well conditioned recurrent neural networks.

Kronecker Recurrent Units


This paper studies the undesired phenomena of over-sensitivity of representations learned by deep networks to semantically-irrelevant changes in data.

We identify a cause for this shortcoming in the classical Variational Auto-encoder (VAE) objective, the evidence lower bound (ELBO).

We show that the ELBO fails to control the behaviour of the encoder out of the support of the empirical data distribution and this behaviour of the VAE can lead to extreme errors in the learned representation.

This is a key hurdle in the effective use of representations for data-efficient learning and transfer.

To address this problem, we propose to augment the data with specifications that enforce insensitivity of the representation with respect to families of transformations.

To incorporate these specifications, we propose a regularization method that is based on a selection mechanism that creates a fictive data point by explicitly perturbing an observed true data point.

For certain choices of parameters, our formulation naturally leads to the minimization of the entropy regularized Wasserstein distance between representations.

We illustrate our approach on standard datasets and experimentally show that significant improvements in the downstream adversarial accuracy can be achieved by learning robust representations completely in an unsupervised manner, without a reference to a particular downstream task and without a costly supervised adversarial training procedure.

We propose a method for computing adversarially robust representations in an entirely unsupervised way.

Adversarially Robust Representations with Smooth Encoders


We propose a novel score-based approach to learning a directed acyclic graph (DAG) from observational data.

We adapt a recently proposed continuous constrained optimization formulation to allow for nonlinear relationships between variables using neural networks.

This extension allows to model complex interactions while being more global in its search compared to other greedy approaches.

In addition to comparing our method to existing continuous optimization methods, we provide missing empirical comparisons to nonlinear greedy search methods.

On both synthetic and real-world data sets, this new method outperforms current continuous methods on most tasks while being competitive with existing greedy search methods on important metrics for causal inference.

We are proposing a new score-based approach to structure/causal learning leveraging neural networks and a recent continuous constrained formulation to this problem

Gradient-Based Neural DAG Learning


We study the problem of designing provably optimal adversarial noise algorithms that induce misclassification in settings where a learner aggregates decisions from multiple classifiers.

Given the demonstrated vulnerability of state-of-the-art models to adversarial examples, recent efforts within the field of robust machine learning have focused on the use of ensemble classifiers as a way of boosting the robustness of individual models.

In this paper, we design provably optimal attacks against a set of classifiers.

We demonstrate how this problem can be framed as finding strategies at equilibrium in a two player, zero sum game between a learner and an adversary and consequently illustrate the need for randomization in adversarial attacks.

The main technical challenge we consider is the design of best response oracles that can be implemented in a Multiplicative Weight Updates framework to find equilibrium strategies in the zero-sum game.

We develop a series of scalable noise generation algorithms for deep neural networks, and show that it outperforms state-of-the-art attacks on various image classification tasks.

Although there are generally no guarantees for deep learning, we show this is a well-principled approach in that it is provably optimal for linear classifiers.

The main insight is a geometric characterization of the decision space that reduces the problem of designing best response oracles to minimizing a quadratic function over a set of convex polytopes.

Paper analyzes the problem of designing adversarial attacks against multiple classifiers, introducing algorithms that are optimal for linear classifiers and which provide state-of-the-art results for deep learning.

Optimal Attacks against Multiple Classifiers


Multiagent systems where the agents interact among themselves and with an stochastic environment can be formalized as stochastic games.

We study a subclass of these games, named Markov potential games (MPGs), that appear often in economic and engineering applications when the agents share some common resource.

We consider MPGs with continuous state-action variables, coupled constraints and nonconvex rewards.

Previous analysis followed a variational approach that is only valid for very simple cases (convex rewards, invertible dynamics, and no coupled constraints); or considered deterministic dynamics and provided open-loop (OL) analysis, studying strategies that consist in predefined action sequences, which are not optimal for stochastic environments.

We present a closed-loop (CL) analysis for MPGs and consider parametric policies that depend on the current state and where agents adapt to stochastic transitions.

We provide easily verifiable, sufficient and necessary conditions for a stochastic game to be an MPG, even for complex parametric functions (e.g., deep neural networks); and show that a closed-loop Nash equilibrium (NE) can be found (or at least approximated) by solving a related optimal control problem (OCP).

This is useful since solving an OCP---which is a single-objective problem---is usually much simpler than solving the original set of coupled OCPs that form the game---which is a multiobjective control problem.

This is a considerable improvement over the previously standard approach for the CL analysis of MPGs, which gives no approximate solution if no NE belongs to the chosen parametric family, and which is practical only for simple parametric forms.

We illustrate the theoretical contributions with an example by applying our approach to a noncooperative communications engineering game.

We then solve the game with a deep reinforcement learning algorithm that learns policies that closely approximates an exact variational NE of the game.

We present general closed loop analysis for Markov potential games and show that deep reinforcement learning can be used for learning approximate closed-loop Nash equilibrium.

Learning Parametric Closed-Loop Policies for Markov Potential Games


We make the following striking observation: fully convolutional VAE models trained on 32x32 ImageNet can generalize well, not just to 64x64 but also to far larger photographs, with no changes to the model.

We use this property, applying fully convolutional models to lossless compression, demonstrating a method to scale the VAE-based 'Bits-Back with ANS' algorithm for lossless compression to large color photographs, and achieving state of the art for compression of full size ImageNet images.

We release Craystack, an open source library for convenient prototyping of lossless compression using probabilistic models, along with full implementations of all of our compression results.

We scale up lossless compression with latent variables, beating existing approaches on full-size ImageNet images.

HiLLoC: lossless image compression with hierarchical latent variable models

latent variable
compression
lossless
image


State of the art computer vision models have been shown to be vulnerable to small adversarial perturbations of the input.

In other words, most images in the data distribution are both correctly classified by the model and are very close to a visually similar misclassified image.

Despite substantial research interest, the cause of the phenomenon is still poorly understood and remains unsolved.

We hypothesize that this counter intuitive behavior is a naturally occurring result of the high dimensional geometry of the data manifold.

As a first step towards exploring this hypothesis, we study a simple synthetic dataset of classifying between two concentric high dimensional spheres.

For this dataset we show a fundamental tradeoff between the amount of test error and the average distance to nearest error.

In particular, we prove that any model which misclassifies a small constant fraction of a sphere will be vulnerable to adversarial perturbations of size $O(1/\sqrt{d})$.

Surprisingly, when we train several different architectures on this dataset, all of their error sets naturally approach this theoretical bound.

As a result of the theory, the vulnerability of neural networks to small adversarial perturbations is a logical consequence of the amount of test error observed.

We hope that our theoretical analysis of this very simple case will point the way forward to explore how the geometry of complex real-world data sets leads to adversarial examples.

We hypothesize that the vulnerability of image models to small adversarial perturbation is a naturally occurring result of the high dimensional geometry of the data manifold. We explore and theoretically prove this hypothesis for a simple synthetic dataset.

Adversarial Spheres


It has been established that diverse behaviors spanning the controllable subspace of a Markov decision process can be trained by rewarding a policy for being distinguishable from other policies.

However, one limitation of this formulation is the difficulty to generalize beyond the finite set of behaviors being explicitly learned, as may be needed in subsequent tasks.

Successor features provide an appealing solution to this generalization problem, but require defining the reward function as linear in some grounded feature space.

In this paper, we show that these two techniques can be combined, and that each method solves the other's primary limitation.

To do so we introduce Variational Intrinsic Successor FeatuRes (VISR), a novel algorithm which learns controllable features that can be leveraged to provide enhanced generalization and fast task inference through the successor features framework.

We empirically validate VISR on the full Atari suite, in a novel setup wherein the rewards are only exposed briefly after a long unsupervised phase.

Achieving human-level performance on 12 games and beating all baselines, we believe VISR represents a step towards agents that rapidly learn from limited feedback.

We introduce Variational Intrinsic Successor FeatuRes (VISR), a novel algorithm which learns controllable features that can be leveraged to provide fast task inference through the successor features framework.

Fast Task Inference with Variational Intrinsic Successor Features

variational intrinsic successor feature
fast task inference


Predicting structured outputs such as semantic segmentation relies on expensive per-pixel annotations to learn strong supervised models like convolutional neural networks.

However, these models trained on one data domain may not generalize well to other domains unequipped with annotations for model finetuning.

To avoid the labor-intensive process of annotation, we develop a domain adaptation method to adapt the source data to the unlabeled target domain.

To this end, we propose to learn discriminative feature representations of patches based on label histograms in the source domain, through the construction of a disentangled space.

With such representations as guidance, we then use an adversarial learning scheme to push the feature representations in target patches to the closer distributions in source ones.

In addition, we show that our framework can integrate a global alignment process with the proposed patch-level alignment and achieve state-of-the-art performance on semantic segmentation.

Extensive ablation studies and experiments are conducted on numerous benchmark datasets with various settings, such as synthetic-to-real and cross-city scenarios.

A domain adaptation method for structured output via learning patch-level discriminative feature representations

Domain Adaptation for Structured Output via Disentangled Patch Representations

structured output via
domain adaptation
representation


Many machine learning algorithms are vulnerable to almost imperceptible perturbations of their inputs.

So far it was unclear how much risk adversarial perturbations carry for the safety of real-world machine learning applications because most methods used to generate such perturbations rely either on detailed model information (gradient-based attacks) or on confidence scores such as class probabilities (score-based attacks), neither of which are available in most real-world scenarios.

In many such cases one currently needs to retreat to transfer-based attacks which rely on cumbersome substitute models, need access to the training data and can be defended against.

Here we emphasise the importance of attacks which solely rely on the final model decision.

Such decision-based attacks are (1) applicable to real-world black-box models such as autonomous cars, (2) need less knowledge and are easier to apply than transfer-based attacks and (3) are more robust to simple defences than gradient- or score-based attacks.

Previous attacks in this category were limited to simple models or simple datasets.

Here we introduce the Boundary Attack, a decision-based attack that starts from a large adversarial perturbation and then seeks to reduce the perturbation while staying adversarial.

The attack is conceptually simple, requires close to no hyperparameter tuning, does not rely on substitute models and is competitive with the best gradient-based attacks in standard computer vision tasks like ImageNet.

We apply the attack on two black-box algorithms from Clarifai.com.

The Boundary Attack in particular and the class of decision-based attacks in general open new avenues to study the robustness of machine learning models and raise new questions regarding the safety of deployed machine learning systems.

An implementation of the attack is available as part of Foolbox (https://github.com/bethgelab/foolbox).

A novel adversarial attack that can directly attack real-world black-box machine learning models without transfer.

Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models

black-box machine learning model
adversarial attack


We demonstrate that it is possible to train large recurrent language models with user-level differential privacy guarantees with only a negligible cost in predictive accuracy.

Our work builds on recent advances in the training of deep networks on user-partitioned data and privacy accounting for stochastic gradient descent.

In particular, we add user-level privacy protection to the federated averaging algorithm, which makes large step updates from user-level data.

Our work demonstrates that given a dataset with a sufficiently large number of users (a requirement easily met by even small internet-scale datasets), achieving differential privacy comes at the cost of increased computation, rather than in decreased utility as in most prior work.

We find that our private LSTM language models are quantitatively and qualitatively similar to un-noised models when trained on a large dataset.

User-level differential privacy for recurrent neural network language models is possible with a sufficiently large dataset.

Learning Differentially Private Recurrent Language Models


Convolutional neural networks (CNNs) are commonly trained using a fixed spatial image size predetermined for a given model.

Although trained on images of a specific size, it is well established that CNNs can be used to evaluate a wide range of image sizes at test time, by adjusting the size of intermediate feature maps.

In this work, we describe and evaluate a novel mixed-size training regime that mixes several image sizes at training time.

We demonstrate that models trained using our method are more resilient to image size changes and generalize well even on small images.

This allows faster inference by using smaller images at test time.

For instance, we receive a 76.43% top-1 accuracy using ResNet50 with an image size of 160, which matches the accuracy of the baseline model with 2x fewer computations.

Furthermore, for a given image size used at test time, we show this method can be exploited either to accelerate training or the final test accuracy.

For example, we are able to reach a 79.27% accuracy with a model evaluated at a 288 spatial size for a relative improvement of 14% over the baseline.

Training convnets with mixed image size can improve results across multiple sizes at evaluation

Mix & Match: training convnets with mixed image sizes for improved accuracy, speed and scale resiliency


We propose a simple technique for encouraging generative RNNs to plan ahead.

We train a ``backward'' recurrent network to generate a given sequence in reverse order, and we encourage states of the forward model to predict cotemporal states of the backward model.

The backward network is used only during training, and plays no role during sampling or inference.

We hypothesize that our approach eases modeling of long-term dependencies by implicitly forcing the forward states to hold information about the longer-term future (as contained in the backward states).

We show empirically that our approach achieves 9% relative improvement for a speech recognition task, and achieves significant improvement on a COCO caption generation task.

The paper introduces a method of training generative recurrent networks that helps to plan ahead. We run a second RNN in a reverse direction and make a soft constraint between cotemporal forward and backward states.

Twin Networks: Matching the Future for Sequence Generation


Deep generative models seek to recover the process with which the observed data was generated.

They may be used to synthesize new samples or to subsequently extract representations.

Successful approaches in the domain of images are driven by several core inductive biases.

However, a bias to account for the compositional way in which humans structure a visual scene in terms of objects has frequently been overlooked.

In this work we propose to structure the generator of a GAN to consider objects and their relations explicitly, and generate images by means of composition.

This provides a way to efficiently learn a more accurate generative model of real-world images, and serves as an initial step towards learning corresponding object representations.

We evaluate our approach on several multi-object image datasets, and find that the generator learns to identify and disentangle information corresponding to different objects at a representational level.

A human study reveals that the resulting generative model is better at generating images that are more faithful to the reference distribution.

We propose to structure the generator of a GAN to consider objects and their relations explicitly, and generate images by means of composition

A Case for Object Compositionality in Deep Generative Models of Images


Current literature in machine learning holds that unaligned, self-interested agents do not learn to use an emergent communication channel.

We introduce a new sender-receiver game to study emergent communication for this spectrum of partially-competitive scenarios and put special care into evaluation.

We find that communication can indeed emerge in partially-competitive scenarios, and we discover three things that are tied to improving it.

First, that selfish communication is proportional to cooperation, and it naturally occurs for situations that are more cooperative than competitive.

Second, that stability and performance are improved by using LOLA (Foerster et al, 2018), especially in more competitive scenarios.

And third, that discrete protocols lend themselves better to learning cooperative communication than continuous ones.

We manage to emerge communication with selfish agents, contrary to the current view in ML

Selfish Emergent Communication


Deep neural networks (DNNs) typically have enough capacity to fit random data by brute force even when conventional data-dependent regularizations focusing on the geometry of the features are imposed.

We find out that the reason for this is the inconsistency between the enforced geometry and the standard softmax cross entropy loss.

To resolve this, we propose a new framework for data-dependent DNN regularization, the Geometrically-Regularized-Self-Validating neural Networks (GRSVNet).

During training, the geometry enforced on one batch of features is simultaneously validated on a separate batch using a validation loss consistent with the geometry.

We study  a particular case of GRSVNet, the Orthogonal-Low-rank Embedding (OLE)-GRSVNet, which is capable of producing highly discriminative features residing in orthogonal low-rank subspaces.

Numerical experiments show that OLE-GRSVNet outperforms DNNs with conventional regularization when trained on real data.

More importantly, unlike conventional DNNs, OLE-GRSVNet refuses to memorize random data or random labels, suggesting it only learns intrinsic patterns by reducing the memorizing capacity of the baseline DNN.

we propose a new framework for data-dependent DNN regularization that can prevent DNNs from overfitting random data or random labels.

Stop memorizing: A data-dependent regularization framework for intrinsic pattern learning


End-to-end automatic speech recognition (ASR) commonly transcribes audio signals into sequences of characters while its performance is evaluated by measuring the word-error rate (WER).

This suggests that predicting sequences of words directly may be helpful instead.

However, training with word-level supervision can be more difficult due to the sparsity of examples per label class.

In this paper we analyze an end-to-end ASR model that combines a word-and-character representation in a multi-task learning (MTL) framework.

We show that it improves on the WER and study how the word-level model can benefit from character-level supervision by analyzing the learned inductive preference bias of each model component empirically.

We find that by adding character-level supervision, the MTL model interpolates between recognizing more frequent words (preferred by the word-level model) and shorter words (preferred by the character-level model).

Multi-task learning improves word-and-character-level speech recognition by interpolating the preference biases of its components: frequency- and word length-preference.

On the Inductive Bias of Word-Character-Level Multi-Task Learning for Speech Recognition

speech recognition
bias
multi-task learning


Discretizing floating-point vectors is a fundamental step of modern indexing methods.

State-of-the-art techniques learn parameters of the quantizers on training data for optimal performance, thus adapting quantizers to the data.

In this work, we propose to reverse this paradigm and adapt the data to the quantizer: we train a neural net whose last layers form a fixed parameter-free quantizer, such as pre-defined points of a sphere.

As a proxy objective, we design and train a neural network that favors uniformity in the spherical latent space, while preserving the neighborhood structure after the mapping.

For this purpose, we propose a new regularizer derived from the Kozachenko-Leonenko differential entropy estimator and combine it with a locality-aware triplet loss.

Experiments show that our end-to-end approach outperforms most learned quantization methods, and is competitive with the state of the art on widely adopted benchmarks.

Further more, we show that training without the quantization step results in almost no difference in accuracy, but yields a generic catalyser that can be applied with any subsequent quantization technique.

We learn a neural network that uniformizes the input distribution, which leads to competitive indexing performance in high-dimensional space

Spreading vectors for similarity search


Temporal difference (TD) learning is a popular algorithm for policy evaluation in reinforcement learning, but the vanilla TD can substantially suffer from the inherent optimization variance.

A variance reduced TD (VRTD) algorithm was proposed by Korda and La (2015), which applies the variance reduction technique directly to the online TD learning with Markovian samples.

In this work, we first point out the technical errors in the analysis of VRTD in Korda and La (2015), and then provide a mathematically solid analysis of the non-asymptotic convergence of VRTD and its variance reduction performance.

We show that VRTD is guaranteed to converge to a neighborhood of the fixed-point solution of TD at a linear convergence rate.

Furthermore, the variance error (for both i.i.d. and Markovian sampling) and the bias error (for Markovian sampling) of VRTD are significantly reduced by the batch size of variance reduction in comparison to those of vanilla TD.

This paper provides a rigorous study of the variance reduced TD learning and characterizes its advantage over vanilla TD learning

Reanalysis of Variance Reduced Temporal Difference Learning


We tackle unsupervised domain adaptation by accounting for the fact that different domains may need to be processed differently to arrive to a common feature representation effective for recognition.

To this end, we introduce a deep learning framework where each domain undergoes a different sequence of operations, allowing some, possibly more complex, domains to go through more computations than others.

This contrasts with  state-of-the-art domain adaptation techniques that force all domains to be processed with the same series of operations, even when using multi-stream architectures whose parameters are not shared.

As evidenced by our experiments, the greater flexibility of our method translates to higher accuracy.

Furthermore, it allows us to handle any number of domains simultaneously.

A Multiflow Network is a dynamic architecture for domain adaptation that learns potentially different computational graphs per domain, so as to map them to a common representation where inference can be performed in a domain-agnostic fashion.

Domain Adaptive Multibranch Networks


The practical usage of reinforcement learning agents is often bottlenecked by the duration of training time.

To accelerate training, practitioners often turn to distributed reinforcement learning architectures to parallelize and accelerate the training process.

However, modern methods for scalable reinforcement learning (RL) often tradeoff between the throughput of samples that an RL agent can learn from (sample throughput) and the quality of learning from each sample (sample efficiency).

In these scalable RL architectures, as one increases sample throughput (i.e. increasing parallelization in IMPALA (Espeholt et al., 2018)), sample efficiency drops significantly.

To address this, we propose a new distributed reinforcement learning algorithm, IMPACT.

IMPACT extends PPO with three changes: a target network for stabilizing the surrogate objective, a circular buffer, and truncated importance sampling.

In discrete action-space environments, we show that IMPACT attains higher reward and, simultaneously, achieves up to 30% decrease in training wall-time than that of IMPALA.

For continuous control environments, IMPACT trains faster than existing scalable agents while preserving the sample efficiency of synchronous PPO.

IMPACT helps RL agents train faster by decreasing training wall-clock time and increasing sample efficiency simultaneously.

IMPACT: Importance Weighted Asynchronous Architectures with Clipped Target Networks


In this paper, we show that a simple coloring scheme can improve, both theoretically and empirically, the expressive power of Message Passing Neural Networks (MPNNs).

More specifically, we introduce a graph neural network called Colored Local Iterative Procedure (CLIP) that uses colors to disambiguate identical node attributes, and show that this representation is a universal approximator of continuous functions on graphs with node attributes.

Our method relies on separability, a key topological characteristic that allows to extend well-chosen neural networks into universal representations.

Finally, we show experimentally that CLIP is capable of capturing structural characteristics that traditional MPNNs fail to distinguish, while being state-of-the-art on benchmark graph classification datasets.

This paper introduces a coloring scheme for node disambiguation in graph neural networks based on separability, proven to be a universal MPNN extension.

Coloring graph neural networks for node disambiguation

graph neural network
node disambiguation
color


In this paper we present a method for algorithmic melody generation using a generative adversarial network without recurrent components.

Music generation has been successfully done using recurrent neural networks, where the model learns sequence information that can help create authentic sounding melodies.

Here, we use DCGAN architecture with dilated convolutions and towers to capture sequential information as spatial image information, and learn long-range dependencies in fixed-length melody forms such as Irish traditional reel.

Representing melodies as images with semantic units aligned we can generate them using a DCGAN without any recurrent components.

Non-Sequential Melody Generation


Neural machine translation (NMT) systems have reached state of the art performance in translating text and widely deployed.

Yet little is understood about how these systems function or break.

Here we show that NMT systems are susceptible to producing highly pathological translations that are completely untethered from the source material, which we term hallucinations.

Such pathological translations are problematic because they are are deeply disturbing of user trust and easy to find.

We describe a method t generate hallucinations and show that many common variations of the NMT architecture are susceptible to them.

We study a variety of approaches to reduce the frequency of hallucinations, including data augmentation, dynamical systems and regularization techniques and show that data augmentation significantly reduces hallucination frequency.

Finally, we analyze networks that produce hallucinations and show signatures of hallucinations in the attention matrix and in the stability measures of the decoder.

We introduce and analyze the phenomenon of "hallucinations" in NMT, or spurious translations unrelated to source text, and propose methods to reduce its frequency.

Hallucinations in neural machine translation


For AI systems to garner widespread public acceptance, we must develop methods capable of explaining the decisions of black-box models such as neural networks.

In this work, we identify two issues of current explanatory methods.

First, we show that two prevalent perspectives on explanations—feature-additivity and feature-selection—lead to fundamentally different instance-wise explanations.

In the literature, explainers from different perspectives are currently being directly compared, despite their distinct explanation goals.

The second issue is that current post-hoc explainers have only been thoroughly validated on simple models, such as linear regression, and, when applied to real-world neural networks, explainers are commonly evaluated under the assumption that the learned models behave reasonably.

However, neural networks often rely on unreasonable correlations, even when producing correct decisions.

We introduce a verification framework for explanatory methods under the feature-selection perspective.

Our framework is based on a non-trivial neural network architecture trained on a real-world task, and for which we are able to provide guarantees on its inner workings.

We validate the efficacy of our evaluation by showing the failure modes of current explainers.

We aim for this framework to provide a publicly available,1 off-the-shelf evaluation when the feature-selection perspective on explanations is needed.

An evaluation framework based on a real-world neural network for post-hoc explanatory methods

Can I Trust the Explainer? Verifying Post-Hoc Explanatory Methods


Planning in high-dimensional space remains a challenging problem, even with recent advances in algorithms and computational power.

We are inspired by efference copy and sensory reafference theory from neuroscience.

Our aim is to allow agents to form mental models of their environments for planning.

The cerebellum is emulated with a two-stream, fully connected, predictor network.

The network receives as inputs the efference as well as the features of the current state.

Building on insights gained from knowledge distillation methods, we choose as our features the outputs of a pre-trained network,  yielding a compressed representation of the current state.

The representation is chosen such that it allows for fast search using classical graph search algorithms.

We display the effectiveness of our approach on a viewpoint-matching task using a modified best-first search algorithm.

We present a neuroscience-inspired method based on neural networks for latent space search

EfferenceNets for latent space planning


Experimental evidence indicates that simple models outperform complex deep networks on many unsupervised similarity tasks.

Introducing the concept of an optimal representation space, we provide a simple theoretical resolution to this apparent paradox.

In addition, we present a straightforward procedure that, without any retraining or architectural modifications, allows deep recurrent models to perform equally well (and sometimes better) when compared to shallow models.

To validate our analysis, we conduct a set of consistent empirical evaluations and introduce several new sentence embedding models in the process.

Even though this work is presented within the context of natural language processing, the insights are readily applicable to other domains that rely on distributed representations for transfer tasks.

By introducing the notion of an optimal representation space, we provide a theoretical argument and experimental validation that an unsupervised model for sentences can perform well on both supervised similarity and unsupervised transfer tasks.

Decoding Decoders: Finding Optimal Representation Spaces for Unsupervised Similarity Tasks

optimal representation space
unsupervised
similarity
task


Cloze test is widely adopted in language exams to evaluate students' language proficiency.

In this paper, we propose the first large-scale human-designed cloze test dataset CLOTH in which the questions were used in middle-school and high-school language exams.

With the missing blanks carefully created by teachers and candidate choices purposely designed to be confusing, CLOTH requires a deeper language understanding and a wider attention span than previous automatically generated cloze datasets.

We show humans outperform dedicated designed baseline models by a significant margin, even when the model is trained on sufficiently large external data.

We investigate the source of the performance gap, trace model deficiencies to some distinct properties of CLOTH, and identify the limited ability of comprehending a long-term context to be the key bottleneck.

In addition, we find that human-designed data leads to a larger gap between the model's performance and human performance when compared to automatically generated data.

A cloze test dataset designed by teachers to assess language proficiency

Large-scale Cloze Test Dataset Designed by Teachers


Recent work suggests goal-driven training of neural networks can be used to model neural activity in the brain.

While response properties of neurons in artificial neural networks bear similarities to those in the brain, the network architectures are often constrained to be different.

Here we ask if a neural network can recover both neural representations and, if the architecture is unconstrained and optimized, also the anatomical properties of neural circuits.

We demonstrate this in a system where the connectivity and the functional organization have been characterized, namely, the head direction circuit of the rodent and fruit fly.

We trained recurrent neural networks (RNNs) to estimate head direction through integration of angular velocity.

We found that the two distinct classes of neurons observed in the head direction system, the Ring neurons and the Shifter neurons, emerged naturally in artificial neural networks as a result of training.

Furthermore, connectivity analysis and in-silico neurophysiology revealed structural and mechanistic similarities between artificial networks and the head direction system.

Overall, our results show that optimization of RNNs in a goal-driven task can recapitulate the structure and function of biological circuits, suggesting that artificial neural networks can be used to study the brain at the level of both neural activity and anatomical organization.

Artificial neural networks trained with gradient descent are capable of recapitulating both realistic neural activity and the anatomical organization of a biological circuit.

Emergence of functional and structural properties of the head direction system by optimization of recurrent neural networks


Convolutional Neural Networks (CNNs) are computationally intensive, which limits their application on mobile devices.

Their energy is dominated by the number of multiplies needed to perform the convolutions.

Winograd’s minimal filtering algorithm (Lavin, 2015) and network pruning (Han et al., 2015) can reduce the operation count, but these two methods cannot be straightforwardly combined — applying the Winograd transform fills in the sparsity in both the weights and the activations.

We propose two modifications to Winograd-based CNNs to enable these methods to exploit sparsity.

First, we move the ReLU operation into the Winograd domain to increase the sparsity of the transformed activations.

Second, we prune the weights in the Winograd domain to exploit static weight sparsity.

For models on CIFAR-10, CIFAR-100 and ImageNet datasets, our method reduces the number of multiplications by 10.4x, 6.8x and 10.8x respectively with loss of accuracy less than 0.1%, outperforming previous baselines by 2.0x-3.0x.

We also show that moving ReLU to the Winograd domain allows more aggressive pruning.

Prune and ReLU in Winograd domain for efficient convolutional neural network

Efficient Sparse-Winograd Convolutional Neural Networks


In this paper we present a novel optimization algorithm called Advanced Neuroevolution.

The aim for this algorithm is to train deep neural networks, and eventually act as an alternative to Stochastic Gradient Descent (SGD) and its variants as needed.We evaluated our algorithm on the MNIST dataset, as well as on several global optimization problems such as the Ackley function.

We find the algorithm performing relatively well for both cases, overtaking other global optimization algorithms such as Particle Swarm Optimization (PSO) and Evolution Strategies (ES).

A new algorithm to train deep neural networks. Tested on optimization functions and MNIST.

Advanced Neuroevolution: A gradient-free algorithm to train Deep Neural Networks


Stochastic neural net weights are used in a variety of contexts, including regularization, Bayesian neural nets, exploration in reinforcement learning, and evolution strategies.

Unfortunately, due to the large number of weights, all the examples in a mini-batch typically share the same weight perturbation, thereby limiting the variance reduction effect of large mini-batches.

We introduce flipout, an efficient method for decorrelating the gradients within a mini-batch by implicitly sampling pseudo-independent weight perturbations for each example.

Empirically, flipout achieves the ideal linear variance reduction for fully connected networks, convolutional networks, and RNNs.

We find significant speedups in training neural networks with multiplicative Gaussian perturbations.

We show that flipout is effective at regularizing LSTMs, and outperforms previous methods.

Flipout also enables us to vectorize evolution strategies: in our experiments, a single GPU with flipout can handle the same throughput as at least 40 CPU cores using existing methods, equivalent to a factor-of-4 cost reduction on Amazon Web Services.

We introduce flipout, an efficient method for decorrelating the gradients computed by stochastic neural net weights within a mini-batch by implicitly sampling pseudo-independent weight perturbations for each example.

Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches

pseudo-independent weight perturbation
flipout
efficient


Deep generative models such as Variational AutoEncoder (VAE) and Generative Adversarial Network (GAN) play an increasingly important role in machine learning and computer vision.

However, there are two fundamental issues hindering their real-world applications: the difficulty of conducting variational inference in VAE and the functional absence of encoding real-world samples in GAN.

In this paper, we propose a novel algorithm named Latently Invertible Autoencoder (LIA) to address the above two issues in one framework.

An invertible network and its inverse mapping are symmetrically embedded in the latent space of VAE.

Thus the partial encoder first transforms the input into feature vectors and then the distribution of these feature vectors is reshaped to fit a prior by the invertible network.

The decoder proceeds in the reverse order of the encoder's composite mappings.

A two-stage stochasticity-free training scheme is designed to train LIA via adversarial learning, in the sense that the decoder of LIA is first trained as a standard GAN with the invertible network and then the partial encoder is learned from an autoencoder by detaching the invertible network from LIA.

Experiments conducted on the FFHQ face dataset and three LSUN datasets validate the effectiveness of LIA for inference and generation.

A new model Latently Invertible Autoencoder is proposed to solve the problem of variational inference in VAE using the invertible network and two-stage adversarial training.

LIA: Latently Invertible Autoencoder with Adversarial Learning

latently invertible autoencoder
adversarial


Neural programs are highly accurate and structured policies that perform algorithmic tasks by controlling the behavior of a computation mechanism.

Despite the potential to increase the interpretability and the compositionality of the behavior of artificial agents, it remains difficult to learn from demonstrations neural networks that represent computer programs.

The main challenges that set algorithmic domains apart from other imitation learning domains are the need for high accuracy, the involvement of specific structures of data, and the extremely limited observability.

To address these challenges, we propose to model programs as Parametrized Hierarchical Procedures (PHPs).

A PHP is a sequence of conditional operations, using a program counter along with the observation to select between taking an elementary action, invoking another PHP as a sub-procedure, and returning to the caller.

We develop an algorithm for training PHPs from a set of supervisor demonstrations, only some of which are annotated with the internal call structure, and apply it to efficient level-wise training of multi-level PHPs.

We show in two benchmarks, NanoCraft and long-hand addition, that PHPs can learn neural programs more accurately from smaller amounts of both annotated and unannotated demonstrations.

We introduce the PHP model for hierarchical representation of neural programs, and an algorithm for learning PHPs from a mixture of strong and weak supervision.

Parametrized Hierarchical Procedures for Neural Programming


Deep Reinforcement Learning has managed to achieve state-of-the-art results in learning control policies directly from raw pixels.

However, despite its remarkable success, it fails to generalize, a fundamental component required in a stable Artificial Intelligence system.

Using the Atari game Breakout, we demonstrate the difficulty of a trained agent in adjusting to simple modifications in the raw image, ones that a human could adapt to trivially.

In transfer learning, the goal is to use the knowledge gained from the source task to make the training of the target task faster and better.

We show that using various forms of fine-tuning, a common method for transfer learning, is not effective for adapting to such small visual changes.

In fact, it is often easier to re-train the agent from scratch than to fine-tune a trained agent.

We suggest that in some cases transfer learning can be improved by adding a dedicated component whose goal is to learn to visually map between the known domain and the new one.

Concretely, we use Unaligned Generative Adversarial Networks (GANs) to create a mapping function to translate images in the target task to corresponding images in the source task.

These mapping functions allow us to transform between various variations of the Breakout game, as well as between different levels of a Nintendo game, Road Fighter.

We show that learning this mapping is substantially more efficient than re-training.

A visualization of a trained agent playing Breakout and Road Fighter, with and without the GAN transfer, can be seen in \url{https://streamable.com/msgtm} and \url{https://streamable.com/5e2ka}.

We propose a method of transferring knowledge between related RL tasks using visual mappings, and demonstrate its effectiveness on visual variants of the Atari Breakout game and different levels of Road Fighter, a Nintendo car driving game.

Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation


A commonplace belief in the machine learning community is that using adaptive gradient methods hurts generalization.

We re-examine this belief both theoretically and experimentally, in light of insights and trends from recent years.

We revisit some previous oft-cited experiments and theoretical accounts in more depth, and provide a new set of experiments in larger-scale, state-of-the-art settings.

We conclude that with proper tuning, the improved training performance of adaptive optimizers does not in general carry an overfitting penalty, especially in contemporary deep learning.

Finally, we synthesize a ``user's guide'' to adaptive optimizers, including some proposed modifications to AdaGrad to mitigate some of its empirical shortcomings.

Adaptive gradient methods, when done right, do not incur a generalization penalty.

Revisiting the Generalization of Adaptive Gradient Methods


The ability to generalize quickly from few observations is crucial for intelligent systems.

In this paper we introduce APL, an algorithm that approximates probability distributions by remembering the most surprising observations it has encountered.

These past observations are recalled from an external memory module and processed by a decoder network that can combine information from different memory slots to generalize beyond direct recall.

We show this algorithm can perform as well as state of the art baselines on few-shot classification benchmarks with a smaller memory footprint.

In addition, its memory compression allows it to scale to thousands of unknown labels.

Finally, we introduce a meta-learning reasoning task which is more challenging than direct classification.

In this setting, APL is able to generalize with fewer than one example per class via deductive reasoning.

We introduce a model which generalizes quickly from few observations by storing surprising information and attending over the most relevant data at each time point.

Adaptive Posterior Learning: few-shot learning with a surprise-based memory module


Recent advances in recurrent neural nets (RNNs) have shown much promise in many applications in natural language processing.

For most of these tasks, such as sentiment analysis of customer reviews, a recurrent neural net model parses the entire review before forming a decision.

We argue that reading the entire input is not always necessary in practice, since a lot of reviews are often easy to classify, i.e., a decision can be formed after reading some crucial sentences or words in the provided text.

In this paper, we present an approach of fast reading for text classification.

Inspired by several well-known human reading techniques, our approach implements an intelligent recurrent agent which evaluates the importance of the current snippet in order to decide whether to make a prediction, or to skip some texts, or to re-read part of the sentence.

Our agent uses an RNN module to encode information from the past and the current tokens, and applies a policy module to form decisions.

With an end-to-end training algorithm based on policy gradient, we train and test our agent on several text classification datasets and achieve both higher efficiency and better accuracy compared to previous approaches.

We develop an end-to-end trainable approach for skimming, rereading and early stopping applicable to classification tasks.

Fast and Accurate Text Classification: Skimming, Rereading and Early Stopping


Reinforcement learning agents need to explore their unknown environments to solve the tasks given to them.

The Bayes optimal solution to exploration is intractable for complex environments, and while several exploration methods have been proposed as approximations, it remains unclear what underlying objective is being optimized by existing exploration methods, or how they can be altered to incorporate prior knowledge about the task.

Moreover, it is unclear how to acquire a single exploration strategy that will be useful for solving multiple downstream tasks.

We address these shortcomings by learning a single exploration policy that can quickly solve a suite of downstream tasks in a multi-task setting, amortizing the cost of learning to explore.

We recast exploration as a problem of State Marginal Matching (SMM), where we aim to learn a policy for which the state marginal distribution matches a given target state distribution, which can incorporate prior knowledge about the task.

We optimize the objective by reducing it to a two-player, zero-sum game between a state density model and a parametric policy.

Our theoretical analysis of this approach suggests that prior exploration methods do not learn a policy that does distribution matching, but acquire a replay buffer that performs distribution matching, an observation that potentially explains these prior methods' success in single-task settings.

On both simulated and real-world tasks, we demonstrate that our algorithm explores faster and adapts more quickly than prior methods.

We view exploration in RL as a problem of matching a marginal distribution over states.

Efficient Exploration via State Marginal Matching


The effectiveness of Convolutional Neural Networks stems in large part from their ability to exploit the translation invariance that is inherent in many learning problems.

Recently, it was shown that CNNs can exploit other invariances, such as rotation invariance, by using group convolutions instead of planar convolutions.

However, for reasons of performance and ease of implementation, it has been necessary to limit the group convolution to transformations that can be applied to the filters without interpolation.

Thus, for images with square pixels, only integer translations, rotations by multiples of 90 degrees, and reflections are admissible.

Whereas the square tiling provides a 4-fold rotational symmetry, a hexagonal tiling of the plane has a 6-fold rotational symmetry.

In this paper we show how one can efficiently implement planar convolution and group convolution over hexagonal lattices, by re-using existing highly optimized convolution routines.

We find that, due to the reduced anisotropy of hexagonal filters, planar HexaConv provides better accuracy than planar convolution with square filters, given a fixed parameter budget.

Furthermore, we find that the increased degree of symmetry of the hexagonal grid increases the effectiveness of group convolutions, by allowing for more parameter sharing.

We show that our method significantly outperforms conventional CNNs on the AID aerial scene classification dataset, even outperforming ImageNet pre-trained models.

We introduce G-HexaConv, a group equivariant convolutional neural network on hexagonal lattices.

HexaConv


Deep Convolutional Neural Networks (CNNs) have been repeatedly shown to perform well on image classification tasks, successfully recognizing a broad array of objects when given sufficient training data.

Methods for object localization, however, are still in need of substantial improvement.

Common approaches to this problem involve the use of a sliding window, sometimes at multiple scales, providing input to a deep CNN trained to classify the contents of the window.

In general, these approaches are time consuming, requiring many classification calculations.

In this paper, we offer a fundamentally different approach to the localization of recognized objects in images.

Our method is predicated on the idea that a deep CNN capable of recognizing an object must implicitly contain knowledge about object location in its connection weights.

We provide a simple method to interpret classifier weights in the context of individual classified images.

This method involves the calculation of the derivative of network generated activation patterns, such as the activation of output class label units, with regard to each in- put pixel, performing a sensitivity analysis that identifies the pixels that, in a local sense, have the greatest influence on internal representations and object recognition.

These derivatives can be efficiently computed using a single backward pass through the deep CNN classifier, producing a sensitivity map of the image.

We demonstrate that a simple linear mapping can be learned from sensitivity maps to bounding box coordinates, localizing the recognized object.

Our experimental results, using real-world data sets for which ground truth localization information is known, reveal competitive accuracy from our fast technique.

Proposing a novel object localization(detection) approach based on interpreting the deep CNN using internal representation and network's thoughts

FAST OBJECT LOCALIZATION VIA SENSITIVITY ANALYSIS


We present trellis networks, a new architecture for sequence modeling.

On the one hand, a trellis network is a temporal convolutional network with special structure, characterized by weight tying across depth and direct injection of the input into deep layers.

On the other hand, we show that truncated recurrent networks are equivalent to trellis networks with special sparsity structure in their weight matrices.

Thus trellis networks with general weight matrices generalize truncated recurrent networks.

We leverage these connections to design high-performing trellis networks that absorb structural and algorithmic elements from both recurrent and convolutional models.

Experiments demonstrate that trellis networks outperform the current state of the art methods on a variety of challenging benchmarks, including word-level language modeling and character-level language modeling tasks, and stress tests designed to evaluate long-term memory retention.

The code is available at https://github.com/locuslab/trellisnet .

Trellis networks are a new sequence modeling architecture that bridges recurrent and convolutional models and sets a new state of the art on word- and character-level language modeling.

Trellis Networks for Sequence Modeling


We propose an end-to-end framework for training domain specific models (DSMs) to obtain both high accuracy and computational efficiency for object detection tasks.

DSMs are trained with distillation and focus on achieving high accuracy at a limited domain (e.g. fixed view of an intersection).

We argue that DSMs can capture essential features well even with a small model size, enabling higher accuracy and efficiency than traditional techniques.

In addition, we improve the training efficiency by reducing the dataset size by culling easy to classify images from the training set.

For the limited domain, we observed that compact DSMs significantly surpass the accuracy of COCO trained models of the same size.

By training on a compact dataset, we show that with an accuracy drop of only 3.6%, the training time can be reduced by 93%.

High object-detection accuracy can be obtained by training domain specific compact models and the training can be very short.

Training Domain Specific Models for Energy-Efficient Object Detection


We compare the model-free reinforcement learning with the model-based approaches through the lens of the expressive power of neural networks for policies, $Q$-functions, and dynamics.

We show, theoretically and empirically, that even for one-dimensional continuous state space, there are many MDPs whose optimal $Q$-functions and policies are much more complex than the dynamics.

We hypothesize many real-world MDPs also have a similar property.

For these MDPs, model-based planning is a favorable algorithm, because the resulting policies can approximate the optimal policy significantly better than a neural network parameterization can, and model-free or model-based policy optimization rely on policy parameterization.

Motivated by the theory, we apply a simple multi-step model-based bootstrapping planner (BOOTS) to bootstrap a weak $Q$-function into a stronger policy.

Empirical results show that applying BOOTS on top of model-based or model-free policy optimization algorithms at the test time improves the performance on MuJoCo benchmark tasks.

We compare deep model-based and model-free RL algorithms by studying the approximability of $Q$-functions, policies, and dynamics by neural networks.

Bootstrapping the Expressivity with Model-based Planning


Common-sense physical reasoning is an essential ingredient for any intelligent agent operating in the real-world.

For example, it can be used to simulate the environment, or to infer the state of parts of the world that are currently unobserved.

In order to match real-world conditions this causal knowledge must be learned without access to supervised data.

To address this problem we present a novel method that learns to discover objects and model their physical interactions from raw visual images in a purely unsupervised fashion.

It incorporates prior knowledge about the compositional nature of human perception to factor interactions between object-pairs and learn efficiently.

On videos of bouncing balls we show the superior modelling capabilities of our method compared to other unsupervised neural approaches that do not incorporate such prior knowledge.

We demonstrate its ability to handle occlusion and show that it can extrapolate learned knowledge to scenes with different numbers of objects.

We introduce a novel approach to common-sense physical reasoning that learns to discover objects and model their physical interactions from raw visual images in a purely unsupervised fashion

Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactions


The idea that neural networks may exhibit a bias towards simplicity has a long history.

Simplicity bias provides a way to quantify this intuition.

It predicts, for a broad class of input-output maps which can describe many systems in science and engineering, that simple outputs are exponentially more likely to occur upon uniform random sampling of inputs than complex outputs are.

This simplicity bias behaviour has been observed for systems ranging from the RNA sequence to secondary structure map, to systems of coupled differential equations, to models of plant growth.

Deep neural networks can be viewed as a mapping from the space of parameters (the weights) to the space of functions (how inputs get transformed to  outputs by the network).

We show that this parameter-function map obeys the necessary  conditions for simplicity bias, and numerically show that it is hugely biased towards functions with low descriptional complexity.

We also demonstrate a Zipf like power-law probability-rank relation.

A bias towards simplicity may help explain why neural nets generalize so well.

A very strong bias towards simple outpouts is observed in many simple input-ouput maps. The parameter-function map of deep networks is found to be biased in the same way.

Simplicity bias in the parameter-function map of deep neural networks

parameter-function map deep
bias
network


Imitation Learning (IL) is an appealing approach to learn desirable autonomous behavior.

However, directing IL to achieve arbitrary goals is difficult.

In contrast, planning-based algorithms use dynamics models and reward functions to achieve goals.

Yet, reward functions that evoke desirable behavior are often difficult to specify.

In this paper, we propose "Imitative Models" to combine the benefits of IL and goal-directed planning.

Imitative Models are probabilistic predictive models of desirable behavior able to plan interpretable expert-like trajectories to achieve specified goals.

We derive families of flexible goal objectives, including constrained goal regions, unconstrained goal sets, and energy-based goals.

We show that our method can use these objectives to successfully direct behavior.

Our method substantially outperforms six IL approaches and a planning-based approach in a dynamic simulated autonomous driving task, and is efficiently learned from expert demonstrations without online data collection.

We also show our approach is robust to poorly-specified goals, such as goals on the wrong side of the road.

In this paper, we propose Imitative Models to combine the benefits of IL and goal-directed planning: probabilistic predictive models of desirable behavior able to plan interpretable expert-like trajectories to achieve specified goals.

Deep Imitative Models for Flexible Inference, Planning, and Control