The prohibitive energy cost of running high-performance Convolutional Neural Networks (CNNs) has been limiting their deployment on resource-constrained platforms including mobile and wearable devices.

We propose a CNN for energy-aware dynamic routing, called the EnergyNet, that achieves adaptive-complexity inference based on the inputs, leading to an overall reduction of run time energy cost without noticeably losing (or even improving) accuracy.

That is achieved by proposing an energy loss that captures both computational and data movement costs.

We combine it with the accuracy-oriented loss, and learn a dynamic routing policy for skipping certain layers in the networks, that optimizes the hybrid loss.

Our empirical results demonstrate that, compared to the baseline CNNs, EnergyNetcan trim down the energy cost up to 40% and 65%, during inference on the CIFAR10 and Tiny ImageNet testing sets, respectively, while maintaining the same testing accuracies.

It is further encouraging to observe that the energy awareness might serve as a training regularization and can even improve prediction accuracy:  our models can achieve 0.7% higher top-1 testing accuracy than the baseline on CIFAR-10 when saving up to 27% energy, and 1.0% higher top-5 testing accuracy on Tiny ImageNet when saving up to 50% energy, respectively.

This paper proposes a new CNN model that combines energy cost with a dynamic routing strategy to enable adaptive energy-efficient inference.

EnergyNet: Energy-Efficient Dynamic Inference


Log-linear models models are widely used in machine learning, and in particular are ubiquitous in deep learning architectures in the form of the softmax.

While exact inference and learning of these requires linear time, it can be done approximately in sub-linear time with strong concentrations guarantees.

In this work, we present LSH Softmax, a method to perform sub-linear learning and inference of the softmax layer in the deep learning setting.

Our method relies on the popular Locality-Sensitive Hashing to build a well-concentrated gradient estimator, using nearest neighbors and uniform samples.

We also present an inference scheme in sub-linear time for LSH Softmax using the Gumbel distribution.

On language modeling, we show that Recurrent Neural Networks trained with LSH Softmax perform on-par with computing the exact softmax while requiring sub-linear computations.

we present LSH Softmax, a softmax approximation layer for sub-linear learning and inference with strong theoretical guarantees; we showcase both its applicability and efficiency by evaluating on a real-world task: language modeling.

LSH Softmax: Sub-Linear Learning and Inference of the Softmax Layer in Deep Architectures

sub-linear learning inference
lsh softmax
layer


Can the success of reinforcement learning methods for simple combinatorial optimization problems be extended to multi-robot sequential assignment planning?

In addition to the challenge of achieving near-optimal performance in large problems, transferability to an unseen number of robots and tasks is another key challenge for real-world applications.

In this paper, we suggest a method that achieves the first success in both challenges for robot/machine scheduling problems.

Our method comprises of three components.

First, we show any robot scheduling problem can be expressed as a random probabilistic graphical model (PGM).

We develop a mean-field inference method for random PGM and use it for Q-function inference.

Second, we show that transferability can be achieved by carefully designing two-step sequential encoding of problem state.

Third, we resolve the computational scalability issue of fitted Q-iteration by suggesting a heuristic auction-based Q-iteration fitting method enabled by transferability we achieved.

We apply our method to discrete-time, discrete space problems (Multi-Robot Reward Collection (MRRC)) and scalably achieve 97% optimality with transferability.

This optimality is maintained under stochastic contexts.

By extending our method to continuous time, continuous space formulation, we claim to be the first learning-based method with scalable performance in any type of multi-machine scheduling problems; our method scalability achieves comparable performance to popular metaheuristics in Identical parallel machine scheduling (IPMS) problems.

RL can solve (stochastic) multi-robot/scheduling problems scalably and transferably using graph embedding

Learning scalable and transferable multi-robot/machine sequential assignment planning via graph embedding


Curriculum learning consists in learning a difficult task by first training on an easy version of it, then on more and more difficult versions and finally on the difficult task.

To make this learning efficient, given a curriculum and the current learning state of an agent, we need to find what are the good next tasks to train the agent on.

Teacher-Student algorithms assume that the good next tasks are the ones on which the agent is making the fastest progress or digress.

However, two problematic situations where the agent is mainly trained on tasks it can't learn yet or it already learnt may occur.

Therefore, we introduce a new algorithm using min max ordered curriculums that assumes that the good next tasks are the ones that are learnable but not learnt yet.

It outperforms Teacher-Student algorithms on small curriculums and significantly outperforms them on sophisticated ones with numerous tasks.

We present a new algorithm for learning by curriculum based on the notion of mastering rate that outperforms previous algorithms.

Learning of Sophisticated Curriculums by viewing them as Graphs over Tasks


The fields of artificial intelligence and neuroscience have a long history of fertile bi-directional interactions.

On the one hand, important inspiration for the development of artificial intelligence systems has come from the study of natural systems of intelligence, the mammalian neocortex in particular.

On the other, important inspiration for models and theories of the brain have emerged from artificial intelligence research.

A central question at the intersection of these two areas is concerned with the processes by which neocortex learns, and the extent to which they are analogous to the back-propagation training algorithm of deep networks.

Matching the data efficiency, transfer and generalisation properties of neocortical learning remains an area of active research in the field of deep learning.

Recent advances in our understanding of neuronal, synaptic and dendritic physiology of the neocortex suggest new approaches for unsupervised representation learning, perhaps through a new class of objective functions, which could act alongside or in lieu of back-propagation.

Such local learning rules have implicit rather than explicit objectives with respect to the training data, facilitating domain adaptation and generalisation.

Incorporating them into deep networks for representation learning could better leverage unlabelled datasets to offer significant improvements in data efficiency of downstream supervised readout learning, and reduce susceptibility to adversarial perturbations, at the cost of a more restricted domain of applicability.

Inspiration from local dendritic processes of neocortical learning to make unsupervised learning great again.

Neocortical plasticity: an unsupervised cake but no free lunch


Real-world Question Answering (QA) tasks consist of thousands of words that often represent many facts and entities.

Existing models based on LSTMs require a large number of parameters to support external memory and do not generalize well for long sequence inputs.

Memory networks attempt to address these limitations by storing information to an external memory module but must examine all inputs in the memory.

Hence, for longer sequence inputs the intermediate memory components proportionally scale in size resulting in poor inference times and high computation costs.

In this paper, we present Adaptive Memory Networks (AMN) that process input question pairs to dynamically construct a network architecture optimized for lower inference times.

During inference, AMN parses input text into entities within different memory slots.

However, distinct from previous approaches, AMN is a dynamic network architecture that creates variable numbers of memory banks weighted by question relevance.

Thus, the decoder can select a variable number of memory banks to construct an answer using fewer banks, creating a runtime trade-off between accuracy and speed.

AMN is enabled by first, a novel bank controller that makes discrete decisions with high accuracy and second, the capabilities of a dynamic framework (such as PyTorch) that allow for dynamic network sizing and efficient variable mini-batching.

In our results, we demonstrate that our model learns to construct a varying number of memory banks based on task complexity and achieves faster inference times for standard bAbI tasks, and modified bAbI tasks.

We achieve state of the art accuracy over these tasks with an average 48% lower entities are examined during inference.

Adaptive Memory Networks


When a bilingual student learns to solve word problems in math, we expect the student to be able to solve these problem in both languages the student is fluent in, even if the math lessons were only taught in one language.

However, current representations in machine learning are language dependent.

In this work, we present a method to decouple the language from the problem by learning language agnostic representations and therefore allowing training a model in one language and applying to a different one in a zero shot fashion.

We learn these representations by taking inspiration from linguistics, specifically the Universal Grammar hypothesis and learn universal latent representations that are language agnostic (Chomsky, 2014; Montague, 1970).

We demonstrate the capabilities of these representations by showing that the models trained on a single language using language agnostic representations achieve very similar accuracies in other languages.

By taking inspiration from linguistics, specifically the Universal Grammar hypothesis, we learn language agnostic universal representations which we can utilize to do zero-shot learning across languages.

Towards Language Agnostic Universal Representations


Generative models with both discrete and continuous latent variables are highly motivated by the structure of many real-world data sets.

They present, however, subtleties in training often manifesting in the discrete latent variable not being leveraged.

In this paper, we show why such models struggle to train using traditional log-likelihood maximization, and that they are amenable to training using the Optimal Transport framework of Wasserstein Autoencoders.

We find our discrete latent variable to be fully leveraged by the model when trained, without any modifications to the objective function or significant fine tuning.

Our model generates comparable samples to other approaches while using relatively simple neural networks, since the discrete latent variable carries much of the descriptive burden.

Furthermore, the discrete latent provides significant control over generation.

This paper shows that the Wasserstein distance objective enables the training of latent variable models with discrete latents in a case where the Variational Autoencoder objective fails to do so.

Improving Gaussian mixture latent variable model convergence with Optimal Transport


While machine learning models achieve human-comparable performance on sequential data, exploiting structured knowledge is still a challenging problem.

Spatio-temporal graphs have been proved to be a useful tool to abstract interaction graphs and previous works exploits carefully designed feed-forward architecture to preserve such structure.

We argue to scale such network design to real-world problem, a model needs to automatically learn a meaningful representation of the possible relations.

Learning such interaction structure is not trivial: on the one hand, a model has to discover the hidden relations between different problem factors in an unsupervised way; on the other hand, the mined relations have to be interpretable.

In this paper, we propose an attention module able to project a graph sub-structure in a fixed size embedding, preserving the influence that the neighbours exert on a given vertex.

On a comprehensive evaluation done on real-world as well as toy task, we found our model competitive against strong baselines.

A graph neural network able to automatically learn and leverage a dynamic interactive graph structure

An Attention-Based Model for Learning Dynamic Interaction Networks


We introduce NAMSG, an adaptive first-order algorithm for training neural networks.

The method is efficient in computation and memory, and is straightforward to implement.

It computes the gradients at configurable remote observation points, in order to expedite the convergence by adjusting the step size for directions with different curvatures in the stochastic setting.

It also scales the updating vector elementwise by a nonincreasing preconditioner to take the advantages of AMSGRAD.

We analyze the convergence properties for both convex and nonconvex problems by modeling the training process as a dynamic system, and provide a strategy to select the observation factor without grid search.

A data-dependent regret bound is proposed to guarantee the convergence in the convex setting.

The method can further achieve a O(log(T)) regret bound for strongly convex functions.

Experiments demonstrate that NAMSG works well in practical problems and compares favorably to popular adaptive methods, such as ADAM, NADAM, and AMSGRAD.

A new algorithm for training neural networks that compares favorably to popular adaptive methods.

NAMSG: An Efficient Method for Training Neural Networks


Recent advances in Generative Adversarial Networks facilitated by improvements to the framework and successful application to various problems has resulted in extensions to multiple domains.

IRGAN attempts to leverage the framework for Information-Retrieval (IR), a task that can be described as modeling the correct conditional probability distribution p(d|q) over the documents (d), given the query (q).

The work that proposes IRGAN claims that optimizing their minimax loss function will result in a generator which can learn the distribution, but their setup and baseline term steer the model away from an exact adversarial formulation, and this work attempts to point out certain inaccuracies in their formulation.

Analyzing their loss curves gives insight into possible mistakes in the loss functions and better performance can be obtained by using the co-training like setup we propose, where two models are trained in a co-operative rather than an adversarial fashion.

Points out problems in loss function used in IRGAN, a recently proposed GAN framework for Information Retrieval. Further, a model motivated by co-training is proposed, which achieves better performance.

Dissecting an Adversarial framework for Information Retrieval


Collaborative personalization, such as through learned user representations (embeddings), can improve the prediction accuracy of neural-network-based models significantly.

We propose Federated User Representation Learning (FURL), a simple, scalable, privacy-preserving and resource-efficient way to utilize existing neural personalization techniques in the Federated Learning (FL) setting.

FURL divides model parameters into federated and private parameters.

Private parameters, such as private user embeddings, are trained locally, but unlike federated parameters, they are not transferred to or averaged on the server.

We show theoretically that this parameter split does not affect training for most model personalization approaches.

Storing user embeddings locally not only preserves user privacy, but also improves memory locality of personalization compared to on-server training.

We evaluate FURL on two datasets, demonstrating a significant improvement in model quality with 8% and 51% performance increases, and approximately the same level of performance as centralized training with only 0% and 4% reductions.

Furthermore, we show that user embeddings learned in FL and the centralized setting have a very similar structure, indicating that FURL can learn collaboratively through the shared parameters while preserving user privacy.

We propose Federated User Representation Learning (FURL), a simple, scalable, privacy-preserving and bandwidth-efficient way to utilize existing neural personalization techniques in the Federated Learning (FL) setting.

Federated User Representation Learning


Recurrent models for sequences have been recently successful at many tasks, especially for language modeling

Nevertheless, it remains challenging to extract good representations from

For instance, even though language has a clear hierarchical structure going from characters

through words to sentences, it is not apparent in current language models.

We propose to improve the representation in sequence models by

augmenting current approaches with an autoencoder that is forced to compress

the sequence through an intermediate discrete latent space.

though this discrete representation we introduce an improved semantic hashing technique.

We show that this technique performs well on a newly proposed quantitative efficiency measure.

We also analyze latent codes produced by the model showing how they correspond to

Finally, we present an application of the autoencoder-augmented

model to generating diverse translations.

Autoencoders for text with a new method for using discrete latent space.

Discrete Autoencoders for Sequence Models


Auto-encoding and generative models have made tremendous successes in image and signal representation learning and generation.

These models, however, generally employ the full Euclidean space or a bounded subset (such as $[0,1]^l$) as the latent space, whose trivial geometry is often too simplistic to meaningfully reflect the structure of the data.

This paper aims at exploring a nontrivial geometric structure of the latent space for better data representation.

Inspired by differential geometry, we propose \textbf{Chart Auto-Encoder (CAE)}, which captures the manifold structure of the data with multiple charts and transition functions among them.

CAE translates the mathematical definition of manifold through parameterizing the entire data set as a collection of overlapping charts, creating local latent representations.

These representations are an enhancement of the single-charted latent space commonly employed in auto-encoding models, as they reflect the intrinsic structure of the manifold.

Therefore, CAE achieves a more accurate approximation of data and generates realistic new ones.

We conduct experiments with synthetic and real-life data to demonstrate the effectiveness of the proposed CAE.

Manifold-structured latent space for generative models

Chart Auto-Encoders for Manifold Structured  Data


We tackle the problem of modeling sequential visual phenomena.

Given examples of a phenomena that can be divided into discrete time steps, we aim to take an input from any such time and realize this input at all other time steps in the sequence.

Furthermore, we aim to do this \textit{without} ground-truth aligned sequences --- avoiding the difficulties needed for gathering aligned data.

This generalizes the unpaired image-to-image problem from generating pairs to generating sequences.

We extend cycle consistency to \textit{loop consistency} and alleviate difficulties associated with learning in the resulting long chains of computation.

We show competitive results compared to existing image-to-image techniques when modeling several different data sets including the Earth's seasons and aging of human faces.

LoopGAN extends cycle length in CycleGAN to enable unaligned sequential transformation for more than two time steps.

Unaligned Image-to-Sequence Transformation with Loop Consistency


We propose Stochastic Weight Averaging in Parallel (SWAP), an algorithm to accelerate DNN training.

Our algorithm uses large mini-batches to compute an approximate solution quickly and then refines it by averaging the weights of multiple models computed independently and in parallel.

The resulting models generalize equally well as those trained with small mini-batches but are produced in a substantially shorter time.

We demonstrate the reduction in training time and the good generalization performance of the resulting models on the computer vision datasets CIFAR10, CIFAR100, and ImageNet.

We propose SWAP, a distributed algorithm for large-batch training of neural networks.

Stochastic Weight Averaging in Parallel: Large-Batch Training That Generalizes Well


A common way to speed up training of large convolutional networks is to add  computational units.

Training is then performed using data-parallel synchronous Stochastic Gradient Descent (SGD) with a mini-batch divided between computational units.

With an increase in the number of nodes, the batch size grows.

However,  training with a large batch  often results in lower model accuracy.

We argue that the current recipe for large batch training (linear learning rate scaling with warm-up) is not general enough and training may diverge.

To overcome these optimization difficulties, we propose a new training algorithm based on Layer-wise Adaptive Rate Scaling (LARS).

Using LARS, we scaled AlexNet  and ResNet-50 to a batch size of 16K.

A new large batch training algorithm  based on Layer-wise Adaptive Rate Scaling (LARS); using LARS, we scaled AlexNet  and ResNet-50 to a batch of 16K.

Large Batch Training of Convolutional Networks with Layer-wise Adaptive Rate Scaling

layer-wise adaptive rate scaling
large batch training


Finding an embedding space for a linear approximation of a nonlinear dynamical system enables efficient system identification and control synthesis.

The Koopman operator theory lays the foundation for identifying the nonlinear-to-linear coordinate transformations with data-driven methods.

Recently, researchers have proposed to use deep neural networks as a more expressive class of basis functions for calculating the Koopman operators.

These approaches, however, assume a fixed dimensional state space; they are therefore not applicable to scenarios with a variable number of objects.

In this paper, we propose to learn compositional Koopman operators, using graph neural networks to encode the state into object-centric embeddings and using a block-wise linear transition matrix to regularize the shared structure across objects.

The learned dynamics can quickly adapt to new environments of unknown physical parameters and produce control signals to achieve a specified goal.

Our experiments on manipulating ropes and controlling soft robots show that the proposed method has better efficiency and generalization ability than existing baselines.

Learning compositional Koopman operators for efficient system identification and model-based control.

Learning Compositional Koopman Operators for Model-Based Control

learn compositional koopman operator
model-based control


We derive reverse-mode (or adjoint) automatic differentiation for solutions of stochastic differential equations (SDEs), allowing time-efficient and constant-memory computation of pathwise gradients, a continuous-time analogue of the reparameterization trick.

Specifically, we construct a backward SDE whose solution is the gradient and provide conditions under which numerical solutions converge.

We also combine our stochastic adjoint approach with a stochastic variational inference scheme for continuous-time SDE models, allowing us to learn distributions over functions using stochastic gradient descent.

Our latent SDE model achieves competitive performance compared to existing approaches on time series modeling.

We present a constant memory gradient computation procedure through solutions of stochastic differential equations (SDEs) and apply the method for learning latent SDE models.

Scalable Gradients and Variational Inference for Stochastic Differential Equations

stochastic differential equation
gradient


It is clear that users should own and control their data and privacy.

Utility providers are also becoming more interested in guaranteeing data privacy.

Therefore, users and providers can and should collaborate in privacy protecting challenges, and this paper addresses this new paradigm.

We propose a framework where the user controls what characteristics of the data they want to share (utility) and what they want to keep private (secret), without necessarily asking the utility provider to change its existing machine learning algorithms.

We first analyze the space of privacy-preserving representations and derive natural information-theoretic bounds on the utility-privacy trade-off when disclosing a sanitized version of the data X. We present explicit learning architectures to learn privacy-preserving representations that approach this bound in a data-driven fashion.

We describe important use-case scenarios where the utility providers are willing to collaborate with the sanitization process.

We study space-preserving transformations where the utility provider can use the same algorithm on original and sanitized data, a critical and novel attribute to help service providers accommodate varying privacy requirements with a single set of utility algorithms.

We illustrate this framework through  the implementation of three use cases; subject-within-subject, where we tackle the problem of having a face identity detector that works only on a consenting subset of users, an important application, for example, for mobile devices activated by face recognition; gender-and-subject, where we preserve facial verification while hiding the gender attribute for users who choose to do so; and emotion-and-gender, where we hide independent variables, as is the case of hiding gender while preserving emotion detection.

Learning privacy-preserving transformations from data. A collaborative approach

Learning data-derived privacy preserving representations from information metrics


Solving tasks with sparse rewards is one of the most important challenges in reinforcement learning.

In the single-agent setting, this challenge has been addressed by introducing intrinsic rewards that motivate agents to explore unseen regions of their state spaces.

Applying these techniques naively to the multi-agent setting results in agents exploring independently, without any coordination among themselves.

We argue that learning in cooperative multi-agent settings can be accelerated and improved if agents coordinate with respect to what they have explored.

In this paper we propose an approach for learning how to dynamically select between different types of intrinsic rewards which consider not just what an individual agent has explored, but all agents, such that the agents can coordinate their exploration and maximize extrinsic returns.

Concretely, we formulate the approach as a hierarchical policy where a high-level controller selects among sets of policies trained on different types of intrinsic rewards and the low-level controllers learn the action policies of all agents under these specific rewards.

We demonstrate the effectiveness of the proposed approach in a multi-agent gridworld domain with sparse rewards, and then show that our method scales up to more complex settings by evaluating on the VizDoom platform.

We propose several intrinsic reward functions for encouraging coordinated exploration in multi-agent problems, and introduce an approach to dynamically selecting the best exploration method for a given task, online.

Coordinated Exploration via Intrinsic Rewards for Multi-Agent Reinforcement Learning

multi-agent
intrinsic reward
coordinate exploration


Object recognition in real-world requires handling long-tailed or even open-ended data.

An ideal visual system needs to reliably recognize the populated visual concepts and meanwhile efficiently learn about emerging new categories with a few training instances.

Class-balanced many-shot learning and few-shot learning tackle one side of this problem, via either learning strong classifiers for populated categories or learning to learn few-shot classifiers for the tail classes.

In this paper, we investigate the problem of generalized few-shot learning (GFSL) -- a model during the deployment is required to not only learn about "tail" categories with few shots, but simultaneously classify the "head" and "tail" categories.

We propose the Classifier Synthesis Learning (CASTLE), a learning framework that learns how to synthesize calibrated few-shot classifiers in addition to the multi-class classifiers of ``head'' classes, leveraging a shared neural dictionary.

CASTLE sheds light upon the inductive GFSL through optimizing one clean and effective GFSL learning objective.

It demonstrates superior performances than existing GFSL algorithms and strong baselines on MiniImageNet and TieredImageNet data sets.

More interestingly, it outperforms previous state-of-the-art methods when evaluated on standard few-shot learning.

We propose to learn synthesizing few-shot classifiers and many-shot classifiers using one single objective function for GFSL.

Learning Classifier Synthesis for Generalized Few-Shot Learning


Machine learning workloads are often expensive to train, taking weeks to converge.

The current generation of frameworks relies on custom back-ends in order to achieve efficiency, making it impractical to train models on less common hardware where no such back-ends exist.

Knossos builds on recent work that avoids the need for hand-written libraries, instead compiles machine learning models in much the same way one would compile other kinds of software.

In order to make the resulting code efficient, the Knossos complier directly optimises the abstract syntax tree of the program.

However in contrast to traditional compilers that employ hand-written optimisation passes, we take a rewriting approach driven by the $A^\star$ search algorithm and a learn value function that evaluates future potential cost reduction of taking various rewriting actions to the program.

We show that Knossos can automatically learned optimisations that past compliers had to implement by hand.

Furthermore, we demonstrate that Knossos can achieve wall time reduction compared to a hand-tuned compiler on a suite of machine learning programs, including basic linear algebra and convolutional networks.

The Knossos compiler has minimal dependencies and can be used on any architecture that supports a \Cpp toolchain.

Since cost model the proposed algorithm optimises can be tailored to a particular hardware architecture, the proposed approach can potentially applied to a variety of hardware.

We combine A* search with reinforcement learning to speed up machine learning code

Knossos: Compiling AI with AI


Grasping an object and precisely stacking it on another is a difficult task for traditional robotic control or hand-engineered approaches.

Here we examine the problem in simulation and provide techniques aimed at solving it via deep reinforcement learning.

We introduce two straightforward extensions to the Deep Deterministic Policy Gradient algorithm (DDPG), which make it significantly more data-efficient and scalable.

Our results show that by making extensive use of off-policy data and replay, it is possible to find high-performance control policies.

Further, our results hint that it may soon be feasible to train successful stacking policies by collecting interactions on real robots.

Data-efficient deep reinforcement learning can be used to learning precise stacking policies.

Data-efficient Deep Reinforcement Learning for Dexterous Manipulation

data-efficient deep reinforcement learning


In recent years deep reinforcement learning has been shown to be adept at solving sequential decision processes with high-dimensional state spaces such as in the Atari games.

Many reinforcement learning problems, however, involve high-dimensional discrete action spaces as well as high-dimensional state spaces.

In this paper, we develop a novel policy gradient methodology for the case of large multidimensional discrete action spaces.

We propose two approaches for creating parameterized policies: LSTM parameterization and a Modified MDP (MMDP) giving rise to Feed-Forward Network (FFN) parameterization.

Both of these approaches provide expressive models to which backpropagation can be applied for training.

We then consider entropy bonus, which is typically added to the reward function to enhance exploration.

In the case of high-dimensional action spaces, calculating the entropy and the gradient of the entropy requires enumerating all the actions in the action space and running forward and backpropagation for each action, which may be computationally infeasible.

We develop several novel unbiased estimators for the entropy bonus and its gradient.

Finally, we test our algorithms on two environments: a multi-hunter multi-rabbit grid game and a multi-agent multi-arm bandit problem.

policy parameterizations and unbiased policy entropy estimators for MDP with large multidimensional discrete action space

Policy Gradient For Multidimensional Action Spaces: Action Sampling and Entropy Bonus

action space
policy
entropy
multidimensional


Neural networks offer high-accuracy solutions to a range of problems, but are computationally costly to run in production systems.

We propose a technique called Deep Learning Approximation to take an already-trained neural network model and build a faster (and almost equally accurate) network by manipulating the network structure and coefficients without requiring re-training or access to the training data.

Speedup is achieved by applying a sequential series of independent optimizations that reduce the floating-point operations (FLOPs) required to perform a forward pass.

An optimal lossy approximation is chosen for each layer by weighing the relative accuracy loss and FLOP reduction.

On PASCAL VOC 2007 with the YOLO network, we show an end-to-end 2x speedup in a network forward pass with a $5$\% drop in mAP that can be re-gained by finetuning, enabling this network (and others like it) to be deployed in compute-constrained systems.

Decompose weights to use fewer FLOPs with SVD

Deep Learning Approximation: Zero-Shot Neural Network Speedup


Asynchronous distributed methods are a popular way to reduce the communication and synchronization costs of large-scale optimization.

Yet, for all their success, little is known about their convergence guarantees in the challenging case of general non-smooth, non-convex objectives, beyond cases where closed-form proximal operator solutions are available.

This is all the more surprising since these objectives are the ones appearing in the training of deep neural networks.

In this paper, we introduce the first convergence analysis covering asynchronous methods in the case of general non-smooth, non-convex objectives.

Our analysis applies to stochastic sub-gradient descent methods both with and without block variable partitioning, and both with and without momentum.

It is phrased in the context of a general probabilistic model of asynchronous scheduling accurately adapted to modern hardware properties.

We validate our analysis experimentally in the context of training deep neural network architectures.

We show their overall successful asymptotic convergence as well as exploring how momentum, synchronization, and partitioning all affect performance.

Asymptotic convergence for stochastic subgradien method with momentum under general parallel asynchronous computation for general nonconvex nonsmooth optimization

Asynchronous Stochastic Subgradient Methods for General Nonsmooth Nonconvex Optimization

method general stochastic optimization asynchronous nonsmooth nonconvex


Stochastic neural networks with discrete random variables are an important class of models for their expressivity and interpretability.

Since direct differentiation and backpropagation is not possible, Monte Carlo gradient estimation techniques have been widely employed for training such models.

Efficient stochastic gradient estimators, such Straight-Through and Gumbel-Softmax, work well for shallow models with one or two stochastic layers.

Their performance, however, suffers with increasing model complexity.

In this work we focus on stochastic networks with multiple layers of Boolean latent variables.

To analyze such such networks, we employ the framework of harmonic analysis for Boolean functions.

We use it to derive an analytic formulation for the source of bias in the biased Straight-Through estimator.

Based on the analysis we propose \emph{FouST}, a simple gradient estimation algorithm that relies on three simple bias reduction steps.

Extensive experiments show that FouST performs favorably compared to state-of-the-art biased estimators, while being much faster than unbiased ones.

To the best of our knowledge FouST is the first gradient estimator to train up very deep stochastic neural networks, with up to 80 deterministic and 11 stochastic layers.

We present a low-bias estimator for Boolean stochastic variable models with many stochastic layers.

Low Bias Gradient Estimates for Very Deep Boolean Stochastic Networks


We propose a novel generative adversarial network for visual attributes manipulation (ManiGAN), which is able to semantically modify the visual attributes of given images using natural language descriptions.

The key to our method is to design a novel co-attention module to combine text and image information rather than simply concatenating two features along the channel direction.

Also, a detail correction module is proposed to rectify mismatched attributes of the synthetic image, and to reconstruct text-unrelated contents.

Finally, we propose a new metric for evaluating manipulation results, in terms of both the generation of text-related attributes and the reconstruction of text-unrelated contents.

Extensive experiments on benchmark datasets demonstrate the advantages of our proposed method, regarding the effectiveness of image manipulation and the capability of generating high-quality results.

We propose a novel method to manipulate given images using natural language descriptions.

ManiGAN: Text-Guided Image Manipulation


Optimal Transport (OT) naturally arises in many machine learning applications, where we need to handle cross-modality data from multiple sources.

Yet the heavy computational burden limits its wide-spread uses.

To address the scalability issue, we propose an implicit generative learning-based framework called SPOT (Scalable Push-forward of Optimal Transport).

Specifically, we approximate the optimal transport plan by a pushforward of a reference distribution, and cast the optimal transport problem into a minimax problem.

We then can solve OT problems efficiently using primal dual stochastic gradient-type algorithms.

We also show that we can recover the density of the optimal transport plan using neural ordinary differential equations.

Numerical experiments on both synthetic and real datasets illustrate that SPOT is robust and has favorable convergence behavior.

SPOT also allows us to efficiently sample from the optimal transport plan, which benefits downstream applications such as domain adaptation.

Use GAN-based method to scalably solve optimal transport

On Scalable and Efficient Computation of Large Scale Optimal Transport


In this work, we propose a novel formulation of planning which views it as a probabilistic inference problem over future optimal trajectories.

This enables us to use sampling methods, and thus, tackle planning in continuous domains using a fixed computational budget.

We design a new algorithm,  Sequential Monte Carlo Planning, by leveraging classical methods in Sequential Monte Carlo and Bayesian smoothing in the context of control as inference.

Furthermore, we show that Sequential Monte Carlo Planning can capture multimodal policies and can quickly learn continuous control tasks.

Leveraging control as inference and Sequential Monte Carlo methods, we proposed a probabilistic planning algorithm.

Probabilistic Planning with Sequential Monte Carlo methods

sequential monte carlo method
probabilistic planning


Training Generative Adversarial Networks (GANs) is notoriously challenging.

We propose and study an architectural modification, self-modulation, which improves GAN performance across different data sets, architectures, losses, regularizers, and hyperparameter settings.

Intuitively, self-modulation allows the intermediate feature maps of a generator to change as a function of the input noise vector.

While reminiscent of other conditioning techniques, it requires no labeled data.

In a large-scale empirical study we observe a relative decrease of 5%-35% in FID.

Furthermore, all else being equal, adding this modification to the generator leads to improved performance in 124/144 (86%) of the studied settings.

Self-modulation is a simple architectural change that requires no additional parameter tuning, which suggests that it can be applied readily to any GAN.

A simple GAN modification that improves performance across many losses, architectures, regularization schemes, and datasets.

On Self Modulation for Generative Adversarial Networks


Extreme Classification Methods have become of paramount importance, particularly for Information Retrieval (IR) problems, owing to the development of smart algorithms that are scalable to industry challenges.

One of the prime class of models that aim to solve the memory and speed challenge of extreme multi-label learning is Group Testing.

Multi-label Group Testing (MLGT) methods construct label groups by grouping original labels either randomly or based on some similarity and then train smaller classifiers to first predict the groups and then recover the original label vectors.

Recently, a novel approach called MACH (Merged Average Classifiers via Hashing) was proposed which projects the huge label vectors to a small and manageable count-min sketch (CMS) matrix and then learns to predict this matrix to recover the original prediction probabilities.

Thereby, the model memory scales O(logK) for K classes.

MACH is a simple algorithm which works exceptionally well in practice.

Despite this simplicity of MACH, there is a big gap between the theoretical understanding of the trade-offs with MACH.

Leveraging the theory of count-min sketch we provide precise quantification of the memory-identifiablity tradeoffs.

We extend the theory to the case of multi-label classification, where the dependencies make the estimators hard to calculate in closed forms.

To mitigate this issue, we propose novel quadratic approximation using the Inclusion-Exclusion Principle.

Our estimator has significantly lower reconstruction error than the typical CMS estimator across various values of number of classes K, label sparsity and compression ratio.

How to estimate original probability vector for millions of classes from count-min sketch measurements -  a theoretical and practical setup.

A Deep Dive into Count-Min Sketch for Extreme Classification in Logarithmic Memory


Neural networks are commonly used as models for classification for a wide variety of tasks.

Typically, a learned affine transformation is placed at the end of such models, yielding a per-class value used for classification.

This classifier can have a vast number of parameters, which grows linearly with the number of possible classes, thus requiring increasingly more resources.

In this work we argue that this classifier can be fixed, up to a global scale constant, with little or no loss of accuracy for most tasks, allowing memory and computational benefits.

Moreover, we show that by initializing the classifier with a Hadamard matrix we can speed up inference as well.

We discuss the implications for current understanding of neural network models.

You can fix the classifier in neural networks without losing accuracy

Fix your classifier: the marginal value of training the last weight layer


This paper introduces NEMO, an approach to unsupervised object detection that uses motion---instead of image labels---as a cue to learn object detection.

To discriminate between motion of the target object and other changes in the image, it relies on negative examples that show the scene without the object.

The required data can be collected very easily by recording two short videos, a positive one showing the object in motion and a negative one showing the scene without the object.

Without any additional form of pretraining or supervision and despite of occlusions, distractions, camera motion, and adverse lighting, those videos are sufficient to learn object detectors that can be applied to new videos and even generalize to unseen scenes and camera angles.

In a baseline comparison, unsupervised object detection outperforms off-the shelf template matching and tracking approaches that are given an initial bounding box of the object.

The learned object representations are also shown to be accurate enough to capture the relevant information from manipulation task demonstrations, which makes them applicable to learning from demonstration in robotics.

An example of object detection that was learned from 3 minutes of video can be found here: http://y2u.be/u_jyz9_ETz4

Learning to detect objects without image labels from 3 minutes of video

Found by NEMO: Unsupervised Object Detection from Negative Examples and Motion


Recent powerful pre-trained language models have achieved remarkable performance on most of the popular datasets for reading comprehension.

It is time to introduce more challenging datasets to push the development of this field towards more comprehensive reasoning of text.

In this paper, we introduce a new Reading Comprehension dataset requiring logical reasoning (ReClor) extracted from standardized graduate admission examinations.

As earlier studies suggest, human-annotated datasets usually contain biases, which are often exploited by models to achieve high accuracy without truly understanding the text.

In order to comprehensively evaluate the logical reasoning ability of models on ReClor, we propose to identify biased data points and separate them into EASY set while the rest as HARD set.

Empirical results show that the state-of-the-art models have an outstanding ability to capture biases contained in the dataset with high accuracy on EASY set.

However, they struggle on HARD set with poor performance near that of random guess, indicating more research is needed to essentially enhance the logical reasoning ability of current models.

We introduce ReClor, a reading comprehension dataset requiring logical reasoning, and find that current state-of-the-art models struggle with real logical reasoning with poor performance near that of random guess.

ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning

reclor reading comprehension dataset require logical reasoning


This paper explores the scenarios under which

an attacker can claim that ‘Noise and access to

the softmax layer of the model is all you need’

to steal the weights of a convolutional neural network

were able to achieve 96% test accuracy using

the stolen MNIST model and 82% accuracy using

theft-susceptibility of the weights is indicative

of the complexity of the dataset and propose a

this dissemination is to not just showcase how far

knowing the architecture can take you in terms of

model stealing, but to also draw attention to this

rather idiosyncratic weight learnability aspects of

some initial results obtained with using

the Ising probability distribution in lieu of the i.i.d.

Input only noise , glean the softmax outputs, steal the weights

Garbage in, model out: Weight theft with just noise


We propose a new form of an autoencoding model which incorporates the best properties of variational autoencoders (VAE) and generative adversarial networks (GAN).

It is known that GAN can produce very realistic samples while VAE does not suffer from mode collapsing problem.

Our model optimizes λ-Jeffreys divergence between the model distribution and the true data distribution.

We show that it takes the best properties of VAE and GAN objectives.

One of these parts can be optimized by using the standard adversarial training, and the second one is the very objective of the VAE model.

However, the straightforward way of substituting the VAE loss does not work well if we use an explicit likelihood such as Gaussian or Laplace which have limited flexibility in high dimensions and are unnatural for modelling images in the space of pixels.

To tackle this problem we propose a novel approach to train the VAE model with an implicit likelihood by an adversarially trained discriminator.

In an extensive set of experiments on CIFAR-10 and TinyImagent datasets, we show that our model achieves the state-of-the-art generation and reconstruction quality and demonstrate how we can balance between mode-seeking and mode-covering behaviour of our model by adjusting the weight λ in our objective.

Implicit λ-Jeffreys Autoencoders: Taking the Best of Both Worlds


Learning-to-learn or meta-learning leverages data-driven inductive bias to increase the efficiency of learning on a novel task.

This approach encounters difficulty when transfer is not mutually beneficial, for instance, when tasks are sufficiently dissimilar or change over time.

Here, we use the connection between gradient-based meta-learning and hierarchical Bayes to propose a mixture of hierarchical Bayesian models over the parameters of an arbitrary function approximator such as a neural network.

Generalizing the model-agnostic meta-learning (MAML) algorithm, we present a stochastic expectation maximization procedure to jointly estimate parameter initializations for gradient descent as well as a latent assignment of tasks to initializations.

This approach better captures the diversity of training tasks as opposed to consolidating inductive biases into a single set of hyperparameters.

Our experiments demonstrate better generalization on the standard miniImageNet benchmark for 1-shot classification.

We further derive a novel and scalable non-parametric variant of our method that captures the evolution of a task distribution over time as demonstrated on a set of few-shot regression tasks.

We use the connection between gradient-based meta-learning and hierarchical Bayes to learn a mixture of meta-learners that is appropriate for a heterogeneous and evolving task distribution.

Modulating transfer between tasks in gradient-based meta-learning


We introduce a new routing algorithm for capsule networks, in which a child capsule is routed to a parent based only on agreement between the parent's state and the child's vote.

Unlike previously proposed routing algorithms, the parent's ability to reconstruct the child is not explicitly taken into account to update the routing probabilities.

This simplifies the routing procedure and improves performance on benchmark datasets such as CIFAR-10 and CIFAR-100.

The new mechanism 1) designs routing via inverted dot-product attention; 2) imposes Layer Normalization as normalization; and 3) replaces sequential iterative routing with concurrent iterative routing.

Besides outperforming existing capsule networks, our model performs at-par with a powerful CNN (ResNet-18), using less than 25% of the parameters.

On a different task of recognizing digits from overlayed digit images, the proposed capsule model performs favorably against CNNs given the same number of layers and neurons per layer.

We believe that our work raises the possibility of applying capsule networks to complex real-world tasks.

We present a new routing method for Capsule networks, and it performs at-par with ResNet-18 on CIFAR-10/ CIFAR-100.

Capsules with Inverted Dot-Product Attention Routing


We introduce Doc2Dial, an end-to-end framework for generating conversational data grounded in business documents via crowdsourcing.

Such data can be used to train automated dialogue agents performing customer care tasks for the enterprises or organizations.

In particular, the framework takes the documents as input and generates the tasks for obtaining the annotations for simulating dialog flows.

The dialog flows are used to guide the collection of utterances produced by crowd workers.

The outcomes include dialogue data grounded in the given documents, as well as various types of annotations that help ensure the quality of the data and the flexibility to (re)composite dialogues.

We introduce Doc2Dial, an end-to-end framework for generating conversational data grounded in business documents via crowdsourcing for train automated dialogue agents

Doc2Dial: a Framework for Dialogue Composition Grounded in Business Documents

ground business document
doc2dial
framework
dialogue


Capturing high-level structure in audio waveforms is challenging because a single second of audio spans tens of thousands of timesteps.

While long-range dependencies are difficult to model directly in the time domain, we show that they can be more tractably modelled in two-dimensional time-frequency representations such as spectrograms.

By leveraging this representational advantage, in conjunction with a highly expressive probabilistic model and a multiscale generation procedure, we design a model capable of generating high-fidelity audio samples which capture structure at timescales which time-domain models have yet to achieve.

We demonstrate that our model captures longer-range dependencies than time-domain models such as WaveNet across a diverse set of unconditional generation tasks, including single-speaker speech generation, multi-speaker speech generation, and music generation.

We introduce an autoregressive generative model for spectrograms and demonstrate applications to speech and music generation

MelNet: A Generative Model for Audio in the Frequency Domain


Deep convolutional network architectures are often assumed to guarantee generalization for small image translations and deformations.

In this paper we show that modern CNNs (VGG16, ResNet50, and InceptionResNetV2) can drastically change their output when an image is translated in the image plane by a few pixels, and that this failure of generalization also happens with other realistic small image transformations.

Furthermore,  we see these failures to generalize more frequently in more modern networks.

We show that these failures are related to the fact that the architecture of modern CNNs ignores the classical sampling theorem so that generalization is not guaranteed.

We also show that biases in the statistics of commonly used image datasets makes it unlikely that CNNs will learn to be invariant to these transformations.

Taken together our results suggest that the performance of CNNs in object recognition falls far short of the generalization capabilities of humans.

Modern deep CNNs are not invariant to translations, scalings and other realistic image transformations, and this lack of invariance is related to the subsampling operation and the biases contained in image datasets.

Why do deep convolutional networks generalize so poorly to small image transformations?


We present a real-time method for synthesizing highly complex human motions using a novel training regime we call the auto-conditioned Recurrent Neural Network (acRNN).

Recently, researchers have attempted to synthesize new motion by using autoregressive techniques, but existing methods tend to freeze or diverge after a couple of seconds due to an accumulation of errors that are fed back into the network.

Furthermore, such methods have only been shown to be reliable for relatively simple human motions, such as walking or running.

In contrast, our approach can synthesize arbitrary motions with highly complex styles, including dances or martial arts in addition to locomotion.

The acRNN is able to accomplish this by explicitly accommodating for autoregressive noise accumulation during training.

Our work is the first to our knowledge that demonstrates the ability to generate over 18,000 continuous frames (300 seconds) of new complex human motion w.r.t. different styles.

Synthesize complex and extended human motions using an auto-conditioned LSTM network

Auto-Conditioned Recurrent Networks for Extended Complex Human Motion Synthesis

extended
human motion
network
auto-conditioned
complex


{\em Saliency methods} attempt to explain a deep net's decision by assigning a {\em score} to each feature/pixel in the input, often doing this credit-assignment via the gradient of the output with respect to input.

Recently \citet{adebayosan} questioned the validity of many of these methods since they do not pass simple {\em sanity checks}, which test whether the scores shift/vanish when  layers of the trained net are randomized, or when the net is retrained using random labels for inputs.

%Surprisingly, the tested methods did not pass these checks: the explanations were relatively unchanged

.

We propose a simple fix to existing saliency methods that helps them pass sanity checks, which we call {\em competition for pixels}.

This involves computing saliency maps for all possible labels in the classification task, and using a simple competition among them to identify and remove less relevant pixels from the map.

Some theoretical justification is provided for it  and its performance is empirically demonstrated on several popular methods.

We devise a mechanism called competition among pixels that allows (approximately) complete saliency methods to pass the sanity checks.

A Simple Technique to Enable Saliency Methods to Pass the Sanity Checks


Classification systems typically act in isolation, meaning they are required to implicitly memorize the characteristics of all candidate classes in order to classify.

The cost of this is increased memory usage and poor sample efficiency.

We propose a model which instead verifies using reference images during the classification process, reducing the burden of memorization.

The model uses iterative non-differentiable queries in order to classify an image.

We demonstrate that such a model is feasible to train and can match baseline accuracy while being more parameter efficient.

However, we show that finding the correct balance between image recognition and verification is essential to pushing the model towards desired behavior, suggesting that a pipeline of recognition followed by verification is a more promising approach towards designing more powerful networks with simpler architectures.

Image classification via iteratively querying for reference image from a candidate class with a RNN and use CNN to compare to the input image

Are You Sure YouWant To Do That? Classification with Interpretable Queries


To reduce memory footprint and run-time latency, techniques such as neural net-work pruning and binarization have been explored separately.

However, it is un-clear how to combine the best of the two worlds to get extremely small and efficient models

.  In this paper, we, for the first time, define the filter-level pruning problem for binary neural networks, which cannot be solved by simply migrating existing structural pruning methods for full-precision models

.  A novel learning-based approach is proposed to prune filters in our main/subsidiary network frame-work, where the main network is responsible for learning representative features to optimize the prediction performance, and the subsidiary component works as a filter selector on the main network

. To avoid gradient mismatch when training the subsidiary component, we propose a layer-wise and bottom-up scheme

.  We also provide the theoretical and experimental comparison between our learning-based and greedy rule-based methods

.  Finally, we empirically demonstrate the effectiveness of our approach applied on several binary models,  including binarizedNIN, VGG-11, and ResNet-18, on various image classification datasets

.  For bi-nary ResNet-18 on ImageNet, we use 78.6% filters but can achieve slightly better test error 49.87% (50.02%-0.15%) than the original model

we define the filter-level pruning problem for binary neural networks for the first time and propose method to solve it.

A Main/Subsidiary Network Framework for Simplifying Binary Neural Networks


Wide adoption of complex RNN based models is hindered by their inference performance, cost and memory requirements.

To address this issue, we develop AntMan, combining structured sparsity with low-rank decomposition synergistically, to reduce model computation, size and execution time of RNNs while attaining desired accuracy.

AntMan extends knowledge distillation based training to learn the compressed models efficiently.

Our evaluation shows that AntMan offers up to 100x computation reduction with less than 1pt accuracy drop for language and machine reading comprehension models.

Our evaluation also shows that for a given accuracy target, AntMan produces 5x smaller models than the state-of-art.

Lastly, we show that AntMan offers super-linear speed gains compared to theoretical speedup, demonstrating its practical value on commodity hardware.

Reducing computation and memory complexity of RNN models by up to 100x using sparse low-rank compression modules, trained via knowledge distillation.

AntMan: Sparse Low-Rank Compression To Accelerate RNN Inference


Graph-structured data such as social networks, functional brain networks, gene regulatory networks, communications networks have brought the interest in generalizing deep learning techniques to graph domains.

In this paper, we are interested to design neural networks for graphs with variable length in order to solve learning problems such as vertex classification, graph classification, graph regression, and graph generative tasks.

Most existing works have focused on recurrent neural networks (RNNs) to learn meaningful representations of graphs, and more recently new convolutional neural networks (ConvNets) have been introduced.

In this work, we want to compare rigorously these two fundamental families of architectures to solve graph learning tasks.

We review existing graph RNN and ConvNet architectures, and propose natural extension of LSTM and ConvNet to graphs with arbitrary size.

Then, we design a set of analytically controlled experiments on two basic graph problems, i.e. subgraph matching and graph clustering, to test the different architectures.

Numerical results show that the proposed graph ConvNets are 3-17% more accurate and 1.5-4x faster than graph RNNs.

Graph ConvNets are also 36% more accurate than variational (non-learning) techniques.

Finally, the most effective graph ConvNet architecture uses gated edges and residuality.

Residuality plays an essential role to learn multi-layer architectures as they provide a 10% gain of performance.

We compare graph RNNs and graph ConvNets, and we consider the most generic class of graph ConvNets with residuality.

Residual Gated Graph ConvNets


Complex-value neural networks are not a new concept, however, the use of real-values has often been favoured over complex-values due to difficulties in training and accuracy of results.

Existing literature ignores the number of parameters used.

We compared complex- and real-valued neural networks using five activation functions.

We found that when real and complex neural networks are compared using simple classification tasks, complex neural networks perform equal to or slightly worse than real-value neural networks.

However, when specialised architecture is used, complex-valued neural networks outperform real-valued neural networks.

Therefore, complex–valued neural networks should be used when the input data is also complex or it can be meaningfully to the complex plane,  or when the network architecture uses the structure defined by using complex numbers.

Comparison of complex- and real-valued multi-layer perceptron with respect to the number of real-valued parameters.

Complex- and Real-Valued Neural Network Architectures


The rise of graph-structured data such as social networks, regulatory networks, citation graphs, and functional brain networks, in combination with resounding success of deep learning in various applications, has brought the interest in generalizing deep learning models to non-Euclidean domains.

In this paper, we introduce a new spectral domain convolutional architecture for deep learning on graphs.

The core ingredient of our model is a new class of parametric rational complex functions (Cayley polynomials) allowing to efficiently compute spectral filters on graphs that specialize on frequency bands of interest.

Our model generates rich spectral filters that are localized in space, scales linearly with the size of the input data for sparsely-connected graphs, and can handle different constructions of Laplacian operators.

Extensive experimental results show the superior performance of our approach on spectral image classification, community detection, vertex classification and matrix completion tasks.

A spectral graph convolutional neural network with spectral zoom properties.

CAYLEYNETS: SPECTRAL GRAPH CNNS WITH COMPLEX RATIONAL FILTERS


We present FasterSeg, an automatically designed semantic segmentation network with not only state-of-the-art performance but also faster speed than current methods.

Utilizing neural architecture search (NAS), FasterSeg is discovered from a novel and broader search space integrating multi-resolution branches, that has been recently found to be vital in manually designed segmentation models.

To better calibrate the balance between the goals of high accuracy and low latency, we propose a decoupled and fine-grained latency regularization, that effectively overcomes our observed phenomenons that the searched networks are prone to "collapsing" to low-latency yet poor-accuracy models.

Moreover, we seamlessly extend FasterSeg to a new collaborative search (co-searching) framework, simultaneously searching for a teacher and a student network in the same single run.

The teacher-student distillation further boosts the student model’s accuracy.

Experiments on popular segmentation benchmarks demonstrate the competency of FasterSeg.

For example, FasterSeg can run over 30% faster than the closest manually designed competitor on Cityscapes, while maintaining comparable accuracy.

We present a real-time segmentation model automatically discovered by a multi-scale NAS framework, achieving 30% faster than state-of-the-art models.

FasterSeg: Searching for Faster Real-time Semantic Segmentation


This paper introduces R2D3, an agent that makes efficient use of demonstrations to solve hard exploration problems in partially observable environments with highly variable initial conditions.

We also introduce a suite of eight tasks that combine these three properties, and show that R2D3 can solve several of the tasks where other state of the art methods (both with and without demonstrations) fail to see even a single successful trajectory after tens of billions of steps of exploration.

We introduce R2D3, an agent that makes efficient use of demonstrations to solve hard exploration problems in partially observable environments with highly variable initial conditions.

Making Efficient Use of Demonstrations to Solve Hard Exploration Problems

make efficient demonstration solve hard exploration problem


We investigate the learned dynamical landscape of a recurrent neural network solving a simple task requiring the interaction of two memory mechanisms: long- and short-term.

Our results show that while long-term memory is implemented by asymptotic attractors, sequential recall is now additionally implemented by oscillatory dynamics in a transverse subspace to the basins of attraction of these stable steady states.

Based on our observations, we propose how different types of memory mechanisms can coexist and work together in a single neural network, and discuss possible applications to the fields of artificial intelligence and neuroscience.

We investigate how a recurrent neural network successfully learns a task combining long-term memory and sequential recall.

An Investigation of Memory in Recurrent Neural Networks


The problem of exploration in reinforcement learning is well-understood in the tabular case and many sample-efficient algorithms are known.

Nevertheless, it is often unclear how the algorithms in the tabular setting can be extended to tasks with large state-spaces where generalization is required.

Recent promising developments generally depend on problem-specific density models or handcrafted features.

In this paper we introduce a simple approach for exploration that allows us to develop theoretically justified algorithms in the tabular case but that also give us intuitions for new algorithms applicable to settings where function approximation is required.

Our approach and its underlying theory is based on the substochastic successor representation, a concept we develop here.

While the traditional successor representation is a representation that defines state generalization by the similarity of successor states, the substochastic successor representation is also able to implicitly count the number of times each state (or feature) has been observed.

This extension connects two until now disjoint areas of research.

We show in traditional tabular domains (RiverSwim and SixArms) that our algorithm empirically performs as well as other sample-efficient algorithms.

We then describe a deep reinforcement learning algorithm inspired by these ideas and show that it matches the performance of recent pseudo-count-based methods in hard exploration Atari 2600 games.

We propose the idea of using the norm of the successor representation an exploration bonus in reinforcement learning. In hard exploration Atari games, our the deep RL algorithm matches the performance of recent pseudo-count-based methods.

Count-Based Exploration with the Successor Representation


Deep generative modeling using flows has gained popularity owing to the tractable exact log-likelihood estimation with efficient training and synthesis process.

However, flow models suffer from the challenge of having high dimensional latent space, same in dimension as the input space.

An effective solution to the above challenge as proposed by Dinh et al. (2016) is a multi-scale architecture, which is based on iterative early factorization of a part of the total dimensions at regular intervals.

Prior works on generative flows involving a multi-scale architecture perform the dimension factorization based on a static masking.

We propose a novel multi-scale architecture that performs data dependent factorization to decide which dimensions should pass through more flow layers.

To facilitate the same, we introduce a heuristic based on the contribution of each dimension to the total log-likelihood which encodes the importance of the dimensions.

Our proposed heuristic is readily obtained as part of the flow training process, enabling versatile implementation of our likelihood contribution based multi-scale architecture for generic flow models.

We present such an implementation for the original flow introduced in Dinh et al. (2016), and demonstrate improvements in log-likelihood score and sampling quality on standard image benchmarks.

We also conduct ablation studies to compare proposed method with other options for dimension factorization.

Data-dependent factorization of dimensions in a multi-scale architecture based on contribution to the total log-likelihood

Likelihood Contribution based Multi-scale Architecture for Generative Flows

multi-scale architecture
base
contribution


Understanding the flow of information in Deep Neural Networks (DNNs) is a challenging problem that has gain increasing attention over the last few years.

While several methods have been proposed to explain network predictions, there have been only a few attempts to compare them from a theoretical perspective.

What is more, no exhaustive empirical comparison has been performed in the past.

In this work we analyze four gradient-based attribution methods and formally prove conditions of equivalence and approximation between them.

By reformulating two of these methods, we construct a unified framework which enables a direct comparison, as well as an easier implementation.

Finally, we propose a novel evaluation metric, called Sensitivity-n and test the gradient-based attribution methods alongside with a simple perturbation-based attribution method on several datasets in the domains of image and text classification, using various network architectures.

Four existing backpropagation-based attribution methods are fundamentally similar. How to assess it?

Towards better understanding of gradient-based attribution methods for Deep Neural Networks


We study SGD and Adam for estimating a rank one signal planted in matrix or tensor noise.

The extreme simplicity of the problem setup allows us to isolate the effects of various factors: signal to noise ratio, density of critical points, stochasticity and initialization.

We observe a surprising phenomenon: Adam seems to get stuck in local minima as soon as polynomially many critical points appear (matrix case), while SGD escapes those.

However, when the number of critical points degenerates to exponentials (tensor case), then both algorithms get trapped.

Theory tells us that at fixed SNR the problem becomes intractable for large $d$ and in our experiments SGD does not escape this.

We exhibit the benefits of warm starting in those situations.

We conclude that in this class of problems, warm starting cannot be replaced by stochasticity in gradients to find the basin of attraction.

SGD and Adam under single spiked model for tensor PCA

Stochastic algorithms under single spiked models


This paper presents a method to explain the knowledge encoded in a convolutional neural network (CNN) quantitatively and semantically.

How to analyze the specific rationale of each prediction made by the CNN presents one of key issues of understanding neural networks, but it is also of significant practical values in certain applications.

In this study, we propose to distill knowledge from the CNN into an explainable additive model, so that we can use the explainable model to provide a quantitative explanation for the CNN prediction.

We analyze the typical bias-interpreting problem of the explainable model and develop prior losses to guide the learning of the explainable additive model.

Experimental results have demonstrated the effectiveness of our method.

Explaining Neural Networks Semantically and Quantitatively

semantically
quantitatively
neural network
explain


We present methodology for using dynamic evaluation to improve neural sequence models.

Models are adapted to recent history via a gradient descent based mechanism, causing them to assign higher probabilities to re-occurring sequential patterns.

Dynamic evaluation outperforms existing adaptation approaches in our comparisons.

Dynamic evaluation improves the state-of-the-art word-level perplexities on the Penn Treebank and WikiText-2 datasets to 51.1 and 44.3 respectively, and the state-of-the-art character-level cross-entropies on the text8 and Hutter Prize datasets to 1.19 bits/char and 1.08 bits/char respectively.

Paper presents dynamic evaluation methodology for adaptive sequence modelling

Dynamic Evaluation of Neural Sequence Models


We propose a novel, projection based way to incorporate the conditional information into the discriminator of GANs that respects the role of the conditional information in the underlining probabilistic model.

This approach is in contrast with most frameworks of conditional GANs used in application today, which use the conditional information by concatenating the (embedded) conditional vector to the feature vectors.

With this modification, we were able to significantly improve the quality of the class conditional image generation on ILSVRC2012 (ImageNet) dataset from the current state-of-the-art result, and we achieved this with a single pair of a discriminator and a generator.

We were also able to extend the application to super-resolution and succeeded in producing highly discriminative super-resolution images.

This new structure also enabled high quality category transformation based on parametric functional transformation of conditional batch normalization layers in the generator.

cGANs with Projection Discriminator


Learning theory tells us that more data is better when minimizing the generalization error of identically distributed training and test sets.

However, when training and test distribution differ, this distribution shift can have a significant effect.

With a novel perspective on function transfer learning, we are able to lower bound the change of performance when transferring from training to test set with the Wasserstein distance between the embedded training and test set distribution.

We find that there is a trade-off affecting performance between how invariant a function is to changes in training and test distribution and how large this shift in distribution is.

Empirically across several data domains, we substantiate this viewpoint by showing that test performance correlates strongly with the distance in data distributions between training and test set.

Complementary to the popular belief that more data is always better, our results highlight the utility of also choosing a training data distribution that is close to the test data distribution when the learned function is not invariant to such changes.

The Frechet Distance between train and test distribution correlates with the change in performance for functions that are not invariant to the shift.

The Frechet Distance of training and test distribution predicts the generalization gap


We introduce a new procedural dynamic system that can generate a variety of shapes that often appear as curves, but technically, the figures are plots of many points.

We name them spiroplots and show how this new system relates to other procedures or processes that generate figures.

Spiroplots are an extremely simple process but with a surprising visual variety.

We prove some fundamental properties and analyze some instances to see how the geometry or topology of the input determines the generated figures.

We show that some spiroplots have a finite cycle and return to the initial situation, whereas others will produce new points infinitely often.

This paper is accompanied by a JavaScript app that allows anyone to generate spiroplots.

A new, very simple dynamic system is introduced that generates pretty patterns; properties are proved and possibilities are explored

Spiroplots: a new discrete-time dynamical system to generate curve patterns


Unsupervised image-to-image translation aims to learn a mapping between several visual domains by using unpaired training pairs.

Recent studies have shown remarkable success in image-to-image translation for multiple domains but they suffer from two main limitations: they are either built from several two-domain mappings that are required to be learned independently and/or they generate low-diversity results, a phenomenon known as model collapse.

To overcome these limitations, we propose a method named GMM-UNIT based on a content-attribute disentangled representation, where the attribute space is fitted with a GMM.

Each GMM component represents a domain, and this simple assumption has two prominent advantages.

First, the dimension of the attribute space does not grow linearly with the number of domains, as it is the case in the literature.

Second, the continuous domain encoding allows for interpolation between domains and for extrapolation to unseen domains.

Additionally, we show how GMM-UNIT can be constrained down to different methods in the literature, meaning that GMM-UNIT is a unifying framework for unsupervised image-to-image translation.

GMM-UNIT is an image-to-image translation model that maps an image to multiple domains in a stochastic fashion.

GMM-UNIT: Unsupervised Multi-Domain and Multi-Modal Image-to-Image Translation via Attribute Gaussian Mixture Modelling


We present Compositional Attention Networks, a novel fully differentiable neural network architecture, designed to facilitate explicit and expressive reasoning.

While many types of neural networks are effective at learning and generalizing from massive quantities of data, this model moves away from monolithic black-box architectures towards a design that provides a strong prior for iterative reasoning, enabling it to support explainable and structured learning, as well as generalization from a modest amount of data.

The model builds on the great success of existing recurrent cells such as LSTMs: It sequences a single recurrent Memory, Attention, and Control (MAC) cell, and by careful design imposes structural constraints on the operation of each cell and the interactions between them, incorporating explicit control and soft attention mechanisms into their interfaces.

We demonstrate the model's strength and robustness on the challenging CLEVR dataset for visual reasoning, achieving a new state-of-the-art 98.9% accuracy, halving the error rate of the previous best model.

More importantly, we show that the new model is more computationally efficient, data-efficient, and requires an order of magnitude less time and/or data to achieve good results.

We present a novel architecture, based on dynamic memory, attention and composition for the task of machine reasoning.

Compositional Attention Networks for Machine Reasoning


Variational Auto-Encoders (VAEs) are designed to capture compressible information about a dataset.

As a consequence the information stored in the latent space is seldom sufficient to reconstruct a particular image.

To help understand the type of information stored in the latent space we train a GAN-style decoder constrained to produce images that the VAE encoder will map to the same region of latent space.

This allows us to ''imagine'' the information captured in the latent space.

We argue that this is necessary to make a VAE into a truly generative model.

We use our GAN to visualise the latent space of a standard VAE and of a $\beta$-VAE.

To understand the information stored in the latent space, we train a GAN-style decoder constrained to produce images that the VAE encoder will map to the same region of latent space.

Imagining the Latent Space of a Variational Auto-Encoders


Human brain function as measured by functional magnetic resonance imaging

In response, understanding the individual variability

of brain function and its association with behavior has become one of the

major concerns in modern cognitive neuroscience.

view that generative models provide a useful tool for understanding this variability.

To this end, this manuscript presents two novel generative models trained

on real neuroimaging data which synthesize task-dependent functional brain images.

Brain images are high dimensional tensors which exhibit structured spatial

Thus, both models are 3D conditional Generative Adversarial networks

(GANs) which apply Convolutional Neural Networks (CNNs) to learn an

abstraction of brain image representations.

brain images are diverse, yet task dependent.

we utilize the generated synthetic brain volumes as additional training data to improve

downstream fMRI classifiers (also known as decoding, or brain reading).

Our approach achieves significant improvements for a variety of datasets, classifi-

Our classification results provide a quantitative

evaluation of the quality of the generated images, and also serve as an additional

Two novel GANs are constructed to generate high-quality 3D fMRI brain images and synthetic brain images greatly help to improve downstream classification tasks.

Hallucinating brains with artificial brains


Transferring representations from large-scale supervised tasks to downstream tasks have shown outstanding results in Machine Learning in both Computer Vision and natural language processing (NLP).

One particular example can be sequence-to-sequence models for Machine Translation (Neural Machine Translation - NMT).

It is because, once trained in a multilingual setup, NMT systems can translate between multiple languages and are also capable of performing zero-shot translation between unseen source-target pairs at test time.

In this paper, we first investigate if we can extend the zero-shot transfer capability of multilingual NMT systems to cross-lingual NLP tasks (tasks other than MT, e.g. sentiment classification and natural language inference).

We demonstrate a simple framework by reusing the encoder from a multilingual NMT system, a multilingual Encoder-Classifier, achieves remarkable zero-shot cross-lingual classification performance, almost out-of-the-box on three downstream benchmark tasks - Amazon Reviews, Stanford sentiment treebank (SST) and Stanford natural language inference (SNLI).

In order to understand the underlying factors contributing to this finding, we conducted a series of analyses on the effect of the shared vocabulary, the training data type for NMT models, classifier complexity, encoder representation power, and model generalization on zero-shot performance.

Our results provide strong evidence that the representations learned from multilingual NMT systems are widely applicable across languages and tasks, and the high, out-of-the-box classification performance is correlated with the generalization capability of such systems.

Zero-shot cross-lingual transfer by using multilingual neural machine translation

On Zero-shot Cross-lingual Transfer of Multilingual Neural Machine Translation

zero-shot cross-lingual transfer multilingual neural machine translation


This paper addresses the scalability challenge of architecture search by formulating the task in a differentiable manner.

Unlike conventional approaches of applying evolution or reinforcement learning over a discrete and non-differentiable search space, our method is based on the continuous relaxation of the architecture representation, allowing efficient search of the architecture using gradient descent.

Extensive experiments on CIFAR-10, ImageNet, Penn Treebank and WikiText-2 show that our algorithm excels in discovering high-performance convolutional architectures for image classification and recurrent architectures for language modeling, while being orders of magnitude faster than state-of-the-art non-differentiable techniques.

We propose a differentiable architecture search algorithm for both convolutional and recurrent networks, achieving competitive performance with the state of the art using orders of magnitude less computation resources.

DARTS: Differentiable Architecture Search


Despite considerable advances in neural language modeling, it remains an open question what the best decoding strategy is for text generation from a language model (e.g.  to generate a story).

The counter-intuitive empirical observation is that even though the use of likelihood as training objective leads to high quality models for a broad range of language understanding tasks, maximization-based decoding methods such as beam search lead to degeneration — output text that is bland, incoherent, or gets stuck in repetitive loops.

To address this we propose Nucleus Sampling, a simple but effective method to draw considerably  higher  quality  text  out  of  neural  language  models.

Our  approach avoids text degeneration by truncating the unreliable tail of the probability distribution, sampling from the dynamic  nucleus  of  tokens containing the vast majority of the probability mass.

To properly examine current maximization-based and stochastic decoding methods, we compare generations from each of these methods to the distribution of human text along several axes such as likelihood, diversity, and repetition.

Our results show that (1) maximization is an inappropriate decoding objective for open-ended text generation, (2) the probability distributions of the best current language models have an unreliable tail which needs to be truncated during generation and (3) Nucleus Sampling is the best decoding strategy for generating long-form text that is both high-quality — as measured by human evaluation — and as diverse as human-written text.

Current language generation systems either aim for high likelihood and devolve into generic repetition or miscalibrate their stochasticity—we provide evidence of both and propose a solution: Nucleus Sampling.

The Curious Case of Neural Text Degeneration


Up until very recently, inspired by a mass of researches on adversarial examples for computer vision, there has been a growing interest in designing adversarial attacks for Natural Language Processing (NLP) tasks, followed by very few works of adversarial defenses for NLP.

To our knowledge, there exists no defense method against the successful synonym substitution based attacks that aim to satisfy all the lexical, grammatical, semantic constraints and thus are hard to perceived by humans.

We contribute to fill this gap and propose a novel adversarial defense method called Synonym Encoding Method (SEM), which inserts an encoder before the input layer of the model and then trains the model to eliminate adversarial perturbations.

Extensive experiments demonstrate that SEM can efficiently defend current best synonym substitution based adversarial attacks with little decay on the accuracy for benign examples.

To better evaluate SEM, we also design a strong attack method called Improved Genetic Algorithm (IGA) that adopts the genetic metaheuristic for synonym substitution based attacks.

Compared with existing genetic based adversarial attack, IGA can achieve higher attack success rate while maintaining the transferability of the adversarial examples.

The first text adversarial defense method in word level, and the improved generic based attack method against synonyms substitution based attacks.

Natural Language Adversarial Attack and Defense in Word Level


A major drawback of backpropagation through time (BPTT) is the difficulty of learning long-term dependencies, coming from having to propagate credit information backwards through every single step of the forward computation.

This makes BPTT both computationally impractical and biologically implausible.

For this reason, full backpropagation through time is rarely used on long sequences, and truncated backpropagation through time is used as a heuristic.

However, this usually leads to biased estimates of the gradient in which longer term dependencies are ignored.

Addressing this issue, we propose an alternative algorithm, Sparse Attentive Backtracking, which might also be related to principles used by brains to learn long-term dependencies.

Sparse Attentive Backtracking learns an attention mechanism over the hidden states of the past and selectively backpropagates through paths with high attention weights.

This allows the model to learn long term dependencies while only backtracking for a small number of time steps, not just from the recent past but also from attended relevant past states.

Towards Efficient Credit Assignment in Recurrent Networks without Backpropagation Through Time

Sparse Attentive Backtracking: Long-Range Credit Assignment in Recurrent Networks


We propose a novel  adversarial learning framework in this work.

Existing adversarial learning methods involve two separate networks, i.e., the structured prediction models and the discriminative models, in the training.

The information captured by discriminative models complements that in the structured prediction models, but few existing researches have studied on utilizing such information to improve structured prediction models at the inference stage.

In this work, we propose to refine the predictions of structured prediction models by effectively integrating discriminative models into the prediction.

Discriminative models are treated as energy-based models.

Similar to the adversarial learning, discriminative models are trained to estimate scores which measure the quality of predicted outputs, while structured prediction models are trained to predict contrastive outputs with maximal energy scores.

In this way, the gradient vanishing problem is ameliorated, and thus we are able to perform inference by following the ascent gradient directions of discriminative models to refine structured prediction models.

The proposed method is able to handle a range of tasks, \emph{e.g.}, multi-label classification and image segmentation.

Empirical results on these two tasks validate the effectiveness of our learning method.

We propose a novel adversarial learning framework for structured prediction, in which discriminative models can be used to refine structured prediction models at the inference stage.

Learning Discriminators as Energy Networks in Adversarial Learning


We propose RaPP, a new methodology for novelty detection by utilizing hidden space activation values obtained from a deep autoencoder.

Precisely, RaPP compares input and its autoencoder reconstruction not only in the input space but also in the hidden spaces.

We show that if we feed a reconstructed input to the same autoencoder again, its activated values in a hidden space are equivalent to the corresponding reconstruction in that hidden space given the original input.

In order to aggregate the hidden space activation values, we propose two metrics, which enhance the novelty detection performance.

Through extensive experiments using diverse datasets, we validate that RaPP improves novelty detection performances of autoencoder-based approaches.

Besides, we show that RaPP outperforms recent novelty detection methods evaluated on popular benchmarks.

A new methodology for novelty detection by utilizing hidden space activation values obtained from a deep autoencoder.

RaPP: Novelty Detection with Reconstruction along Projection Pathway


Learning preferences of users over plan traces can be a challenging task given a large number of features and limited queries that we can ask a single user.

Additionally, the preference function itself can be quite convoluted and non-linear.

Our approach uses feature-directed active learning to gather the necessary information about plan trace preferences.

This data is used to train a simple feedforward neural network to learn preferences over the sequential data.

We evaluate the impact of active learning on the number of traces that are needed to train a model that is accurate and interpretable.

This evaluation is done by comparing the aforementioned feedforward network to a more complex neural network model that uses LSTMs and is trained with a larger dataset without active learning.

Learning preferences over plan traces using active learning.

Feature-directed Active Learning for Learning User Preferences


Reinforcement learning is a promising framework for solving control problems, but its use in practical situations is hampered by the fact that reward functions are often difficult to engineer.

Specifying goals and tasks for autonomous machines, such as robots, is a significant challenge: conventionally, reward functions and goal states have been used to communicate objectives.

But people can communicate objectives to each other simply by describing or demonstrating them.

How can we build learning algorithms that will allow us to tell machines what we want them to do?

In this work, we investigate the problem of grounding language commands as reward functions using inverse reinforcement learning, and argue that language-conditioned rewards are more transferable than language-conditioned policies to new environments.

We propose language-conditioned reward learning (LC-RL), which grounds language commands as a reward function represented by a deep neural network.

We demonstrate that our model learns rewards that transfer to novel tasks and environments on realistic, high-dimensional visual environments with natural language commands, whereas directly learning a language-conditioned policy leads to poor performance.

We ground language commands in a high-dimensional visual environment by learning language-conditioned rewards using inverse reinforcement learning.

From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Following


Many biological learning systems such as the mushroom body, hippocampus, and cerebellum are built from sparsely connected networks of neurons.

For a new understanding of such networks, we study the function spaces induced by sparse random features and characterize what functions may and may not be learned.

A network with d inputs per neuron is found to be equivalent to an additive model of order d, whereas with a degree distribution the network combines additive terms of different orders.

We identify three specific advantages of sparsity: additive function approximation is a powerful inductive bias that limits the curse of dimensionality, sparse networks are stable to outlier noise in the inputs, and sparse random features are scalable.

Thus, even simple brain architectures can be powerful function approximators.

Finally, we hope that this work helps popularize kernel theories of networks among computational neuroscientists.

We advocate for random features as a theory of biological neural networks, focusing on sparsely connected networks

Additive function approximation in the brain