Existing multi-agent reinforcement learning (MARL) communication methods have relied on a trusted third party (TTP) to distribute reward to agents, leaving them inapplicable in peer-to-peer environments.

This paper proposes reward distribution using {\em Neuron as an Agent} (NaaA) in MARL without a TTP with two key ideas:

Auction theory is introduced because inter-agent reward distribution is insufficient for optimization.

Agents in NaaA maximize their profits (the difference between reward and cost) and, as a theoretical result, the auction mechanism is shown to have agents autonomously evaluate counterfactual returns as the values of other agents.

NaaA enables representation trades in peer-to-peer environments, ultimately regarding unit in neural networks as agents.

Finally, numerical experiments (a single-agent environment from OpenAI Gym and a multi-agent environment from ViZDoom) confirm that NaaA framework optimization leads to better performance in reinforcement learning.

Neuron as an Agent (NaaA) enable us to train multi-agent communication without a trusted third party.

Neuron as an Agent


Increasing model size when pretraining natural language representations often results in improved performance on downstream tasks.

However, at some point further model increases become harder due to GPU/TPU memory limitations, longer training times, and unexpected model degradation.

To address these problems,  we present two parameter-reduction techniques to lower memory consumption and increase the training speed of BERT.

Comprehensive empirical evidence shows that our proposed methods lead to models that scale much better compared to the original BERT.

We also use a self-supervised loss that focuses on modeling inter-sentence coherence, and show it consistently helps downstream tasks with multi-sentence inputs.

As a result, our best model establishes new state-of-the-art results on the GLUE, RACE, and SQuAD benchmarks while having fewer parameters compared to BERT-large.

A new pretraining method that establishes new state-of-the-art results on the GLUE, RACE, and SQuAD benchmarks while having fewer parameters compared to BERT-large.

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations


Structured tabular data is the most commonly used form of data in industry according to a Kaggle ML and DS Survey.

Gradient Boosting Trees, Support Vector Machine, Random Forest, and Logistic Regression are typically used for classification tasks on tabular data.

The recent work of Super Characters method using two-dimensional word embeddings achieved state-of-the-art results in text classification tasks, showcasing the promise of this new approach.

In this paper, we propose the SuperTML method, which borrows the idea of Super Characters method and two-dimensional embeddings to address the problem of classification on tabular data.

For each input of tabular data, the features are first projected into two-dimensional embeddings like an image, and then this image is fed into fine-tuned ImageNet CNN models for classification.

Experimental results have shown that the proposed SuperTML method have achieved state-of-the-art results on both large and small datasets.

Deep learning on structured tabular data using two-dimensional word embedding with fine-tuned ImageNet pre-trained CNN model.

SuperTML: Domain Transfer from Computer Vision to Structured Tabular Data through Two-Dimensional Word Embedding

tabular datum two-dimensional word embed


Predictive coding, within theoretical neuroscience, and variational autoencoders, within machine learning, both involve latent Gaussian models and variational inference.

While these areas share a common origin, they have evolved largely independently.

We outline connections and contrasts between these areas, using their relationships to identify new parallels between machine learning and neuroscience.

We then discuss specific frontiers at this intersection: backpropagation, normalizing flows, and attention, with mutual benefits for both fields.

connections between predictive coding and VAEs + new frontiers

Predictive Coding, Variational Autoencoders, and Biological Connections


Adaptive optimization methods such as AdaGrad, RMSprop and Adam have been proposed to achieve a rapid training process with an element-wise scaling term on learning rates.

Though prevailing, they are observed to generalize poorly compared with SGD or even fail to converge due to unstable and extreme learning rates.

Recent work has put forward some algorithms such as AMSGrad to tackle this issue but they failed to achieve considerable improvement over existing methods.

In our paper, we demonstrate that extreme learning rates can lead to poor performance.

We provide new variants of Adam and AMSGrad, called AdaBound and AMSBound respectively, which employ dynamic bounds on learning rates to achieve a gradual and smooth transition from adaptive methods to SGD and give a theoretical proof of convergence.

We further conduct experiments on various popular tasks and models, which is often insufficient in previous work.

Experimental results show that new variants can eliminate the generalization gap between adaptive methods and SGD and maintain higher learning speed early in training at the same time.

Moreover, they can bring significant improvement over their prototypes, especially on complex deep networks.

The implementation of the algorithm can be found at https://github.com/Luolc/AdaBound .

Novel variants of optimization methods that combine the benefits of both adaptive and non-adaptive methods.

Adaptive Gradient Methods with Dynamic Bound of Learning Rate


An important problem that arises in reinforcement learning and Monte Carlo methods is estimating quantities defined by the stationary distribution of a Markov chain.

In many real-world applications, access to the underlying transition operator is limited to a fixed set of data that has already been collected, without additional interaction with the environment being available.

We show that consistent estimation remains possible in this scenario, and that effective estimation can still be achieved in important applications.

Our approach is based on estimating a ratio that corrects for the discrepancy between the stationary and empirical distributions, derived from fundamental properties of the stationary distribution, and exploiting constraint reformulations based on variational divergence minimization.

The resulting algorithm, GenDICE, is straightforward and effective.

We prove the consistency of the method under general conditions, provide a detailed error analysis, and demonstrate strong empirical performance on benchmark tasks, including off-line PageRank and off-policy policy evaluation.

In this paper, we proposed a novel algorithm, GenDICE, for general stationary distribution correction estimation, which can handle both discounted and average off-policy evaluation on multiple behavior-agnostic samples.

GenDICE: Generalized Offline Estimation of Stationary Values


The power of neural networks lies in their ability to generalize to unseen data, yet the underlying reasons for this phenomenon remain elusive.

Numerous rigorous attempts have been made to explain generalization, but available bounds are still quite loose, and analysis does not always lead to true understanding.

The goal of this work is to make generalization more intuitive.

Using visualization methods, we discuss the mystery of generalization, the geometry of loss landscapes, and how the curse (or, rather, the blessing) of dimensionality causes optimizers to settle into minima that generalize well.

An intuitive empirical and visual exploration of the generalization properties of deep neural networks.

The Blessing of Dimensionality: An Empirical Study of Generalization


We argue that symmetry is an important consideration in addressing the problem

of systematicity and investigate two forms of symmetry relevant to symbolic processes.

We implement this approach in terms of convolution and show that it can

be used to achieve effective generalisation in three toy problems: rule learning,

We use convolution to make neural networks behave more like symbolic systems.

Symmetry and Systematicity


Key equatorial climate phenomena such as QBO and ENSO have never been adequately explained as deterministic processes.

This in spite of recent research showing growing evidence of predictable behavior.

This study applies the fundamental Laplace tidal equations with simplifying assumptions along the equator — i.e. no Coriolis force and a small angle approximation.

The solutions to the partial differential equations are highly non-linear related to Navier-Stokes and only search approaches can be used to fit to the data.

Analytical Formulation of Equatorial Standing Wave Phenomena: Application to QBO and ENSO

Nonlinear Differential Equations with external forcing


In the field of Continual Learning, the objective is to learn several tasks one after the other without access to the data from previous tasks.

Several solutions have been proposed to tackle this problem but they usually  assume that the user knows which of the tasks to perform at test time on a particular sample, or rely on small samples from previous data and most of them suffer of a substantial drop in accuracy when updated with batches of only one class at a time.

In this article, we propose a new method, OvA-INN, which is able to learn one class at a time and without storing any of the previous data.

To achieve this, for each class, we train a specific Invertible Neural Network to output the zero vector for its class.

At test time, we can predict the class of a sample by identifying which network outputs the vector with the smallest norm.

With this method, we show that we can take advantage of pretrained models by stacking an invertible network on top of a features extractor.

This way, we are able to outperform state-of-the-art approaches that rely on features learning for the Continual Learning of MNIST and CIFAR-100 datasets.

In our experiments, we are reaching 72% accuracy on CIFAR-100 after training our model one class at a time.

We propose to train an Invertible Neural Network for each class to perform class-by-class Continual Learning.

OvA-INN: Continual Learning with Invertible Neural Networks

invertible neural network
continual learning


Human-computer conversation systems have attracted much attention in Natural Language Processing.

Conversation systems can be roughly divided into two categories: retrieval-based and generation-based systems.

Retrieval systems search a user-issued utterance (namely a query) in a large conversational repository and return a reply that best matches the query.

Generative approaches synthesize new replies.

Both ways have certain advantages but suffer from their own disadvantages.

We propose a novel ensemble of retrieval-based and generation-based conversation system.

The retrieved candidates, in addition to the original query, are fed to a reply generator via a neural network, so that the model is aware of more information.

The generated reply together with the retrieved ones then participates in a re-ranking process to find the final reply to output.

Experimental results show that such an ensemble system outperforms each single module by a large margin.

A novel ensemble of retrieval-based and generation-based for open-domain conversation systems.

An Ensemble of Retrieval-Based and Generation-Based Human-Computer Conversation Systems.


Deep neural networks trained on a wide range of datasets demonstrate impressive transferability.

Deep features appear general in that they are applicable to many datasets and tasks.

Such property is in prevalent use in real-world applications.

A neural network pretrained on large datasets, such as ImageNet, can significantly boost generalization and accelerate training if fine-tuned to a smaller target dataset.

Despite its pervasiveness, few effort has been devoted to uncovering the reason of transferability in deep feature representations.

This paper tries to understand transferability from the perspectives of improved generalization, optimization and the feasibility of transferability.

1) Transferred models tend to find flatter minima, since their weight matrices stay close to the original flat region of pretrained parameters when transferred to a similar target dataset;

2) Transferred representations make the loss landscape more favorable with improved Lipschitzness, which accelerates and stabilizes training substantially.

The improvement largely attributes to the fact that the principal component of gradient is suppressed in the pretrained parameters, thus stabilizing the magnitude of gradient in back-propagation.

3) The feasibility of transferability is related to the similarity of both input and label.

And a surprising discovery is that the feasibility is also impacted by the training stages in that the transferability first increases during training, and then declines.

We further provide a theoretical analysis to verify our observations.

Understand transferability from the perspectives of improved generalization, optimization and the feasibility of transferability.

Towards Understanding the Transferability of Deep Representations


We address the following question: How redundant is the parameterisation of ReLU networks?

Specifically, we consider transformations of the weight space which leave the function implemented by the network intact.

Two such transformations are known for feed-forward architectures: permutation of neurons within a layer, and positive scaling of all incoming weights of a neuron coupled with inverse scaling of its outgoing weights.

In this work, we show for architectures with non-increasing widths that permutation and scaling are in fact the only function-preserving weight transformations.

For any eligible architecture we give an explicit construction of a neural network such that any other network that implements the same function can be obtained from the original one by the application of permutations and rescaling.

The proof relies on a geometric understanding of boundaries between linear regions of ReLU networks, and we hope the developed mathematical tools are of independent interest.

We prove that there exist ReLU networks whose parameters are almost uniquely determined by the function they implement.

Functional vs. parametric equivalence of ReLU networks


A general problem that received considerable recent attention is how to perform multiple tasks in the same network, maximizing both efficiency and prediction accuracy.

A popular approach consists of a multi-branch architecture on top of a

shared backbone, jointly trained on a weighted sum of losses.

However, in many cases, the shared representation results in non-optimal performance, mainly due to an interference between conflicting gradients of uncorrelated tasks.

Recent approaches address this problem by a channel-wise modulation of the feature-maps along the shared backbone, with task specific vectors, manually or dynamically tuned.

Taking this approach a step further, we propose a novel architecture which

modulate the recognition network channel-wise, as well as spatial-wise, with an efficient top-down image-dependent computation scheme.

Our architecture uses no task-specific branches, nor task specific modules.

Instead, it uses a top-down modulation network that is shared between all of the tasks.

We show the effectiveness of our scheme by achieving on par or better results than alternative approaches on both correlated and uncorrelated sets of tasks.

We also demonstrate our advantages in terms of model size, the addition of novel tasks and interpretability.

We propose a top-down modulation network for multi-task learning applications with several advantages over current schemes.

Task-Based Top-Down Modulation Network for Multi-Task-Learning Applications


Clustering algorithms have wide applications and play an important role in data analysis fields including time series data analysis.

The performance of a clustering algorithm depends on the features extracted from the data.

However, in time series analysis, there has been a problem that the conventional methods based on the signal shape are unstable for phase shift, amplitude and signal length variations.

In this paper, we propose a new clustering algorithm focused on the dynamical system aspect of the signal using recurrent neural network and variational Bayes method.

Our experiments show that our proposed algorithm has a robustness against above variations and boost the classification performance.

Novel time series data clustring algorithm based on dynamical system features.

Dynamical Clustering of Time Series Data Using Multi-Decoder RNN Autoencoder


Two important topics in deep learning both involve incorporating humans into the modeling process: Model priors transfer information from humans to a model by regularizing the model's parameters; Model attributions transfer information from a model to humans by explaining the model's behavior.

Previous work has taken important steps to connect these topics through various forms of gradient regularization.

We find, however, that existing methods that use attributions to align a model's behavior with human intuition are ineffective.

We develop an efficient and theoretically grounded feature attribution method, expected gradients, and a novel framework, attribution priors, to enforce prior expectations about a model's behavior during training.

We demonstrate that attribution priors are broadly applicable by instantiating them on three different types of data: image data, gene expression data, and health care data.

Our experiments show that models trained with attribution priors are more intuitive and achieve better generalization performance than both equivalent baselines and existing methods to regularize model behavior.

A method for encouraging axiomatic feature attributions of a deep model to match human intuition.

Learning Explainable Models Using Attribution Priors


Recurrent neural networks (RNNs) have shown excellent performance in processing sequence data.

However, they are both complex and memory intensive due to their recursive nature.

These limitations make RNNs difficult to embed on mobile devices requiring real-time processes with limited hardware resources.

To address the above issues, we introduce a method that can learn binary and ternary weights during the training phase to facilitate hardware implementations of RNNs.

As a result, using this approach replaces all multiply-accumulate operations by simple accumulations, bringing significant benefits to custom hardware in terms of silicon area and power consumption.

On the software side, we evaluate the performance (in terms of accuracy) of our method using long short-term memories (LSTMs) and gated recurrent units (GRUs) on various sequential models including sequence classification and language modeling.

We demonstrate that our method achieves competitive results on the aforementioned tasks while using binary/ternary weights during the runtime.

On the hardware side, we present custom hardware for accelerating the recurrent computations of LSTMs with binary/ternary weights.

Ultimately, we show that LSTMs with binary/ternary weights can achieve up to 12x memory saving and 10x inference speedup compared to the full-precision hardware implementation design.

We propose high-performance LSTMs with binary/ternary weights, that can greatly reduce implementation complexity

Learning Recurrent Binary/Ternary Weights


This paper addresses unsupervised few-shot object recognition, where all training images are unlabeled and do not share classes with labeled support images for few-shot recognition in testing.

We use a new GAN-like deep architecture aimed at unsupervised learning of an image representation which will encode latent object parts and thus generalize well to unseen classes in our few-shot recognition task.

Our unsupervised training integrates adversarial, self-supervision, and deep metric learning.

First, we extend the vanilla GAN with reconstruction loss to enforce the discriminator capture the most relevant characteristics of "fake" images generated from randomly sampled codes.

Second, we compile a training set of triplet image examples for estimating the triplet loss in metric learning by using an image masking procedure suitably designed to identify latent object parts.

Hence, metric learning ensures that the deep representation of images showing similar object classes which share some parts are closer than the representations of images which do not have common parts.

Our results show that we significantly outperform the state of the art, as well as get similar performance to the common episodic training for fully-supervised few-shot learning on the Mini-Imagenet and Tiered-Imagenet datasets.

We address the problem of unsupervised few-shot object recognition, where all training images are unlabeled and do not share classes with test images.

Unsupervised Few-shot Object Recognition by Integrating Adversarial, Self-supervision, and Deep Metric Learning of Latent Parts


Existing black-box attacks on deep neural networks (DNNs) so far have largely focused on transferability, where an adversarial instance generated for a locally trained model can “transfer” to attack other learning models.

In this paper, we propose novel Gradient Estimation black-box attacks for adversaries with query access to the target model’s class probabilities, which do not rely on transferability.

We also propose strategies to decouple the number of queries required to generate each adversarial sample from the dimensionality of the input.

An iterative variant of our attack achieves close to 100% adversarial success rates for both targeted and untargeted attacks on DNNs.

We carry out extensive experiments for a thorough comparative evaluation of black-box attacks and show that the proposed Gradient Estimation attacks outperform all transferability based black-box attacks we tested on both MNIST and CIFAR-10 datasets, achieving adversarial success rates similar to well known, state-of-the-art white-box attacks.

We also apply the Gradient Estimation attacks successfully against a real-world content moderation classiﬁer hosted by Clarifai.

Furthermore, we evaluate black-box attacks against state-of-the-art defenses.

We show that the Gradient Estimation attacks are very effective even against these defenses.

Query-based black-box attacks on deep neural networks with adversarial success rates matching white-box attacks

Exploring the Space of Black-box Attacks on Deep Neural Networks


We propose a novel subgraph image representation for classification of network fragments with the target being their parent networks.

The graph image representation is based on 2D image embeddings of adjacency matrices.

We use this image representation in two modes.

First, as the input to a machine learning algorithm.

Second, as the input to a pure transfer learner.

Our conclusions from multiple datasets are that

1. deep learning using structured image features performs the best compared to graph kernel and classical features based methods; and,

2. pure transfer learning works effectively with minimum interference from the user and is robust against small data.

We convert subgraphs into structured images and classify them using 1. deep learning and 2. transfer learning (Caffe) and achieve stunning results.

Network Signatures from Image Representation of Adjacency Matrices: Deep/Transfer Learning for Subgraph Classification


This paper explores the use of self-ensembling for visual domain adaptation problems.

Our technique is derived from the mean teacher variant (Tarvainen et. al 2017) of temporal ensembling (Laine et al. 2017), a technique that achieved state of the art results in the area of semi-supervised learning.

We introduce a number of modifications to their approach for challenging domain adaptation scenarios and evaluate its effectiveness.

Our approach achieves state of the art results in a variety of benchmarks, including our winning entry in the VISDA-2017 visual domain adaptation challenge.

In small image benchmarks, our algorithm not only outperforms prior art, but can also achieve accuracy that is close to that of a classifier trained in a supervised fashion.

Self-ensembling based algorithm for visual domain adaptation, state of the art results, won VisDA-2017 image classification domain adaptation challenge.

Self-ensembling for visual domain adaptation


It is easy for people to imagine what a man with pink hair looks like, even if they have never seen such a person before.

We call the ability to create images of novel semantic concepts visually grounded imagination.

In this paper, we show how we can modify variational auto-encoders to perform this task.

Our method uses a novel training objective, and a novel product-of-experts inference network, which can handle partially specified (abstract) concepts in a principled and efficient way.

We also propose a set of easy-to-compute evaluation metrics that capture our intuitive notions of what it means to have good visual imagination, namely correctness, coverage, and compositionality (the 3 C’s).

Finally, we perform a detailed comparison of our method with two existing joint image-attribute VAE methods (the JMVAE method of Suzuki et al., 2017 and the BiVCCA method of Wang et al., 2016) by applying them to two datasets: the MNIST-with-attributes dataset (which we introduce here), and the CelebA dataset (Liu et al., 2015).

A VAE-variant which can create diverse images corresponding to novel concrete or abstract "concepts" described using attribute vectors.

Generative Models of Visually Grounded Imagination


We introduce "Search with Amortized Value Estimates" (SAVE), an approach for combining model-free Q-learning with model-based Monte-Carlo Tree Search (MCTS).

In SAVE, a learned prior over state-action values is used to guide MCTS, which estimates an improved set of state-action values.

The new Q-estimates are then used in combination with real experience to update the prior.

This effectively amortizes the value computation performed by MCTS, resulting in a cooperative relationship between model-free learning and model-based search.

SAVE can be implemented on top of any Q-learning agent with access to a model, which we demonstrate by incorporating it into agents that perform challenging physical reasoning tasks and Atari.

SAVE consistently achieves higher rewards with fewer training steps, and---in contrast to typical model-based search approaches---yields strong performance with very small search budgets.

By combining real experience with information computed during search, SAVE demonstrates that it is possible to improve on both the performance of model-free learning and the computational cost of planning.

We propose a model-based method called "Search with Amortized Value Estimates" (SAVE) which leverages both real and planned experience by combining Q-learning with Monte-Carlo Tree Search, achieving strong performance with very small search budgets.

Combining Q-Learning and Search with Amortized Value Estimates


Two main families of reinforcement learning algorithms, Q-learning and policy gradients, have recently been proven to be equivalent when using a softmax relaxation on one part, and an entropic regularization on the other.

We relate this result to the well-known convex duality of Shannon entropy and the softmax function.

Such a result is also known as the Donsker-Varadhan formula.

This provides a short proof of the equivalence.

We then interpret this duality further, and use ideas of convex analysis to prove a new policy inequality relative to soft Q-learning.

A short proof of the equivalence of soft Q-learning and policy gradients.

Representing Entropy : A short proof of the equivalence between soft Q-learning and policy gradients

proof equivalence soft q-learning policy gradient


Computer simulation provides an automatic and safe way for training robotic control

policies to achieve complex tasks such as locomotion.

trained in simulation usually does not transfer directly to the real hardware due

to the differences between the two environments.

randomization is a promising approach, but it usually assumes that the target environment

is close to the distribution of the training environments, thus relying

heavily on accurate system identification.

approach that leverages domain randomization for transferring control policies to

The key idea that, instead of learning a single policy in

the simulation, we simultaneously learn a family of policies that exhibit different

When tested in the target environment, we directly search for the best

policy in the family based on the task performance, without the need to identify

We evaluate our method on five simulated robotic control

problems with different discrepancies in the training and testing environment

and demonstrate that our method can overcome larger modeling errors compared

to training a robust policy or an adaptive policy.

We propose a policy transfer algorithm that can overcome large and challenging discrepancies in the system dynamics such as latency, actuator modeling error, etc.

Policy Transfer with Strategy Optimization


We propose vq-wav2vec to learn discrete representations of audio segments through a wav2vec-style self-supervised context prediction task.

The algorithm uses either a gumbel softmax or online k-means clustering to quantize the dense representations.

Discretization enables the direct application of algorithms from the NLP community which require discrete inputs.

Experiments show that BERT pre-training achieves a new state of the art on TIMIT phoneme classification and WSJ speech recognition.

Learn how to quantize speech signal and apply algorithms requiring discrete inputs to audio data such as BERT.

vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations


Deep Reinforcement Learning algorithms lead to agents that can solve difficult decision making problems in complex environments.

However, many difficult multi-agent competitive games, especially real-time strategy games are still considered beyond the capability of current deep reinforcement learning algorithms, although there has been a recent effort to change this \citep{openai_2017_dota, vinyals_2017_starcraft}.

Moreover, when the opponents in a competitive game are suboptimal, the current \textit{Nash Equilibrium} seeking, self-play algorithms are often unable to generalize their strategies to opponents that play strategies vastly different from their own.

This suggests that a learning algorithm that is beyond conventional self-play is necessary.

We develop Hierarchical Agent with Self-play (HASP), a learning approach for obtaining hierarchically structured policies that can achieve higher performance than conventional self-play on competitive games through the use of a diverse pool of sub-policies we get from Counter Self-Play (CSP).

We demonstrate that the ensemble policy generated by HASP can achieve better performance while facing unseen opponents that use sub-optimal policies.

On a motivating iterated Rock-Paper-Scissor game and a partially observable real-time strategic game (http://generals.io/), we are led to the conclusion that HASP can perform better than conventional self-play as well as achieve 77% win rate against FloBot, an open-source agent which has ranked at position number 2 on the online leaderboards.

We develop Hierarchical Agent with Self-play (HASP), a learning approach for obtaining hierarchically structured policies that can achieve high performance than conventional self-play on competitive real-time strategic games.

Hierarchical Deep Reinforcement Learning Agent with Counter Self-play  on Competitive Games

agent competitive learn hierarchical game self-play


We introduce adaptive input representations for neural language modeling which extend the adaptive softmax of Grave et al. (2017) to input representations of variable capacity.

There are several choices on how to factorize the input and output layers, and whether to model words, characters or sub-word units.

We perform a systematic comparison of popular choices for a self-attentional architecture.

Our experiments show that models equipped with adaptive embeddings are more than twice as fast to train than the popular character input CNN while having a lower number of parameters.

On the WikiText-103 benchmark we achieve 18.7 perplexity, an improvement of 10.5 perplexity compared to the previously best published result and on the Billion Word benchmark, we achieve 23.02 perplexity.

Variable capacity input word embeddings and SOTA on WikiText-103, Billion Word benchmarks.

Adaptive Input Representations for Neural Language Modeling


In this paper, we consider the problem of detecting object under occlusion.

Most object detectors formulate bounding box regression as a unimodal task (i.e., regressing a single set of bounding box coordinates independently).

However, we observe that the bounding box borders of an occluded object can have multiple plausible configurations.

Also, the occluded bounding box borders have correlations with visible ones.

Motivated by these two observations, we propose a deep multivariate mixture of Gaussians model for bounding box regression under occlusion.

The mixture components potentially learn different configurations of an occluded part, and the covariances between variates help to learn the relationship between the occluded parts and the visible ones.

Quantitatively, our model improves the AP of the baselines by 3.9% and 1.2% on CrowdHuman and MS-COCO respectively with almost no computational or memory overhead.

Qualitatively, our model enjoys explainability since we can interpret the resulting bounding boxes via the covariance matrices and the mixture components.

Deep Multivariate Mixture of Gaussians for Object Detection under Occlusion

deep multivariate mixture gaussian
occlusion


Adversarial examples are a pervasive phenomenon of machine learning models where seemingly imperceptible perturbations to the input lead to misclassifications for otherwise statistically accurate models.

Adversarial training, one of the most successful empirical defenses to adversarial examples, refers to training on adversarial examples generated within a geometric constraint set.

The most commonly used geometric constraint is an $L_p$-ball of radius $\epsilon$ in some norm.

We introduce adversarial training with Voronoi constraints, which replaces the $L_p$-ball constraint with the Voronoi cell for each point in the training set.

We show that adversarial training with Voronoi constraints produces robust models which significantly improve over the state-of-the-art on MNIST and are competitive on CIFAR-10.

We replace the Lp ball constraint with the Voronoi cells of the training data to produce more robust models.

Adversarial Training with Voronoi Constraints


Recent research efforts enable study for natural language grounded navigation in photo-realistic environments, e.g., following natural language instructions or dialog.

However, existing methods tend to overfit training data in seen environments and fail to generalize well in previously unseen environments.

In order to close the gap between seen and unseen environments, we aim at learning a generalizable navigation model from two novel perspectives:

(1) we introduce a multitask navigation model that can be seamlessly trained on both Vision-Language Navigation (VLN) and Navigation from Dialog History (NDH) tasks, which benefits from richer natural language guidance and effectively transfers knowledge across tasks;

(2) we propose to learn environment-agnostic representations for navigation policy that are invariant among environments, thus generalizing better on unseen environments.

Extensive experiments show that our environment-agnostic multitask navigation model significantly reduces the performance gap between seen and unseen environments and outperforms the baselines on unseen environments by 16% (relative measure on success rate) on VLN and 120% (goal progress) on NDH, establishing the new state of the art for NDH task.

We propose to learn a more generalized policy for natural language grounded navigation tasks via environment-agnostic multitask learning.

Generalized Natural Language Grounded Navigation via Environment-agnostic Multitask Learning


In this paper we propose to perform model ensembling in a multiclass or a multilabel learning setting using Wasserstein (W.) barycenters.

Optimal transport metrics, such as the Wasserstein distance, allow incorporating semantic side information such as word embeddings.

Using W. barycenters to find the consensus between models allows us to balance confidence and semantics in finding the agreement between the models.

We show applications of Wasserstein ensembling in attribute-based classification, multilabel learning and image captioning generation.

These results show that the W. ensembling is a viable alternative to the basic geometric or arithmetic mean ensembling.

we propose to use Wasserstein barycenters for semantic model ensembling

Wasserstein Barycenter Model Ensembling


While deep learning has been incredibly successful in modeling tasks with large, carefully curated labeled datasets, its application to problems with limited labeled data remains a challenge.

The aim of the present work is to improve the label efficiency of large neural networks operating on audio data through a combination of multitask learning and self-supervised learning on unlabeled data.

We trained an end-to-end audio feature extractor based on WaveNet that feeds into simple, yet versatile task-specific neural networks.

We describe several easily implemented self-supervised learning tasks that can operate on any large, unlabeled audio corpus.

We demonstrate that, in scenarios with limited labeled training data, one can significantly improve the performance of three different supervised classification tasks individually by up to 6% through simultaneous training with these additional self-supervised tasks.

We also show that incorporating data augmentation into our multitask setting leads to even further gains in performance.

Label-efficient audio classification via multi-task learning and self-supervision

LABEL-EFFICIENT AUDIO CLASSIFICATION THROUGH MULTITASK LEARNING AND SELF-SUPERVISION

label-efficient audio classification
learning self-supervision


Recent neural network and language models have begun to rely on softmax distributions with an extremely large number of categories.

In this context calculating the softmax normalizing constant is prohibitively expensive.

This has spurred a growing literature of efficiently computable but biased estimates of the softmax.

In this paper we present the first two unbiased algorithms for maximizing the softmax likelihood whose work per iteration is independent of the number of classes and datapoints (and does not require extra work at the end of each epoch).

We compare our unbiased methods' empirical performance to the state-of-the-art on seven real world datasets, where they comprehensively outperform all competitors.

Propose first methods for exactly optimizing the softmax distribution using stochastic gradient with runtime independent on the number of classes or datapoints.

Unbiased scalable softmax optimization


Crafting adversarial examples on discrete inputs like text sequences is fundamentally different from generating such examples for continuous inputs like images.

This paper tries to answer the question: under a black-box setting, can we create adversarial examples automatically to effectively fool deep learning classifiers on texts by making imperceptible changes?

Previous efforts mostly replied on using gradient evidence, and they are less effective either due to finding the nearest neighbor word (wrt meaning) automatically is difficult or relying heavily on hand-crafted linguistic rules.

We, instead, use Monte Carlo tree search (MCTS) for finding the most important few words to perturb and perform homoglyph attack by replacing one character in each selected word with a symbol of identical shape.

Our novel algorithm, we call MCTSBug, is black-box and extremely effective at the same time.

Our experimental results indicate that MCTSBug can fool deep learning classifiers at the success rates of 95% on seven large-scale benchmark datasets, by perturbing only a few characters.

Surprisingly, MCTSBug, without relying on gradient information at all, is more effective than the gradient-based white-box baseline.

Thanks to the nature of homoglyph attack, the generated adversarial perturbations are almost imperceptible to human eyes.

Use Monte carlo Tree Search and Homoglyphs to generate indistinguishable adversarial samples on text data

MCTSBug: Generating Adversarial Text Sequences via Monte Carlo Tree Search and Homoglyph Attack

monte carlo tree search
generate
adversarial
text


  We introduce a model that learns to convert simple hand drawings
  into graphics programs written in a subset of \LaTeX.~

The model
  combines techniques from deep learning and program synthesis.

We
  learn a convolutional neural network that proposes plausible drawing
  primitives that explain an image.

These drawing primitives are like
  a trace of the set of primitive commands issued by a graphics
  program.

We learn a model that uses program synthesis techniques to
  recover a graphics program from that trace.

These programs have
  constructs like variable bindings, iterative loops, or simple kinds
  of conditionals.

With a graphics program in hand, we can correct
  errors made by the deep network and extrapolate drawings.

Taken
  together these results are a step towards agents that induce useful,
  human-readable programs from perceptual input.

Learn to convert a hand drawn sketch into a high-level program

Learning to Infer Graphics Programs from Hand-Drawn Images


Adversarial examples remain an issue for contemporary neural networks.

This paper draws on Background Check (Perello-Nieto et al., 2016), a technique in model calibration, to assist two-class neural networks in detecting adversarial examples, using the one dimensional difference between logit values as the underlying measure.

This method interestingly tends to achieve the highest average recall on image sets that are generated with large perturbation vectors, which is unlike the existing literature on adversarial attacks (Cubuk et al., 2017).

The proposed method does not need knowledge of the attack parameters or methods at training time, unlike a great deal of the literature that uses deep learning based methods to detect adversarial examples, such as Metzen et al. (2017), imbuing the proposed method with additional flexibility.

This paper uses principles from the field of calibration in machine learning on the logits of a neural network to defend against adversarial attacks

Calibration of neural network logit vectors to combat adversarial attacks

neural network
adversarial attack
calibration
logit


We present a novel multi-task training approach to learning multilingual distributed representations of text.

Our system learns word and sentence embeddings jointly by training a multilingual skip-gram model together with a cross-lingual sentence similarity model.

We construct sentence embeddings by processing word embeddings with an LSTM and by taking an average of the outputs.

Our architecture can transparently use both monolingual and sentence aligned bilingual corpora to learn multilingual embeddings, thus covering a vocabulary significantly larger than the vocabulary of the bilingual corpora alone.

Our model shows competitive performance in a standard cross-lingual document classification task.

We also show the effectiveness of our method in a low-resource scenario.

We jointly train a multilingual skip-gram model and a cross-lingual sentence similarity model to learn high quality multilingual text embeddings that perform well in the low resource scenario.

Multitask learning of Multilingual Sentence Representations


Generative Adversarial Networks (GANs) are a very powerful framework for generative modeling.

However, they are often hard to train, and learning of GANs often becomes unstable.

Wasserstein GAN (WGAN) is a promising framework to deal with the instability problem as it has a good convergence property.

One drawback of the WGAN is that it evaluates the Wasserstein distance in the dual domain, which requires some approximation, so that it may fail to optimize the true Wasserstein distance.

In this paper, we propose evaluating the exact empirical optimal transport cost efficiently in the primal domain and performing gradient descent with respect to its derivative to train the generator network.

Experiments on the MNIST dataset show that our method is significantly stable to converge, and achieves the lowest Wasserstein distance among the WGAN variants at the cost of some sharpness of generated images.

Experiments on the 8-Gaussian toy dataset show that better gradients for the generator are obtained in our method.

In addition, the proposed method enables more flexible generative modeling than WGAN.

We have proposed a flexible generative model that learns stably by directly minimizing exact empirical Wasserstein distance.

Generative model based on minimizing exact empirical Wasserstein distance

minimize exact empirical wasserstein distance


Neural Architecture Search (NAS) is an exciting new field which promises to be as much as a game-changer as Convolutional Neural Networks were in 2012.

Despite many great works leading to substantial improvements on a variety of tasks, comparison between different methods is still very much an open issue.

While most algorithms are tested on the same datasets, there is no shared experimental protocol followed by all.

As such, and due to the under-use of ablation studies, there is a lack of clarity regarding why certain methods are more effective than others.

Our first contribution is a benchmark of 8 NAS methods on 5 datasets.

To overcome the hurdle of comparing methods with different search spaces, we propose using a method’s relative improvement over the randomly sampled average architecture, which effectively removes advantages arising from expertly engineered search spaces or training protocols.

Surprisingly, we find that many NAS techniques struggle to significantly beat the average architecture baseline.

We perform further experiments with the commonly used DARTS search space in order to understand the contribution of each component in the NAS pipeline.

(i) the use of tricks in the evaluation protocol has a predominant impact on the reported performance of architectures;

(ii) the cell-based search space has a very narrow accuracy range, such that the seed has a considerable impact on architecture rankings;

(iii) the hand-designed macrostructure (cells) is more important than the searched micro-structure (operations); and

(iv) the depth-gap is a real phenomenon, evidenced by the change in rankings between 8 and 20 cell architectures.

To conclude, we suggest best practices, that we hope will prove useful for the community and help mitigate current NAS pitfalls, e.g. difficulties in reproducibility and comparison of search methods.

code used is available at https://github.com/antoyang/NAS-Benchmark.

A study of how different components in the NAS pipeline contribute to the final accuracy. Also, a benchmark of 8 methods on 5 datasets.

NAS evaluation is frustratingly hard


Identifying analogies across domains without supervision is a key task for artificial intelligence.

Recent advances in cross domain image mapping have concentrated on translating images across domains.

Although the progress made is impressive, the visual fidelity many times does not suffice for identifying the matching sample from the other domain.

In this paper, we tackle this very task of finding exact analogies between datasets i.e. for every image from domain A find an analogous image in domain B. We present a matching-by-synthesis approach: AN-GAN, and show that it outperforms current techniques.

We further show that the cross-domain mapping task can be broken into two parts: domain alignment and learning the mapping function.

The tasks can be iteratively solved, and as the alignment is improved, the unsupervised translation function reaches quality comparable to full supervision.

Finding correspondences between domains by performing matching/mapping iterations

Identifying Analogies Across Domains


Effectively inferring discriminative and coherent latent topics of short texts is a critical task for many real world applications.

Nevertheless, the task has been proven to be a great challenge for traditional topic models due to the data sparsity problem induced by the characteristics of short texts.

Moreover, the complex inference algorithm also become a bottleneck for these traditional models to rapidly explore variations.

In this paper, we propose a novel model called Neural Variational Sparse Topic Model (NVSTM) based on a sparsity-enhanced topic model named Sparse Topical Coding (STC).

In the model, the auxiliary word embeddings are utilized to improve the generation of representations.

The Variational Autoencoder (VAE) approach is applied to inference the model efficiently, which makes the model easy to explore extensions for its black-box inference process.

Experimental results onWeb Snippets, 20Newsgroups, BBC and Biomedical datasets show the effectiveness and efficiency of the model.

a neural sparsity-enhanced topic model based on VAE

Neural Variational Sparse Topic Model


Neural Tangents is a library for working with infinite-width neural networks.

It provides a high-level API for specifying complex and hierarchical neural network architectures.

These networks can then be trained and evaluated either at finite-width as usual, or in their infinite-width limit.

For the infinite-width networks, Neural Tangents performs exact inference either via Bayes' rule or gradient descent, and generates the corresponding Neural Network Gaussian Process and Neural Tangent kernels.

Additionally, Neural Tangents provides tools to study gradient descent training dynamics of wide but finite networks.

The entire library runs out-of-the-box on CPU, GPU, or TPU.

All computations can be automatically distributed over multiple accelerators with near-linear scaling in the number of devices.

In addition to the repository below, we provide an accompanying interactive Colab notebook at

Neural Tangents: Fast and Easy Infinite Neural Networks in Python


Symbolic logic allows practitioners to build systems that perform rule-based reasoning which is interpretable and which can easily be augmented with prior knowledge.

However, such systems are traditionally difficult to apply to problems involving natural language due to the large linguistic variability of language.

Currently, most work in natural language processing focuses on neural networks which learn distributed representations of words and their composition, thereby performing well in the presence of large linguistic variability.

We propose to reap the benefits of both approaches by applying a combination of neural networks and logic programming to natural language question answering.

We propose to employ an external, non-differentiable Prolog prover which utilizes a similarity function over pretrained sentence encoders.

We fine-tune these representations via Evolution Strategies with the goal of multi-hop reasoning on natural language.

This allows us to create a system that can apply rule-based reasoning to natural language and induce domain-specific natural language rules from training data.

We evaluate the proposed system on two different question answering tasks, showing that it complements two very strong baselines – BIDAF (Seo et al., 2016a) and FASTQA (Weissenborn et al.,2017) – and outperforms both when used in an ensemble.

We introduce NLProlog, a system that performs rule-based reasoning on natural language by leveraging pretrained sentence embeddings and fine-tuning with Evolution Strategies, and apply it to two multi-hop Question Answering tasks.

NLProlog: Reasoning with Weak Unification for Natural Language Question Answering

natural language
question answer
nlprolog


Training labels are expensive to obtain and may be of varying quality, as some may be from trusted expert labelers while others might be from heuristics or other sources of weak supervision such as crowd-sourcing.

This creates a fundamental quality-versus-quantity trade-off in the learning process.

Do we learn from the small amount of high-quality data or the potentially large amount of weakly-labeled data?

We argue that if the learner could somehow know and take the label-quality into account, we could get the best of both worlds.

To this end, we introduce “fidelity-weighted learning” (FWL), a semi-supervised student-teacher approach for training deep neural networks using weakly-labeled data.

FWL modulates the parameter updates to a student network, trained on the task we care about on a per-sample basis according to the posterior confidence of its label-quality estimated by a teacher, who has access to limited samples with high-quality labels.

We propose Fidelity-weighted Learning, a semi-supervised teacher-student approach for training neural networks using weakly-labeled data.

Learning from Samples of Variable Quality


This paper is focused on investigating and demystifying an intriguing robustness phenomena in over-parameterized neural network training.

In particular we provide empirical and theoretical evidence that first order methods such as gradient descent are provably robust to noise/corruption on a constant fraction of the labels despite over-parameterization under a rich dataset model.

i) First, we show that in the first few iterations where the updates are still in the vicinity of the initialization these algorithms only fit to the correct labels essentially ignoring the noisy labels.

ii) Secondly, we prove that to start to overfit to the noisy labels these algorithms must stray rather far from from the initial model which can only occur after many more iterations.

Together, these show that gradient descent with early stopping is provably robust to label noise and shed light on empirical robustness of deep networks as well as commonly adopted early-stopping heuristics.

We prove that gradient descent is robust to label corruption despite over-parameterization under a rich dataset model.

Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks


Weight pruning has been introduced as an efficient model compression technique.

Even though pruning removes significant amount of weights in a network, memory requirement reduction was limited since conventional sparse matrix formats require significant amount of memory to store index-related information.

Moreover, computations associated with such sparse matrix formats are slow because sequential sparse matrix decoding process does not utilize highly parallel computing systems efficiently.

As an attempt to compress index information while keeping the decoding process parallelizable, Viterbi-based pruning was suggested.

Decoding non-zero weights, however, is still sequential in Viterbi-based pruning.

In this paper, we propose a new sparse matrix format in order to enable a highly parallel decoding process of the entire sparse matrix.

The proposed sparse matrix is constructed by combining pruning and weight quantization.

For the latest RNN models on PTB and WikiText-2 corpus, LSTM parameter storage requirement is compressed 19x using the proposed sparse matrix format compared to the baseline model.

Compressed weight and indices can be reconstructed into a dense matrix fast using Viterbi encoders.

Simulation results show that the proposed scheme can feed parameters to processing elements 20 % to 106 % faster than the case where the dense matrix values directly come from DRAM.

We present a new weight encoding scheme which enables high compression ratio and fast sparse-to-dense matrix conversion.

Double Viterbi: Weight Encoding for High Compression Ratio and Fast On-Chip Reconstruction for Deep Neural Network


The use of deep learning models as priors for compressive sensing tasks presents new potential for inexpensive seismic data acquisition.

An appropriately designed Wasserstein generative adversarial network is designed based on a generative adversarial network architecture trained on several historical surveys, capable of learning the statistical properties of the seismic wavelets.

The usage of validating and performance testing of compressive sensing are three steps.

First, the existence of a sparse representation with different compression rates for seismic surveys is studied.

Then, non-uniform samplings are studied, using the proposed methodology.

Finally, recommendations for non-uniform seismic survey grid, based on the evaluation of reconstructed seismic images and metrics, is proposed.

The primary goal of the proposed deep learning model is to provide the foundations of an optimal design for seismic acquisition, with less loss in imaging quality.

Along these lines, a compressive sensing design of a non-uniform grid over an asset in Gulf of Mexico, versus a traditional seismic survey grid which collects data uniformly at every few feet, is suggested, leveraging the proposed method.

Improved a GAN based pixel inpainting network for compressed seismic image recovery andproposed a non-uniform sampling survey recommendatio, which can be easily applied to medical and other domains for compressive sensing technique.

Generative Inpainting Network Applications on Seismic Image Compression


In recent years we have seen fast progress on a number of benchmark problems in AI, with modern methods achieving near or super human performance in Go, Poker and Dota.

One common aspect of all of these challenges is that they are by design adversarial or, technically speaking, zero-sum.

In contrast to these settings, success in the real world commonly requires humans to collaborate and communicate with others, in settings that are, at least partially, cooperative.

In the last year, the card game Hanabi has been established as a new benchmark environment for AI to fill this gap.

In particular, Hanabi is interesting to humans since it is entirely focused on theory of mind, i.e. the ability to effectively reason over the intentions, beliefs and point of view of other agents when observing their actions.

Learning to be informative when observed by others is an interesting challenge for Reinforcement Learning (RL): Fundamentally, RL requires agents to explore in order to discover good policies.

However, when done naively, this randomness will inherently make their actions less informative to others during training.

We present a new deep multi-agent RL method, the Simplified Action Decoder (SAD), which resolves this contradiction exploiting the centralized training phase.

During training SAD allows other agents to not only observe the (exploratory) action chosen, but agents instead also observe the greedy action of their team mates.

By combining this simple intuition with an auxiliary task for state prediction and best practices for multi-agent learning, SAD establishes a new state of the art for 2-5 players on the self-play part of the Hanabi challenge.

We develop Simplified Action Decoder, a simple MARL algorithm that beats previous SOTA on Hanabi by a big margin across 2- to 5-player games.

Simplified Action Decoder for Deep Multi-Agent Reinforcement Learning


We present an end-to-end trainable approach for optical character recognition (OCR) on printed documents.

It is based on predicting a two-dimensional character grid ('chargrid') representation of a document image as a semantic segmentation task.

To identify individual character instances from the chargrid, we regard characters as objects and use object detection techniques from computer vision.

We demonstrate experimentally that our method outperforms previous state-of-the-art approaches in accuracy while being easily parallelizable on GPU (thereby being significantly faster), as well as easier to train.

End-to-end trainable Optical Character Recognition on printed documents; we achieve state-of-the-art results, beating Tesseract4 on benchmark datasets both in terms of accuracy and runtime, using a purely computer vision based approach.

Chargrid-OCR: End-to-end trainable Optical Character Recognition through Semantic Segmentation and Object Detection

end-to-end trainable optical character recognition


We propose NovoGrad, an adaptive stochastic gradient descent method with layer-wise gradient normalization and decoupled weight decay.

In our experiments on neural networks for image classification, speech recognition, machine translation, and language modeling, it performs on par or better than well tuned SGD with momentum and Adam/AdamW.

Additionally, NovoGrad (1) is robust to the choice of learning rate and weight initialization, (2) works well in a large batch setting, and (3) has two times smaller memory footprint than Adam.

NovoGrad -  an adaptive SGD method with layer-wise gradient normalization and decoupled weight decay.

Training Deep Networks with Stochastic Gradient Normalized by Layerwise Adaptive Second Moments


Most generative models of audio directly generate samples in one of two domains: time or frequency.

While sufficient to express any signal, these representations are inefficient, as they do not utilize existing knowledge of how sound is generated and perceived.

A third approach (vocoders/synthesizers) successfully incorporates strong domain knowledge of signal processing and perception, but has been less actively researched due to limited expressivity and difficulty integrating with modern auto-differentiation-based machine learning methods.

In this paper, we introduce the Differentiable Digital Signal Processing (DDSP) library, which enables direct integration of classic signal processing elements with deep learning methods.

Focusing on audio synthesis, we achieve high-fidelity generation without the need for large autoregressive models or adversarial losses, demonstrating that DDSP enables utilizing strong inductive biases without losing the expressive power of neural networks.

Further, we show that combining interpretable modules permits manipulation of each separate model component, with applications such as independent control of pitch and loudness, realistic extrapolation to pitches not seen during training, blind dereverberation of room acoustics, transfer of extracted room acoustics to new environments, and transformation of timbre between disparate sources.

In short, DDSP enables an interpretable and modular approach to generative modeling, without sacrificing the benefits of deep learning.

The library will is available at https://github.com/magenta/ddsp and we encourage further contributions from the community and domain experts.

Better audio synthesis by combining interpretable DSP with end-to-end learning.

DDSP: Differentiable Digital Signal Processing


Spectral Graph Convolutional Networks (GCNs) are a generalization of convolutional networks to learning on graph-structured data.

Applications of spectral GCNs have been successful, but limited to a few problems where the graph is fixed, such as shape correspondence and node classification.

In this work, we address this limitation by revisiting a particular family of spectral graph networks, Chebyshev GCNs, showing its efficacy in solving graph classification tasks with a variable graph structure and size.

Current GCNs also restrict graphs to have at most one edge between any pair of nodes.

To this end, we propose a novel multigraph network that learns from multi-relational graphs.

We explicitly model different types of edges: annotated edges, learned edges with abstract meaning, and hierarchical edges.

We also experiment with different ways to fuse the representations extracted from different edge types.

This restriction is sometimes implied from a dataset, however, we relax this restriction for all kinds of datasets.

We achieve state-of-the-art results on a variety of chemical, social, and vision graph classification benchmarks.

A novel approach to graph classification based on spectral graph convolutional networks and its extension to multigraphs with learnable relations and hierarchical structure. We show state-of-the art results on chemical, social and image datasets.

Spectral Convolutional Networks on Hierarchical Multigraphs

convolutional network
hierarchical
spectral
multigraph


Deep neural networks have been recently demonstrated to be vulnerable to backdoor attacks.

Specifically, by altering a small set of training examples, an adversary is able to install a backdoor that can be used during inference to fully control the model’s behavior.

While the attack is very powerful, it crucially relies on the adversary being able to introduce arbitrary, often clearly mislabeled, inputs to the training set and can thus be detected even by fairly rudimentary data filtering.

In this paper, we introduce a new approach to executing backdoor attacks, utilizing adversarial examples and GAN-generated data.

The key feature is that the resulting poisoned inputs appear to be consistent with their label and thus seem benign even upon human inspection.

We show how to successfully perform backdoor attacks without changing training labels.

Clean-Label Backdoor Attacks


Despite their ability to memorize large datasets, deep neural networks often achieve good generalization performance.

However, the differences between the learned solutions of networks which generalize and those which do not remain unclear.

Additionally, the tuning properties of single directions (defined as the activation of a single unit or some linear combination of units in response to some input) have been highlighted, but their importance has not been evaluated.

Here, we connect these lines of inquiry to demonstrate that a network’s reliance on single directions is a good predictor of its generalization performance, across networks trained on datasets with different fractions of corrupted labels, across ensembles of networks trained on datasets with unmodified labels, across different hyper- parameters, and over the course of training.

While dropout only regularizes this quantity up to a point, batch normalization implicitly discourages single direction reliance, in part by decreasing the class selectivity of individual units.

Finally, we find that class selectivity is a poor predictor of task importance, suggesting not only that networks which generalize well minimize their dependence on individual units by reducing their selectivity, but also that individually selective units may not be necessary for strong network performance.

We find that deep networks which generalize poorly are more reliant on single directions than those that generalize well, and evaluate the impact of dropout and batch normalization, as well as class selectivity on single direction reliance.

On the importance of single directions for generalization


Typical amortized inference in variational autoencoders is specialized for a single probabilistic query.

Here we propose an inference network architecture that generalizes to unseen probabilistic queries.

Instead of an encoder-decoder pair, we can train a single inference network directly from data, using a cost function that is stochastic not only over samples, but also over queries.

We can use this network to perform the same inference tasks as we would in an undirected graphical model with hidden variables, without having to deal with the intractable partition function.

The results can be mapped to the learning of an actual undirected model, which is a notoriously hard problem.

Our network also marginalizes nuisance variables as required.

We show that our approach generalizes to unseen probabilistic queries on also unseen test data, providing fast and flexible inference.

Experiments show that this approach outperforms or matches PCD and AdVIL on 9 benchmark datasets.

Instead of learning the parameters of a graphical model from data, learn an inference network that can answer the same probabilistic queries.

Learning undirected models via query training


A plethora of computer vision tasks, such as optical flow and image alignment, can be formulated as non-linear optimization problems.

Before the resurgence of deep learning, the dominant family for solving such optimization problems was numerical optimization, e.g, Gauss-Newton (GN).

More recently, several attempts were made to formulate learnable GN steps as cascade regression architectures.

In this paper, we investigate recent machine learning architectures, such as deep neural networks with residual connections, under the above perspective.

To this end, we first demonstrate how residual blocks (when considered as discretization of ODEs) can be viewed as GN steps.

Then, we go a step further and propose a new residual block, that is reminiscent of Newton's method in numerical optimization and exhibits faster convergence.

We thoroughly evaluate the proposed Newton-ResNet by conducting experiments on image and speech classification and image generation, using 4 datasets.

All the experiments demonstrate that Newton-ResNet requires less parameters to achieve the same performance with the original ResNet.

We demonstrate how residual blocks can be viewed as Gauss-Newton steps; we propose a new residual block that exploits second order information.

Newton Residual Learning


In competitive situations, agents may take actions to achieve their goals that unwittingly facilitate an opponent’s goals.

consider a domain where three agents operate: (1) a user (human), (2) an attacker (human or a software) agent and (3) an observer (a software) agent.

The user and the attacker compete to achieve different goals.

When there is a disparity in the domain knowledge the user and the attacker possess, the attacker may use the user’s unfamiliarity with the domain to

In this situation, the observer, whose goal is to support the user may need to intervene, and this intervention needs to occur online, on-time and be accurate.

We formalize the online plan intervention problem and propose a solution that uses a decision tree classifier to identify intervention points in situations where agents unwittingly facilitate an opponent’s goal.

We trained a classifier using domain-independent features extracted from the observer’s decision space to evaluate the “criticality” of the current state.

The trained model is then used in an online setting on IPC benchmarks to identify observations that warrant intervention.

Our contributions lay a foundation for further work in the area of deciding when to intervene.

We introduce a machine learning model that uses domain-independent features to estimate the criticality of the current state to cause a known undesirable state.

Domain-independent Plan Intervention When Users Unwittingly Facilitate Attacks


Deep generative models can emulate the perceptual properties of complex image datasets, providing a latent representation of the data.

However, manipulating such representation to perform meaningful and controllable transformations in the data space remains challenging without some form of supervision.

While previous work has focused on exploiting statistical independence to \textit{disentangle} latent factors, we argue that such requirement can be advantageously relaxed and propose instead a non-statistical framework that relies on identifying a modular organization of the network, based on counterfactual manipulations.

Our experiments support that modularity between groups of channels is achieved to a certain degree on a variety of generative models.

This allowed the design of targeted interventions on complex image datasets, opening the way to applications such as computationally efficient style transfer and the automated assessment of robustness to contextual changes in pattern recognition systems.

We develop a framework to find modular internal representations in generative models and manipulate then to generate counterfactual examples.

Counterfactuals uncover the modular structure of deep generative models


Catastrophic forgetting poses a grand challenge for continual learning systems, which prevents neural networks from protecting old knowledge while learning new tasks sequentially.

We propose a Differentiable Hebbian Plasticity (DHP) Softmax layer which adds a fast learning plastic component to the slow weights of the softmax output layer.

The DHP Softmax behaves as a compressed episodic memory that reactivates existing memory traces, while creating new ones.

We demonstrate the flexibility of our model by combining it with existing well-known consolidation methods to prevent catastrophic forgetting.

We evaluate our approach on the Permuted MNIST and Split MNIST benchmarks, and introduce Imbalanced Permuted MNIST — a dataset that combines the challenges of class imbalance and concept drift.

Our model requires no additional hyperparameters and outperforms comparable baselines by reducing forgetting.

Hebbian plastic weights can behave as a compressed episodic memory storage in neural networks; improving their ability to alleviate catastrophic forgetting in continual learning.

Differentiable Hebbian Plasticity for Continual Learning


While real brain networks exhibit functional modularity, we investigate whether functional mod- ularity also exists in Deep Neural Networks (DNN) trained through back-propagation.

Under the hypothesis that DNN are also organized in task-specific modules, in this paper we seek to dissect a hidden layer into disjoint groups of task-specific hidden neurons with the help of relatively well- studied neuron attribution methods.

By saying task-specific, we mean the hidden neurons in the same group are functionally related for predicting a set of similar data samples, i.e. samples with similar feature patterns.

We argue that such groups of neurons which we call Functional Modules can serve as the basic functional unit in DNN.

We propose a preliminary method to identify Functional Modules via bi- clustering attribution scores of hidden neurons.

We find that first, unsurprisingly, the functional neurons are highly sparse, i.e., only a small sub- set of neurons are important for predicting a small subset of data samples and, while we do not use any label supervision, samples corresponding to the same group (bicluster) show surprisingly coherent feature patterns.

We also show that these Functional Modules perform a critical role in discriminating data samples through ablation experiment.

We develop an approach to parcellate a hidden layer in DNN into functionally related groups, by applying spectral coclustering on the attribution scores of hidden neurons.

Checking Functional Modularity in DNN By Biclustering Task-specific Hidden Neurons


Power-efficient CNN Domain Specific Accelerator (CNN-DSA) chips are currently available for wide use in mobile devices.

These chips are mainly used in computer vision applications.

However, the recent work of Super Characters method for text classification and sentiment analysis tasks using two-dimensional CNN models has also achieved state-of-the-art results through the method of transfer learning from vision to text.

In this paper, we implemented the text classification and sentiment analysis applications on mobile devices using CNN-DSA chips.

Compact network representations using one-bit and three-bits precision for coefficients and five-bits for activations are used in the CNN-DSA chip with power consumption less than 300mW.

For edge devices under memory and compute constraints, the network is further compressed by approximating the external Fully Connected (FC) layers within the CNN-DSA chip.

At the workshop, we have two system demonstrations for NLP tasks.

The first demo classifies the input English Wikipedia sentence into one of the 14 classes.

The second demo classifies the Chinese online-shopping review into positive or negative.

Deploy text classification and sentiment analysis applications for English and Chinese on a 300mW CNN accelerator chip for on-device application scenarios.

System Demo for Transfer Learning from Vision to Language using Domain Specific CNN Accelerator for On-Device NLP Applications


Comparing the inferences of diverse candidate models is an essential part of model checking and escaping local optima.

To enable efficient comparison, we introduce an amortized variational inference framework that can perform fast and reliable posterior estimation across models of the same architecture.

Our Any Parameter Encoder (APE) extends the encoder neural network common in amortized inference to take both a data feature vector and a model parameter vector as input.

APE thus reduces posterior inference across unseen data and models to a single forward pass.

In experiments comparing candidate topic models for synthetic data and product reviews, our Any Parameter Encoder yields comparable posteriors to more expensive methods in far less time, especially when the encoder architecture is designed in model-aware fashion.

We develop VAEs where the encoder takes a model parameter vector as input, so we can do rapid inference for many models

Rapid Model Comparison by Amortizing Across Models


The ability to transfer knowledge to novel environments and tasks is a sensible desiderata for general learning agents.

Despite the apparent promises, transfer in RL is still an open and little exploited research area.

In this paper, we take a brand-new perspective about transfer: we suggest that the ability to assign credit unveils structural invariants in the tasks that can be transferred to make RL more sample efficient.

Our main contribution is Secret, a novel approach to transfer learning for RL that uses a backward-view credit assignment mechanism based on a self-attentive architecture.

Two aspects are key to its generality: it learns to assign credit as a separate offline supervised process and exclusively modifies the reward function.

Consequently, it can be supplemented by transfer methods that do not modify the reward function and it can be plugged on top of any RL algorithm.

Secret is a transfer method for RL based on the transfer of credit assignment.

Self-Attentional Credit Assignment for Transfer in Reinforcement Learning


Recent advances in Neural Variational Inference allowed for a renaissance in latent variable models in a variety of domains involving high-dimensional data.

In this paper, we introduce two generic Variational Inference frameworks for generative models of Knowledge Graphs; Latent Fact Model and Latent Information Model.

While traditional variational methods derive an analytical approximation for the intractable distribution over the latent variables, here we construct an inference network conditioned on the symbolic representation of entities and relation types in the Knowledge Graph, to provide the variational distributions.

The new framework can create models able to discover underlying probabilistic semantics for the symbolic representation by utilising parameterisable distributions which permit training by back-propagation in the context of neural variational inference, resulting in a highly-scalable method.

Under a Bernoulli sampling framework, we provide an alternative justification for commonly used techniques in large-scale stochastic variational inference, which drastically reduces training time at a cost of an additional approximation to the variational lower bound.

The generative frameworks are flexible enough to allow training under any prior distribution that permits a re-parametrisation trick, as well as under any scoring function that permits maximum likelihood estimation of the parameters.

Experiment results display the potential and efficiency of this framework by improving upon multiple benchmarks with Gaussian prior representations.

Working toward generative knowledge graph models to better estimate predictive uncertainty in knowledge inference.

Neural Variational Inference For Embedding Knowledge Graphs


We present a deep generative model, named Monge-Amp\`ere flow, which builds on continuous-time gradient flow arising from the Monge-Amp\`ere equation in optimal transport theory.

The generative map from the latent space to the data space follows a dynamical system, where a learnable potential function guides a compressible fluid to flow towards the target density distribution.

Training of the model amounts to solving an optimal control problem.

The Monge-Amp\`ere flow has tractable likelihoods and supports efficient sampling and inference.

One can easily impose symmetry constraints in the generative model by designing suitable scalar potential functions.

We apply the approach to unsupervised density estimation of the MNIST dataset and variational calculation of the two-dimensional Ising model at the critical point.

This approach brings insights and techniques from Monge-Amp\`ere equation, optimal transport, and fluid dynamics into reversible flow-based generative models.

A gradient flow based dynamical system for invertible generative modeling

Monge-Amp\`ere Flow for Generative Modeling


Graph Neural Networks (GNNs) are a class of deep models that operates on data with arbitrary topology and order-invariant structure represented as graphs.

We introduce an efficient memory layer for GNNs that can learn to jointly perform graph representation learning and graph pooling.

We also introduce two new networks based on our memory layer: Memory-Based Graph Neural Network (MemGNN) and Graph Memory Network (GMN) that can learn hierarchical graph representations by coarsening the graph throughout the layers of memory.

The experimental results demonstrate that the proposed models achieve state-of-the-art results in six out of seven graph classification and regression benchmarks.

We also show that the learned representations could correspond to chemical features in the molecule data.

We introduce an efficient memory layer that can learn representation and coarsen input graphs simultaneously without relying on message passing.

Memory-Based Graph Networks


Deep neural networks require extensive computing resources, and can not be efficiently applied to embedded devices such as mobile phones, which seriously limits their applicability.

To address this problem, we propose a novel encoding scheme by using {-1,+1} to decompose quantized neural networks (QNNs) into multi-branch binary networks, which can be efficiently implemented by bitwise operations (xnor and bitcount) to achieve model compression, computational acceleration and resource saving.

Our method can achieve at most ~59 speedup and ~32 memory saving over its full-precision counterparts.

Therefore, users can easily achieve different encoding precisions arbitrarily according to their requirements and hardware resources.

Our mechanism is very suitable for the use of FPGA and ASIC in terms of data storage and computation, which provides a feasible idea for smart chips.

We validate the effectiveness of our method on both large-scale image classification (e.g., ImageNet) and object detection tasks.

A novel encoding scheme of using {-1, +1} to decompose QNNs into multi-branch binary networks, in which we used bitwise operations (xnor and bitcount) to achieve model compression, computational acceleration and resource saving.

Efficient Computation of Quantized Neural Networks by {−1, +1} Encoding Decomposition


We propose a novel method for incorporating conditional information into a generative adversarial network (GAN) for structured prediction tasks.

This method is based on fusing features from the generated and conditional information in feature space and allows the discriminator to better capture higher-order statistics from the data.

This method also increases the strength of the signals passed through the network where the real or generated data and the conditional data agree.

The proposed method is conceptually simpler than the joint convolutional neural network - conditional Markov random field (CNN-CRF) models and enforces higher-order consistency without being limited to a very specific class of high-order potentials.

Experimental results demonstrate that this method leads to improvement on a variety of different structured prediction tasks including image synthesis, semantic segmentation, and depth estimation.

We propose a novel way to incorporate conditional image information into the discriminator of GANs using feature fusion that can be used for structured prediction tasks.

Structured Prediction using cGANs with Fusion Discriminator

structured prediction
fusion
discriminator


Intelligent creatures can explore their environments and learn useful skills without supervision.

In this paper, we propose ``Diversity is All You Need''(DIAYN), a method for learning useful skills without a reward function.

Our proposed method learns skills by maximizing an information theoretic objective using a maximum entropy policy.

On a variety of simulated robotic tasks, we show that this simple objective results in the unsupervised emergence of diverse skills, such as walking and jumping.

In a number of reinforcement learning benchmark environments, our method is able to learn a skill that solves the benchmark task despite never receiving the true task reward.

We show how pretrained skills can provide a good parameter initialization for downstream tasks, and can be composed hierarchically to solve complex, sparse reward tasks.

Our results suggest that unsupervised discovery of skills can serve as an effective pretraining mechanism for overcoming challenges of exploration and data efficiency in reinforcement learning.

We propose an algorithm for learning useful skills without a reward function, and show how these skills can be used to solve downstream tasks.

Diversity is All You Need: Learning Skills without a Reward Function


Lexical ambiguity, i.e., the presence of two or more meanings for a single word, is an inherent and challenging problem for machine translation systems.

Even though the use of recurrent neural networks and attention mechanisms are expected to solve this problem, machine translation systems are not always able to correctly translate lexically ambiguous sentences.

In this work, I attempt to resolve the problem of lexical ambiguity in English--Japanese neural machine translation systems by combining a pretrained Bidirectional Encoder Representations from Transformer (BERT) language model that can produce contextualized word embeddings and a Transformer translation model, which is a state-of-the-art architecture for the machine translation task.

These two proposed architectures have been shown to be more effective in translating ambiguous sentences than a vanilla Transformer model and the Google Translate system.

Furthermore, one of the proposed models, the Transformer_BERT-WE, achieves a higher BLEU score compared to the vanilla Transformer model in terms of general translation, which is concrete proof that the use of contextualized word embeddings from BERT can not only solve the problem of lexical ambiguity, but also boost the translation quality in general.

The paper solves a lexical ambiguity problem caused from homonym in neural translation by BERT.

Resolving Lexical Ambiguity in English–Japanese Neural Machine Translation


This paper focuses on the synthetic generation of human mobility data in urban areas.

We present a novel and scalable application of Generative Adversarial Networks (GANs) for modeling and generating human mobility data.

We leverage actual ride requests from ride sharing/hailing services from four major cities in the US to train our GANs model.

Our model captures the spatial and temporal variability of the ride-request patterns observed for all four cities on any typical day and over any typical week.

Previous works have succinctly characterized the spatial and temporal properties of human mobility data sets using the fractal dimensionality and the densification power law, respectively, which we utilize to validate our GANs-generated synthetic data sets.

Such synthetic data sets can avoid privacy concerns and be extremely useful for researchers and policy makers on urban mobility and intelligent transportation.

This paper focuses on the synthetic generation of human mobility data in urban areas using GANs.

Using GANs for Generation of Realistic City-Scale Ride Sharing/Hailing Data Sets


While deep neural networks are a highly successful model class, their large memory footprint puts considerable strain on energy consumption, communication bandwidth, and storage requirements.

Consequently, model size reduction has become an utmost goal in deep learning.

Following the classical bits-back argument, we encode the network weights using a random sample, requiring only a number of bits corresponding to the Kullback-Leibler divergence between the sampled variational distribution and the encoding distribution.

By imposing a constraint on the Kullback-Leibler divergence, we are able to explicitly control the compression rate, while optimizing the expected loss on the training set.

The employed encoding scheme can be shown to be close to the optimal information-theoretical lower bound, with respect to the employed variational family.

On benchmarks LeNet-5/MNIST and VGG-16/CIFAR-10, our approach yields the best test performance for a fixed memory budget, and vice versa, it achieves the highest compression rates for a fixed test performance.

This paper proposes an effective coding scheme for neural networks that encodes a random set of weights from a variational distribution.

Minimal Random Code Learning: Getting Bits Back from Compressed Model Parameters


Deep neural networks provide state-of-the-art performance for image denoising, where the goal is to recover a near noise-free image from a noisy image.

The underlying principle is that neural networks trained on large datasets have empirically been shown to be able to generate natural images well from a low-dimensional latent representation of the image.

Given such a generator network, or prior, a noisy image can be denoised by finding the closest image in the range of the prior.

However, there is little theory to justify this success, let alone to predict the denoising performance as a function of the networks parameters.

In this paper we consider the problem of denoising an image from additive Gaussian noise, assuming the image is well described by a deep neural network with ReLu activations functions, mapping a k-dimensional latent space to an n-dimensional image.

We state and analyze a simple gradient-descent-like iterative algorithm that minimizes a non-convex loss function, and provably removes a fraction of (1 - O(k/n)) of the noise energy.

We also demonstrate in numerical experiments that this denoising performance is, indeed, achieved by generative priors learned from data.

By analyzing an algorithms minimizing a non-convex loss, we show that all but a small fraction of noise can be removed from an image using a deep neural network based generative prior.

Deep Denoising: Rate-Optimal Recovery of Structured Signals with a Deep Prior


Deep learning yields great results across many fields,

from speech recognition, image classification, to translation.

But for each problem, getting a deep model to work well involves

research into the architecture and a long period of tuning.

We present a single model that yields good results on a number

is trained concurrently on ImageNet, multiple translation tasks,

image captioning (COCO dataset), a speech recognition corpus,

Our model architecture incorporates building blocks from multiple

It contains convolutional layers, an attention mechanism,

Each of these computational blocks is crucial for a subset of

crucial for a task, we observe that adding it never hurts performance

and in most cases improves it on all tasks.

We also show that tasks with less data benefit largely from joint

training with other tasks, while performance on large tasks degrades

Large scale multi-task architecture solves ImageNet and translation together and shows transfer learning.

Large Scale Multi-Domain Multi-Task Learning with MultiModel


Machine learning algorithms for controlling devices will need to learn quickly, with few trials.

Such a goal can be attained with concepts borrowed from continental philosophy and formalized using tools from the mathematical theory of categories.

Illustrations of this approach are presented on a cyberphysical system: the slot car game, and also on Atari 2600 games.

Continental-philosophy-inspired approach to learn with few data.

Learning to play slot cars and Atari 2600 games in just minutes


Audio signals are sampled at high temporal resolutions, and learning to synthesize audio requires capturing structure across a range of timescales.

Generative adversarial networks (GANs) have seen wide success at generating images that are both locally and globally coherent, but they have seen little application to audio generation.

In this paper we introduce WaveGAN, a first attempt at applying GANs to unsupervised synthesis of raw-waveform audio.

WaveGAN is capable of synthesizing one second slices of audio waveforms with global coherence, suitable for sound effect generation.

Our experiments demonstrate that—without labels—WaveGAN learns to produce intelligible words when trained on a small-vocabulary speech dataset, and can also synthesize audio from other domains such as drums, bird vocalizations, and piano.

We compare WaveGAN to a method which applies GANs designed for image generation on image-like audio feature representations, finding both approaches to be promising.

Learning to synthesize raw waveform audio with GANs

Adversarial Audio Synthesis


The difficulty of obtaining sufficient labeled data for supervised learning has motivated domain adaptation, in which a classifier is trained in one domain, source domain, but operates in another, target domain.

Reducing domain discrepancy has improved the performance, but it is hampered by the embedded features that do not form clearly separable and aligned clusters.

We address this issue by propagating labels using a manifold structure, and by enforcing cycle consistency to align the clusters of features in each domain more closely.

Specifically, we prove that cycle consistency leads the embedded features distant from all but one clusters if the source domain is ideally clustered.

We additionally utilize more information from approximated local manifold and pursue local manifold consistency for more improvement.

Results for various domain adaptation scenarios show tighter clustering and an improvement in classification accuracy.

A novel domain adaptation method to align manifolds from source and target domains using label propagation for better accuracy.

Domain Adaptation Through Label Propagation: Learning Clustered and Aligned Features

label propagation
domain adaptation
align


We propose a new method for training neural networks online in a bandit setting.

Similar to prior work, we model the uncertainty only in the last layer of the network, treating the rest of the network as a feature extractor.

This allows us to successfully balance between exploration and exploitation due to the efficient, closed-form uncertainty estimates available for linear models.

To train the rest of the network, we take advantage of the posterior we have over the last layer, optimizing over all values in the last layer distribution weighted by probability.

We derive a closed form, differential approximation to this objective and show empirically over various models and datasets that training the rest of the network in this fashion leads to both better online and offline performance when compared to other methods.

This paper proposes a new method for neural network learning in online bandit settings by marginalizing over the last layer

Neural Network Bandit Learning by Last Layer Marginalization


Despite a growing literature on explaining neural networks, no consensus has been reached on how to explain a neural network decision or how to evaluate an explanation.

Our contributions in this paper are twofold.

First, we investigate schemes to combine explanation methods and reduce model uncertainty to obtain a single aggregated explanation.

The aggregation is more robust and aligns better with the neural network than any single explanation method..

Second, we propose a new approach to evaluating explanation methods that circumvents the need for manual evaluation and is not reliant on the alignment of neural networks and humans decision processes.

We show in theory and in practice that combining multiple explanation methods for DNN benefits the explanation.

Aggregating explanation methods for neural networks stabilizes explanations


We present SOSELETO (SOurce SELEction for Target Optimization), a new method for exploiting a source dataset to solve a classification problem on a target dataset.

SOSELETO is based on the following simple intuition: some source examples are more informative than others for the target problem.

To capture this intuition, source samples are each given weights; these weights are solved for jointly with the source and target classification problems via a bilevel optimization scheme.

The target therefore gets to choose the source samples which are most informative for its own classification task.

Furthermore, the bilevel nature of the optimization acts as a kind of regularization on the target, mitigating overfitting.

SOSELETO may be applied to both classic transfer learning, as well as the problem of training on datasets with noisy labels; we show state of the art results on both of these problems.

Learning with limited training data by exploiting "helpful" instances from a rich data source.

SOSELETO: A Unified Approach to Transfer Learning and Training with Noisy Labels