Deep reinforcement learning (RL) methods generally engage in exploratory behavior through noise injection in the action space.

An alternative is to add noise directly to the agent's parameters, which can lead to more consistent exploration and a richer set of behaviors.

Methods such as evolutionary strategies use parameter perturbations, but discard all temporal structure in the process and require significantly more samples.

Combining parameter noise with traditional RL methods allows to combine the best of both worlds.

We demonstrate that both off- and on-policy methods benefit from this approach through experimental comparison of DQN, DDPG, and TRPO on high-dimensional discrete action environments as well as continuous control tasks.

Parameter space noise allows reinforcement learning algorithms to explore by perturbing parameters instead of actions, often leading to significantly improved exploration performance.

Parameter Space Noise for Exploration


Pre-trained deep neural network language models such as ELMo, GPT, BERT and XLNet have recently achieved state-of-the-art performance on a variety of language understanding tasks.

However, their size makes them impractical for a number of scenarios, especially on mobile and edge devices.

In particular, the input word embedding matrix accounts for a significant proportion of the model's memory footprint, due to the large input vocabulary and embedding dimensions.

Knowledge distillation techniques have had success at compressing large neural network models, but they are ineffective at yielding student models with vocabularies different from the original teacher models.

We introduce a novel knowledge distillation technique for training a student model with a significantly smaller vocabulary as well as lower embedding and hidden state dimensions.

Specifically, we employ a dual-training mechanism that trains the teacher and student models simultaneously to obtain optimal word embeddings for the student vocabulary.

We combine this approach with learning shared projection matrices that transfer layer-wise knowledge from the teacher model to the student model.

Our method is able to compress the BERT-BASE model by more than 60x, with only a minor drop in downstream task metrics, resulting in a language model with a footprint of under 7MB.

Experimental results also demonstrate higher compression efficiency and accuracy when compared with other state-of-the-art compression techniques.

We present novel distillation techniques that enable training student models with different vocabularies and compress BERT by 60x with minor performance drop.

Extreme Language Model Compression with Optimal Subwords and Shared Projections


Human reasoning involves recognising common underlying principles across many examples by utilising variables.

The by-products of such reasoning are invariants that capture patterns across examples such as "if someone went somewhere then they are there" without mentioning specific people or places.

Humans learn what variables are and how to use them at a young age, and the question this paper addresses is whether machines can also learn and use variables solely from examples without requiring human pre-engineering.

We propose Unification Networks that incorporate soft unification into neural networks to learn variables and by doing so lift examples into invariants that can then be used to solve a given task.

We evaluate our approach on four datasets to demonstrate that learning invariants captures patterns in the data and can improve performance over baselines.

End-to-end learning of invariant representations with variables across examples such as if someone went somewhere then they are there.

Learning Invariants through Soft Unification


We propose a new learning-based approach to solve ill-posed inverse problems in imaging.

We address the case where ground truth training samples are rare and the problem is severely ill-posed---both because of the underlying physics and because we can only get few measurements.

This setting is common in geophysical imaging and remote sensing.

We show that in this case the common approach to directly learn the mapping from the measured data to the reconstruction becomes unstable.

Instead, we propose to first learn an ensemble of simpler mappings from the data to projections of the unknown image into random piecewise-constant subspaces.

We then combine the projections to form a final reconstruction by solving a deconvolution-like problem.

We show experimentally that the proposed method is more robust to measurement noise and corruptions not seen during training than a directly learned inverse.

We solve ill-posed inverse problems with scarce ground truth examples by estimating an ensemble of random projections of the model instead of the model itself.

Random mesh projectors for inverse problems


Designing architectures for deep neural networks requires expert knowledge and substantial computation time.

We propose a technique to accelerate architecture selection by learning an auxiliary HyperNet that generates the weights of a main model conditioned on that model's architecture.

By comparing the relative validation performance of networks with HyperNet-generated weights, we can effectively search over a wide range of architectures at the cost of a single training run.

To facilitate this search, we develop a flexible mechanism based on memory read-writes that allows us to define a wide range of network connectivity patterns, with ResNet, DenseNet, and FractalNet blocks as special cases.

We validate our method (SMASH) on CIFAR-10 and CIFAR-100, STL-10, ModelNet10, and Imagenet32x32, achieving competitive performance with similarly-sized hand-designed networks.

A technique for accelerating neural architecture selection by approximating the weights of each candidate architecture instead of training them individually.

SMASH: One-Shot Model Architecture Search through HyperNetworks


Textual entailment (or NLI) data has proven useful as pretraining data for tasks requiring language understanding, even when building on an already-pretrained model like RoBERTa.

The standard protocol for collecting NLI was not designed for the creation of pretraining data, and it is likely far from ideal for this purpose.

With this application in mind we propose four alternative protocols, each aimed at improving either the ease with which annotators can produce sound training examples or the quality and diversity of those examples.

Using these alternatives and a simple MNLIbased baseline, we collect and compare five new 9k-example training sets.

Our primary results are largely negative, with none of these new methods showing major improvements in transfer learning.

However, we make several observations that should inform future work on NLI data, such as that the use of automatically provided seed sentences for inspiration improves the quality of the resulting data on most measures, and all of the interventions we investigated dramatically reduce previously observed issues with annotation artifacts.

We propose four new ways of collecting NLI data. Some help slightly as pretraining data, all help reduce annotation artifacts.

Collecting Entailment Data for Pretraining: New Protocols and Negative Results


Predicting the future in real-world settings, particularly from raw sensory observations such as images, is exceptionally challenging.

Real-world events can be stochastic and unpredictable, and the high dimensionality and complexity of natural images requires the predictive model to build an intricate understanding of the natural world.

Many existing methods tackle this problem by making simplifying assumptions about the environment.

One common assumption is that the outcome is deterministic and there is only one plausible future.

This can lead to low-quality predictions in real-world settings with stochastic dynamics.

In this paper, we develop a stochastic variational video prediction (SV2P) method that predicts a different possible future for each sample of its latent variables.

To the best of our knowledge, our model is the first to provide effective stochastic multi-frame prediction for real-world video.

We demonstrate the capability of the proposed method in predicting detailed future frames of videos on multiple real-world datasets, both action-free and action-conditioned.

We find that our proposed method produces substantially improved video predictions when compared to the same model without stochasticity, and to other stochastic video prediction methods.

Our SV2P implementation will be open sourced upon publication.

Stochastic variational video prediction in real-world settings.

Stochastic Variational Video Prediction


In multi-agent systems, complex interacting behaviors arise due to the high correlations among agents.

However, previous work on modeling multi-agent interactions from demonstrations is primarily constrained by assuming the independence among policies and their reward structures.

In this paper, we cast the multi-agent interactions modeling problem into a multi-agent imitation learning framework with explicit modeling of correlated policies by approximating opponents’ policies, which can recover agents' policies that can regenerate similar interactions.

Consequently, we develop a Decentralized Adversarial Imitation Learning algorithm with Correlated policies (CoDAIL), which allows for decentralized training and execution.

Various experiments demonstrate that CoDAIL can better regenerate complex interactions close to the demonstrators and outperforms state-of-the-art multi-agent imitation learning methods.

Our code is available at \url{https://github.com/apexrl/CoDAIL}.

Modeling complex multi-agent interactions under multi-agent imitation learning framework with explicit modeling of correlated policies by approximating opponents’ policies.

Multi-Agent Interactions Modeling with Correlated Policies

correlate policy
multi-agent interaction
model


Plagiarism and text reuse become more available with the Internet development.

Therefore it is important to check scientific papers for the fact of cheating, especially in Academia.

Existing systems of plagiarism detection show the good performance and have a huge source databases.

Thus now it is not enough just to copy the text as is from the source document to get the original work.

Therefore, another type of plagiarism become popular -- cross-lingual plagiarism.

We present a CrossLang system for such kind of plagiarism detection for English-Russian language pair.

A system for cross-lingual (English-Russian) plagiarism detection

CrossLang: the system of cross-lingual plagiarism detection

system cross-lingual
plagiarism detection


Data parallelism has become a dominant method to scale Deep Neural Network (DNN) training across multiple nodes.

Since the synchronization of the local models or gradients can be a bottleneck for large-scale distributed training, compressing communication traffic has gained widespread attention recently.

Among several recent proposed compression algorithms,
Residual Gradient Compression (RGC) is one of the most successful approaches---it can significantly compress the transmitting message size (0.1% of the gradient size) of each node and still preserve accuracy.

However, the literature on compressing deep networks focuses almost exclusively on achieving good compression rate, while the efficiency of RGC in real implementation has been less investigated.

In this paper, we develop an RGC method that achieves significant training time improvement in real-world multi-GPU systems.

Our proposed RGC system design called RedSync, introduces a set of optimizations to reduce communication bandwidth while introducing limited overhead.

We examine the performance of RedSync on two different multiple GPU platforms, including a supercomputer and a multi-card server.

Our test cases include image classification on Cifar10 and ImageNet, and language modeling tasks on Penn Treebank and Wiki2 datasets.

For DNNs featured with high communication to computation ratio, which has long been considered with poor scalability, RedSync shows significant performance improvement.

We proposed an implementation to accelerate DNN data parallel training by reducing communication bandwidth requirement.

RedSync : Reducing Synchronization Traffic for Distributed Deep Learning


Backpropagation is driving today's artificial neural networks.

However, despite extensive research, it remains unclear if the brain implements this algorithm.

Among neuroscientists, reinforcement learning (RL) algorithms are often seen as a realistic alternative.

However, the convergence rate of such learning scales poorly with the number of involved neurons.

Here we propose a hybrid learning approach, in which each neuron uses an RL-type strategy to learn how to approximate the gradients that backpropagation would provide.

We show that our approach learns to approximate the gradient, and can match the performance of gradient-based learning on fully connected and convolutional networks.

Learning feedback weights provides a biologically plausible mechanism of achieving good performance, without the need for precise, pre-specified learning rules.

Perturbations can be used to learn feedback weights on large fully connected and convolutional networks.

Learning to solve the credit assignment problem


Most deep neural networks (DNNs) require complex models to achieve high performance.

Parameter quantization is widely used for reducing the implementation complexities.

Previous studies on quantization were mostly based on extensive simulation using training data.

We choose a different approach and attempt to measure the per-parameter capacity of DNN models and interpret the results to obtain insights on optimum quantization of parameters.

This research uses artificially generated data and generic forms of fully connected DNNs, convolutional neural networks, and recurrent neural networks.

We conduct memorization and classification tests to study the effects of the number and precision of the parameters on the performance.

The model and the per-parameter capacities are assessed by measuring the mutual information between the input and the classified output.

We also extend the memorization capacity measurement results to image classification and language modeling tasks.

To get insight for parameter quantization when performing real tasks, the training and test performances are compared.

We suggest the sufficient number of bits for representing weights of DNNs and the optimum bits are conservative when solving real problems.

Capacity of Deep Neural Networks under Parameter Quantization


Inspired by the combination of feedforward and iterative computations in the visual cortex, and taking advantage of the ability of denoising autoencoders to estimate the score of a joint distribution, we propose a novel approach to iterative inference for capturing and exploiting the complex joint distribution of output variables conditioned on some input variables.

This approach is applied to image pixel-wise segmentation, with the estimated conditional score used to perform gradient ascent towards a mode of the estimated conditional distribution.

This extends previous work on score estimation by denoising autoencoders to the case of a conditional distribution, with a novel use of a corrupted feedforward predictor replacing Gaussian corruption.

An advantage of this approach over more classical ways to perform iterative inference for structured outputs, like conditional random fields (CRFs), is that it is not any more necessary to define an explicit energy function linking the output variables.

To keep computations tractable, such energy function parametrizations are typically fairly constrained, involving only a few neighbors of each of the output variables in each clique.

We experimentally find that the proposed iterative inference from conditional score estimation by conditional denoising autoencoders performs better than comparable models based on CRFs or those not using any explicit modeling of the conditional joint distribution of outputs.

Refining segmentation proposals by performing iterative inference with conditional denoising autoencoders.

Image Segmentation by Iterative Inference from Conditional Score Estimation

iterative inference conditional
segmentation


Inspired by the success of self attention mechanism and Transformer architecture

in sequence transduction and image generation applications, we propose novel self

attention-based architectures to improve the performance of adversarial latent code-

Adversarial latent code-based text generation

has recently gained a lot of attention due to their promising results.

we take a step to fortify the architectures used in these setups, specifically AAE

We benchmark two latent code-based methods (AAE and ARAE)

compression dataset is utilized to compare our method with these methods using

various objective and subjective measures.

proposed (self) attention-based models outperform the state-of-the-art in adversarial

We propose a self-attention based GAN architecture for unconditional text generation and improve on previous adversarial code-based results.

SALSA-TEXT : SELF ATTENTIVE LATENT SPACE BASED ADVERSARIAL TEXT GENERATION


Watermarks have been used for various purposes.

Recently, researchers started to look into using them for deep neural networks.

Some works try to hide attack triggers on their adversarial samples when attacking neural networks and others want to watermark neural networks to prove their ownership against plagiarism.

Implanting a backdoor watermark module into a neural network is getting more attention from the community.

In this paper, we present a general purpose encoder-decoder joint training method, inspired by generative adversarial networks (GANs).

Unlike GANs, however, our encoder and decoder neural networks cooperate to find the best watermarking scheme given data samples.

In other words, we do not design any new watermarking strategy but our proposed two neural networks will find the best suited method on their own.

After being trained, the decoder can be implanted into other neural networks to attack or protect them (see Appendix for their use cases and real implementations).

To this end, the decoder should be very tiny in order not to incur any overhead when attached to other neural networks but at the same time provide very high decoding success rates, which is very challenging.

Our joint training method successfully solves the problem and in our experiments maintain almost 100\% encoding-decoding success rates for multiple datasets with very little modifications on data samples to hide watermarks.

We also present several real-world use cases in Appendix.

We propose a novel watermark encoder-decoder neural networks. They perform a cooperative game to define their own watermarking scheme. People do not need to design watermarking methods any more.

Fatty and Skinny: A Joint Training Method of Watermark Encoder and Decoder


We derive an unbiased estimator for expectations over discrete random variables based on sampling without replacement, which reduces variance as it avoids duplicate samples.

We show that our estimator can be derived as the Rao-Blackwellization of three different estimators.

Combining our estimator with REINFORCE, we obtain a policy gradient estimator and we reduce its variance using a built-in control variate which is obtained without additional model evaluations.

The resulting estimator is closely related to other gradient estimators.

Experiments with a toy problem, a categorical Variational Auto-Encoder and a structured prediction problem show that our estimator is the only estimator that is consistently among the best estimators in both high and low entropy settings.

We derive a low-variance, unbiased gradient estimator for expectations over discrete random variables based on sampling without replacement

Estimating Gradients for Discrete Random Variables by Sampling without Replacement

discrete random variable
sample without replacement
gradient


We introduce a parameter sharing scheme, in which different layers of a convolutional neural network (CNN) are defined by a learned linear combination of parameter tensors from a global bank of templates.

Restricting the number of templates yields a flexible hybridization of traditional CNNs and recurrent networks.

Compared to traditional CNNs, we demonstrate substantial parameter savings on standard image classification tasks, while maintaining accuracy.

Our simple parameter sharing scheme, though defined via soft weights, in practice often yields trained networks with near strict recurrent structure; with negligible side effects, they convert into networks with actual loops.

Training these networks thus implicitly involves discovery of suitable recurrent architectures.

Though considering only the aspect of recurrent links, our trained networks achieve accuracy competitive with those built using state-of-the-art neural architecture search (NAS) procedures.

Our hybridization of recurrent and convolutional networks may also represent a beneficial architectural bias.

Specifically, on synthetic tasks which are algorithmic in nature, our hybrid networks both train faster and extrapolate better to test examples outside the span of the training set.

We propose a method that enables CNN folding to create recurrent connections

Learning Implicitly Recurrent CNNs Through Parameter Sharing


Gradient clipping is a widely-used technique in the training of deep networks, and is generally motivated from an optimisation lens: informally, it controls the dynamics of iterates, thus enhancing the rate of convergence to a local minimum.

This intuition has been made precise in a line of recent works, which show that suitable clipping  can yield significantly faster convergence than vanilla gradient descent.

In this paper, we propose a new lens for studying gradient clipping, namely, robustness: informally, one expects clipping to provide robustness to noise, since one does not overly trust any single sample.

Surprisingly, we prove that  for the common problem of label noise in classification, standard gradient clipping does not in general provide robustness.

On the other hand, we show that  a simple variant of gradient clipping is provably robust, and corresponds to suitably modifying the underlying loss function.

This yields a simple, noise-robust alternative to the standard cross-entropy loss which performs well empirically.

Gradient clipping doesn't endow robustness to label noise, but a simple loss-based variant does.

Can gradient clipping mitigate label noise?


Among deep generative models, flow-based models, simply referred as \emph{flow}s in this paper, differ from other models in that they provide tractable likelihood.

Besides being an evaluation metric of synthesized data, flows are supposed to be robust against out-of-distribution~(OoD) inputs since they do not discard any information of the inputs.

However, it has been observed that flows trained on FashionMNIST assign higher likelihoods to OoD samples from MNIST.

This counter-intuitive observation raises the concern about the robustness of flows' likelihood.

In this paper, we explore the correlation between flows' likelihood and image semantics.

We choose two typical flows as the target models: Glow, based on coupling transformations, and pixelCNN, based on autoregressive transformations.

Our experiments reveal surprisingly weak correlation between flows' likelihoods and image semantics: the predictive likelihoods of flows can be heavily affected by trivial transformations that keep the image semantics unchanged, which we call semantic-invariant transformations~(SITs).

We explore three SITs~(all small pixel-level modifications): image pixel translation, random noise perturbation, latent factors zeroing~(limited to flows using multi-scale architecture, e.g. Glow).

These findings, though counter-intuitive, resonate with the fact that the predictive likelihood of a flow is the joint probability of all the image pixels.

So flows' likelihoods, modeling on pixel-level intensities, is not able to indicate the existence likelihood of the high-level image semantics.

We call for attention that it may be \emph{abuse} if we use the predictive likelihoods of flows for OoD samples detection.

show experimental evidences about the weak correlation between flows' likelihoods and image semantics.

Exploring the Correlation between Likelihood of Flow-based Generative Models and Image Semantics


This paper investigates strategies that defend against adversarial-example attacks on image-classification systems by transforming the inputs before feeding them to the system.

Specifically, we study applying image transformations such as bit-depth reduction, JPEG compression, total variance minimization, and image quilting before feeding the image to a convolutional network classifier.

Our experiments on ImageNet show that total variance minimization and image quilting are very effective defenses in practice, in particular, when the network is trained on transformed images.

The strength of those defenses lies in their non-differentiable nature and their inherent randomness, which makes it difficult for an adversary to circumvent the defenses.

Our best defense eliminates 60% of strong gray-box and 90% of strong black-box attacks by a variety of major attack methods.

We apply a model-agnostic defense strategy against adversarial examples and achieve 60% white-box accuracy and 90% black-box accuracy against major attack algorithms.

Countering Adversarial Images using Input Transformations


In this paper, we propose the Asynchronous Accelerated Nonuniform Randomized Block Coordinate Descent algorithm (A2BCD).

We prove A2BCD converges linearly to a solution of the convex minimization problem at the same rate as NU_ACDM, so long as the maximum delay is not too large.

This is the first asynchronous Nesterov-accelerated algorithm that attains any provable speedup.

Moreover, we then prove that these algorithms both have optimal complexity.

Asynchronous algorithms complete much faster iterations, and A2BCD has optimal complexity.

Hence we observe in experiments that A2BCD is the top-performing coordinate descent algorithm, converging up to 4-5x faster than NU_ACDM on some data sets in terms of wall-clock time.

To motivate our theory and proof techniques, we also derive and analyze a continuous-time analog of our algorithm and prove it converges at the same rate.

We prove the first-ever convergence proof of an asynchronous accelerated algorithm that attains a speedup.

A2BCD: Asynchronous Acceleration with Optimal Complexity


A framework for efficient Bayesian inference in probabilistic programs is introduced by embedding a sampler inside a variational posterior approximation.

Its strength lies in both ease of implementation and automatically tuning sampler parameters to speed up mixing time.

Several strategies to approximate the evidence lower bound (ELBO) computation are introduced, including a rewriting of the ELBO objective.

Experimental evidence is shown by performing experiments on an unconditional VAE on density estimation tasks; solving an influence diagram in a high-dimensional space with a conditional variational autoencoder (cVAE) as a deep Bayes classifier; and state-space models for time-series data.

We embed SG-MCMC samplers inside a variational approximation

Variationally Inferred Sampling Through a Refined Bound


The point estimates of ReLU classification networks, arguably the most widely used neural network architecture, have recently been shown to have arbitrarily high confidence far away from the training data.

This architecture is thus not robust, e.g., against out-of-distribution data.

Approximate Bayesian posteriors on the weight space have been empirically demonstrated to improve predictive uncertainty in deep learning.

The theoretical analysis of such Bayesian approximations is limited, including for ReLU classification networks.

We present an analysis of approximate Gaussian posterior distributions on the weights of ReLU networks.

We show that even a simplistic (thus cheap), non-Bayesian Gaussian distribution fixes the asymptotic overconfidence issue.

Furthermore, when a Bayesian method, even if a simple one, is employed to obtain the Gaussian, the confidence becomes better calibrated.

This theoretical result motivates a range of Laplace approximations along a fidelity-cost trade-off.

We validate these findings empirically via experiments using common deep ReLU networks.

We argue theoretically that by simply assuming the weights of a ReLU network to be Gaussian distributed (without even a Bayesian formalism) could fix this issue; for a more calibrated uncertainty, a simple Bayesian method could already be sufficient.

Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks


Word alignments are useful for tasks like statistical and neural machine translation (NMT) and annotation projection.

Statistical word aligners perform well, as do methods that extract alignments jointly with translations in NMT.

However, most approaches require parallel training data and quality decreases as less training data is available.

We propose word alignment methods that require little or no parallel data.

The key idea is to leverage multilingual word embeddings – both static and contextualized – for word alignment.

Our multilingual embeddings are created from monolingual data only without relying on any parallel data or dictionaries.

We find that traditional statistical aligners are outperformed by contextualized embeddings – even in scenarios with abundant parallel data.

For example, for a set of 100k parallel sentences, contextualized embeddings achieve a word alignment F1 that is more than 5% higher (absolute) than eflomal.

We use representations trained without any parallel data for creating word alignments.

Leveraging Static and Contextualized Embeddings for Word Alignments


Recent studies have shown the vulnerability of reinforcement learning (RL) models in noisy settings.

The sources of noises differ across scenarios.

For instance, in practice, the observed reward channel is often subject to noise (e.g., when observed rewards are collected through sensors), and thus observed rewards may not be credible as a result.

Also, in applications such as robotics, a deep reinforcement learning (DRL) algorithm can be manipulated to produce arbitrary errors.

In this paper, we consider noisy RL problems where observed rewards by RL agents are generated with a reward confusion matrix.

We call such observed rewards as perturbed rewards.

We develop an unbiased reward estimator aided robust RL framework that enables RL agents to learn in noisy environments while observing only perturbed rewards.

Our framework draws upon approaches for supervised learning with noisy data.

The core ideas of our solution include estimating a reward confusion matrix and defining a set of unbiased surrogate rewards.

We prove the convergence and sample complexity of our approach.

Extensive experiments on different DRL platforms show that policies based on our estimated surrogate reward can achieve higher expected rewards, and converge faster than existing baselines.

For instance, the state-of-the-art PPO algorithm is able to obtain 67.5% and 46.7% improvements in average on five Atari games, when the error rates are 10% and 30% respectively.

A new approach for learning with noisy rewards in reinforcement learning

Reinforcement Learning with Perturbed Rewards


Training recurrent neural networks (RNNs) on long sequences using backpropagation through time (BPTT) remains a fundamental challenge.

It has been shown that adding a local unsupervised loss term into the optimization objective makes the training of RNNs on long sequences more effective.

While the importance of an unsupervised task can in principle be controlled by a coefficient in the objective function, the gradients with respect to the unsupervised loss term still influence all the hidden state dimensions, which might cause important information about the supervised task to be degraded or erased.

Compared to existing semi-supervised sequence learning methods, this paper focuses upon a traditionally overlooked mechanism -- an architecture with explicitly designed private and shared hidden units designed to mitigate the detrimental influence of the auxiliary unsupervised loss over the main supervised task.

We achieve this by dividing RNN hidden space into a private space for the supervised task and a shared space for both the supervised and unsupervised tasks.

We present extensive experiments with the proposed framework on several long sequence modeling benchmark datasets.

Results indicate that the proposed framework can yield performance gains in RNN models where long term dependencies are notoriously challenging to deal with.

This paper focuses upon a traditionally overlooked mechanism -- an architecture with explicitly designed private and shared hidden units designed to mitigate the detrimental influence of the auxiliary unsupervised loss over the main supervised task.

RNNs with Private and Shared Representations for Semi-Supervised Sequence Learning


We investigate multi-task learning approaches which use a shared feature representation for all tasks.

To better understand the transfer of task information, we study an architecture with a shared module for all tasks and a separate output module for each task.

We study the theory of this setting on linear and ReLU-activated models.

Our key observation is that whether or not tasks' data are well-aligned can significantly affect the performance of multi-task learning.

We show that misalignment between task data can cause negative transfer (or hurt performance) and provide sufficient conditions for positive transfer.

Inspired by the theoretical insights, we show that aligning tasks' embedding layers leads to performance gains for multi-task training and transfer learning on the GLUE benchmark and sentiment analysis tasks; for example, we obtained a 2.35% GLUE score average improvement on 5 GLUE tasks over BERT LARGE using our alignment method.

We also design an SVD-based task re-weighting scheme and show that it improves the robustness of multi-task training on a multi-label image dataset.

A Theoretical Study of Multi-Task Learning with Practical Implications for Improving Multi-Task Training and Transfer Learning

Understanding and Improving Information Transfer in Multi-Task Learning


We review three limitations of BLEU and ROUGE – the most popular metrics

used to assess reference summaries against hypothesis summaries, come up with

criteria for what a good metric should behave like and propose concrete ways to

assess the performance of a metric in detail and show the potential of Transformers-based Language Models to assess reference summaries against hypothesis summaries.

New method for assessing the quaility of similarity evaluators and showing potential of Transformer-based language models in replacing BLEU and ROUGE.

Towards Neural Similarity Evaluator


In this paper, we explore meta-learning for few-shot text classification.

Meta-learning has shown strong performance in computer vision, where low-level patterns are transferable across learning tasks.

However, directly applying this approach to text is challenging–lexical features highly informative for one task maybe insignificant for another.

Thus, rather than learning solely from words, our model also leverages their distributional signatures, which encode pertinent word occurrence patterns.

Our model is trained within a meta-learning framework to map these signatures into attention scores, which are then used to weight the lexical representations of words.

We demonstrate that our model consistently outperforms prototypical networks learned on lexical knowledge (Snell et al., 2017) in both few-shot text classification and relation classification by a significant margin across six benchmark datasets (19.96% on average in 1-shot classification).

Meta-learning methods used for vision, directly applied to NLP, perform worse than nearest neighbors on new classes; we can do better with distributional signatures.

Few-shot Text Classification with Distributional Signatures


The description of neural computations in the field of neuroscience relies on two competing views:

(i) a classical single-cell view that relates the activity of individual neurons to sensory or behavioural variables, and focuses on how different cell classes map onto computations;

(ii) a more recent population view that instead characterises computations in terms of collective neural trajectories, and focuses on the dimensionality of these trajectories as animals perform tasks.

How the two key concepts of cell classes and low-dimensional trajectories interact to shape neural computations is however currently not understood.

Here we address this question by combining machine-learning tools for training RNNs with reverse-engineering and theoretical analyses of network dynamics.

We introduce a novel class of theoretically tractable recurrent networks: low-rank, mixture of Gaussian RNNs.

In these networks, the rank of the connectivity controls the dimensionality of the dynamics, while the number of components in the Gaussian mixture corresponds to the number of cell classes.

Using back-propagation, we determine the minimum rank and number of cell classes needed to implement neuroscience tasks of increasing complexity.

We then exploit mean-field theory to reverse-engineer the obtained solutions and identify the respective roles of dimensionality and cell classes.

We show that the rank determines the phase-space available for dynamics that implement input-output mappings, while having multiple cell classes allows networks to flexibly switch between different types of dynamics in the available phase-space.

Our results have implications for the analysis of neuroscience experiments and the development of explainable AI.

A theoretical analysis of a new class of RNNs, trained on neuroscience tasks, allows us to identify the role of dynamical dimensionality and cell classes in neural computations.

Disentangling the roles of dimensionality and cell classes in neural computations

dimensionality cell class neural computation


We propose the fusion discriminator, a single unified framework for incorporating conditional information into a generative adversarial network (GAN) for a variety of distinct structured prediction tasks, including image synthesis, semantic segmentation, and depth estimation.

Much like commonly used convolutional neural network - conditional Markov random field (CNN-CRF) models, the proposed method is able to enforce higher-order consistency in the model, but without being limited to a very specific class of potentials.

The method is conceptually simple and flexible, and our experimental results demonstrate improvement on several diverse structured prediction tasks.

We propose the fusion discriminator, a novel architecture for incorporating conditional information into the discriminator of GANs for structured prediction tasks.

Structured Prediction using cGANs with Fusion Discriminator

fusion discriminator
structured prediction


Model-based reinforcement learning (MBRL) has been shown to be a powerful framework for data-efficiently learning control of continuous tasks.

Recent work in MBRL has mostly focused on using more advanced function approximators and planning schemes, leaving the general framework virtually unchanged since its conception.

In this paper, we identify a fundamental issue of the standard MBRL framework -- what we call the objective mismatch issue.

Objective mismatch arises when one objective is optimized in the hope that a second, often uncorrelated, metric will also be optimized.

In the context of MBRL, we characterize the objective mismatch between training the forward dynamics model w.r.t. the likelihood of the one-step ahead prediction, and the overall goal of improving performance on a downstream control task.

For example, this issue can emerge with the realization that dynamics models effective for a specific task do not necessarily need to be globally accurate, and vice versa globally accurate models might not be sufficiently accurate locally to obtain good control performance on a specific task.

In our experiments, we study this objective mismatch issue and demonstrate that the likelihood of the one-step ahead prediction is not always correlated with downstream control performance.

This observation highlights a critical flaw in the current MBRL framework which will require further research to be fully understood and addressed.

We propose an initial method to mitigate the mismatch issue by re-weighting dynamics model training.

Building on it, we conclude with a discussion about other potential directions of future research for addressing this issue.

We define, explore, and begin to address the objective mismatch issue in model-based reinforcement learning.

Objective Mismatch in Model-based Reinforcement Learning

model-based reinforcement learning
objective mismatch


There has recently been a heated debate (e.g. Schwartz-Ziv & Tishby (2017), Saxe et al. (2018), Noshad et al. (2018), Goldfeld et al. (2018)) about measuring the information flow in Deep Neural Networks using techniques from information theory.

It is claimed that Deep Neural Networks in general have good generalization capabilities since they not only learn how to map from an input to an output but also how to compress information about the training data input (Schwartz-Ziv & Tishby, 2017).

That is, they abstract the input information and strip down any unnecessary or over-specific information.

If so, the message compression method, Information Bottleneck (IB), could be used as a natural comparator for network performance, since this method gives an optimal information compression boundary.

This claim was then later denounced as well as reaffirmed (e.g.  Saxe et al. (2018), Achille et al. (2017), Noshad et al. (2018)), as the employed method of mutual information measuring is not actually measuring information but clustering of the internal layer representations (Goldfeld et al. (2018)).

In this paper, we will present a detailed explanation of the development in the Information Plain (IP), which is a plot-type that compares mutual information to judge compression (Schwartz-Ziv & Tishby (2017)), when noise is retroactively added (using binning estimation).

We also explain why different activation functions show different trajectories on the IP.

Further, we have looked into the effect of clustering on the network loss through early and perfect stopping using the Information Plane and how clustering can be used to help network pruning.

We give a detailed explanation of the trajectories in the information plane and investigate its usage for neural network design (pruning)

Revisiting the Information Plane


Formal understanding of the inductive bias behind deep convolutional networks, i.e. the relation between the network's architectural features and the functions it is able to model, is limited.

In this work, we establish a fundamental connection between the fields of quantum physics and deep learning, and use it for obtaining novel theoretical observations regarding the inductive bias of convolutional networks.

Specifically, we show a structural equivalence between the function realized by a convolutional arithmetic circuit (ConvAC) and a quantum many-body wave function, which facilitates the use of quantum entanglement measures as quantifiers of a deep network's expressive ability to model correlations.

Furthermore, the construction of a deep ConvAC in terms of a quantum Tensor Network is enabled.

This allows us to perform a graph-theoretic analysis of a convolutional network, tying its expressiveness to a min-cut in its underlying graph.

We demonstrate a practical outcome in the form of a direct control over the inductive bias via the number of channels (width) of each layer.

We empirically validate our findings on standard convolutional networks which involve ReLU activations and max pooling.

The description of a deep convolutional network in well-defined graph-theoretic tools and the structural connection to quantum entanglement, are two interdisciplinary bridges that are brought forth by this work.

Employing quantum entanglement measures for quantifying correlations in deep learning, and using the connection to fit the deep network's architecture to correlations in the data.

Deep Learning and Quantum Entanglement: Fundamental Connections with Implications to Network Design

deep learning
quantum entanglement
connection
network


Deep learning algorithms are increasingly used in modeling chemical processes.

However, black box predictions without rationales have limited used in practical applications, such as drug design.

To this end, we learn to identify molecular substructures -- rationales -- that are associated with the target chemical property (e.g., toxicity).

The rationales are learned in an unsupervised fashion, requiring no additional information beyond the end-to-end task.

We formulate this problem as a reinforcement learning problem over the molecular graph, parametrized by two convolution networks corresponding to the rationale selection and prediction based on it, where the latter induces the reward function.

We evaluate the approach on two benchmark toxicity datasets.

We demonstrate that our model sustains high performance under the additional constraint that predictions strictly follow the rationales.

Additionally, we validate the extracted rationales through comparison against those described in chemical literature and through synthetic experiments.

We use a reinforcement learning over molecular graphs to generate rationales for interpretable molecular property prediction.

Using Deep Reinforcement Learning to Generate Rationales for Molecules

generate rationale
reinforcement learning


In recent years, substantial progress has been made on graph convolutional networks (GCN).

In this paper, for the first time, we theoretically analyze the connections between GCN and matrix factorization (MF), and unify GCN as matrix factorization with co-training and unitization.

Moreover, under the guidance of this theoretical analysis, we propose an alternative model to GCN named Co-training and Unitized Matrix Factorization (CUMF).

The correctness of our analysis is verified by thorough experiments.

The experimental results show that CUMF achieves similar or superior performances compared to GCN.

In addition, CUMF inherits the benefits of MF-based methods to naturally support constructing mini-batches, and is more friendly to distributed computing comparing with GCN.

The distributed CUMF on semi-supervised node classification significantly outperforms distributed GCN methods.

Thus, CUMF greatly benefits large scale and complex real-world applications.

We unify graph convolutional networks as co-training and unitized matrix factorization.

Unifying Graph Convolutional Networks as Matrix Factorization

unify graph convolutional network
matrix factorization


Molecular graph generation is a fundamental problem for drug discovery and has been attracting growing attention.

The problem is challenging since it requires not only generating chemically valid molecular structures but also optimizing their chemical properties in the meantime.

Inspired by the recent progress in deep generative models, in this paper we propose a flow-based autoregressive model for graph generation called GraphAF.

GraphAF combines the advantages of both autoregressive and flow-based approaches and enjoys: (1) high model flexibility for data density estimation; (2) efficient parallel computation for training; (3) an iterative sampling process, which allows leveraging chemical domain knowledge for valency checking.

Experimental results show that GraphAF is able to generate 68\% chemically valid molecules even without chemical knowledge rules and 100\% valid molecules with chemical rules.

The training process of GraphAF is two times faster than the existing state-of-the-art approach GCPN.

After fine-tuning the model for goal-directed property optimization with reinforcement learning, GraphAF achieves state-of-the-art performance on both chemical property optimization and constrained property optimization.

A flow-based autoregressive model for molecular graph generation. Reaching state-of-the-art results on molecule generation and properties optimization.

GraphAF: a Flow-based Autoregressive Model for Molecular Graph Generation

flow-based autoregressive model molecular graph generation


We study two types of preconditioners and preconditioned stochastic gradient descent (SGD) methods in a unified framework.

We call the first one the Newton type due to its close relationship to the Newton method, and the second one the Fisher type as its preconditioner is closely related to the inverse of Fisher information matrix.

Both preconditioners can be derived from one framework, and efficiently estimated on any matrix Lie groups designated by the user using natural or relative gradient descent minimizing certain preconditioner estimation criteria.

Many existing preconditioners and methods, e.g., RMSProp, Adam, KFAC, equilibrated SGD, batch normalization, etc., are special cases of or closely related to either the Newton type or the Fisher type ones.

Experimental results on relatively large scale machine learning  problems are reported for performance study.

We propose a new framework for preconditioner learning, derive new forms of preconditioners and learning methods, and reveal the relationship to methods like RMSProp, Adam, Adagrad, ESGD, KFAC, batch normalization, etc.

Preconditioner on Matrix Lie Group for SGD


We present EDA: easy data augmentation techniques for boosting performance on text classification tasks.

EDA consists of four simple but powerful operations: synonym replacement, random insertion, random swap, and random deletion.

On five text classification tasks, we show that EDA improves performance for both convolutional and recurrent neural networks.

EDA demonstrates particularly strong results for smaller datasets; on average, across five datasets, training with EDA while using only 50% of the available training set achieved the same accuracy as normal training with all available data.

We also performed extensive ablation studies and suggest parameters for practical use.

Simple text augmentation techniques can significantly boost performance on text classification tasks, especially for small datasets.

EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks

boost performance text classification task


We propose a model that is able to perform physical parameter estimation of systems from video, where the differential equations governing the scene dynamics are known, but labeled states or objects are not available.

Existing physical scene understanding methods require either object state supervision, or do not integrate with differentiable physics to learn interpretable system parameters and states.

We address this problem through a \textit{physics-as-inverse-graphics} approach that brings together vision-as-inverse-graphics and differentiable physics engines, where objects and explicit state and velocity representations are discovered by the model.

This framework allows us to perform long term extrapolative video prediction, as well as vision-based model-predictive control.

Our approach significantly outperforms related unsupervised methods in long-term future frame prediction of systems with interacting objects (such as ball-spring or 3-body gravitational systems), due to its ability to build dynamics into the model as an inductive bias.

We further show the value of this tight vision-physics integration by demonstrating data-efficient learning of vision-actuated model-based control for a pendulum system.

We also show that the controller's interpretability provides unique capabilities in goal-driven control and physical reasoning for zero-data adaptation.

Physics-as-Inverse-Graphics: Unsupervised Physical Parameter Estimation from Video


This paper proposes ASAL, a new pool based active learning method that generates high entropy samples.

Instead of directly annotating the synthetic samples, ASAL searches similar samples from the pool and includes them for training.

Hence, the quality of new samples is high and annotations are reliable.

ASAL is particularly suitable for large data sets because it achieves a better run-time complexity (sub-linear) for sample selection than traditional uncertainty sampling (linear).

We present a comprehensive set of experiments on two data sets and show that ASAL outperforms similar methods and clearly exceeds the established baseline (random sampling).

In the discussion section we analyze in which situations ASAL performs best and why it is sometimes hard to outperform random sample selection.

To the best of our knowledge this is the first adversarial active learning technique that is applied for multiple class problems using deep convolutional classifiers and demonstrates superior performance than random sample selection.

ASAL is a pool based active learning method that generates high entropy samples and retrieves matching samples from the pool in sub-linear time.

Adversarial Sampling for Active Learning


We point out important problems with the common practice of using the best single model performance for comparing deep learning architectures, and we propose a method that corrects these flaws.

Each time a model is trained, one gets a different result due to random factors in the training process, which include random parameter initialization and random data shuffling.

Reporting the best single model performance does not appropriately address this stochasticity.

We propose a normalized expected best-out-of-n performance (Boo_n) as a way to correct these problems.

A Boo(n) for Evaluating Architecture Performance


Unsupervised domain adaptation aims to generalize the hypothesis trained in a source domain to an unlabeled target domain.

One popular approach to this problem is to learn domain-invariant embeddings for both domains.

In this work, we study, theoretically and empirically, the effect of the embedding complexity on generalization to the target domain.

In particular, this complexity affects an upper bound on the target risk; this is reflected in experiments, too.

Next, we specify our theoretical framework to multilayer neural networks.

As a result, we develop a strategy that mitigates sensitivity to the embedding complexity, and empirically achieves performance on par with or better than the best layer-dependent complexity tradeoff.

We study the effect of the embedding complexity in learning domain-invariant representations and develop a strategy that mitigates sensitivity to it.

The Role of Embedding Complexity in Domain-invariant Representations

embed complexity
domain-invariant representation


We propose a new architecture termed Dual Adversarial Transfer Network (DATNet) for addressing low-resource Named Entity Recognition (NER).

Specifically, two variants of DATNet, i.e., DATNet-F and DATNet-P, are proposed to explore effective feature fusion between high and low resource.

To address the noisy and imbalanced training data, we propose a novel Generalized Resource-Adversarial Discriminator (GRAD).

Additionally, adversarial training is adopted to boost model generalization.

We examine the effects of different components in DATNet across domains and languages and show that significant improvement can be obtained especially for low-resource data.

Without augmenting any additional hand-crafted features, we achieve new state-of-the-art performances on CoNLL and Twitter NER---88.16% F1 for Spanish, 53.43% F1 for WNUT-2016, and 42.83% F1 for WNUT-2017.

We propose a new  architecture termed Dual Adversarial Transfer Network (DATNet) for addressing low-resource Named Entity Recognition (NER) and achieve new state-of-the-art performances on CoNLL and Twitter NER.

DATNet: Dual Adversarial Transfer for Low-resource Named Entity Recognition

low-resource name entity recognition
dual adversarial transfer
datnet


Generative adversarial networks (GANs) evolved into one of the most successful unsupervised techniques for generating realistic images.

Even though it has recently been shown that GAN training converges, GAN models often end up in local Nash equilibria that are associated with mode collapse or otherwise fail to model the target distribution.

We introduce Coulomb GANs, which pose the GAN learning problem as a potential field, where generated samples are attracted to training set samples but repel each other.

The discriminator learns a potential field while the generator decreases the energy by moving its samples along the vector (force) field determined by the gradient of the potential field.

Through decreasing the energy, the GAN model learns to generate samples according to the whole target distribution and does not only cover some of its modes.

We prove that Coulomb GANs possess only one Nash equilibrium which is optimal in the sense that the model distribution equals the target distribution.

We show the efficacy of Coulomb GANs on LSUN bedrooms, CelebA faces, CIFAR-10 and the Google Billion Word text generation.

Coulomb GANs can optimally learn a distribution by posing the distribution learning problem as optimizing a potential field

Coulomb GANs: Provably Optimal Nash Equilibria via Potential Fields


Predicting properties of nodes in a graph is an important problem with applications in a variety of domains.

Graph-based Semi Supervised Learning (SSL) methods aim to address this problem by labeling a small subset of the nodes as seeds, and then utilizing the graph structure to predict label scores for the rest of the nodes in the graph.

Recently, Graph Convolutional Networks (GCNs) have achieved impressive performance on the graph-based SSL task.

In addition to label scores, it is also desirable to have a confidence score associated with them.

Unfortunately, confidence estimation in the context of GCN has not been previously explored.

We fill this important gap in this paper and propose ConfGCN, which estimates labels scores along with their confidences jointly in GCN-based setting.

ConfGCN uses these estimated confidences to determine the influence of one node on another during neighborhood aggregation, thereby acquiring anisotropic capabilities.

Through extensive analysis and experiments on standard benchmarks, we find that ConfGCN is able to significantly outperform state-of-the-art baselines.

We have made ConfGCN’s source code available to encourage reproducible research.

We propose a confidence based Graph Convolutional Network for Semi-Supervised Learning.

Confidence-based Graph Convolutional Networks for Semi-Supervised Learning

graph convolutional network semi-supervised learning


In this paper we establish rigorous benchmarks for image classifier robustness.

Our first benchmark, ImageNet-C, standardizes and expands the corruption robustness topic, while showing which classifiers are preferable in safety-critical applications.

Then we propose a new dataset called ImageNet-P which enables researchers to benchmark a classifier's robustness to common perturbations.

Unlike recent robustness research, this benchmark evaluates performance on common corruptions and perturbations not worst-case adversarial perturbations.

We find that there are negligible changes in relative corruption robustness from AlexNet classifiers to ResNet classifiers.

Afterward we discover ways to enhance corruption and perturbation robustness.

We even find that a bypassed adversarial defense provides substantial common perturbation robustness.

Together our benchmarks may aid future work toward networks that robustly generalize.

We propose ImageNet-C to measure classifier corruption robustness and ImageNet-P to measure perturbation robustness

Benchmarking Neural Network Robustness to Common Corruptions and Perturbations


The paper explores a novel methodology in source code obfuscation through the application of text-based recurrent neural network network (RNN) encoder-decoder models in ciphertext generation and key generation.

are incorporated into the model architecture to generate obfuscated code, generate the deobfuscation key, and live execution.

Quantitative benchmark comparison to existing obfuscation methods indicate significant improvement in stealth and execution cost for the proposed solution, and experiments regarding the model’s properties yield positive results regarding its character variation, dissimilarity to the original codebase, and consistent length of obfuscated code.

Obfuscate code using seq2seq networks, and execute using the obfuscated code and key pair

DeepObfusCode: Source Code Obfuscation Through Sequence-to-Sequence Networks


We propose a method for joint image and per-pixel annotation synthesis with GAN.

We demonstrate that GAN has good high-level representation of target data that can be easily projected to semantic segmentation masks.

This method can be used to create a training dataset for teaching separate semantic segmentation network.

Our experiments show that such segmentation network successfully generalizes on real data.

Additionally, the method outperforms supervised training when the number of training samples is small, and works on variety of different scenes and classes.

The source code of the proposed method will be publicly available.

GAN-based method for joint image and per-pixel annotation synthesis

Teaching GAN to generate per-pixel annotation


Vision-Language Navigation (VLN) is the task where an agent is commanded to navigate in photo-realistic unknown environments with natural language instructions.

Previous research on VLN is primarily conducted on the Room-to-Room (R2R) dataset with only English instructions.

The ultimate goal of VLN, however, is to serve people speaking arbitrary languages.

Towards multilingual VLN with numerous languages, we collect a cross-lingual R2R dataset, which extends the original benchmark with corresponding Chinese instructions.

But it is time-consuming and expensive to collect large-scale human instructions for every existing language.

Based on the newly introduced dataset, we propose a general cross-lingual VLN framework to enable instruction-following navigation for different languages.

We first explore the possibility of building a cross-lingual agent when no training data of the target language is available.

The cross-lingual agent is equipped with a meta-learner to aggregate cross-lingual representations and a visually grounded cross-lingual alignment module to align textual representations of different languages.

Under the zero-shot learning scenario, our model shows competitive results even compared to a model trained with all target language instructions.

In addition, we introduce an adversarial domain adaption loss to improve the transferring ability of our model when given a certain amount of target language data.

Our methods and dataset demonstrate the potentials of building a cross-lingual agent to serve speakers with different languages.

We introduce a new task and dataset on cross-lingual vision-language navigation, and propose a general cross-lingual VLN framework for the task.

Cross-Lingual Vision-Language Navigation


Deep generative models have advanced the state-of-the-art in semi-supervised classification, however their capacity for deriving useful discriminative features in a completely unsupervised fashion for classification in difficult real-world data sets, where adequate manifold separation is required has not been adequately explored.

Most methods rely on defining a pipeline of deriving features via generative modeling and then applying clustering algorithms, separating the modeling and discriminative processes.

We propose a deep hierarchical generative model which uses a mixture of discrete and continuous distributions to learn to effectively separate the different data manifolds and is trainable end-to-end.

We show that by specifying the form of the discrete variable distribution we are imposing a specific structure on the model's latent representations.

We test our model's discriminative performance on the task of CLL diagnosis against baselines from the field of computational FC, as well as the Variational Autoencoder literature.

Unsupervised classification via deep generative modeling with controllable feature learning evaluated in a difficult real world task

Towards Unsupervised Classification with Deep Generative Models


Representation Learning over graph structured data has received significant attention recently due to its ubiquitous applicability.

However, most advancements have been made in static graph settings while efforts for jointly learning dynamic of the graph and dynamic on the graph are still in an infant stage.

Two fundamental questions arise in learning over dynamic graphs:

(i) How to elegantly model dynamical processes over graphs?

(ii) How to leverage such a model to effectively encode evolving graph information into low-dimensional representations?

We present DyRep - a novel modeling framework for dynamic graphs that posits representation learning as a latent mediation process bridging two observed processes namely -- dynamics of the network (realized as topological evolution) and dynamics on the network (realized as activities between nodes).

Concretely, we propose a two-time scale deep temporal point process model that captures the interleaved dynamics of the observed processes.

This model is further parameterized by a temporal-attentive representation network that encodes temporally evolving structural information into node representations which in turn drives the nonlinear evolution of the observed graph dynamics.

Our unified framework is trained using an efficient unsupervised procedure and has capability to generalize over unseen nodes.

We demonstrate that DyRep outperforms state-of-the-art baselines for dynamic link prediction and time prediction tasks and present extensive qualitative insights into our framework.

Models Representation Learning over dynamic graphs as latent hidden process bridging two observed processes of Topological Evolution of and Interactions on dynamic graphs.

DyRep: Learning Representations over Dynamic Graphs


With the success of modern machine learning, it is becoming increasingly important to understand and control how learning algorithms interact.

Unfortunately, negative results from game theory show there is little hope of understanding or controlling general n-player games.

We therefore introduce smooth markets (SM-games), a class of n-player games with pairwise zero sum interactions.

SM-games codify a common design pattern in machine learning that includes some GANs, adversarial training, and other recent algorithms.

We show that SM-games are amenable to analysis and optimization using first-order methods.

We introduce a class of n-player games suited to gradient-based methods.

Smooth markets: A basic mechanism for organizing gradient-based learners


While counter machines have received little attention in theoretical computer science since the 1960s, they have recently achieved a newfound relevance to the field of natural language processing (NLP).

Recent work has suggested that some strong-performing recurrent neural networks utilize their memory as counters.

Thus, one potential way to understand the sucess of these networks is to revisit the theory of counter computation.

Therefore, we choose to study the abilities of real-time counter machines as formal grammars.

We first show that several variants of the counter machine converge to express the same class of formal languages.

We also prove that counter languages are closed under complement, union, intersection, and many other common set operations.

Next, we show that counter machines cannot evaluate boolean expressions, even though they can weakly validate their syntax.

This has implications for the interpretability and evaluation of neural network systems: successfully matching syntactic patterns does not guarantee that a counter-like model accurately represents underlying semantic structures.

Finally, we consider the question of whether counter languages are semilinear.

This work makes general contributions to the theory of formal languages that are of particular interest for the interpretability of recurrent neural networks.

We study the class of formal languages acceptable by real-time counter automata, a model of computation related to some types of recurrent neural networks.

On the Linguistic Capacity of Real-time Counter Automata


Attentional, RNN-based encoder-decoder models for abstractive summarization have achieved good performance on short input and output sequences.

For longer documents and summaries however these models often include repetitive and incoherent phrases.

We introduce a neural network model with a novel intra-attention that attends over the input and continuously generated output separately, and a new training method that combines standard supervised word prediction and reinforcement learning (RL).

Models trained only with supervised learning often exhibit "exposure bias" - they assume ground truth is provided at each step during training.

However, when standard word prediction is combined with the global sequence prediction training of RL the resulting summaries become more readable.

We evaluate this model on the CNN/Daily Mail and New York Times datasets.

Our model obtains a 41.16 ROUGE-1 score on the CNN/Daily Mail dataset, an improvement over previous state-of-the-art models.

Human evaluation also shows that our model produces higher quality summaries.

A summarization model combining a new intra-attention and reinforcement learning method to increase summary ROUGE scores and quality for long sequences.

A Deep Reinforced Model for Abstractive Summarization


Knowledge Distillation (KD) is a common method for transferring the ``knowledge'' learned by one machine learning model (the teacher) into another model (the student), where typically, the teacher has a greater capacity (e.g., more parameters or higher bit-widths).

To our knowledge, existing methods overlook the fact that although the student absorbs extra knowledge from the teacher, both models share the same input data -- and this data is the only medium by which the teacher's knowledge can be demonstrated.

Due to the difference in model capacities, the student may not benefit fully from the same data points on which the teacher is trained.

On the other hand, a human teacher may demonstrate a piece of knowledge with individualized examples adapted to a particular student, for instance, in terms of her cultural background and interests.

Inspired by this behavior, we design data augmentation agents with distinct roles to facilitate knowledge distillation.

Our data augmentation agents generate distinct training data for the teacher and student, respectively.

We focus specifically on KD when the teacher network has greater precision (bit-width) than the student network.

We find empirically that specially tailored data points enable the teacher's knowledge to be demonstrated more effectively to the student.

We compare our approach with existing KD methods on training popular neural architectures and demonstrate that role-wise data augmentation improves the effectiveness of KD over strong prior approaches.

The code for reproducing our results will be made publicly available.

We study whether and how adaptive data augmentation and knowledge distillation can be leveraged simultaneously in a synergistic manner for better training student networks.

Role-Wise Data Augmentation for Knowledge Distillation

datum augmentation knowledge distillation


Neural network models have shown excellent fluency and performance when applied to abstractive summarization.

Many approaches to neural abstractive summarization involve the introduction of significant inductive bias, such as pointer-generator architectures, coverage, and partially extractive procedures, designed to mimic human summarization.

We show that it is possible to attain competitive performance by instead directly viewing summarization as language modeling.

We introduce a simple procedure built upon pre-trained decoder-transformers to obtain competitive ROUGE scores using a language modeling loss alone, with no beam-search or other decoding-time optimization, and instead rely on efficient nucleus sampling and greedy decoding.

We introduce a simple procedure to repurpose pre-trained transformer-based language models to perform abstractive summarization well.

Repurposing Decoder-Transformer Language Models for Abstractive Summarization

language model
abstractive summarization
repurpose


This paper addresses the problem of incremental domain adaptation (IDA).

We assume each domain comes sequentially, and that we could only access data in the current domain.

The goal of IDA is  to build a unified model performing well on all the encountered domains.

We propose to augment a recurrent neural network (RNN) with a directly parameterized memory bank, which is retrieved by an attention mechanism at each step of RNN transition.

The memory bank provides a natural way of IDA: when adapting our model to a new domain, we progressively add new slots to the memory bank, which increases the model capacity.

We learn the new memory slots and fine-tune existing parameters by back-propagation.

Experiments show that our approach significantly outperforms naive fine-tuning and previous work on IDA, including elastic weight consolidation and the progressive neural network.

Compared with expanding hidden states, our approach is more robust for old domains, shown by both empirical and theoretical results.

We present a neural memory-based architecture for incremental domain adaptation, and provide theoretical and empirical results.

Progressive Memory Banks for Incremental Domain Adaptation


In compressed sensing, a primary problem to solve is to reconstruct a high dimensional sparse signal from a small number of observations.

In this work, we develop a new sparse signal recovery algorithm using reinforcement learning (RL) and Monte CarloTree Search (MCTS).

Similarly to orthogonal matching pursuit (OMP), our RL+MCTS algorithm chooses the support of the signal sequentially.

The key novelty is that the proposed algorithm learns how to choose the next support as opposed to following a pre-designed rule as in OMP.

Empirical results are provided to demonstrate the superior performance of the proposed RL+MCTS algorithm over existing sparse signal recovery algorithms.

Formulating sparse signal recovery as a sequential decision making problem, we develop a method based on RL and MCTS that learns a policy to discover the support of the sparse signal.

Learning to Recover Sparse Signals


Many real-world sequential decision-making problems can be formulated as optimal control with high-dimensional observations and unknown dynamics.

A promising approach is to embed the high-dimensional observations into a lower-dimensional latent representation space, estimate the latent dynamics model, then utilize this model for control in the latent space.

An important open question is how to learn a representation that is amenable to existing control algorithms?

In this paper, we focus on learning representations for locally-linear control algorithms, such as iterative LQR (iLQR).

By formulating and analyzing the representation learning problem from an optimal control perspective, we establish three underlying principles that the learned representation should comprise: 1) accurate prediction in the observation space, 2) consistency between latent and observation space dynamics, and 3) low curvature in the latent space transitions.

These principles naturally correspond to a loss function that consists of three terms: prediction, consistency, and curvature (PCC).

Crucially, to make PCC tractable, we derive an amortized variational bound for the PCC loss function.

Extensive experiments on benchmark domains demonstrate that the new variational-PCC learning algorithm benefits from significantly more stable and reproducible training, and leads to superior control performance.

Further ablation studies give support to the importance of all three PCC components for learning a good latent space for control.

Learning embedding for control with high-dimensional observations

Prediction, Consistency, Curvature: Representation Learning for Locally-Linear Control


The interplay between inter-neuronal network topology and cognition has been studied deeply by connectomics researchers and network scientists, which is crucial towards understanding the remarkable efficacy of biological neural networks.

Curiously, the deep learning revolution that revived neural networks has not paid much attention to topological aspects.

The architectures of deep neural networks (DNNs) do not resemble their biological counterparts in the topological sense.

We bridge this gap by presenting initial results of Deep Connectomics Networks (DCNs) as DNNs with topologies inspired by real-world neuronal networks.

We show high classification accuracy obtained by DCNs whose architecture was inspired by the biological neuronal networks of C. Elegans and the mouse visual cortex.

Initial findings in the intersection of network neuroscience and deep learning. C. Elegans and a mouse visual cortex learn to recognize handwritten digits.

Deep Connectomics Networks: Neural Network Architectures Inspired by Neuronal Networks


Convolutional neural networks and recurrent neural networks are designed with network structures well suited to the nature of spacial and sequential data respectively.

However, the structure of standard feed-forward neural networks (FNNs) is simply a stack of fully connected layers, regardless of the feature correlations in data.

In addition, the number of layers and the number of neurons are manually tuned on validation data, which is time-consuming and may lead to suboptimal networks.

In this paper, we propose an unsupervised structure learning method for learning parsimonious deep FNNs.

Our method determines the number of layers, the number of neurons at each layer, and the sparse connectivity between adjacent layers automatically from data.

The resulting models are called Backbone-Skippath Neural Networks (BSNNs).

Experiments on 17 tasks show that, in comparison with FNNs,  BSNNs can achieve better or comparable classification performance with much fewer parameters.

The interpretability of BSNNs is also shown to be better than that of FNNs.

An unsupervised structure learning method for Parsimonious Deep Feed-forward Networks.

Learning Parsimonious Deep Feed-forward Networks

parsimonious deep feed-forward network
learn


Bayesian inference is used extensively to infer and to quantify the uncertainty in a field of interest from a measurement of a related field when the two are linked by a mathematical model.

Despite its many applications, Bayesian inference faces challenges when inferring fields that have discrete representations of large dimension, and/or have prior distributions that are difficult to characterize mathematically.

In this work we demonstrate how the approximate distribution learned by a generative adversarial network (GAN) may be used as a prior in a Bayesian update to address both these challenges.

We demonstrate the efficacy of this approach by inferring and quantifying uncertainty in a physics-based inverse problem and an inverse problem arising in computer vision.

In this latter example, we also demonstrate how the knowledge of the spatial variation of uncertainty may be used to select an optimal strategy of placing the sensors (i.e. taking measurements), where information about the image is revealed one sub-region at a time.

Using GANs as priors for efficient Bayesian inference of complex fields.

GAN priors for Bayesian inference


We argue that the widely used Omniglot and miniImageNet benchmarks are too simple because their class semantics do not vary across episodes, which defeats their intended purpose of evaluating few-shot classification methods.

The class semantics of Omniglot is invariably “characters” and the class semantics of miniImageNet, “object category”.

Because the class semantics are so similar, we propose a new method called Centroid Networks which can achieve surprisingly high accuracies on Omniglot and miniImageNet without using any labels at metaevaluation time.

Our results suggest that those benchmarks are not adapted for supervised few-shot classification since the supervision itself is not necessary during meta-evaluation.

The Meta-Dataset, a collection of 10 datasets, was recently proposed as a harder few-shot classification benchmark.

Using our method, we derive a new metric, the Class Semantics Consistency Criterion, and use it to quantify the difficulty of Meta-Dataset.

Finally, under some restrictive assumptions, we show that Centroid Networks is faster and more accurate than a state-of-the-art learning-to-cluster method (Hsu et al., 2018).

Omniglot and miniImageNet are too simple for few-shot learning because we can solve them without using labels during meta-evaluation, as demonstrated with a method called centroid networks

Are Few-shot Learning Benchmarks Too Simple ?


We learn to identify decision states, namely the parsimonious set of states where decisions meaningfully affect the future states an agent can reach in an environment.

We utilize the VIC framework, which maximizes an agent’s `empowerment’, ie the ability to reliably reach a diverse set of states -- and formulate a sandwich bound on the empowerment objective that allows identification of decision states. Unlike previous work, our decision states are discovered without extrinsic rewards -- simply by interacting with the world.

Our results show that our decision states are:

2) lead to better exploration on downstream goal-driven tasks in partially observable environments.

Identify decision states (where agent can take actions that matter) without reward supervision, use it for transfer.

DS-VIC: Unsupervised Discovery of Decision States for Transfer in RL


Inverse problems are ubiquitous in natural sciences and refer to the challenging task of inferring complex and potentially multi-modal posterior distributions over hidden parameters given a set of observations.

Typically, a model of the physical process in the form of differential equations is available but leads to intractable inference over its parameters.

While the forward propagation of parameters through the model simulates the evolution of the system, the inverse problem of finding the parameters given the sequence of states is not unique.

In this work, we propose a generalisation of the Bayesian optimisation framework to approximate inference.

The resulting method learns approximations to the posterior distribution by applying Stein variational gradient descent on top of estimates from a Gaussian process model.

Preliminary results demonstrate the method's performance on likelihood-free inference for reinforcement learning environments.

An approach to combine variational inference and Bayesian optimisation to solve complicated inverse problems

Distributional Bayesian optimisation for variational inference on black-box simulators

variational inference
bayesian optimisation


We present a data driven approach to construct a library of feedback motion primitives for non-holonomic vehicles that guarantees bounded error in following arbitrarily long trajectories.

This ensures that motion re-planning can be avoided as long as disturbances to the vehicle remain within a certain bound and also potentially when the obstacles are displaced within a certain bound.

The library is constructed along local abstractions of the dynamics that enables addition of new motion primitives through abstraction refinement.

We provide sufficient conditions for construction of such robust motion primitives for a large class of nonlinear dynamics, including commonly used models, such as the standard Reeds-Shepp model.

The algorithm is applied for motion planning and control of a rover with slipping without its prior modelling.

We show that under some assumptions on vehicle dynamics and environment uncertainty it is possible to automatically synthesize motion primitives that do not accumulate error over time.

Data-driven construction of robust motion primitives for non-holonomic vehicles


Deep Convolutional Networks (DCNs) have been shown to be sensitive to Universal Adversarial Perturbations (UAPs): input-agnostic perturbations that fool a model on large portions of a dataset.

These UAPs exhibit interesting visual patterns, but this phenomena is, as yet, poorly understood.

Our work shows that visually similar procedural noise patterns also act as UAPs.

In particular, we demonstrate that different DCN architectures are sensitive to Gabor noise patterns.

This behaviour, its causes, and implications deserve further in-depth study.

Existing Deep Convolutional Networks in image classification tasks are sensitive to Gabor noise patterns, i.e. small structured changes to the input cause large changes to the output.

Sensitivity of Deep Convolutional Networks to Gabor Noise


Deep neural networks (DNNs) perform well on a variety of tasks despite the fact that most used in practice are vastly overparametrized and even capable of perfectly fitting randomly labeled data.

Recent evidence suggests that developing "compressible" representations is key for adjusting the complexity of overparametrized networks to the task at hand and avoiding overfitting (Arora et al., 2018; Zhou et al., 2018).

In this paper, we provide new empirical evidence that supports this hypothesis, identifying two independent mechanisms that emerge when the network’s width is increased: robustness (having units that can be removed without affecting accuracy) and redundancy (having units with similar activity).

In a series of experiments with AlexNet, ResNet and Inception networks in the CIFAR-10 and ImageNet datasets, and also using shallow networks with synthetic data, we show that DNNs consistently increase either their robustness, their redundancy, or both at greater widths for a comprehensive set of hyperparameters.

These results suggest that networks in the deep learning regime adjust their effective capacity by developing either robustness or redundancy.

Probing robustness and redundancy in deep neural networks reveals capacity-constraining features which help to explain non-overfitting.

Robustness and/or Redundancy Emerge in Overparametrized Deep Neural Networks

deep neural network
redundancy
robustness


Deep networks realize complex mappings that are often understood by their locally linear behavior at or around points of interest.

For example, we use the derivative of the mapping with respect to its inputs for sensitivity analysis, or to explain (obtain coordinate relevance for) a prediction.

One key challenge is that such derivatives are themselves inherently unstable.

In this paper, we propose a new learning problem to encourage deep networks to have stable derivatives over larger regions.

While the problem is challenging in general, we focus on networks with piecewise linear activation functions.

Our algorithm consists of an inference step that identifies a region around a point where linear approximation is provably stable, and an optimization step to expand such regions.

We propose a novel relaxation to scale the algorithm to realistic models.

We illustrate our method with residual and recurrent networks on image and sequence datasets.

A scalable algorithm to establish robust derivatives of deep networks w.r.t. the inputs.

Towards Robust, Locally Linear Deep Networks


Recent years have witnessed two seemingly opposite developments of deep convolutional neural networks (CNNs).

On one hand, increasing the density of CNNs by adding cross-layer connections achieve higher accuracy.

On the other hand, creating sparsity structures through regularization and pruning methods enjoys lower computational costs.

In this paper, we bridge these two by proposing a new network structure with locally dense yet externally sparse connections.

This new structure uses dense modules, as basic building blocks and then sparsely connects these modules via a novel algorithm during the training process.

Experimental results demonstrate that the locally dense yet externally sparse structure could acquire competitive performance on benchmark tasks (CIFAR10, CIFAR100, and ImageNet) while keeping the network structure slim.

In this paper, we explore an internal dense yet external sparse network structure of deep neural networks and analyze its key properties.

Learning Internal Dense But External Sparse Structures of Deep Neural Network

structure deep neural network
internal dense
external sparse


Statistical inference methods are fundamentally important in machine learning.

Most state-of-the-art inference algorithms are

variants of Markov chain Monte Carlo (MCMC) or variational inference (VI).

However, both methods struggle with limitations in practice: MCMC methods can be computationally demanding; VI methods may have large bias.

In this work, we aim to improve upon MCMC and VI by a novel hybrid method based on the idea of reducing simulation bias of finite-length MCMC chains using gradient-based optimisation.

The proposed method can generate low-biased samples by increasing the length of MCMC simulation and optimising the MCMC hyper-parameters, which offers attractive balance between approximation bias and computational efficiency.

We show that our method produces promising results on popular benchmarks when compared to recent hybrid methods of MCMC and VI.

Ergodic Inference: Accelerate Convergence by Optimisation


We show that information about whether a neural network's output will be correct or incorrect is present in the outputs of the network's intermediate layers.

To demonstrate this effect, we train a new "meta" network to predict from either the final output of the underlying "base" network or the output of one of the base network's intermediate layers whether the base network will be correct or incorrect for a particular input.

We find that, over a wide range of tasks and base networks, the meta network can achieve accuracies ranging from 65% - 85% in making this determination.

Information about whether a neural network's output will be correct or incorrect is somewhat present in the outputs of the network's intermediate layers.

Predicting the accuracy of neural networks from final and intermediate layer outputs

intermediate layer
neural network
output


We develop a new algorithm for imitation learning from a single expert demonstration.

In contrast to many previous one-shot imitation learning approaches, our algorithm does not assume access to more than one expert demonstration during the training phase.

Instead, we leverage an exploration policy to acquire unsupervised trajectories, which are then used to train both an encoder and a context-aware imitation policy.

The optimization procedures for the encoder, imitation learner, and exploration policy are all tightly linked.

This linking creates a feedback loop wherein the exploration policy collects new demonstrations that challenge the imitation learner, while the encoder attempts to help the imitation policy to the best of its abilities.

We evaluate our algorithm on 6 MujoCo robotics tasks.

Unsupervised self-imitation algorithm capable of inference from a single expert demonstration.

One Demonstration Imitation Learning


Significant work has been dedicated to developing methods for communicating reasons for decision-making within au-

tomated scheduling systems to human users.

However, much less focus has been placed on communicating reasons for why

scheduling systems are unable to arrive at a feasible solution when over-constrained.

We introduce the agent resource-constrained project scheduling problem (ARCPSP), an ex-

tension of the resource-constrained project scheduling problem which includes a conception of agents that execute tasks

We outline a generic framework, based on efficiently enumerating minimal unsatisfiable sets (MUS) and

maximal satisfiable sets (MSS), to produce small descriptions of the source of infeasibility.

mented with potential relaxations that would fix the infeasibility found within the problem instance.

this method may be applied to the ARCPSP and demonstrate how to generate different types of explanations for an over-

We develop a framework for generating human-understandable explanations for why infeasibility is occurring in over-constrained instances of a class of resource-constrained scheduling problems.

Human-Understandable Explanations of Infeasibility for Resource-Constrained Scheduling Problems


Adversarial perturbations cause a shift in the salient features of an image, which may result in a misclassification.

We demonstrate that gradient-based saliency approaches are unable to capture this shift, and develop a new defense which detects adversarial examples based on learnt saliency models instead.

We study two approaches: a CNN trained to distinguish between natural and adversarial images using the saliency masks produced by our learnt saliency model, and a CNN trained on the salient pixels themselves as its input.

On MNIST, CIFAR-10 and ASSIRA, our defenses are able to detect various adversarial attacks, including strong attacks such as C&W and DeepFool, contrary to gradient-based saliency and detectors which rely on the input image.

The latter are unable to detect adversarial images when the L_2- and L_infinity- norms of the perturbations are too small.

Lastly, we find that the salient pixel based detector improves on saliency map based detectors as it is more robust to white-box attacks.

We show that gradients are unable to capture shifts in saliency due to adversarial perturbations and present an alternative adversarial defense using learnt saliency models that is effective against both black-box and white-box attacks.

Model-based Saliency for the Detection of Adversarial Examples


Disentangled encoding is an important step towards a better representation learning.

However, despite the numerous efforts, there still is no clear winner that captures the independent features of the data in an unsupervised fashion.

In this work we empirically evaluate the performance of six unsupervised disentanglement approaches on the mpi3d toy dataset curated and released for the NeurIPS 2019 Disentanglement Challenge.

The methods investigated in this work are Beta-VAE, Factor-VAE, DIP-I-VAE, DIP-II-VAE, Info-VAE, and Beta-TCVAE.

The capacities of all models were progressively increased throughout the training and the hyper-parameters were kept intact across experiments.

The methods were evaluated based on five disentanglement metrics, namely, DCI, Factor-VAE, IRS, MIG, and SAP-Score.

Within the limitations of this study, the Beta-TCVAE approach was found to outperform its alternatives with respect to the normalized sum of metrics.

However, a qualitative study of the encoded latents reveal that there is not a consistent correlation between the reported metrics and the disentanglement potential of the model.

A Preliminary Study of Disentanglement  With Insights on the Inadequacy of Metrics


Most of the prior work on multi-agent reinforcement learning (MARL) achieves optimal collaboration by directly learning a policy for each agent to maximize a common reward.

In this paper, we aim to address this from a different angle.

In particular, we consider scenarios where there are self-interested agents (i.e., worker agents) which have their own minds (preferences, intentions, skills, etc.) and can not be dictated to perform tasks they do not want to do.

For achieving optimal coordination among these agents, we train a super agent (i.e., the manager) to manage them by first inferring their minds based on both current and past observations and then initiating contracts to assign suitable tasks to workers and promise to reward them with corresponding bonuses so that they will agree to work together.

The objective of the manager is to maximize the overall productivity as well as minimize payments made to the workers for ad-hoc worker teaming.

To train the manager, we propose Mind-aware Multi-agent Management Reinforcement Learning (M^3RL), which consists of agent modeling and policy learning.

We have evaluated our approach in two environments, Resource Collection and Crafting, to simulate multi-agent management problems with various task settings and multiple designs for the worker agents.

The experimental results have validated the effectiveness of our approach in modeling worker agents' minds online, and in achieving optimal ad-hoc teaming with good generalization and fast adaptation.

We propose Mind-aware Multi-agent Management Reinforcement Learning (M^3RL) for training a manager to motivate self-interested workers to achieve optimal collaboration by assigning suitable contracts to them.

M^3RL: Mind-aware Multi-agent Management Reinforcement Learning

mind-aware multi-agent management reinforcement learning


Inferring temporally coherent data features is crucial for a large variety of learning tasks.

We propose a network architecture that introduces temporal recurrent connections for the internal  state of the widely used residual blocks.

We demonstrate that, with these connections, convolutional neural networks can more robustly learn stable temporal states that persist between evaluations.

We demonstrate their potential for inferring high-quality super-resolution images from low resolution images produced with real-time renderers.

This data arises in a wide range of applications,  and is particularly challenging as it contains a strongly aliased signal.

Hence, the data differs substantially from the smooth inputs encountered  in natural videos, and existing techniques do not succeed at producing acceptable image quality.

We additionally propose a series of careful adjustments of typical generative adversarial architectures for video super-resolution to arrive at a first model that can produce detailed, yet temporally coherent images from an aliased stream of inputs from a real-time renderer.

A method for persistent latent states in ResBlocks demonstrated for super-resolution of alised image sequences.

Depth-Recurrent Residual Connections for Super-Resolution of Real-Time Renderings