Many tasks in artificial intelligence require the collaboration of multiple agents.

We exam deep reinforcement learning for multi-agent domains.

Recent research efforts often take the form of two seemingly conflicting perspectives, the decentralized perspective, where each agent is supposed to have its own controller; and the centralized perspective, where one assumes there is a larger model controlling all agents.

In this regard, we revisit the idea of the master-slave architecture by incorporating both perspectives within one framework.

Such a hierarchical structure naturally leverages advantages from one another.

The idea of combining both perspective is intuitive and can be well motivated from many real world systems, however, out of a variety of possible realizations, we highlights three key ingredients, i.e. composed action representation, learnable communication and independent reasoning.

With network designs to facilitate these explicitly, our proposal consistently outperforms latest competing methods both in synthetics experiments and when applied to challenging StarCraft  micromanagement tasks.

We revisit the idea of the master-slave architecture in multi-agent deep reinforcement learning and outperforms state-of-the-arts.

Revisiting The Master-Slave Architecture In Multi-Agent Deep Reinforcement Learning

master-slave architecture multi-agent deep reinforcement learning


We study the implicit bias of gradient descent methods in solving a binary classification problem over a linearly separable dataset.

The classifier is described by a nonlinear ReLU model and the objective function adopts the exponential loss function.

We first characterize the landscape of the loss function and show that there can exist spurious asymptotic local minima besides asymptotic global minima.

We then show that gradient descent (GD) can converge to either a global or a local max-margin direction, or may diverge from the desired max-margin direction in a general context.

For stochastic gradient descent (SGD), we show that it converges in expectation to either the global or the local max-margin direction if SGD converges.

We further explore the implicit bias of these algorithms in learning a multi-neuron network under certain stationary conditions, and show that the learned classifier maximizes the margins of each sample pattern partition under the ReLU activation.

We study the implicit bias of gradient methods in solving a binary classification problem with nonlinear ReLU models.

When Will Gradient Methods Converge to Max-margin Classifier under ReLU Models?


Model pruning has become a useful technique that improves the computational efficiency of deep learning, making it possible to deploy solutions in resource-limited scenarios.

A widely-used practice in relevant work assumes that a smaller-norm parameter or feature plays a less informative role at the inference time.

In this paper, we propose a channel pruning technique for accelerating the computations of deep convolutional neural networks (CNNs) that does not critically rely on this assumption.

Instead, it focuses on direct simplification of the channel-to-channel computation graph of a CNN without the need of performing a computationally difficult and not-always-useful task of making high-dimensional tensors of CNN structured sparse.

Our approach takes two stages: first to adopt an end-to-end stochastic training method that eventually forces the outputs of some channels to be constant, and then to prune those constant channels from the original neural network by adjusting the biases of their impacting layers such that the resulting compact model can be quickly fine-tuned.

Our approach is mathematically appealing from an optimization perspective and easy to reproduce.

We experimented our approach through several image learning benchmarks and demonstrate its interest- ing aspects and competitive performance.

A CNN model pruning method using ISTA and rescaling trick to enforce sparsity of scaling parameters in batch normalization.

Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers


Stochastic gradient descent (SGD) has been the dominant optimization method for training deep neural networks due to its many desirable properties.

One of the more remarkable and least understood quality of SGD is that it generalizes relatively well

on unseen data even when the neural network has millions of parameters.

We hypothesize that in certain cases it is desirable to relax its intrinsic generalization properties and introduce an extension of SGD called deep gradient boosting (DGB).

The key idea of DGB is that back-propagated gradients inferred using the chain rule can be viewed as pseudo-residual targets of a gradient boosting problem.

Thus at each layer of a neural network the weight update is calculated by solving the corresponding boosting problem using a linear base learner.

The resulting weight update formula can also be viewed as a normalization procedure of the data that arrives at each layer during the forward pass.

When implemented as a separate input normalization layer (INN) the new architecture shows improved performance on image recognition tasks when compared to the same architecture without normalization layers.

As opposed to batch normalization (BN), INN has no learnable parameters however it matches its performance on CIFAR10 and ImageNet classification tasks.

What can we learn about training neural networks if we treat each layer as a gradient boosting problem?

Deep Gradient Boosting -- Layer-wise Input Normalization of Neural Networks


We propose to extend existing deep reinforcement learning  (Deep RL) algorithms by allowing them to additionally choose sequences of actions as a part of their policy.

This modification forces the network to anticipate the reward of action sequences, which, as we show, improves the exploration leading to better convergence.

Our proposal is simple, flexible, and can be easily incorporated into any Deep RL framework.

We show the power of our scheme by consistently outperforming the state-of-the-art GA3C algorithm on several popular Atari Games.

Anticipation improves convergence of deep reinforcement learning.

Anticipatory Asynchronous Advantage Actor-Critic (A4C): The power of Anticipation in Deep Reinforcement Learning

deep reinforcement learning
anticipation


To optimize a neural network one often thinks of optimizing its parameters, but it is ultimately a matter of optimizing the function that maps inputs to outputs.

Since a change in the parameters might serve as a poor proxy for the change in the function, it is of some concern that primacy is given to parameters but that the correspondence has not been tested.

Here, we show that it is simple and computationally feasible to calculate distances between functions in a $L^2$ Hilbert space.

We examine how typical networks behave in this space, and compare how parameter $\ell^2$ distances compare to function $L^2$ distances between various points of an optimization trajectory.

We find that the two distances are nontrivially related.

In particular, the $L^2/\ell^2$ ratio decreases throughout optimization, reaching a steady value around when test error plateaus.

We then investigate how the $L^2$ distance could be applied directly to optimization.

We first propose that in multitask learning, one can avoid catastrophic forgetting by directly limiting how much the input/output function changes between tasks.

Secondly, we propose a new learning rule that constrains the distance a network can travel through $L^2$-space in any one update.

This allows new examples to be learned in a way that minimally interferes with what has previously been learned.

These applications demonstrate how one can measure and regularize function distances directly, without relying on parameters or local approximations like loss curvature.

We find movement in function space is not proportional to movement in parameter space during optimization. We propose a new natural-gradient style optimizer to address this.

Measuring and regularizing networks in function space


The information bottleneck (IB) problem tackles the issue of obtaining relevant compressed representations T of some random variable X for the task of predicting Y. It is defined as a constrained optimization problem which maximizes the information the representation has about the task, I(T;Y), while ensuring that a minimum level of compression r is achieved (i.e., I(X;T) <= r).

For practical reasons the problem is usually solved by maximizing the IB Lagrangian for many values of the Lagrange multiplier, therefore drawing the IB curve (i.e., the curve of maximal I(T;Y) for a given I(X;Y)) and selecting the representation of desired predictability and compression.

It is known when Y is a deterministic function of X, the IB curve cannot be explored and other Lagrangians have been proposed to tackle this problem (e.g., the squared IB Lagrangian).

(i) present a general family of Lagrangians which allow for the exploration of the IB curve in all scenarios;

(ii) prove that if these Lagrangians are used, there is a one-to-one mapping between the Lagrange multiplier and the desired compression rate r for known IB curve shapes, hence, freeing from the burden of solving the optimization problem for many values of the Lagrange multiplier.

We introduce a general family of Lagrangians that allow exploring the IB curve in all scenarios. When these are used, and the IB curve is known, one can optimize directly for a performance/compression level directly.

The Convex Information Bottleneck Lagrangian


We propose the Neural Logic Machine (NLM), a neural-symbolic architecture for both inductive learning and logic reasoning.

NLMs exploit the power of both neural networks---as function approximators, and logic programming---as a symbolic processor for objects with properties, relations, logic connectives, and quantifiers.

After being trained on small-scale tasks (such as sorting short arrays), NLMs can recover lifted rules, and generalize to large-scale tasks (such as sorting longer arrays).

In our experiments, NLMs achieve perfect generalization in a number of tasks, from relational reasoning tasks on the family tree and general graphs, to decision making tasks including sorting arrays, finding shortest paths, and playing the blocks world.

Most of these tasks are hard to accomplish for neural networks or inductive logic programming alone.

Neural Logic Machines


Sequence-to-sequence (seq2seq) neural models have been actively investigated for abstractive summarization.

Nevertheless, existing neural abstractive systems frequently generate factually incorrect summaries and are vulnerable to adversarial information, suggesting a crucial lack of semantic understanding.

In this paper, we propose a novel semantic-aware neural abstractive summarization model that learns to generate high quality summaries through semantic interpretation over salient content.

A novel evaluation scheme with adversarial samples is introduced to measure how well a model identifies off-topic information, where our model yields significantly better performance than the popular pointer-generator summarizer.

Human evaluation also confirms that our system summaries are uniformly more informative and faithful as well as less redundant than the seq2seq model.

We propose a semantic-aware neural abstractive summarization model and a novel automatic summarization evaluation scheme that measures how well a model identifies off-topic information from adversarial samples.

Robust Neural Abstractive Summarization Systems and Evaluation against Adversarial Information

neural abstractive summarization
adversarial
evaluation
information


The recent work of Super Characters method using two-dimensional word embedding achieved state-of-the-art results in text classification tasks, showcasing the promise of this new approach.

This paper borrows the idea of Super Characters method and two-dimensional embedding, and proposes a method of generating conversational response for open domain dialogues.

The experimental results on a public dataset shows that the proposed SuperChat method generates high quality responses.

An interactive demo is ready to show at the workshop.

And code will be available at github soon.

Print the input sentence and current response sentence onto an image and use fine-tuned ImageNet CNN model to predict the next response word.

SuperChat: Dialogue Generation by Transfer Learning from Vision to Language using Two-dimensional Word Embedding and Pretrained ImageNet CNN Models


Convolutional neural networks (CNNs) have been generally acknowledged as one of the driving forces for the advancement of computer vision.

Despite their promising performances on many tasks, CNNs still face major obstacles on the road to achieving ideal machine intelligence.

One is that CNNs are complex and hard to interpret.

Another is that standard CNNs require large amounts of annotated data, which is sometimes very hard to obtain, and it is desirable to be able to learn them from few examples.

In this work, we address these limitations of CNNs by developing novel, simple, and interpretable models for few-shot learn- ing. Our models are based on the idea of encoding objects in terms of visual concepts, which are interpretable visual cues represented by the feature vectors within CNNs.

We first adapt the learning of visual concepts to the few-shot setting, and then uncover two key properties of feature encoding using visual concepts, which we call category sensitivity and spatial pattern.

Motivated by these properties, we present two intuitive models for the problem of few-shot learning.

Experiments show that our models achieve competitive performances, while being much more flexible and interpretable than alternative state-of-the-art few-shot learning methods.

We conclude that using visual concepts helps expose the natural capability of CNNs for few-shot learning.

We enable ordinary CNNs for few-shot learning by exploiting visual concepts which are interpretable visual cues learnt within CNNs.

Unleashing the Potential of CNNs for Interpretable Few-Shot Learning


Recently various neural networks have been proposed for irregularly structured data such as graphs and manifolds.

To our knowledge, all existing graph networks have discrete depth.

Inspired by neural ordinary differential equation (NODE) for data in the Euclidean domain, we extend the idea of continuous-depth models to graph data, and propose graph ordinary differential equation (GODE).

The derivative of hidden node states are parameterized with a graph neural network, and the output states are the solution to this ordinary differential equation.

We demonstrate two end-to-end methods for efficient training of GODE: (1) indirect back-propagation with the adjoint method; (2) direct back-propagation through the ODE solver, which accurately computes the gradient.

We demonstrate that direct backprop outperforms the adjoint method in experiments.

We then introduce a family of bijective blocks, which enables $\mathcal{O}(1)$ memory consumption.

We demonstrate that GODE can be easily adapted to different existing graph neural networks and improve accuracy.

We validate the performance of GODE in both semi-supervised node classification tasks and graph classification tasks.

Our GODE model achieves a continuous model in time, memory efficiency, accurate gradient estimation, and generalizability with different graph networks.

Apply ordinary differential equation model on graph structured data

Ordinary differential equations on graph networks


Unsupervised image-to-image translation is a recently proposed task of translating an image to a different style or domain given only unpaired image examples at training time.

In this paper, we formulate a new task of unsupervised video-to-video translation, which poses its own unique challenges.

Translating video implies learning not only the appearance of objects and scenes but also realistic motion and transitions between consecutive frames.

We investigate the performance of per-frame video-to-video translation using existing image-to-image translation networks, and propose a spatio-temporal 3D translator as an alternative solution to this problem.

We evaluate our 3D method on multiple synthetic datasets, such as moving colorized digits, as well as the realistic segmentation-to-video GTA dataset and a new CT-to-MRI volumetric images translation dataset.

Our results show that frame-wise translation produces realistic results on a single frame level but underperforms significantly on the scale of the whole video compared to our three-dimensional translation approach, which is better able to learn the complex structure of video and motion and continuity of object appearance.

Proposed new task, datasets and baselines; 3D Conv CycleGAN preserves object properties across frames; batch structure in frame-level methods matters.

Unsupervised Video-to-Video Translation


Capsule networks are constrained by the parameter-expensive nature of their layers, and the general lack of provable equivariance guarantees.

We present a variation of capsule networks that aims to remedy this.

We identify that learning all pair-wise part-whole relationships between capsules of successive layers is inefficient.

Further, we also realise that the choice of prediction networks and the routing mechanism are both key to equivariance.

Based on these, we propose an alternative framework for capsule networks that learns to projectively encode the manifold of pose-variations, termed the space-of-variation (SOV), for every capsule-type of each layer.

This is done using a trainable, equivariant function defined over a grid of group-transformations.

Thus, the prediction-phase of routing involves projection into the SOV of a deeper capsule using the corresponding function.

As a specific instantiation of this idea, and also in order to reap the benefits of increased parameter-sharing, we use type-homogeneous group-equivariant convolutions of shallower capsules in this phase.

We also introduce an equivariant routing mechanism based on degree-centrality.

We show that this particular instance of our general model is equivariant, and hence preserves the compositional representation of an input under transformations.

We conduct several experiments on standard object-classification datasets that showcase the increased transformation-robustness, as well as general performance, of our model to several capsule baselines.

A new scalable, group-equivariant model for capsule networks that preserves compositionality under transformations, and is empirically more transformation-robust to older capsule network models.

Building Deep Equivariant Capsule Networks


Recently deep neural networks have shown their capacity to memorize training data, even with noisy labels, which hurts generalization performance.

To mitigate this issue, we propose a simple but effective method that is robust to noisy labels, even with severe noise.

Our objective involves a variance regularization term that implicitly penalizes the Jacobian norm of the neural network on the whole training set (including the noisy-labeled data), which encourages generalization and prevents overfitting to the corrupted labels.

Experiments on noisy benchmarks demonstrate that our approach achieves state-of-the-art performance with a high tolerance to severe noise.

The paper proposed a simple yet effective baseline for learning with noisy labels.

A Simple yet Effective Baseline for Robust Deep Learning with Noisy Labels

simple yet effective baseline
noisy label


Recent research suggests that neural machine translation achieves parity with professional human translation on the WMT Chinese--English news translation task.

We empirically test this claim with alternative evaluation protocols, contrasting the evaluation of single sentences and entire documents.

In a pairwise ranking experiment, human raters assessing adequacy and fluency show a stronger preference for human over machine translation when evaluating documents as compared to isolated sentences.

Our findings emphasise the need to shift towards document-level evaluation as machine translation improves to the degree that errors which are hard or impossible to spot at the sentence-level become decisive in discriminating quality of different translation outputs.

Raters prefer adequacy in human over machine translation when evaluating entire documents, but not when evaluating single sentences.

Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation


Imitation learning aims to inversely learn a policy from expert demonstrations, which has been extensively studied in the literature for both single-agent setting with Markov decision process (MDP) model, and multi-agent setting with Markov game (MG) model.

However, existing approaches for general multi-agent Markov games are not applicable to multi-agent extensive Markov games, where agents make asynchronous decisions following a certain order, rather than simultaneous decisions.

We propose a novel framework for asynchronous multi-agent generative adversarial imitation learning (AMAGAIL) under general extensive Markov game settings, and the learned expert policies are proven to guarantee subgame perfect equilibrium (SPE), a more general and stronger equilibrium than Nash equilibrium (NE).

The experiment results demonstrate that compared to state-of-the-art baselines, our AMAGAIL model can better infer the policy of each expert agent using their demonstration data collected from asynchronous decision-making scenarios (i.e., extensive Markov games).

This paper extends the multi-agent generative adversarial imitation learning to extensive-form Markov games.

ASYNCHRONOUS MULTI-AGENT GENERATIVE ADVERSARIAL IMITATION LEARNING


Self-training is one of the earliest and simplest semi-supervised methods.

The key idea is to augment the original labeled dataset with unlabeled data paired with the model’s prediction.

Self-training has mostly been well-studied to classification problems.

However, in complex sequence generation tasks such as machine translation, it is still not clear how self-training woks due to the compositionality of the target space.

In this work, we first show that it is not only possible but recommended to apply self-training in sequence generation.

Through careful examination of the performance gains, we find that the noise added on the hidden states (e.g. dropout) is critical to the success of self-training, as this acts like a regularizer which forces the model to yield similar predictions for similar inputs from unlabeled data.

To further encourage this mechanism, we propose to inject noise to the input space, resulting in a “noisy” version of self-training.

Empirical study on standard benchmarks across machine translation and text summarization tasks under different resource settings shows that noisy self-training is able to effectively utilize unlabeled data and improve the baseline performance by large margin.

We revisit self-training as a semi-supervised learning method for neural sequence generation problem, and show that self-training can be quite successful with injected noise.

Revisiting Self-Training for Neural Sequence Generation

neural sequence generation
revisit self-training


We present an end-to-end trained memory system that quickly adapts to new data and generates samples like them.

Inspired by Kanerva's sparse distributed memory, it has a robust  distributed reading and writing mechanism.

The memory is analytically tractable, which enables optimal on-line compression via a Bayesian update-rule.

We formulate it as a hierarchical conditional generative model, where memory provides a rich data-dependent prior distribution.

Consequently, the top-down memory and bottom-up perception are combined to produce the code representing an observation.

Empirically, we demonstrate that the adaptive memory significantly improves generative models trained on both the Omniglot and CIFAR datasets.

Compared with the Differentiable Neural Computer (DNC) and its variants, our memory model has greater capacity and is significantly easier to train.

A generative memory model that combines slow-learning neural networks and a fast-adapting linear Gaussian model as memory.

The Kanerva Machine: A Generative Distributed Memory


Pruning large neural networks while maintaining their performance is often desirable due to the reduced space and time complexity.

In existing methods, pruning is done within an iterative optimization procedure with either heuristically designed pruning schedules or additional hyperparameters, undermining their utility.

In this work, we present a new approach that prunes a given network once at initialization prior to training.

To achieve this, we introduce a saliency criterion based on connection sensitivity that identifies structurally important connections in the network for the given task.

This eliminates the need for both pretraining and the complex pruning schedule while making it robust to architecture variations.

After pruning, the sparse network is trained in the standard way.

Our method obtains extremely sparse networks with virtually the same accuracy as the reference network on the MNIST, CIFAR-10, and Tiny-ImageNet classification tasks and is broadly applicable to various architectures including convolutional, residual and recurrent networks.

Unlike existing methods, our approach enables us to demonstrate that the retained connections are indeed relevant to the given task.

We present a new approach, SNIP, that is simple, versatile and interpretable; it prunes irrelevant connections for a given task at single-shot prior to training and is applicable to a variety of neural network models without modifications.

SNIP: SINGLE-SHOT NETWORK PRUNING BASED ON CONNECTION SENSITIVITY


Transfer learning uses trained weights from a source model as the initial weightsfor the training of a target dataset.

A well chosen source with a large numberof labeled data leads to significant improvement in accuracy.

We demonstrate atechnique that automatically labels large unlabeled datasets so that they can trainsource models for transfer learning.

We experimentally evaluate this method, usinga baseline dataset of human-annotated ImageNet1K labels, against five variationsof this technique.

We show that the performance of these automatically trainedmodels come within 17% of baseline on average.

A technique for automatically labeling large unlabeled datasets so that they can train source models for transfer learning and its experimental evaluation.

Automatic Labeling of Data for Transfer Learning


Recent studies in attention modules have enabled higher performance in computer vision tasks by capturing global contexts and accordingly attending important features.

In this paper, we propose a simple and highly parametrically efficient module named Tree-structured Attention Module (TAM) which recursively encourages neighboring channels to collaborate in order to produce a spatial attention map as an output.

Unlike other attention modules which try to capture long-range dependencies at each channel, our module focuses on imposing non-linearities be- tween channels by utilizing point-wise group convolution.

This module not only strengthens representational power of a model but also acts as a gate which controls signal flow.

Our module allows a model to achieve higher performance in a highly parameter-efficient manner.

We empirically validate the effectiveness of our module with extensive experiments on CIFAR-10/100 and SVHN datasets.

With our proposed attention module employed, ResNet50 and ResNet101 models gain 2.3% and 1.2% accuracy improvement with less than 1.5% parameter over- head.

Our PyTorch implementation code is publicly available.

Our paper proposes an attention module which captures inter-channel relationships and offers large performance gains.

Tree-structured Attention Module for Image Classification


A significant challenge for the practical application of reinforcement learning toreal world problems is the need to specify an oracle reward function that correctly defines a task.

Inverse reinforcement learning (IRL) seeks to avoid this challenge by instead inferring a reward function from expert behavior.

While appealing, it can be impractically expensive to collect datasets of demonstrations that cover the variation common in the real world (e.g. opening any type of door).

Thus in practice, IRL must commonly be performed with only a limited set of demonstrations where it can be exceedingly difficult to unambiguously recover a reward function.

In this work, we exploit the insight that demonstrations from other tasks can be used to constrain the set of possible reward functions by learning a "prior" that is specifically optimized for the ability to infer expressive reward functions from limited numbers of demonstrations.

We demonstrate that our method can efficiently recover rewards from images for novel tasks and provide intuition as to how our approach is analogous to learning a prior.

The applicability of inverse reinforcement learning is often hampered by the expense of collecting expert demonstrations; this paper seeks to broaden its applicability by incorporating prior task information through meta-learning.

Few-Shot Intent Inference via Meta-Inverse Reinforcement Learning


Recent work has focused on combining kernel methods and deep learning.

With this in mind, we introduce Deepström networks -- a new architecture of neural networks which we use to replace top dense layers of standard convolutional architectures with an approximation of a kernel function by relying on the Nyström approximation.

It is compatible with any kernel function and it allows exploiting multiple kernels.

We show that Deepström networks reach state-of-the-art performance on standard datasets like SVHN and CIFAR100.

One benefit of the method lies in its limited number of learnable parameters which make it particularly suited for small training set sizes, e.g. from 5 to 20 samples per class.

Finally we illustrate two ways of using multiple kernels, including a multiple Deepström  setting, that exploits a kernel on each feature map output by the convolutional part of the model.

A new neural architecture where top dense layers of standard convolutional architectures are replaced with an approximation of a kernel function by relying on the Nyström approximation.

Deepström Networks


The main goal of this short paper is to inform the neural art community at large on the ethical ramifications of using models trained on the imagenet dataset, or using seed images from classes 445 -n02892767- [’bikini, two-piece’] and 459- n02837789- [’brassiere, bra, bandeau’] of the same.

We discovered that many of the images belong to these classes were verifiably pornographic, shot in a non-consensual setting, voyeuristic and also entailed underage nudity.

Akin to the \textit{ivory carving-illegal poaching} and \textit{diamond jewelry art-blood diamond} nexuses, we posit there is a similar moral conundrum at play here and would like to instigate a conversation amongst the neural artists in the community.

There's non-consensual and pornographic images in the ImageNet dataset

The blood diamond effect in neural art: On ethically troublesome images of the imagenet dataset


Inductive and unsupervised graph learning is a critical technique for predictive or information retrieval tasks where label information is difficult to obtain.

It is also challenging to make graph learning inductive and unsupervised at the same time, as learning processes guided by reconstruction error based loss functions inevitably demand graph similarity evaluation that is usually computationally intractable.

In this paper, we propose a general framework SEED (Sampling, Encoding, and Embedding Distributions) for inductive and unsupervised representation learning on graph structured objects.

Instead of directly dealing with the computational challenges raised by graph similarity evaluation, given an input graph, the SEED framework samples a number of subgraphs whose reconstruction errors could be efficiently evaluated, encodes the subgraph samples into a collection of subgraph vectors, and employs the embedding of the subgraph vector distribution as the output vector representation for the input graph.

By theoretical analysis, we demonstrate the close connection between SEED and graph isomorphism.

Using public benchmark datasets, our empirical study suggests the proposed SEED framework is able to achieve up to 10% improvement, compared with competitive baseline methods.

This paper proposed a novel framework for graph similarity learning in inductive and unsupervised scenario.

Inductive and Unsupervised Representation Learning on Graph Structured Objects


Neural population responses to sensory stimuli can exhibit both nonlinear stimulus- dependence and richly structured shared variability.

Here, we show how adversarial training can be used to optimize neural encoding models to capture both the deterministic and stochastic components of neural population data.

To account for the discrete nature of neural spike trains, we use the REBAR method to estimate unbiased gradients for adversarial optimization of neural encoding models.

We illustrate our approach on population recordings from primary visual cortex.

We show that adding latent noise-sources to a convolutional neural network yields a model which captures both the stimulus-dependence and noise correlations of the population activity.

We show how neural encoding models can be trained to capture both the signal and spiking variability of neural population data using GANs.

Adversarial Training of Neural Encoding Models on Population Spike Trains

neural encoding model
spike
population
train


A weakly supervised learning based clustering framework is proposed in this paper.

As the core of this framework, we introduce a novel multiple instance learning task based on a bag level label called unique class count (ucc), which is the number of unique classes among all instances inside the bag.

In this task, no annotations on individual instances inside the bag are needed during training of the models.

We mathematically prove that with a perfect ucc classifier, perfect clustering of individual instances inside the bags is possible even when no annotations on individual instances are given during training.

We have constructed a neural network based ucc classifier and experimentally shown that the clustering performance of our framework with our weakly supervised ucc classifier is comparable to that of fully supervised learning models where labels for all instances are known.

Furthermore, we have tested the applicability of our framework to a real world task of semantic segmentation of breast cancer metastases in histological lymph node sections and shown that the performance of our weakly supervised framework is comparable to the performance of a fully supervised Unet model.

A weakly supervised learning based clustering framework performs comparable to that of fully supervised learning models by exploiting unique class count.

Weakly Supervised Clustering by Exploiting Unique Class Count

exploit unique class count
weakly supervise


Model-free reinforcement learning (RL) methods are succeeding in a growing number of tasks, aided by recent advances in deep learning.

However, they tend to suffer from high sample complexity, which hinders their use in real-world domains.

Alternatively, model-based reinforcement learning promises to reduce sample complexity, but tends to require careful tuning and to date have succeeded mainly in restrictive domains where simple models are sufficient for learning.

In this paper, we analyze the behavior of vanilla model-based reinforcement learning methods when deep neural networks are used to learn both the model and the policy, and show that the learned policy tends to exploit regions where insufficient data is available for the model to be learned, causing instability in training.

To overcome this issue, we propose to use an ensemble of models to maintain the model uncertainty and regularize the learning process.

We further show that the use of likelihood ratio derivatives yields much more stable learning than backpropagation through time.

Altogether, our approach Model-Ensemble Trust-Region Policy Optimization (ME-TRPO) significantly reduces the sample complexity compared to model-free deep RL methods on challenging continuous control benchmark tasks.


The high computational and parameter complexity of neural networks makes their training very slow and difficult to deploy on energy and storage-constrained comput- ing systems.

Many network complexity reduction techniques have been proposed including fixed-point implementation.

However, a systematic approach for design- ing full fixed-point training and inference of deep neural networks remains elusive.

We describe a precision assignment methodology for neural network training in which all network parameters, i.e., activations and weights in the feedforward path, gradients and weight accumulators in the feedback path, are assigned close to minimal precision.

The precision assignment is derived analytically and enables tracking the convergence behavior of the full precision training, known to converge a priori.

Thus, our work leads to a systematic methodology of determining suit- able precision for fixed-point training.

The near optimality (minimality) of the resulting precision assignment is validated empirically for four networks on the CIFAR-10, CIFAR-100, and SVHN datasets.

The complexity reduction arising from our approach is compared with other fixed-point neural network designs.

We analyze and determine the precision requirements for training neural networks when all tensors, including back-propagated signals and weight accumulators, are quantized to fixed-point format.

Per-Tensor Fixed-Point Quantization of the Back-Propagation Algorithm


Machine learning (ML) research has investigated prototypes: examples that are representative of the behavior to be learned.

We systematically evaluate five methods for identifying prototypes, both ones previously introduced as well as new ones we propose, finding all of them to provide meaningful but different interpretations.

Through a human study, we confirm that all five metrics are well matched to human intuition.

Examining cases where the metrics disagree offers an informative perspective on the properties of data and algorithms used in learning, with implications for data-corpus construction, efficiency, adversarial robustness, interpretability, and other ML aspects.

In particular, we confirm that the "train on hard" curriculum approach can improve accuracy on many datasets and tasks, but that it is strictly worse when there are many mislabeled or ambiguous examples.

We can identify prototypical and outlier examples in machine learning that are quantifiably very different, and make use of them to improve many aspects of neural networks.

Prototypical Examples in Deep Learning: Metrics, Characteristics, and Utility


In this work, we propose the Sparse Deep Scattering Croisé Network (SDCSN) a novel architecture based on the Deep Scattering Network (DSN).

The DSN is achieved by cascading  wavelet transform convolutions with a complex modulus and a time-invariant operator.

crossing multiple wavelet family transforms to increase the feature diversity while avoiding any learning.

Thus providing a more informative latent representation and benefit from the development of highly specialized wavelet filters over the last decades.

Beside, by combining all the different wavelet representations, we reduce the amount of prior information needed regarding the signals at hand.

Secondly, we develop an optimal thresholding strategy for over-complete filter banks that regularizes the network and controls instabilities such as inherent non-stationary noise in the signal.

Our systematic and principled solution sparsifies the latent representation of the network by acting as a local mask distinguishing between activity and noise.

Thus, we propose to enhance the DSN by increasing the variance of the scattering coefficients representation as well as improve its robustness with respect to non-stationary noise.

We show that our new approach is more robust and outperforms the DSN on a bird detection task.

We propose to enhance the Deep Scattering Network  in order to improve control and stability of any given machine learning pipeline by proposing a continuous wavelet thresholding scheme

Sparse Deep Scattering Croisé Network


We propose a neural clustering model that jointly learns both latent features and how they cluster.

Unlike similar methods our model does not require a predefined number of clusters.

Using a supervised approach, we agglomerate latent features towards randomly sampled targets within the same space whilst progressively removing the targets until we are left with only targets which represent cluster centroids.

To show the behavior of our model across different modalities we apply our model on both text and image data and very competitive results on MNIST.

Finally, we also provide results against baseline models for fashion-MNIST, the 20 newsgroups dataset, and a Twitter dataset we ourselves create.

Neural clustering without needing a number of clusters

Neural Clustering By Predicting And Copying Noise


Recent work on explanation generation for decision-making problems has viewed the explanation process as one of model reconciliation where an AI agent brings the human mental model (of its capabilities, beliefs, and goals) to the same page with regards to a task at hand.

This formulation succinctly captures many possible types of explanations, as well as explicitly addresses the various properties -- e.g. the social aspects, contrastiveness, and selectiveness -- of explanations studied in social sciences among human-human interactions.

However, it turns out that the same process can be hijacked into producing "alternative explanations" -- i.e. explanations that are not true but still satisfy all the properties of a proper explanation.

In previous work, we have looked at how such explanations may be perceived by the human in the loop and alluded to one possible way of generating them.

In this paper, we go into more details of this curious feature of the model reconciliation process and discuss similar implications to the overall notion of explainable decision-making.

Model Reconciliation is an established framework for plan explanations, but can be easily hijacked to produce lies.

(How) Can AI Bots Lie?


We consider new variants of optimization algorithms.

Our algorithms are based on the observation that mini-batch of stochastic gradients in consecutive iterations do not change drastically and consequently may be predictable.

Inspired by the similar setting in online learning literature called Optimistic Online learning, we propose two new optimistic algorithms for AMSGrad and Adam, respectively, by  exploiting the predictability of gradients.

The new algorithms combine the idea of momentum method, adaptive gradient method, and algorithms in Optimistic Online learning, which leads to speed up in training deep neural nets in practice.

We consider new variants of optimization algorithms for training deep nets.

Optimistic Acceleration for Optimization


Using Recurrent Neural Networks (RNNs) in sequence modeling tasks is promising in delivering high-quality results but challenging to meet stringent latency requirements because of the memory-bound execution pattern of RNNs.

We propose a big-little dual-module inference to dynamically skip unnecessary memory access and computation to speedup RNN inference.

Leveraging the error-resilient feature of nonlinear activation functions used in RNNs, we propose to use a lightweight little module that approximates the original RNN layer, which is referred to as the big module, to compute activations of the insensitive region that are more error-resilient.

The expensive memory access and computation of the big module can be reduced as the results are only used in the sensitive region.

Our method can reduce the overall memory access by 40% on average and achieve 1.54x to 1.75x speedup on CPU-based server platform with negligible impact on model quality.

We accelerate RNN inference by dynamically reducing redundant memory access using a mixture of accurate and approximate modules.

Dual-module Inference for Efficient Recurrent Neural Networks


Fine-grained Entity Recognition (FgER) is the task of detecting and classifying entity mentions to a large set of types spanning diverse domains such as biomedical, finance and sports.

We  observe  that  when  the  type  set  spans  several  domains,  detection  of  entity mention becomes a limitation for supervised learning models.

The primary reason being lack  of  dataset  where  entity  boundaries  are  properly  annotated  while  covering  a  large spectrum of entity types.

We propose Heuristics Allied with Distant Supervision (HAnDS) framework to automatically construct a quality dataset suitable for the FgER task.

HAnDS framework exploits the high interlink among Wikipedia  and  Freebase  in  a  pipelined  manner,  reducing  annotation  errors  introduced by naively using distant supervision approach.

Using HAnDS framework,  we create two datasets, one suitable for building FgER systems recognizing up to 118 entity types based on the FIGER type hierarchy and another for up to 1115 entity types based on the TypeNet hierarchy.

Our extensive empirical experimentation warrants the quality of the generated datasets.

Along with this, we also provide a manually annotated dataset for benchmarking FgER systems.

We initiate a push towards building ER systems to recognize thousands of types by providing a method to automatically construct suitable datasets based on the type hierarchy.

Fine-grained Entity Recognition with Reduced False Negatives and Large Type Coverage


Implementing correct method invocation is an important task for software developers.

However, this is challenging work, since the structure of method invocation can be complicated.

In this paper, we propose InvocMap, a code completion tool allows developers to obtain an implementation of multiple method invocations from a list of method names inside code context.

InvocMap is able to predict the nested method invocations which their names didn’t appear in the list of input method names given by developers.

To achieve this, we analyze the Method Invocations by four levels of abstraction.

We build a Machine Translation engine to learn the mapping from the first level to the third level of abstraction of multiple method invocations, which only requires developers to manually add local variables from generated expression to get the final code.

We evaluate our proposed approach on six popular libraries: JDK, Android, GWT, Joda-Time, Hibernate, and Xstream.

With the training corpus of 2.86 million method invocations extracted from 1000 Java Github projects and the testing corpus extracted from 120 online forums code snippets, InvocMap achieves the accuracy rate up to 84 in F1- score depending on how much information of context provided along with method names, that shows its potential for auto code completion.

This paper proposes a theory of classifying Method Invocations by different abstraction levels and conducting a statistical approach for code completion from method name to method invocation.

INVOCMAP: MAPPING METHOD NAMES TO METHOD INVOCATIONS VIA MACHINE LEARNING


Adversaries in neural networks have drawn much attention since their first debut.

While most existing methods aim at deceiving image classification models into misclassification or crafting attacks for specific object instances in the object setection tasks, we focus on creating universal adversaries to fool object detectors and hide objects from the detectors.

The adversaries we examine are universal in three ways:

(1) They are not specific for specific object instances;

(3) They can further transfer to different unknown models.

To achieve this, we propose two novel techniques to improve the transferability of the adversaries: \textit{piling-up} and \textit{monochromatization}.

Both techniques prove to simplify the patterns of generated adversaries, and ultimately result in higher transferability.

We focus on creating universal adversaries to fool object detectors and hide objects from the detectors.

Hiding Objects from Detectors: Exploring Transferrable Adversarial Patterns


This work presents the Poincaré Wasserstein Autoencoder, a reformulation of

the recently proposed Wasserstein autoencoder framework on a non-Euclidean

manifold, the Poincaré ball model of the hyperbolic space H n .

latent space to be hyperbolic, we can use its intrinsic hierarchy to impose structure

on the learned latent space representations.

hierarchies, we can recover the structure in a low-dimensional latent space.

also demonstrate the model in the visual domain to analyze some of its properties

and show competitive results on a graph link prediction task.

Wasserstein Autoencoder with hyperbolic latent space

Poincaré Wasserstein Autoencoder


In this paper, we introduce a method to compress intermediate feature maps of deep neural networks (DNNs) to decrease memory storage and bandwidth requirements during inference.

Unlike previous works, the proposed method is based on converting fixed-point activations into vectors over the smallest GF(2) finite field followed by nonlinear dimensionality reduction (NDR) layers embedded into a DNN.

Such an end-to-end learned representation finds more compact feature maps by exploiting quantization redundancies within the fixed-point activations along the channel or spatial dimensions.

We apply the proposed network architecture to the tasks of ImageNet classification and PASCAL VOC object detection.

Compared to prior approaches, the conducted experiments show a factor of 2 decrease in memory requirements with minor degradation in accuracy while adding only bitwise computations.

Feature map compression method that converts quantized activations into binary vectors followed by nonlinear dimensionality reduction layers embedded into a DNN

DNN Feature Map Compression using Learned Representation over GF(2)


Adversarial training is one of the strongest defenses against adversarial attacks, but it requires adversarial examples to be generated for every mini-batch during optimization.

The expense of producing these examples during training often precludes adversarial training from use on complex image datasets.

In this study, we explore the mechanisms by which adversarial training improves classifier robustness, and show that these mechanisms can be effectively mimicked using simple regularization methods, including label smoothing and logit squeezing.

Remarkably, using these simple regularization methods in combination with Gaussian noise injection, we are able to achieve strong adversarial robustness -- often exceeding that of adversarial training -- using no adversarial examples.

Achieving strong adversarial robustness comparable to adversarial training without training on adversarial examples

Label Smoothing and Logit Squeezing: A Replacement for Adversarial Training?


A major goal of unsupervised learning is to discover data representations that are useful for subsequent tasks, without access to supervised labels during training.

Typically, this involves minimizing a surrogate objective, such as the negative log likelihood of a generative model, with the hope that representations useful for subsequent tasks will arise as a side effect.

In this work, we propose instead to directly target later desired tasks by meta-learning an unsupervised learning rule which leads to representations useful for those tasks.

Specifically, we target semi-supervised classification performance, and we meta-learn an algorithm -- an unsupervised weight update rule -- that produces representations useful for this task.

Additionally, we constrain our unsupervised update rule to a be a biologically-motivated, neuron-local function, which enables it to generalize to different neural network architectures, datasets, and data modalities.

We show that the meta-learned update rule produces useful features and sometimes outperforms existing unsupervised learning techniques.

We further show that the meta-learned unsupervised update rule generalizes to train networks with different widths, depths, and nonlinearities.

It also generalizes to train on data with randomly permuted input dimensions and even generalizes from image datasets to a text task.

We learn an unsupervised learning algorithm that produces useful representations from a set of supervised tasks. At test-time, we apply this algorithm to new tasks without any supervision and show performance comparable to a VAE.

Meta-Learning Update Rules for Unsupervised Representation Learning


There is significant recent evidence in supervised learning that, in the over-parametrized setting, wider networks achieve better test error.

In other words, the bias-variance tradeoff is not directly observable when increasing network width arbitrarily.

We investigate whether a corresponding phenomenon is present in reinforcement learning.

We experiment on four OpenAI Gym environments, increasing the width of the value and policy networks beyond their prescribed values.

Our empirical results lend support to this hypothesis.

However, tuning the hyperparameters of each network width separately remains as important future work in environments/algorithms where the optimal hyperparameters vary noticably across widths, confounding the results when the same hyperparameters are used for all widths.

Over-parametrization in width seems to help in deep reinforcement learning, just as it does in supervised learning.

In Support of Over-Parametrization in Deep Reinforcement Learning: an Empirical Study

deep reinforcement learning
over-parametrization


Learning disentangled representations of  data is one of the central themes in unsupervised learning in general and generative modelling in particular.

In this work,  we tackle a slightly more intricate scenario where the observations are generated from a conditional distribution of some known control variate and some latent noise variate.

To this end, we present a hierarchical model and a training method (CZ-GEM) that leverages some of the recent developments in likelihood-based and likelihood-free generative models.

We show that by formulation, CZ-GEM introduces the right inductive biases that ensure the disentanglement of the control from the noise variables, while also keeping the components of the control variate disentangled.

This is achieved without compromising on the quality of the generated samples.

Our approach is simple, general, and can be applied both in supervised and unsupervised settings.

Hierarchical generative model (hybrid of VAE and GAN) that learns a disentangled representation of data without compromising the generative quality.

CZ-GEM:  A  FRAMEWORK  FOR DISENTANGLED REPRESENTATION LEARNING


Deep learning has yielded state-of-the-art performance on many natural language processing tasks including named entity recognition (NER).

However, this typically requires large amounts of labeled data.

In this work, we demonstrate that the amount of labeled training data can be drastically reduced when deep learning is combined with active learning.

While active learning is sample-efficient, it can be computationally expensive since it requires iterative retraining.

To speed this up, we introduce a lightweight architecture for NER, viz., the CNN-CNN-LSTM model consisting of convolutional character and word encoders and a  long short term memory (LSTM) tag decoder.

The model achieves nearly state-of-the-art performance on standard datasets for the task while being computationally much more efficient than best performing models.

We carry out incremental active learning, during the training process, and are able to nearly match state-of-the-art performance with just 25\% of the original training data.

We introduce a lightweight architecture for named entity recognition and carry out incremental active learning, which is able to match state-of-the-art performance with just 25% of the original training data.

Deep Active Learning for Named Entity Recognition


Network quantization is a model compression and acceleration technique that has become essential to neural network deployment.

Most quantization methods per- form fine-tuning on a pretrained network, but this sometimes results in a large loss in accuracy compared to the original network.

We introduce a new technique to train quantization-friendly networks, which can be directly converted to an accurate quantized network without the need for additional fine-tuning.

Our technique allows quantizing the weights and activations of all network layers down to 4 bits, achieving high efficiency and facilitating deployment in practical settings.

Com- pared to other fully quantized networks operating at 4 bits, we show substantial improvements in accuracy, for example 66.68% top-1 accuracy on ImageNet using ResNet-18, compared to the previous state-of-the-art accuracy of 61.52% Louizos et al. (2019) and a full precision reference accuracy of 69.76%.

We performed a thorough set of experiments to test the efficacy of our method and also conducted ablation studies on different aspects of the method and techniques to improve training stability and accuracy.

Our codebase and trained models are available on GitHub.

We train accurate fully quantized networks using a loss function maximizing full precision model accuracy and minimizing the difference between the full precision and quantized networks.

GQ-Net: Training Quantization-Friendly Deep Networks


While much of the work in the design of convolutional networks over the last five years has revolved around the empirical investigation of the importance of depth, filter sizes, and number of feature channels, recent studies have shown that  branching, i.e., splitting the computation along parallel but distinct threads and then aggregating their outputs, represents a new promising dimension for significant improvements in performance.

To combat the complexity of design choices in multi-branch architectures, prior work has adopted simple strategies, such as a fixed branching factor, the same input being fed to all parallel branches, and an additive combination of the outputs produced by all branches at aggregation points.

In this work we remove these predefined choices and propose an algorithm to learn the connections between branches in the network.

Instead of being chosen a priori by the human designer, the multi-branch connectivity is learned simultaneously with the weights of the network by optimizing a single loss function defined with respect to the end task.

We demonstrate our approach on the problem of multi-class image classification using four different datasets where it yields consistently higher accuracy compared to the state-of-the-art ``ResNeXt'' multi-branch network given the same learning capacity.

In this paper we introduced an algorithm to learn the connectivity of deep multi-branch networks. The approach is evaluated on image categorization where it consistently yields accuracy gains over state-of-the-art models that use fixed connectivity.

Connectivity Learning in Multi-Branch Networks


Although deep convolutional networks have achieved improved performance in many natural language tasks, they have been treated as black boxes because they are difficult to interpret.

Especially, little is known about how they represent language in their intermediate layers.

In an attempt to understand the representations of deep convolutional networks trained on language tasks, we show that individual units are selectively responsive to specific morphemes, words, and phrases, rather than responding to arbitrary and uninterpretable patterns.

In order to quantitatively analyze such intriguing phenomenon, we propose a concept alignment method based on how units respond to replicated text.

We conduct analyses with different architectures on multiple datasets for classification and translation tasks and provide new insights into how deep models understand natural language.

We show that individual units in CNN representations learned in NLP tasks are selectively responsive to natural language concepts.

Discovery of Natural Language Concepts in Individual Units of CNNs

natural language concept
individual unit cnn


Applying reinforcement learning (RL) to real-world problems will require reasoning about action-reward correlation over long time horizons.

Hierarchical reinforcement learning (HRL) methods handle this by dividing the task into hierarchies, often with hand-tuned network structure or pre-defined subgoals.

We propose a novel HRL framework TAIC, which learns the temporal abstraction from past experience or expert demonstrations without task-specific knowledge.

We formulate the temporal abstraction problem as learning latent representations of action sequences and present a novel approach of regularizing the latent space by adding information-theoretic constraints.

Specifically, we maximize the mutual information between the latent variables and the state changes.

A visualization of the latent space demonstrates that our algorithm learns an effective abstraction of the long action sequences.

The learned abstraction allows us to learn new tasks on higher level more efficiently.

We convey a significant speedup in convergence over benchmark learning problems.

These results demonstrate that learning temporal abstractions is an effective technique in increasing the convergence rate and sample efficiency of RL algorithms.

We propose a novel HRL framework, in which we formulate the temporal abstraction problem as learning a latent representation  of  action  sequence.

Learning Temporal Abstraction with Information-theoretic Constraints for Hierarchical Reinforcement Learning


Recurrent neural networks (RNNs) have achieved state-of-the-art performance on many diverse tasks, from machine translation to surgical activity recognition, yet training RNNs to capture long-term dependencies remains difficult.

To date, the vast majority of successful RNN architectures alleviate this problem using nearly-additive connections between states, as introduced by long short-term memory (LSTM).

We take an orthogonal approach and introduce MIST RNNs, a NARX RNN architecture that allows direct connections from the very distant past.

1) exhibit superior vanishing-gradient properties in comparison to LSTM and previously-proposed NARX RNNs;

2) are far more efficient than previously-proposed NARX RNN architectures, requiring even fewer computations than LSTM; and

3) improve performance substantially over LSTM and Clockwork RNNs on tasks requiring very long-term dependencies.

We introduce MIST RNNs, which a) exhibit superior vanishing-gradient properties in comparison to LSTM; b) improve performance substantially over LSTM and Clockwork RNNs on tasks requiring very long-term dependencies; and c) are much more efficient than previously-proposed NARX RNNs, with even fewer parameters and operations than LSTM.

Analyzing and Exploiting NARX Recurrent Neural Networks for Long-Term Dependencies


A well-trained model should classify objects with unanimous score for every category.

This requires the high-level semantic features should be alike among samples, despite a wide span in resolution, texture, deformation, etc.

Previous works focus on re-designing the loss function or proposing new regularization constraints on the loss.

In this paper, we address this problem via a new perspective.

For each category, it is assumed that there are two sets in the feature space: one with more reliable information and the other with less reliable source.

We argue that the reliable set could guide the feature learning of the less reliable set during training - in spirit of student mimicking teacher’s behavior and thus pushing towards a more compact class centroid in the high-dimensional space.

Such a scheme also benefits the reliable set since samples become more closer within the same category - implying that it is easilier for the classifier to identify.

We refer to this mutual learning process as feature intertwiner and embed the spirit into object detection.

It is well-known that objects of low resolution are more difficult to detect due to the loss of detailed information during network forward pass.

We thus regard objects of high resolution as the reliable set and objects of low resolution as the less reliable set.

Specifically, an intertwiner is achieved by minimizing the distribution divergence between two sets.

We design a historical buffer to represent all previous samples in the reliable set and utilize them to guide the feature learning of the less reliable set.

The design of obtaining an effective feature representation for the reliable set is further investigated, where we introduce the optimal transport (OT) algorithm into the framework.

Samples in the less reliable set are better aligned with the reliable set with aid of OT metric.

Incorporated with such a plug-and-play intertwiner, we achieve an evident improvement over previous state-of-the-arts on the COCO object detection benchmark.

(Camera-ready version) A feature intertwiner module to leverage features from one accurate set to help the learning of another less reliable set.

Feature Intertwiner for Object Detection


Approaches to continual learning aim to successfully learn a set of related tasks that arrive in an online manner.

Recently, several frameworks have been developed which enable deep learning to be deployed in this learning scenario.

A key modelling decision is to what extent the architecture should be shared across tasks.

On the one hand, separately modelling each task avoids catastrophic forgetting but it does not support transfer learning and leads to large models.

On the other hand, rigidly specifying a shared component and a task-specific part enables task transfer and limits the model size, but it is vulnerable to catastrophic forgetting and restricts the form of task-transfer that can occur.

Ideally, the network should adaptively identify which parts of the network to share in a data driven way.

Here we introduce such an approach called Continual Learning with Adaptive Weights (CLAW), which is based on probabilistic modelling and variational inference.

Experiments show that CLAW achieves state-of-the-art performance on six benchmarks in terms of overall continual learning performance, as measured by classification accuracy, and in terms of addressing catastrophic forgetting.

A continual learning framework which learns to automatically adapt its architecture based on a proposed variational inference algorithm.


High-dimensional data often lie in or close to low-dimensional subspaces.

Sparse subspace clustering methods with sparsity induced by L0-norm, such as L0-Sparse Subspace Clustering (L0-SSC), are demonstrated to be more effective than its L1 counterpart such as Sparse Subspace Clustering (SSC).

However, these L0-norm based subspace clustering methods are restricted to clean data that lie exactly in subspaces.

Real data often suffer from noise and they may lie close to subspaces.

We propose noisy L0-SSC to handle noisy data so as to improve the robustness.

We show that the optimal solution to the optimization problem of noisy L0-SSC achieves subspace detection property (SDP), a key element with which data from different subspaces are separated, under deterministic and randomized models.

Our results provide theoretical guarantee on the correctness of noisy L0-SSC in terms of SDP on noisy data.

We further propose Noisy-DR-L0-SSC which provably recovers the subspaces on dimensionality reduced data.

Noisy-DR-L0-SSC first projects the data onto a lower dimensional space by linear transformation, then performs noisy L0-SSC on the dimensionality reduced data so as to improve the efficiency.

The experimental results demonstrate the effectiveness of noisy L0-SSC and Noisy-DR-L0-SSC.

We propose Noisy-DR-L0-SSC (Noisy Dimension Reduction L0-Sparse Subspace Clustering) to efficiently partition noisy data in accordance to their underlying subspace structure.

Noisy $\ell^{0}$-Sparse Subspace Clustering on Dimensionality Reduced Data


Mode connectivity provides novel geometric insights on analyzing loss landscapes and enables building high-accuracy pathways between well-trained neural networks.

In this work, we propose to employ mode connectivity in loss landscapes to study the adversarial robustness of deep neural networks, and provide novel methods for improving this robustness.

Our experiments cover various types of adversarial attacks applied to different network architectures and datasets.

When network models are tampered with backdoor or error-injection attacks, our results demonstrate that the path connection learned using limited amount of bonafide data can effectively mitigate adversarial effects while maintaining the original accuracy on clean data.

Therefore, mode connectivity provides users with the power to repair backdoored or error-injected models.

We also use mode connectivity to investigate the loss landscapes of regular and robust models against evasion attacks.

Experiments show that there exists a barrier in adversarial robustness loss on the path connecting regular and adversarially-trained models.

A high correlation is observed between the adversarial robustness loss and the largest eigenvalue of the input Hessian matrix, for which theoretical justifications are provided.

Our results suggest that mode connectivity offers a holistic tool and practical means for evaluating and improving adversarial robustness.

A novel approach using mode connectivity in loss landscapes to mitigate adversarial effects, repair tampered models and evaluate adversarial robustness

Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness

mode connectivity loss landscape
adversarial robustness


Generative adversarial networks (GANs) learn to map samples from a noise distribution to a chosen data distribution.

Recent work has demonstrated that GANs are consequently sensitive to, and limited by, the shape of the noise distribution.

For example, a single generator struggles to map continuous noise (e.g. a uniform distribution) to discontinuous output (e.g. separate Gaussians) or complex output (e.g. intersecting parabolas).

We address this problem by learning to generate from multiple models such that the generator's output is actually the combination of several distinct networks.

We contribute a novel formulation of multi-generator models where we learn a prior over the generators conditioned on the noise, parameterized by a neural network.

Thus, this network not only learns the optimal rate to sample from each generator but also optimally shapes the noise received by each generator.

The resulting Noise Prior GAN (NPGAN) achieves expressivity and flexibility that surpasses both single generator models and previous multi-generator models.

A multi-generator GAN framework with an additional network to learn a prior over the input noise.

Improving Multi-Manifold GANs with a Learned Noise Prior


Recent advances in Generative Adversarial Networks (GANs) – in architectural design,  training strategies,  and empirical tricks – have led nearly photorealistic samples on large-scale datasets such as ImageNet.

In fact, for one model in particular, BigGAN, metrics such as Inception Score or Frechet Inception Distance nearly match those of the dataset, suggesting that these models are close to match-ing the distribution of the training set.

Given the quality of these models,  it is worth understanding to what extent these samples can be used for data augmentation, a task expressed as a long-term goal of the GAN research project.

To that end, we train ResNet-50 classifiers using either purely BigGAN images or mixtures of ImageNet and BigGAN images, and test on the ImageNet validation set.

Our  preliminary  results  suggest  both a measured view of  state-of-the-art  GAN quality and highlight limitations of current metrics.

Using only BigGAN images, we find that Top-1 and Top-5 error increased by 120% and 384%, respectively, and furthermore, adding more BigGAN data to the ImageNet training set at best only marginally improves classifier performance.

Finally, we find that neither Inception Score, nor FID, nor combinations thereof are predictive of classification accuracy.

These results suggest that as GANs are beginning to be deployed in downstream tasks, we should create metrics that better measure downstream task performance.

We propose classification performance as one such metric that, in addition to assessing per-class sample quality, is more suited to such downstream tasks.

BigGANs do not capture the ImageNet data distributions and are only modestly successful for data augmentation.

Seeing is Not Necessarily Believing: Limitations of BigGANs for Data Augmentation


Modern federated networks, such as those comprised of wearable devices, mobile phones, or autonomous vehicles, generate massive amounts of data each day.

This wealth of data can help to learn models that can improve the user experience on each device.

However, the scale and heterogeneity of federated data presents new challenges in research areas such as federated learning, meta-learning, and multi-task learning.

As the machine learning community begins to tackle these challenges, we are at a critical time to ensure that developments made in these areas are grounded with realistic benchmarks.

To this end, we propose Leaf, a modular benchmarking framework for learning in federated settings.

Leaf includes a suite of open-source federated datasets, a rigorous evaluation framework, and a set of reference implementations, all geared towards capturing the obstacles and intricacies of practical federated environments.

We present Leaf, a modular benchmarking framework for learning in federated data, with applications to learning paradigms such as federated learning, meta-learning, and multi-task learning.

LEAF: A Benchmark for Federated Settings


Understanding object motion is one of the core problems in computer vision.

It requires segmenting and tracking objects over time.

Significant progress has been made in instance segmentation, but such models cannot track objects, and more crucially, they are unable to reason in both 3D space and time.

We propose a new spatio-temporal embedding loss on videos that generates temporally consistent video instance segmentation.

Our model includes a temporal network that learns to model temporal context and motion, which is essential to produce smooth embeddings over time.

Further, our model also estimates monocular depth, with a self-supervised loss, as the relative distance to an object effectively constrains where it can be next, ensuring a time-consistent embedding.

Finally, we show that our model can accurately track and segment instances, even with occlusions and missed detections, advancing the state-of-the-art on the KITTI Multi-Object and Tracking Dataset.

We introduce a new spatio-temporal embedding loss on videos that generates temporally consistent video instance segmentation, even with occlusions and missed detections, using appearance, geometry, and temporal context.

Learning a Spatio-Temporal Embedding for Video Instance Segmentation

video instance segmentation
spatio-temporal embed


As reinforcement learning continues to drive machine intelligence beyond its conventional boundary, unsubstantial practices in sparse reward environment severely limit further applications in a broader range of advanced fields.

Motivated by the demand for an effective deep reinforcement learning algorithm that accommodates sparse reward environment, this paper presents Hindsight Trust Region Policy Optimization (HTRPO), a method that efficiently utilizes interactions in sparse reward conditions to optimize policies within trust region and, in the meantime, maintains learning stability.

Firstly, we theoretically adapt the TRPO objective function, in the form of the expected return of the policy, to the distribution of hindsight data generated from the alternative goals.

Then, we apply Monte Carlo with importance sampling to estimate KL-divergence between two policies, taking the hindsight data as input.

Under the condition that the distributions are sufficiently close, the KL-divergence is approximated by another f-divergence.

Such approximation results in the decrease of variance and alleviates the instability during policy update.

Experimental results on both discrete and continuous benchmark tasks demonstrate that HTRPO converges significantly faster than previous policy gradient methods.

It achieves effective performances and high data-efficiency for training policies in sparse reward environments.

This paper proposes an advanced policy optimization method with hindsight experience for sparse reward reinforcement learning.


Open-domain question answering (QA) is an important problem in AI and NLP that is emerging as a bellwether for progress on the generalizability of AI methods and techniques.

Much of the progress in open-domain QA systems has been realized through advances in information retrieval methods and corpus construction.

In this paper, we focus on the recently introduced ARC Challenge dataset, which contains 2,590 multiple choice questions authored for grade-school science exams.

These questions are selected to be the most challenging for current QA systems, and current state of the art performance is only slightly better than random chance.

We present a system that reformulates a given question into queries that are used to retrieve supporting text from a large corpus of science-related text.

Our rewriter is able to incorporate background knowledge from ConceptNet and -- in tandem with a generic textual entailment system trained on SciTail that identifies support in the retrieved results -- outperforms several strong baselines on the end-to-end QA task despite only being trained to identify essential terms in the original source question.

We use a generalizable decision methodology over the retrieved evidence and answer candidates to select the best answer.

By combining query reformulation, background knowledge, and textual entailment our system is able to outperform several strong baselines on the ARC dataset.

We explore how using background knowledge with query reformulation can help retrieve better supporting evidence when answering multiple-choice science questions.

Answering Science Exam Questions Using Query Reformulation with Background Knowledge

query reformulation
background knowledge
question
science
answer


Deep CNNs have achieved state-of-the-art performance for numerous machine learning and computer vision tasks in recent years, but as they have become increasingly deep, the number of parameters they use has also increased, making them hard to deploy in memory-constrained environments and difficult to interpret.

Machine learning theory implies that such networks are highly over-parameterised and that it should be possible to reduce their size without sacrificing accuracy, and indeed many recent studies have begun to highlight specific redundancies that can be exploited to achieve this.

In this paper, we take a further step in this direction by proposing a filter-sharing approach to compressing deep CNNs that reduces their memory footprint by repeatedly applying a single convolutional mapping of learned filters to simulate a CNN pipeline.

We show, via experiments on CIFAR-10, CIFAR-100, Tiny ImageNet, and ImageNet that this allows us to reduce the parameter counts of networks based on common designs such as VGGNet and ResNet by a factor proportional to their depth, whilst leaving their accuracy largely unaffected.

At a broader level, our approach also indicates how the scale-space regularities found in visual signals can be leveraged to build neural architectures that are more parsimonious and interpretable.

We compress deep CNNs by reusing a single convolutional layer in an iterative manner, thereby reducing their parameter counts by a factor proportional to their depth, whilst leaving their accuracies largely unaffected

ShardNet: One Filter Set to Rule Them All


We extend the recent results of (Arora et al., 2019) by a spectral analysis of representations corresponding to kernel and neural embeddings.

They showed that in a simple single layer network, the alignment of the labels to the eigenvectors of the corresponding Gram matrix determines both the convergence of the optimization during training as well as the generalization properties.

We generalize their result to kernel and neural representations and show that these extensions improve both optimization and generalization of the basic setup studied in (Arora et al., 2019).

Spectral analysis for understanding how different representations can improve optimization and generalization.

Spectral Analysis of Kernel and Neural Embeddings:  Optimization and Generalization

optimization generalization
spectral analysis


Obtaining high-quality uncertainty estimates is essential for many applications of deep neural networks.

In this paper, we theoretically justify a scheme for estimating uncertainties, based on sampling from a prior distribution.

Crucially, the uncertainty estimates are shown to be conservative in the sense that they never underestimate a posterior uncertainty obtained by a hypothetical Bayesian algorithm.

We also show concentration, implying that the uncertainty estimates converge to zero as we get more data.

Uncertainty estimates obtained from random priors can be adapted to any deep network architecture and trained using standard supervised learning pipelines.

We provide experimental evaluation of random priors on calibration and out-of-distribution detection on typical computer vision tasks, demonstrating that they outperform deep ensembles in practice.

We provide theoretical support to uncertainty estimates for deep learning obtained fitting random priors.

Conservative Uncertainty Estimation By Fitting  Prior Networks


In this paper, we propose an arbitrarily-conditioned data imputation framework built upon variational autoencoders and normalizing flows.

The proposed model is capable of mapping any partial data to a multi-modal latent variational distribution.

Sampling from such a distribution leads to stochastic imputation.

Preliminary evaluation on MNIST dataset shows promising stochastic imputation conditioned on partial images as input.

We propose an arbitrarily-conditioned data imputation framework built upon variational autoencoders and normalizing flows

Arbitrarily-conditioned Data Imputation


This paper studies \emph{model inversion attacks}, in which the access to a model is abused to infer information about the training data.

Since its first introduction by~\citet{fredrikson2014privacy}, such attacks have raised serious concerns given that training data usually contain sensitive information.

Thus far, successful model inversion attacks have only been demonstrated on simple models, such as linear regression and logistic regression.

Previous attempts to invert neural networks, even the ones with simple architectures, have failed to produce convincing results.

We present a novel attack method, termed the \emph{generative model inversion attack}, which can invert deep neural networks with high success rates.

Rather than reconstructing private training data from scratch, we leverage partial public information, which can be very generic, to learn a distributional prior via generative adversarial networks (GANs) and use it to guide the inversion process.

Moreover, we theoretically prove that a model's predictive power and its vulnerability to inversion attacks are indeed two sides of the same coin---highly predictive models are able to establish a strong correlation between features and labels, which coincides exactly with what an adversary exploits to mount the attacks.

Our experiments demonstrate that the proposed attack improves identification accuracy over the existing work by about $75\%$ for reconstructing face images from a state-of-the-art face recognition classifier.

We also show that differential privacy, in its canonical form, is of little avail to protect against our attacks.

We develop a privacy attack that can recover the sensitive input data of a deep net from its output

The Secret Revealer: Generative Model Inversion Attacks Against Deep Neural Networks


Latent space based GAN methods and attention based encoder-decoder architectures have achieved impressive results in text generation and Unsupervised NMT respectively.

Leveraging the two domains, we propose an adversarial latent space based architecture capable of generating parallel sentences in two languages concurrently and translating bidirectionally.

The bilingual generation goal is achieved by sampling from the latent space that is adversarially constrained to be shared between both languages.

First an NMT model is trained, with back-translation and an adversarial setup, to enforce a latent state between the two languages.

The encoder and decoder are shared for the two translation directions.

Next, a GAN is trained to generate ‘synthetic’ code mimicking the languages’ shared latent space.

This code is then fed into the decoder to generate text in either language.

We perform our experiments on Europarl and Multi30k datasets, on the English-French language pair, and document our performance using both Supervised and Unsupervised NMT.

We present a novel method for Bilingual Text Generation producing parallel concurrent sentences in two languages.

Bilingual-GAN: Neural Text Generation and Neural Machine Translation as Two Sides of the Same Coin


As Artificial Intelligence (AI) becomes an integral part of our life, the development of explainable AI, embodied in the decision-making process of an AI or robotic agent, becomes imperative.

For a robotic teammate, the ability to generate explanations to explain its behavior is one of the key requirements of an explainable agency.

Prior work on explanation generation focuses on supporting the reasoning behind the robot's behavior.

These approaches, however, fail to consider the mental workload needed to understand the received explanation.

In other words, the human teammate is expected to understand any explanation provided, often before the task execution, no matter how much information is presented in the explanation.

In this work, we argue that an explanation, especially complex ones, should be made in an online fashion during the execution, which helps spread out the information to be explained and thus reducing the mental workload of humans.

However, a challenge here is that the different parts of an explanation are dependent on each other, which must be taken into account when generating online explanations.

To this end, a general formulation of online explanation generation is presented along with three different implementations satisfying different online properties.

We base our explanation generation method on a model reconciliation setting introduced in our prior work.

Our approaches are evaluated both with human subjects in a standard planning competition (IPC) domain, using NASA Task Load Index (TLX), as well as in simulation with ten different problems across two IPC domains.

We introduce online explanation to consider the cognitive requirement of the human for understanding the generated explanation by the agent.

Online Explanation Generation for Human-Robot Teaming


Deep neural networks use deeper and broader structures to achieve better performance and consequently, use increasingly more GPU memory as well.

However, limited GPU memory restricts many potential designs of neural networks.

In this paper, we propose a reinforcement learning based variable swapping and recomputation algorithm to reduce the memory cost, without sacrificing the accuracy of models.

Variable swapping can transfer variables between CPU and GPU memory to reduce variables stored in GPU memory.

Recomputation can trade time for space by removing some feature maps during forward propagation.

Forward functions are executed once again to get the feature maps before reuse.

However, how to automatically decide which variables to be swapped or recomputed remains a challenging problem.

To address this issue, we propose to use a deep Q-network(DQN) to make plans.

By combining variable swapping and recomputation, our results outperform several well-known benchmarks.

We propose a reinforcement learning based variable swapping and recomputation algorithm to reduce the memory cost.

GPU Memory Management for Deep Neural Networks Using Deep Q-Network


In vanilla backpropagation (VBP), activation function matters considerably in terms of non-linearity and differentiability.

Vanishing gradient has been an important problem related to the bad choice of activation function in deep learning (DL).

This work shows that a differentiable activation function is not necessary any more for error backpropagation.

The derivative of the activation function can be replaced by an iterative temporal differencing (ITD) using fixed random feedback weight alignment (FBA).

Using FBA with ITD, we can transform the VBP into a more biologically plausible approach for learning deep neural network architectures.

We don't claim that ITD works completely the same as the spike-time dependent plasticity (STDP) in our brain but this work can be a step toward the integration of STDP-based error backpropagation in deep learning.

Iterative temporal differencing with fixed random feedback alignment support spike-time dependent plasticity in vanilla backpropagation for deep learning.

temporal differencing fix random feedback alignment support spike-time dependent

plasticity vanilla backpropagation deep learning


Inspired by the recent successes of deep generative models for Text-To-Speech (TTS) such as WaveNet (van den Oord et al., 2016) and Tacotron (Wang et al., 2017), this article proposes the use of a deep generative model tailored for Automatic Speech Recognition (ASR) as the primary acoustic model (AM) for an overall recognition system with a separate language model (LM).

Two dimensions of depth are considered: (1) the use of mixture density networks, both autoregressive and non-autoregressive, to generate density functions capable of modeling acoustic input sequences with much more powerful conditioning than the first-generation generative models for ASR, Gaussian Mixture Models / Hidden Markov Models (GMM/HMMs), and (2) the use of standard LSTMs, in the spirit of the original tandem approach, to produce discriminative feature vectors for generative modeling.

Combining mixture density networks and deep discriminative features leads to a novel dual-stack LSTM architecture directly related to the RNN Transducer (Graves, 2012), but with the explicit functional form of a density, and combining naturally with a separate language model, using Bayes rule.

The generative models discussed here are compared experimentally in terms of log-likelihoods and frame accuracies.

This paper proposes the use of a deep generative acoustic model for automatic speech recognition, combining naturally with other deep sequence-to-sequence modules using Bayes' rule.

A Deep Generative Acoustic Model for Compositional Automatic Speech Recognition

deep generative acoustic model
automatic speech recognition


Recent studies show that widely used Deep neural networks (DNNs) are vulnerable to the carefully crafted adversarial examples.

Many advanced algorithms have been proposed to generate adversarial examples by leveraging the L_p distance for penalizing perturbations.

Different defense methods have also been explored to defend against such adversarial attacks.

While the effectiveness of L_p distance as a metric of perceptual quality remains an active research area, in this paper we will instead focus on a different type of perturbation, namely spatial transformation, as opposed to manipulating the pixel values directly as in prior works.

Perturbations generated through spatial transformation could result in large L_p distance measures, but our extensive experiments show that such spatially transformed adversarial examples are perceptually realistic and more difficult to defend against with existing defense systems.

This potentially provides a new direction in adversarial example generation and the design of corresponding defenses.

We visualize the spatial transformation based perturbation for different examples and show that our technique

can produce realistic adversarial examples with smooth image deformation.

Finally, we visualize the attention of deep networks with different types of adversarial examples to better understand how these examples are interpreted.

We propose a new approach for generating adversarial examples based on spatial transformation, which produces perceptually realistic examples compared to existing attacks.

Spatially Transformed Adversarial Examples


Most existing deep reinforcement learning (DRL) frameworks consider action spaces that are either

Motivated by the project of design Game AI for King of Glory

(KOG), one the world’s most popular mobile game, we consider the scenario with the discrete-continuous

To directly apply existing DLR frameworks, existing approaches

either approximate the hybrid space by a discrete set or relaxing it into a continuous set, which is

In this paper, we propose a parametrized deep Q-network (P-DQN)

for the hybrid action space without approximation or relaxation.

DDPG and can be viewed as an extension of the DQN to hybrid actions.

game KOG validates the efficiency and effectiveness of our method.

A DQN and DDPG hybrid algorithm is proposed to deal with the discrete-continuous hybrid action space.

PARAMETRIZED DEEP Q-NETWORKS LEARNING: PLAYING ONLINE BATTLE ARENA WITH DISCRETE-CONTINUOUS HYBRID ACTION SPACE


Over the past decade, knowledge graphs became popular for capturing structured domain knowledge.

Relational learning models enable the prediction of missing links inside knowledge graphs.

More specifically, latent distance approaches model the relationships among entities via a distance between latent representations.

Translating embedding models (e.g., TransE) are among the most popular latent distance approaches which use one distance function to learn multiple relation patterns.

However, they are mostly inefficient in capturing symmetric relations since the representation vector norm for all the symmetric relations becomes equal to zero.

They also lose information when learning relations with reflexive patterns since they become symmetric and transitive.

We propose the Multiple Distance Embedding model (MDE) that addresses these limitations and a framework which enables collaborative combinations of latent distance-based terms (MDE).

Our solution is based on two principles:

1) using limit-based loss instead of margin ranking loss and

2) by learning independent embedding vectors for each of terms we can collectively train and predict using contradicting distance terms.

We further demonstrate that MDE allows modeling relations with (anti)symmetry, inversion, and composition patterns.

We propose MDE as a neural network model which allows us to map non-linear relations between the embedding vectors and the expected output of the score function.

Our empirical results show that MDE outperforms the state-of-the-art embedding models on several benchmark datasets.

A novel method of modelling Knowledge Graphs based on Distance Embeddings and Neural Networks

MDE: Multiple Distance Embeddings for Link Prediction in Knowledge Graphs


Improved generative adversarial network (Improved GAN) is a successful method of using generative adversarial models to solve the problem of semi-supervised learning.

However, it suffers from the problem of unstable training.

In this paper, we found that the instability is mostly due to the vanishing gradients on the generator.

To remedy this issue, we propose a new method to use collaborative training to improve the stability of semi-supervised GAN with the combination of Wasserstein GAN.

The experiments have shown that our proposed method is more stable than the original Improved GAN and achieves comparable classification accuracy on different data sets.

Improve Training Stability of Semi-supervised Generative Adversarial Networks with Collaborative Training

improve training stability semi-supervised generative adversarial network collaborative


It has long been known that a single-layer fully-connected neural network with an i.i.d.

prior over its parameters is equivalent to a Gaussian process (GP), in the limit of infinite network width.

This correspondence enables exact Bayesian inference for infinite width neural networks on regression tasks by means of evaluating the corresponding GP.

Recently, kernel functions which mimic multi-layer random neural networks have been developed, but only outside of a Bayesian framework.

As such, previous work has not identified that these kernels can be used as covariance functions for GPs and allow fully Bayesian prediction with a deep neural network.

In this work, we derive the exact equivalence between infinitely wide, deep, networks and GPs with a particular covariance function.

We further develop a computationally efficient pipeline to compute this covariance function.

We then use the resulting GP to perform Bayesian inference for deep neural networks on MNIST and CIFAR-10.

We observe that the trained neural network accuracy approaches that of the corresponding GP with increasing layer width, and that the GP uncertainty is strongly correlated with trained network prediction error.

We further find that test performance increases as finite-width trained networks are made wider and  more  similar  to  a  GP,  and  that  the  GP-based  predictions  typically  outperform  those  of  finite-width  networks.

Finally  we  connect  the  prior  distribution over weights and variances in our GP formulation to the recent development of signal propagation in random neural networks.

We show how to make predictions using deep networks, without training deep networks.

Deep Neural Networks as Gaussian Processes