Interactions such as double negation in sentences and scene interactions in images are common forms of complex dependencies captured by state-of-the-art machine learning models.

We propose Mahé, a novel approach to provide Model-Agnostic Hierarchical Explanations of how powerful machine learning models, such as deep neural networks, capture these interactions as either dependent on or free of the context of data instances.

Specifically, Mahé provides context-dependent explanations by a novel local interpretation algorithm that effectively captures any-order interactions, and obtains context-free explanations through generalizing context-dependent interactions to explain global behaviors.

Experimental results show that Mahé obtains improved local interaction interpretations over state-of-the-art methods and successfully provides explanations of interactions that are context-free.

A new framework for context-dependent and context-free explanations of predictions

The authors extend the linear local attribution method LIME for interpreting black box models, and propose a method to discern between context-dependent and context-free interactions.

A method that can provide hierarchical explanations for a model, including both context-dependent and context-free explanations by a local interpretation algorithm.

Can I trust you more? Model-Agnostic Hierarchical Explanations


To realize the promise of ubiquitous embedded deep network inference, it is essential to seek limits of energy and area efficiency.

To this end, low-precision networks offer tremendous promise because both energy and area scale down quadratically with the reduction in precision.

Here, for the first time, we demonstrate ResNet-18, ResNet-34, ResNet-50, ResNet-152, Inception-v3, densenet-161, and VGG-16bn networks on the ImageNet classification benchmark that, at 8-bit precision exceed the accuracy of the full-precision baseline networks after one epoch of finetuning, thereby leveraging the availability of pretrained models.

We also demonstrate ResNet-18, ResNet-34, and ResNet-50 4-bit models that match the accuracy of the full-precision baseline networks -- the highest scores to date.

Surprisingly, the weights of the low-precision networks are very close (in cosine similarity) to the weights of the corresponding baseline networks, making training from scratch unnecessary.

We find that gradient noise due to quantization during training increases with reduced precision, and seek ways to overcome this noise.

The number of iterations required by stochastic gradient descent to achieve a given training error is related to the square of (a) the distance of the initial solution from the final plus (b) the maximum variance of the gradient estimates.

By drawing inspiration from this observation, we (a) reduce solution distance by starting with pretrained fp32 precision baseline networks and fine-tuning, and (b) combat noise introduced by quantizing weights and activations during training, by using larger batches along with matched learning rate annealing.

Sensitivity analysis indicates that these techniques, coupled with proper activation function range calibration, offer a promising heuristic to discover low-precision networks, if they exist, close to fp32 precision baseline networks.

Finetuning after quantization matches or exceeds full-precision state-of-the-art networks at both 8- and 4-bit quantization.

This paper proposes to improve the performance of low-precision models by doing quantization on pre-trained models, using large batches size, and using proper learning rate annealing with longer training time.

A method for low bit quantization to enable inference on efficient hardware that achieves full accuracy on ResNet50 with 4-bit weights and activations, based on observations that fine-tuning at low precision introduces noise in the gradient.

Discovering Low-Precision Networks Close to Full-Precision Networks for Efficient Embedded Inference

quantization network 4-bit efficient full-precision low-precision inference


  Analysis methods which enable us to better understand the
  representations and functioning of neural models of language are
  increasingly needed as deep learning becomes the dominant approach
  in NLP.

Here we present two methods based on Representational
  Similarity Analysis (RSA) and Tree Kernels (TK) which allow us to
  directly quantify how strongly the information encoded in neural
  activation patterns corresponds to information represented by
  symbolic structures such as syntax trees.

We first validate our
  methods on the case of a simple synthetic language for arithmetic
  expressions with clearly defined syntax and semantics, and show that
  they exhibit the expected pattern of results.

We then apply our methods to
  correlate neural representations of English sentences with their
  constituency parse trees.

Two methods based on Representational Similarity Analysis (RSA) and Tree Kernels (TK) which directly quantify how strongly information encoded in neural activation patterns corresponds to information represented by symbolic structures.

Correlating neural and symbolic representations of language


Supervised deep learning requires a large amount of training samples with annotations (e.g. label class for classification task, pixel- or voxel-wised label map for segmentation tasks), which are expensive and time-consuming to obtain.

During the training of a deep neural network, the annotated samples are fed into the network in a mini-batch way, where they are often regarded of equal importance.

However, some of the samples may become less informative during training, as the magnitude of the gradient start to vanish for these samples.

In the meantime, other samples of higher utility or hardness may be more demanded for the training process to proceed and require more exploitation.

To address the challenges of expensive annotations and loss of sample informativeness, here we propose a novel training framework which adaptively selects informative samples that are fed to the training process.

The adaptive selection or sampling is performed based on a hardness-aware strategy in the latent space constructed by a generative model.

To evaluate the proposed training framework, we perform experiments on three different datasets, including MNIST and CIFAR-10 for image classification task and a medical image dataset IVUS for biophysical simulation task.

On all three datasets, the proposed framework outperforms a random sampling method, which demonstrates the effectiveness of our framework.

This paper introduces a framework for data-efficient representation learning by adaptive sampling in latent space.

A method for sequential and adaptive selection of training examples to be presented to the training algorithm, where selection happens in the latent space based on choosing samples in the direction of the gradient of the loss.

A method to efficiently select hard samples during neural network training, achieved via a variational auto-encoder that encodes samples into a latent space.

Efficient Deep Representation Learning by Adaptive Latent Space Sampling

representation learning adaptive
latent space
method
sample
training


Existing methods for AI-generated artworks still struggle with generating high-quality stylized content, where high-level semantics are preserved, or separating fine-grained styles from various artists.

We propose a novel Generative Adversarial Disentanglement Network which can disentangle two complementary factors of variations when only one of them is labelled in general, and fully decompose complex anime illustrations into style and content in particular.

Training such model is challenging, since given a style, various content data may exist but not the other way round.

Our approach is divided into two stages, one that encodes an input image into a style independent content, and one based on a dual-conditional generator.

We demonstrate the ability to generate high-fidelity anime portraits with a fixed content and a large variety of styles from over a thousand artists, and vice versa, using a single end-to-end network and with applications in style transfer.

We show this unique capability as well as superior output to the current state-of-the-art.

An adversarial training-based method for disentangling two complementary sets of variations in a dataset where only one of them is labelled, tested on style vs. content in anime illustrations.

An image generation method combining conditional GANs and conditional VAEs that generates high fidelity anime images with various styles from various artists.

Proposal for a method to learn disentangled style (artist) and content representations in anime.

Disentangling Style and Content in Anime Illustrations

content anime illustration
disentangle style
method
artist


Recent research has shown that CNNs are often overly sensitive to high-frequency textural patterns.

Inspired by the intuition that humans are more sensitive to the lower-frequency (larger-scale) patterns we design a regularization scheme that penalizes large differences between adjacent components within each convolutional kernel.

We apply our regularization onto several popular training methods, demonstrating that the models with the proposed smooth kernels enjoy improved adversarial robustness.

Further, building on recent work establishing connections between adversarial robustness and interpretability, we show that our method appears to give more perceptually-aligned gradients.

We introduce a smoothness regularization for convolutional kernels of CNN that can help improve adversarial robustness and lead to perceptually-aligned gradients

This paper proposes a new regularization scheme that encourages convolutional kernels to be smoother, arguing that reducing neural network reliance on high-frequency components helps robustness against adversarial examples.

The authors propose a method for learning smoother convolutional kernels, specifically, a regularizer penalizing large changes between consecutive pixels of the kernel with the intuition of penalizing the use of high-frequency input components.

Smooth Kernels Improve Adversarial Robustness and Perceptually-Aligned Gradients

improve adversarial robustness
convolutional kernel
perceptually-aligned gradient
high-frequency
help
regularization
component
smooth


Despite an ever growing literature on reinforcement learning algorithms and applications, much less is known about their statistical inference.

In this paper, we investigate the large-sample behaviors of the Q-value estimates with closed-form characterizations of the asymptotic variances.

This allows us to efficiently construct confidence regions for Q-value and optimal value functions, and to develop policies to minimize their estimation errors.

This also leads to a policy exploration strategy that relies on estimating the relative discrepancies among the Q estimates.

Numerical experiments show superior performances of our exploration strategy than other benchmark approaches.

We investigate the large-sample behaviors of the Q-value estimates and proposed an efficient exploration strategy that relies on estimating the relative discrepancies among the Q estimates.

Efficient Inference and Exploration for Reinforcement Learning


Entailment vectors are a principled way to encode in a vector what information is known and what is unknown.

They are designed to model relations where one vector should include all the information in another vector, called entailment.

This paper investigates the unsupervised learning of entailment vectors for the semantics of words.

Using simple entailment-based models of the semantics of words in text (distributional semantics), we induce entailment-vector word embeddings which outperform the best previous results for predicting entailment between words, in unsupervised and semi-supervised experiments on hyponymy.

We train word embeddings based on entailment instead of similarity, successfully predicting lexical entailment.

The paper presents a word embedding algorithm for lexical entailment which follows the work of Henderson and Popa (ACL, 2016).

Unsupervised Learning of Entailment-Vector Word Embeddings


We describe a simple scheme that allows an agent to learn about its environment in an unsupervised manner.

Our scheme pits two versions of the same agent, Alice and Bob, against one another.

Alice proposes a task for Bob to complete; and then Bob attempts to complete the task.

In this work we will focus on two kinds of environments: (nearly) reversible environments and environments that can be reset.

Alice will "propose" the task by doing a sequence of actions and then Bob must undo or repeat them, respectively.

Via an appropriate reward structure, Alice and Bob automatically generate a curriculum of exploration, enabling unsupervised training of the agent.

When Bob is deployed on an RL task within the environment, this unsupervised training reduces the number of supervised episodes needed to learn, and in some cases converges to a higher reward.

Unsupervised learning for reinforcement learning using an automatic curriculum of self-play

A new formulation for exploring the environment in an unsupervised way to aid a specific task later, where one agent proposes increasingly difficult tasks and the learning agent tries to accomplish them.

A self-play model where one agent learns to propose tasks that are easy for them but difficult for an opponent, creating a moving target of self-play objectives and learning curriculum.

Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play

automatic curriculum
learn
task
one agent
self-play
difficult


Many real-world data sets are represented as graphs, such as citation links, social media, and biological interaction.

The volatile graph structure makes it non-trivial to employ convolutional neural networks (CNN's) for graph data processing.

Recently, graph attention network (GAT) has proven a promising attempt by combining graph neural networks with attention mechanism, so as to achieve massage passing in graphs with arbitrary structures.

However, the attention in GAT is computed mainly based on the similarity between the node content, while the structures of the graph remains largely unemployed (except in masking the attention out of one-hop neighbors).

In this paper, we propose an `````````````````````````````"ADaptive Structural Fingerprint" (ADSF) model to fully exploit both topological details of the graph and  content features of the nodes.

The key idea is to contextualize each node with a weighted, learnable receptive field  encoding rich and diverse local graph structures.

By doing this, structural interactions between the nodes can  be inferred accurately, thus improving subsequent attention layer as well as the convergence of learning.

Furthermore, our model provides a useful platform  for different subspaces of node features and various scales of graph structures to ``cross-talk'' with each other through the learning of multi-head attention, being particularly useful in handling complex real-world data.

Encouraging performance is observed on a number of benchmark data sets in node classification.

Exploiting rich strucural details in graph-structued data via adaptive "strucutral fingerprints''

A graph structure based methodology to augment the attention mechanism of graph neural networks, with the main idea to explore interactions between different types of nodes of the local neighborhood of a root node.

This paper extends the idea of self-attention in graph NNs, which is typically based on feature similarity between nodes, to include structural similarity.

Adaptive Structural Fingerprints for Graph Attention Networks

graph idea fingerprint adaptive base network attention structural


Informed and robust decision making in the face of uncertainty is critical for robots that perform physical tasks alongside people.

We formulate this as a Bayesian Reinforcement Learning problem over latent Markov Decision Processes (MDPs).

While Bayes-optimality is theoretically the gold standard, existing algorithms do not scale well to continuous state and action spaces.

We propose a scalable solution that builds on the following insight: in the absence of uncertainty, each latent MDP is easier to solve.

We split the challenge into two simpler components.

First, we obtain an ensemble of clairvoyant experts and fuse their advice to compute a baseline policy.

Second, we train a Bayesian residual policy to improve upon the ensemble's recommendation and learn to reduce uncertainty.

Our algorithm, Bayesian Residual Policy Optimization (BRPO), imports the scalability of policy gradient methods as well as the initialization from prior models.

BRPO significantly improves the ensemble of experts and drastically outperforms existing adaptive RL methods.

We propose a scalable Bayesian Reinforcement Learning algorithm that learns a Bayesian correction over an ensemble of clairvoyant experts to solve problems with complex latent rewards and dynamics.

This paper considers Bayesian Reinforcement Learning problem over latent Markov Decision Processes (MDPs) by making decisions with experts.

In this paper, the authors motivate and propose a learning algorithm, called Bayesian Residual Policy Optimization (BRPO), for Bayesian reinforcement learning problems.

Bayesian Residual Policy Optimization: Scalable Bayesian Reinforcement Learning with Clairvoyant Experts

scalable bayesian reinforcement learning


One of the mysteries in the success of neural networks is randomly initialized first order methods like gradient descent can achieve zero training loss even though the objective function is non-convex and non-smooth.

This paper demystifies this surprising phenomenon for two-layer fully connected ReLU activated neural networks.

For an $m$ hidden node shallow neural network with ReLU activation and $n$ training data, we show as long as $m$ is large enough and no two inputs are parallel, randomly initialized gradient descent converges to a globally optimal solution at a linear convergence rate for the quadratic loss function.

Our analysis relies on the following observation: over-parameterization and random initialization jointly restrict every weight vector to be close to its initialization for all iterations, which allows us to exploit a strong convexity-like property to show that gradient descent converges at a global linear rate to the global optimum.

We believe these insights are also useful in analyzing deep models and other first order methods.

We prove gradient descent achieves zero training loss with a linear rate on over-parameterized neural networks.

This work considers optimizing a two-layer over-parameterized ReLU network with the squared loss and given a data set with arbituary labels.

This paper studies one hidden layer neural networks with square loss, where they show that in over-parameterized setting, random initialization and gradient descent gets to zero loss.

Gradient Descent Provably Optimizes Over-parameterized Neural Networks

over-parameterized neural network
loss
gradient descent
zero
optimize


For many applications, in particular in natural science, the task is to

determine hidden system parameters from a set of measurements.

the forward process from parameter- to measurement-space is well-defined,

whereas the inverse problem is ambiguous: multiple parameter sets can

To fully characterize this ambiguity, the full

posterior parameter distribution, conditioned on an observed measurement,

We argue that a particular class of neural networks

is well suited for this task – so-called Invertible Neural Networks (INNs).

Unlike classical neural networks, which attempt to solve the ambiguous

inverse problem directly, INNs focus on learning the forward process, using

additional latent output variables to capture the information otherwise

Due to invertibility, a model of the corresponding inverse process is

Given a specific measurement and the distribution of

the latent variables, the inverse pass of the INN provides the full posterior

We prove theoretically and verify experimentally, on

artificial data and real-world problems from medicine and astrophysics, that

INNs are a powerful analysis tool to find multi-modalities in parameter space,

uncover parameter correlations, and identify unrecoverable parameters.

To analyze inverse problems with Invertible Neural Networks

The author proposes to use invertible networks to solve ambiguous inverse problems and suggest to not only train the forward model, but also the inverse model with an MMD critic.

The research paper proposes an invertible network with observations for posterior probability of complex input distributions with a theoretical valid bidirectional training scheme.

Analyzing Inverse Problems with Invertible Neural Networks

analyze inverse problem invertible neural network


Decisions made by machine learning systems have increasing influence on the world.

Yet it is common for machine learning algorithms to assume that no such influence exists.

assumption in online learning for applications such as content recommendation, where the (choice of) content displayed can change users' perceptions and preferences, or even drive them away, causing a shift in the distribution of users.

Generally speaking, it is possible for an algorithm to change the distribution of its own inputs.

We introduce the term self-induced distributional shift (SIDS) to describe this phenomenon.

A large body of work in reinforcement learning and causal machine learning aims to deal with distributional shift caused by deploying learning systems previously trained offline.

Our goal is similar, but distinct: we point out that changes to the learning algorithm, such as the introduction of meta-learning, can reveal hidden incentives for distributional shift (HIDS), and aim to diagnose and prevent problems associated with hidden incentives.

We design a simple  environment as a "unit test" for HIDS, as well as a content recommendation environment which allows us to disentangle different types of SIDS.

We demonstrate the potential for HIDS to cause unexpected or undesirable behavior in these environments, and propose and test a mitigation strategy.

Performance metrics are incomplete specifications; the ends don't always justify the means.

The authors show how meta-learning reveals the hidden incentives for distributional shift and propose an approach based on swapping learners between environments to reduce self introduced distributional shift.

The paper generalizes the inherent incentive for the learner to win by making the task easier in meta-learning to a larger class of problems.

Hidden incentives for self-induced distributional shift

hide incentive
distributional shift
learner


In one-class-learning tasks, only the normal case can be modeled with data, whereas the variation of all possible anomalies is too large to be described sufficiently by samples.

Thus, due to the lack of representative data, the wide-spread discriminative approaches cannot cover such learning tasks, and rather generative models, which attempt to learn the input density of the normal cases, are used.

However, generative models suffer from a large input dimensionality (as in images) and are typically inefficient learners.

We propose to learn the data distribution more efficiently with a multi-hypotheses autoencoder.

Moreover, the model is criticized by a discriminator, which prevents artificial data modes not supported by data, and which enforces diversity across hypotheses.

This consistency-based anomaly detection (ConAD) framework allows the reliable identification of outof- distribution samples.

For anomaly detection on CIFAR-10, it yields up to 3.9% points improvement over previously reported results.

On a real anomaly detection task, the approach reduces the error of the baseline models from 6.8% to 1.5%.

We propose an anomaly-detection approach that combines modeling the foreground class via multiple local densities with adversarial training.

The paper proposes a technique to make generative models more robust by making them consistent with the local density.

Consistency-based anomaly detection with adaptive multiple-hypotheses predictions


Generative Adversarial Networks (GAN) can achieve promising performance on learning complex data distributions on different types of data.

In this paper, we first show that a straightforward extension of an existing GAN algorithm is not applicable to point clouds, because the constraint required for discriminators is undefined for set data.

We propose a two fold modification to a GAN algorithm to be able to generate point clouds (PC-GAN).

First, we combine ideas from hierarchical Bayesian modeling and implicit generative models by learning a hierarchical and interpretable sampling process.

A key component of our method is that we train a posterior inference network for the hidden variables.

Second, PC-GAN defines a generic framework that can incorporate many existing GAN algorithms.

We further propose a sandwiching objective, which results in a tighter Wasserstein distance estimate than the commonly used dual form in WGAN.

We validate our claims on the ModelNet40 benchmark dataset and observe that PC- GAN trained by the sandwiching objective achieves better results on test data than existing methods.

We also conduct studies on several tasks, including generalization on unseen point clouds, latent space interpolation, classification, and image to point clouds transformation, to demonstrate the versatility of the proposed PC-GAN algorithm.

We propose a GAN variant which learns to generate point clouds. Different studies have been explores, including tighter Wasserstein distance estimate,  conditional generation, generalization to unseen point clouds and image to point cloud.

This paper proposes using GAN to generate 3D point cloud and introduces a sandwiching objective, averaging the upper and lower bound of Wasserstein distance between distributions.

This paper proposes a new generative model for unordered data, with a particular application to point clouds, which includes an inference method and a novel objective function.

Point Cloud GAN

gan
point cloud
generate
include
wasserstein distance
objective


Existing attention mechanisms, are mostly item-based in that a model is trained to attend to individual items in a collection (the memory) where each item has a predefined, fixed granularity, e.g., a character or a word.

Intuitively, an area in the memory consisting of multiple items can be worth attending to as a whole.

We propose area attention: a way to attend to an area of the memory, where each area contains a group of items that are either spatially adjacent when the memory has a 2-dimensional structure, such as images, or temporally adjacent for 1-dimensional memory, such as natural language sentences.

Importantly, the size of an area, i.e., the number of items in an area or the level of aggregation, is dynamically determined via learning, which can vary depending on the learned coherence of the adjacent items.

By giving the model the option to attend to an area of items, instead of only individual items, a model can attend to information with varying granularity.

Area attention can work along multi-head attention for attending to multiple areas in the memory.

We evaluate area attention on two tasks: neural machine translation (both character and token-level) and image captioning, and improve upon strong (state-of-the-art) baselines in all the cases.

These improvements are obtainable with a basic form of area attention that is parameter free.

In addition to proposing the novel concept of area attention, we contribute an efficient way for computing it by leveraging the technique of summed area tables.

The paper presents a novel approach for attentional mechanisms that can benefit a range of tasks such as machine translation and image captioning.

This paper extends the current attention models from word level to the combination of adjacent words, by applying the models to items made from merged adjacent words.

Area Attention


We identify a phenomenon, which we refer to as *multi-model forgetting*, that occurs when sequentially training multiple deep networks with partially-shared parameters; the performance of previously-trained models degrades as one optimizes a subsequent one, due to the overwriting of shared parameters.

To overcome this, we introduce a statistically-justified weight plasticity loss that regularizes the learning of a model's shared parameters according to their importance for the previous models, and demonstrate its effectiveness when training two models sequentially and for neural architecture search.

Adding weight plasticity in neural architecture search preserves the best models to the end of the search and yields improved results in both natural language processing and computer vision tasks.

We identify a phenomenon, neural brainwashing, and introduce a statistically-justified weight plasticity loss to overcome this.

This paper discusses the phenomena of “neural brainwashing”, which refers to that the performance of one model is affected via another model sharing model parameters.

Overcoming Multi-model Forgetting


Revealing latent structure in data is an active field of research, having introduced exciting technologies such as variational autoencoders and adversarial networks, and is essential to push machine learning towards unsupervised knowledge discovery.

However, a major challenge is the lack of suitable benchmarks for an objective and quantitative evaluation of learned representations.

To address this issue we introduce Morpho-MNIST, a framework that aims to answer: "to what extent has my model learned to represent specific factors of variation in the data?"

We extend the popular MNIST dataset by adding a morphometric analysis enabling quantitative comparison of trained models, identification of the roles of latent variables, and characterisation of sample diversity.

We further propose a set of quantifiable perturbations to assess the performance of unsupervised and supervised methods on challenging tasks such as outlier detection and domain adaptation.

This paper introduces Morpho-MNIST, a collection of shape metrics and perturbations, in a step towards quantitative evaluation of representation learning.

This paper discusses the problem of evaluating and diagnosing the represenatations learnt using a generative model.

Authors present a set of criteria to categorize MNISt digists and a set of interesting perturbations to modify MNIST dataset.

Morpho-MNIST: Quantitative Assessment and Diagnostics for Representation Learning

representation learning
morpho-mnist
perturbation
quantitative


Exploration in environments with sparse rewards is a key challenge for reinforcement learning.

How do we design agents with generic inductive biases so that they can explore in a consistent manner instead of just using local exploration schemes like epsilon-greedy?

We propose an unsupervised reinforcement learning agent which learns a discrete pixel grouping model that preserves spatial geometry of the sensors and implicitly of the environment as well.

We use this representation to derive geometric intrinsic reward functions, like centroid coordinates and area, and learn policies to control each one of them with off-policy learning.

These policies form a basis set of behaviors (options) which allows us explore in a consistent way and use them in a hierarchical reinforcement learning setup to solve for extrinsically defined rewards.

We show that our approach can scale to a variety of domains with competitive performance, including navigation in 3D environments and Atari games with sparse rewards.

structured exploration in deep reinforcement learning via unsupervised visual abstraction discovery and control

The paper introduces visual abstractions that are used for reinforcement learning, where an algorithm learns to "control" each abstraction as well as select the options to achieve the overall task.

Learning to Control Visual Abstractions for Structured Exploration in Deep Reinforcement Learning

structured exploration deep reinforcement learning


Combinatorial optimization is a common theme in computer science.

While in general such problems are NP-Hard, from a practical point of view, locally optimal solutions can be useful.

In some combinatorial problems however, it can be hard to define meaningful solution neighborhoods that connect large portions of the search space, thus hindering methods that search this space directly.

We suggest to circumvent such cases by utilizing a policy gradient algorithm that transforms the problem to the continuous domain, and to optimize a new surrogate objective that renders the former as generic stochastic optimizer.

This is achieved by producing a surrogate objective whose distribution is fixed and predetermined, thus removing the need to fine-tune various hyper-parameters in a case by case manner.

Since we are interested in methods which can successfully recover locally optimal solutions, we use the problem of finding locally maximal cliques as a challenging experimental benchmark, and we report results on a large dataset of graphs that is designed to test clique finding algorithms.

Notably, we show in this benchmark that fixing the distribution of the surrogate is key to consistently recovering locally optimal solutions, and that our surrogate objective leads to an algorithm that outperforms other methods we have tested in a number of measures.

A new policy gradient algorithm designed to approach black-box combinatorial optimization problems. The algorithm relies only on function evaluations, and returns locally optimal solutions with high probability.

The paper proposes an approach to construct surrogate objectives for the application of policy gradient methods to combinatorial optimization with the goal of reducing the need of hyper-parameter tuning.

The paper propose to replace the reward term in the policy gradient algorithm with its centered empirical cumulative distribution.

The Cakewalk Method

policy gradient algorithm
combinatorial optimization
approach
method


Deterministic neural networks (NNs) are increasingly being deployed in safety critical domains, where calibrated, robust and efficient measures of uncertainty are crucial.

While it is possible to train regression networks to output the parameters of a probability distribution by maximizing a Gaussian likelihood function, the resulting model remains oblivious to the underlying confidence of its predictions.

In this paper, we propose a novel method for training deterministic NNs to not only estimate the desired target but also the associated evidence in support of that target.

We accomplish this by  placing evidential priors over our original Gaussian likelihood function and training our NN to infer the hyperparameters of our evidential distribution.

We impose priors during training such that the model is penalized when its predicted evidence is not aligned with the correct output.

Thus the model estimates not only the probabilistic mean and variance of our target but also the underlying uncertainty associated with each of those parameters.

We observe that our evidential regression method learns well-calibrated measures of uncertainty on various benchmarks, scales to complex computer vision tasks, and is robust to adversarial input perturbations.

Fast, calibrated uncertainty estimation for neural networks without sampling

This paper proposes a novel approach to estimate the confidence of predictions in a regression setting, opening the door to online applications with fully integrated uncertainty estimates.

This paper proposed deep evidential regression, a method for training neural networks to not only estimate the output but also the associated evidence in support of that output.

Deep Evidential Uncertainty

neural network
regression
estimate
deep evidential
uncertainty


The Lottery Ticket Hypothesis from Frankle & Carbin (2019) conjectures that, for typically-sized neural networks, it is possible to find small sub-networks which train faster and yield superior performance than their original counterparts.

The proposed algorithm to search for such sub-networks (winning tickets), Iterative Magnitude Pruning (IMP), consistently finds sub-networks with 90-95% less parameters which indeed train faster and better than the overparameterized models they were extracted from, creating potential applications to problems such as transfer learning.

In this paper, we propose a new algorithm to search for winning tickets, Continuous Sparsification, which continuously removes parameters from a network during training, and learns the sub-network's structure with gradient-based methods instead of relying on pruning strategies.

We show empirically that our method is capable of finding tickets that outperforms the ones learned by Iterative Magnitude Pruning, and at the same time providing up to 5 times faster search, when measured in number of training epochs.

We propose a new algorithm that quickly finds winning tickets in neural networks.

This paper proposes a novel objective function that can be used to jointly optimize a classification objective while encouraging sparsification in a network that performs with high accuracy.

This work propose a new iterative pruning methods named Continuous Sparsification, which continuously prunes the current weight until it reaches the target ratio.

Winning the Lottery with Continuous Sparsification

new sparsification network win continuous


In most practical settings and theoretical analyses, one assumes that a model can be trained until convergence.

However, the growing complexity of machine learning datasets and models may violate such assumptions.

Indeed, current approaches for hyper-parameter tuning and neural architecture search tend to be limited by practical resource constraints.

Therefore, we introduce a formal setting for studying training under the non-asymptotic, resource-constrained regime, i.e., budgeted training.

We analyze the following problem: "given a dataset, algorithm, and fixed resource budget, what is the best achievable performance?"

We focus on the number of optimization iterations as the representative resource.

Under such a setting, we show that it is critical to adjust the learning rate schedule according to the given budget.

Among budget-aware learning schedules, we find simple linear decay to be both robust and high-performing.

We support our claim through extensive experiments with state-of-the-art models on ImageNet (image classification), Kinetics (video classification), MS COCO (object detection and instance segmentation), and Cityscapes (semantic segmentation).

We also analyze our results and find that the key to a good schedule is budgeted convergence, a phenomenon whereby the gradient vanishes at the end of each allowed budget.

We also revisit existing approaches for fast convergence and show that budget-aware learning schedules readily outperform such approaches under (the practical but under-explored) budgeted training setting.

Introduce a formal setting for budgeted training and propose a budget-aware linear learning rate schedule

This work presents a technique for tuning the learning rate for Neural Network training when under a fixed number of epochs.

This paper analyzed which learning rate schedule should be used when the number of iteration is limited using an introduced concept of BAS (Budget-Aware Schedule).

Budgeted Training: Rethinking Deep Neural Network Training Under Resource Constraints

training
learn rate schedule
introduce
budget-aware
number
neural network
budget


We present a new approach for efficient exploration which leverages a low-dimensional encoding of the environment learned with a combination of model-based and model-free objectives.

Our approach uses intrinsic rewards that are based on a weighted distance of nearest neighbors in the low dimensional representational space to gauge novelty.

We then leverage these intrinsic rewards for sample-efficient exploration with planning routines in representational space.

One key element of our approach is that we perform more gradient steps in-between every environment step in order to ensure the model accuracy.

We test our approach on a number of maze tasks, as well as a control problem and show that our exploration approach is more sample-efficient compared to strong baselines.

We conduct exploration using intrinsic rewards that are based on a weighted distance of nearest neighbors in representational space.

This paper proposes a method for efficient exploration in tabular MDPs as well as a simple control environment, using deterministic encoders to learn a low dimensional representation of the environment dynamics.

This paper proposes a method of sample-efficient exploration for RL agent using a combination of model-based and model-free approaches with a novelty metric.

Novelty Search in representational space for sample efficient exploration

method
representational space
exploration
efficient
novelty


Neural networks are vulnerable to small adversarial perturbations.

While existing literature largely focused on the vulnerability of learned models, we demonstrate an intriguing phenomenon that adversarial robustness, unlike clean accuracy, is sensitive to the input data distribution.

Even a semantics-preserving transformations on the input data distribution can cause a significantly different robustness for the adversarially trained model that is both trained and evaluated on the new distribution.

We show this by constructing semantically- identical variants for MNIST and CIFAR10 respectively, and show that standardly trained models achieve similar clean accuracies on them, but adversarially trained models achieve significantly different robustness accuracies.

This counter-intuitive phenomenon indicates that input data distribution alone can affect the adversarial robustness of trained neural networks, not necessarily the tasks themselves.

Lastly, we discuss the practical implications on evaluating adversarial robustness, and make initial attempts to understand this complex phenomenon.

Robustness performance of PGD trained models are sensitive to semantics-preserving transformation of image datasets, which implies the trickiness of evaluation of robust learning algorithms in practice.

On the Sensitivity of Adversarial Robustness to Input Data Distributions


Sample inefficiency is a long-lasting problem in reinforcement learning (RL).

The state-of-the-art uses action value function to derive policy while it usually involves an extensive search over the state-action space and unstable optimization.

Towards the sample-efficient RL, we propose ranking policy gradient (RPG), a policy gradient method that learns the optimal rank of a set of discrete actions.

To accelerate the learning of policy gradient methods, we establish the equivalence between maximizing the lower bound of return and imitating a near-optimal policy without accessing any oracles.

These results lead to a general off-policy learning framework, which preserves the optimality, reduces variance, and improves the sample-efficiency.

We conduct extensive experiments showing that when consolidating with the off-policy learning framework, RPG substantially reduces the sample complexity, comparing to the state-of-the-art.

We propose ranking policy gradient that learns the optimal rank of actions to maximize return. We propose a general off-policy learning framework with the properties of optimality preserving, variance reduction, and sample-efficiency.

This paper proposes to reparameterize the policy using a form of ranking to convert the RL problem into a supervised learning problem.

This paper presents a new view on policy gradient methods from the perspective of ranking.

Ranking Policy Gradient


We introduce MultiGrain, a neural network architecture that generates compact image embedding vectors that solve multiple tasks of different granularity: class, instance, and copy recognition.

MultiGrain is trained jointly for classification by optimizing the cross-entropy loss and for instance/copy recognition by optimizing a self-supervised ranking loss.

The self-supervised loss only uses data augmentation and thus does not require additional labels.

Remarkably, the unified embeddings are not only much more compact than using several specialized embeddings, but they also have the same or better accuracy.

When fed to a linear classifier, MultiGrain using ResNet-50 achieves 79.4% top-1 accuracy on ImageNet, a +1.8% absolute improvement over the the current state-of-the-art AutoAugment method.

The same embeddings perform on par with state-of-the-art instance retrieval with images of moderate resolution.

An ablation study shows that our approach benefits from the self-supervision, the pooling method and the mini-batches with repeated augmentations of the same image.

Combining classification and image retrieval in a neural network architecture, we obtain an improvement for both tasks.

This paper proposes a unified embedding for image classification and instance retrieval to enhance the performance for both tasks.

The paper proposes to jointy train a deep neural net for image classification, instance, and copy recognition.

MultiGrain: a unified image embedding for classes and instances

image classification instance
task
unified
retrieval
neural


 In this paper, we investigate mapping the hyponymy relation of
 wordnet to feature vectors.

We aim to model lexical knowledge in such a way that it can be used as
  input in generic machine-learning models, such as phrase entailment
  predictors.

The first one leverages an existing mapping of
  words to feature vectors (fasttext), and attempts to classify
  such vectors as within or outside of each class.

The second model is fully supervised,
  using solely wordnet as a ground truth.

It maps each concept to an
  interval or a disjunction thereof.

On the first model, we approach, but not quite attain state of the
  art performance.

The second model can achieve near-perfect accuracy.

We investigate mapping the hyponymy relation of wordnet to feature vectors

This paper studies how hyponymy between words can be mapped to feature representations.

This paper explores the notion of hyponymy in word vector representations and describes a method of organizing WordNet relations into a tree structure to define hyponymy.

Mapping the hyponymy relation of wordnet onto vector Spaces

map hyponymy relation wordnet
feature
word
representation
vector


Recurrent Neural Networks (RNNs) are powerful autoregressive sequence models for learning prevalent patterns in natural language.

Yet language generated by RNNs often shows several degenerate characteristics that are uncommon in human language; while fluent, RNN language production can be overly generic, repetitive, and even self-contradictory.

We postulate that the objective function optimized by RNN language models, which amounts to the overall perplexity of a text, is not expressive enough to capture the abstract qualities of good generation such as Grice’s Maxims.

In this paper, we introduce a general learning framework that can construct a decoding objective better suited for generation.

Starting with a generatively trained RNN language model, our framework learns to construct a substantially stronger generator by combining several discriminatively trained models that can collectively address the limitations of RNN generation.

Human evaluation demonstrates that text generated by the resulting generator is preferred over  that  of  baselines  by  a  large  margin  and  significantly  enhances  the  overall coherence, style, and information content of the generated text.

We build a stronger natural language generator by discriminatively training scoring functions that rank candidate generations with respect to various qualities of good writing.

This paper proposes to bring together multiple inductive biases that hope to correct for inconsistencies in sequence decoding and proposes to optimize for the parameters of a pre-defined combination of various sub-objectives.

This paper combines RNN language model with several discriminatively trained models to improve the language generation.

This paper proposes to improve RNN language model generation using augmented objectives inspired by Grice's maxims of communication.

Learning to Write by Learning the Objective

improve
rnn language model
discriminatively train
generation
various
objective


In recent years, the efficiency and even the feasibility of traditional load-balancing policies are challenged by the rapid growth of cloud infrastructure with increasing levels of server heterogeneity and increasing size of cloud services and applications.

In such many software-load-balancers heterogeneous systems, traditional solutions, such as JSQ, incur an increasing communication overhead, whereas low-communication alternatives, such as JSQ(d) and the recently proposed JIQ scheme are either unstable or provide poor performance.

We argue that a better low-communication load balancing scheme can be established by allowing each dispatcher to have a different view of the system and keep using JSQ, rather than greedily trying to avoid starvation on a per-decision basis.

accordingly, we introduce the Loosely-Shortest -Queue family of load balancing algorithms.

Roughly speaking, in Loosely-shortest -Queue, each dispatcher keeps a different approximation of the server queue lengths and routes jobs to the shortest among them.

Communication is used only to update the approximations and make sure that they are not too far from the real queue lengths in expectation.

We formally establish the strong stability of any Loosely-Shortest -Queue policy and provide an easy-to-verify sufficient condition for verifying that a policy is Loosely-Shortest -Queue.

We further demonstrate that the Loosely-Shortest -Queue approach allows constructing throughput optimal policies with an arbitrarily low communication budget.

Finally, using extensive simulations that consider homogeneous, heterogeneous and highly skewed heterogeneous systems in scenarios with a single dispatcher as well as with multiple dispatchers, we show that the examined Loosely-Shortest -Queue example policies are always stable as dictated by theory.

Moreover, it exhibits an appealing performance and significantly outperforms well-known low-communication policies, such as JSQ(d) and JIQ, while using a similar communication budget.

Scalable and low communication load balancing solution for heterogeneous-server multi-dispatcher systems with strong theoretical guarantees and promising empirical results.

Load Balancing in Large-Scale Heterogeneous Systems with Multiple Dispatchers


We propose a novel quantitative measure to predict the performance of a deep neural network classifier, where the measure is derived exclusively from the graph structure of the network.

We expect that this measure is a fundamental first step in developing a method to evaluate new network architectures and reduce the reliance on the computationally expensive trial and error or "brute force" optimisation processes involved in model selection.

The measure is derived in the context of multi-layer perceptrons (MLPs), but the definitions are shown to be useful also in the context of deep convolutional neural networks (CNN), where it is able to estimate and compare the relative performance of different types of neural networks, such as VGG, ResNet, and DenseNet.

Our measure is also used to study the effects of some important "hidden" hyper-parameters of the DenseNet architecture, such as number of layers, growth rate and the dimension of 1x1 convolutions in DenseNet-BC.

Ultimately, our measure facilitates the optimisation of the DenseNet design, which shows improved results compared to the baseline.

A quantitative measure to predict the performances of deep neural network models.

The paper proposes a novel quantity that counts the number of path in the neural network which is predictive of the performance of neural networks with the same number of parameters.

The paper presents a method for counting paths in deep neural networks that arguably can be used to measure the performance of the network.

Counting the Paths in Deep Neural Networks as a Performance Predictor

count path deep neural network
performance
measure


There is a stark disparity between the learning rate schedules used in the practice of large scale machine learning and what are considered admissible learning rate schedules prescribed in the theory of stochastic approximation.

Recent results, such as in the 'super-convergence' methods which use oscillating learning rates, serve to emphasize this point even more.

One plausible explanation is that non-convex neural network training procedures are better suited to the use of fundamentally different learning rate  schedules, such as the ``cut the learning rate every constant number of epochs'' method (which more closely resembles an exponentially decaying learning rate schedule); note that this widely used schedule is in stark contrast to the polynomial decay schemes prescribed in the stochastic approximation literature, which are indeed shown to be (worst case) optimal for classes of convex optimization problems.

The main contribution of this work shows that the picture is far more nuanced, where we do not even need to move to non-convex optimization to show other learning rate schemes can be far more effective.

In fact, even for the simple case of stochastic linear regression with a fixed time horizon, the rate achieved by any polynomial decay scheme is sub-optimal compared to the statistical minimax rate (by a factor of condition number); in contrast the ```''cut the learning rate every constant number of epochs'' provides an exponential improvement (depending only logarithmically on the condition number) compared to any polynomial decay scheme.

Finally, it is important to ask if our theoretical insights are somehow fundamentally tied to quadratic loss minimization (where we have circumvented minimax lower bounds for more general convex optimization problems)?

Here, we conjecture that recent results which make the gradient norm small at a near optimal rate, for both convex and non-convex optimization, may also provide more insights into learning rate schedules used in practice.

This paper presents a rigorous study of why practically used learning rate schedules (for a given computational budget) offer significant advantages even though these schemes are not advocated by the classical theory of Stochastic Approximation.

This paper presents a theoretical study of different learning rate schedules that resulted in statistical minimax lower bounds for both polynomial and constant-and-cut schemes.

The paper studies the effect of learning-rate choices for stochastic optimization, focusing on least-mean-squares with decaying stepsizes

Rethinking learning rate schedules for stochastic optimization

present
learning rate schedule
study
stochastic optimization
scheme


We present Value Propagation (VProp), a set of parameter-efficient differentiable planning modules built on Value Iteration which can successfully be trained using reinforcement learning to solve unseen tasks, has the capability to generalize to larger map sizes, and can learn to navigate in dynamic environments.

We show that the modules enable learning to plan when the environment also includes stochastic elements, providing a cost-efficient learning system to build low-level size-invariant planners for a variety of interactive navigation problems.

We evaluate on static and dynamic configurations of MazeBase grid-worlds, with randomly generated environments of several different sizes, and on a StarCraft navigation scenario, with more complex dynamics, and pixels as input.

We present planners based on convnets that are sample-efficient and that generalize to larger instances of navigation and pathfinding problems.

Proposes methods, which can be seen as modifications of Value Iteration Networks (VIN), with some improvements aimed at improving sample efficiency and generalization to large environment sizes.

The paper presents an extension of the original value iteration networks (VIN) by considering a state-dependent transition function.

Value Propagation Networks

value iteration network vin
large
present


Learning high-quality word embeddings is of significant importance in achieving better performance in many down-stream learning tasks.

On one hand, traditional word embeddings are trained on a large scale corpus for general-purpose tasks, which are often sub-optimal for many domain-specific tasks.

On the other hand, many domain-specific tasks do not have a large enough domain corpus to obtain high-quality embeddings.

We observe that domains are not isolated and a small domain corpus can leverage the learned knowledge from many past domains to augment that corpus in order to generate high-quality embeddings.

In this paper, we formulate the learning of word embeddings as a lifelong learning process.

Given knowledge learned from many previous domains and a small new domain corpus, the proposed method can effectively generate new domain embeddings by leveraging a simple but effective algorithm and a meta-learner, where the meta-learner is able to provide word context similarity information at the domain-level.

Experimental results demonstrate that the proposed method can effectively learn new domain embeddings from a small corpus and past domain knowledges\footnote{We will release the code after final revisions.}.

also demonstrate that general-purpose embeddings trained from a large scale corpus are sub-optimal in domain-specific tasks.

learning better domain embeddings via lifelong learning and meta-learning

Presents a lifelong learning method for learning word embeddings.

This paper proposes an approach to learn embeddings in new domains and significantly beats the baseline on an aspect extraction task.

Lifelong Word Embedding via Meta-Learning

lifelong learning
learn
embedding
via
domain
word


Parameter pruning is a promising approach for CNN compression and acceleration by eliminating redundant model parameters with tolerable performance loss.

Despite its effectiveness, existing regularization-based parameter pruning methods usually drive weights towards zero with large and constant regularization factors, which neglects the fact that the expressiveness of CNNs is fragile and needs a more gentle way of regularization for the networks to adapt during pruning.

To solve this problem, we propose a new regularization-based pruning method (named IncReg) to incrementally assign different regularization factors to different weight groups based on their relative importance, whose effectiveness is proved on popular CNNs compared with state-of-the-art methods.

we propose a new regularization-based pruning method (named IncReg) to incrementally assign different regularization factors to different weight groups based on their relative importance.

This paper proposes a regularization-based pruning method to incrementally assign different regularization factors to different weight groups based on their relative importance.

Structured Pruning for Efficient ConvNets via Incremental Regularization

incrementally assign different regularization factor different weight group


Momentum based stochastic gradient methods such as heavy ball (HB) and Nesterov's accelerated gradient descent (NAG) method are widely used in practice for training deep networks and other supervised learning models, as they often provide significant improvements over stochastic gradient descent (SGD).

Rigorously speaking, fast gradient methods have provable improvements over gradient descent only for the deterministic case, where the gradients are exact.

In the stochastic case, the popular explanations for their wide applicability is that when these fast gradient methods are applied in the stochastic case, they partially mimic their exact gradient counterparts, resulting in some practical gain.

This work provides a counterpoint to this belief by proving that there exist simple problem instances where these methods cannot outperform SGD despite the best setting of its parameters.

These negative problem instances are, in an informal sense, generic; they do not look like carefully constructed pathological instances.

These results suggest (along with empirical evidence) that HB or NAG's practical performance gains are a by-product of minibatching.

Furthermore, this work provides a viable (and provable) alternative, which, on the same set of problem instances, significantly improves over HB, NAG, and SGD's performance.

This algorithm, referred to as Accelerated Stochastic Gradient Descent (ASGD), is a simple to implement stochastic algorithm, based on a relatively less popular variant of Nesterov's Acceleration.

Extensive empirical results in this paper show that ASGD has performance gains over HB, NAG, and SGD.

The code for implementing the ASGD Algorithm can be found at https://github.com/rahulkidambi/AccSGD.

Existing momentum/acceleration schemes such as heavy ball method and Nesterov's acceleration employed with stochastic gradients do not improve over vanilla stochastic gradient descent, especially when employed with small batch sizes.

On the insufficiency of existing momentum schemes for Stochastic Optimization


Oversubscription planning (OSP) is the problem of finding plans that maximize the utility value of their end state while staying within a specified cost bound.

Recently, it has been shown that OSP problems can be reformulated as classical planning problems with multiple cost functions but no utilities.

Here we take advantage of this reformulation to show that OSP problems can be solved optimally using the A* search algorithm, in contrast to previous approaches that have used variations on branch-and-bound search.

This allows many powerful techniques developed for classical planning to be applied to OSP problems.

We also introduce novel bound-sensitive heuristics, which are able to reason about the primary cost of a solution while taking into account secondary cost functions and bounds, to provide superior guidance compared to heuristics that do not take these bounds into account.

We implement two such bound-sensitive variants of existing classical planning heuristics, and show experimentally that the resulting search is significantly more informed than comparable heuristics that do not consider bounds.

We show that oversubscription planning tasks can be solved using A* and introduce novel bound-sensitive heuristics for oversubscription planning tasks.

Presents an approach to solve oversubscription planning (OSP) tasks optimally by using a translation to classical planning with multiple cost functions.

The paper proposes modifications to admissible heuristics to make them better informed in a multi-criteria setting where.

A∗ Search and Bound-Sensitive Heuristics for Oversubscription Planning

bound-sensitive heuristic oversubscription planning


Previous work on adversarially robust neural networks requires large training sets and computationally expensive training procedures.

On the other hand, few-shot learning methods are highly vulnerable to adversarial examples.

The goal of our work is to produce networks which both perform well at few-shot tasks and are simultaneously robust to adversarial examples.

We adapt adversarial training for meta-learning, we adapt robust architectural features to small networks for meta-learning, we test pre-processing defenses as an alternative to adversarial training for meta-learning, and we investigate the advantages of robust meta-learning over robust transfer-learning for few-shot tasks.

This work provides a thorough analysis of adversarially robust methods in the context of meta-learning, and we lay the foundation for future work on defenses for few-shot tasks.

We develop meta-learning methods for adversarially robust few-shot learning.

This paper presents a method that enhances the robustness of few-shot learning by introducing adversarial query data attack in the inner-task fine-tuning phase of a meta-learning algorithm.

The authors of this paper propose a novel approach for training a robust few-shot model.

Robust Few-Shot Learning with Adversarially Queried Meta-Learners

robust few-shot learning
method
adversarially
meta-learning
query


Many of our core assumptions about how neural networks operate remain empirically untested.

One common assumption is that convolutional neural networks need to be stable to small translations and deformations to solve image recognition tasks.

For many years, this stability was baked into CNN architectures by incorporating interleaved pooling layers.

Recently, however, interleaved pooling has largely been abandoned.

This raises a number of questions: Are our intuitions about deformation stability right at all?

Is pooling necessary for deformation invariance?

If not, how is deformation invariance achieved in its absence?

In this work, we rigorously test these questions, and find that deformation stability in convolutional networks is more nuanced than it first appears: (1) Deformation invariance is not a binary property, but rather that different tasks require different degrees of deformation stability at different layers.

(2) Deformation stability is not a fixed property of a network and is heavily adjusted over the course of training, largely through the smoothness of the convolutional filters.

(3) Interleaved pooling layers are neither necessary nor sufficient for achieving the optimal form of deformation stability for natural image classification.

(4) Pooling confers \emph{too much} deformation stability for image classification at initialization, and during training, networks have to learn to \emph{counteract} this inductive bias.

Together, these findings provide new insights into the role of interleaved pooling and deformation invariance in CNNs, and demonstrate the importance of rigorous empirical testing of even our most basic assumptions about the working of neural networks.

We find that pooling alone does not determine deformation stability in CNNs and that filter smoothness plays an important role in determining stability.

Pooling Is Neither Necessary nor Sufficient for Appropriate Deformation Stability in CNNs


Deep neural networks (DNNs) have been shown to over-fit a dataset when being trained with noisy labels for a long enough time.

To overcome this problem, we present a simple and effective method self-ensemble label filtering (SELF) to progressively filter out the wrong labels during training.

Our method improves the task performance by gradually allowing supervision only from the potentially non-noisy (clean) labels and stops learning on the filtered noisy labels.

For the filtering, we form running averages of predictions over the entire training dataset using the network output at different training epochs.

We show that these ensemble estimates yield more accurate identification of inconsistent predictions throughout training than the single estimates of the network at the most recent training epoch.

While filtered samples are removed entirely from the supervised training loss, we dynamically leverage them via semi-supervised learning in the unsupervised loss.

We demonstrate the positive effect of such an approach on various image classification tasks under both symmetric and asymmetric label noise and at different noise ratios.

It substantially outperforms all previous works on noise-aware learning across different datasets and can be applied to a broad set of network architectures.

We propose a self-ensemble framework to train more robust deep learning models under noisy labeled datasets.

This paper proposed "self-ensemble label filtering" for learning with noisy labels where the label noise is instance-independent, which yield more accurate identification of inconsistent predictions.

This paper proposes an algorithm for learning from data with noisy labels which alternates between updating the model and removing samples that look like they have noisy labels.

SELF: Learning to Filter Noisy Labels with Self-Ensembling


Long training times of deep neural networks are a bottleneck in machine learning research.

The major impediment to fast training is the quadratic growth of both memory and compute requirements of dense and convolutional layers with respect to their information bandwidth.

Recently, training `a priori' sparse networks has been proposed as a method for allowing layers to retain high information bandwidth, while keeping memory and compute low.

However, the choice of which sparse topology should be used in these networks is unclear.

In this work, we provide a theoretical foundation for the choice of intra-layer topology.

First, we derive a new sparse neural network initialization scheme that allows us to explore the space of very deep sparse networks.

Next, we evaluate several topologies and show that seemingly similar topologies can often have a large difference in attainable accuracy.

To explain these differences, we develop a data-free heuristic that can evaluate a topology independently from the dataset the network will be trained on.

We then derive a set of requirements that make a good topology, and arrive at a single topology that satisfies all of them.

We investigate pruning DNNs before training and provide an answer to which topology should be used for training a priori sparse networks.

The authors propose to replace dense layers with sparsely-connected linear layers and an approach to finding the best topology by measuring how well the sparse layers approximate random weights of their dense counterparts.

The paper proposes a sparse cascade architecture that is a multiplication of several sparse matrices and a specific connectivity pattern that outperforms other provided considerations.

NeuroFabric: Identifying Ideal Topologies for Training A Priori Sparse Networks

train priori sparse network
provide
topology


Deep learning models require extensive architecture design exploration and hyperparameter optimization to perform well on a given task.

The exploration of the model design space is often made by a human expert, and optimized using a combination of grid search and search heuristics over a large space of possible choices.

Neural Architecture Search (NAS) is a Reinforcement Learning approach that has been proposed to automate architecture design.

NAS has been successfully applied to generate Neural Networks that rival the best human-designed architectures.

However, NAS requires sampling, constructing, and training hundreds to thousands of models to achieve well-performing architectures.

This procedure needs to be executed from scratch for each new task.

The application of NAS to a wide set of tasks currently lacks a way to transfer generalizable knowledge across tasks.

In this paper, we present the Multitask Neural Model Search (MNMS) controller.

Our goal is to learn a generalizable framework that can condition model construction on successful model searches for previously seen tasks, thus significantly speeding up the search for new tasks.

We demonstrate that MNMS can conduct an automated architecture search for multiple tasks simultaneously while still learning well-performing, specialized models for each task.

We then show that pre-trained MNMS controllers can transfer learning to new tasks.

By leveraging knowledge from previous searches, we find that pre-trained MNMS models start from a better location in the search space and reduce search time on unseen tasks, while still discovering models that outperform published human-designed models.

We present Multitask Neural Model Search, a Meta-learner that can design models for multiple tasks simultaneously and transfer learning to unseen tasks.

This paper extends Neural Architecture Search to the multi-task learning problem where a task conditioned model search controller is learned to handle multiple tasks simultaneously.

In this paper, authors summarize their work on building a framework, called Multitask Neural Model Search controller, for automated neural network construction across multiple tasks simultaneously.

Transfer Learning to Learn with Multitask Neural Model Search

multitask neural model search
multiple task simultaneously
transfer learn
controller


This work studies the problem of modeling non-linear visual processes by leveraging deep generative architectures for learning linear, Gaussian models of observed sequences.

We propose a joint learning framework, combining a multivariate autoregressive model and deep convolutional generative networks.

After justification of theoretical assumptions of inearization, we propose an architecture that allows Variational Autoencoders and Generative Adversarial Networks to simultaneously learn the non-linear observation as well as the linear state-transition model from a sequence of observed frames.

Finally, we demonstrate our approach on conceptual toy examples and dynamic textures.

We model non-linear visual processes as autoregressive noise via generative deep learning.

Proposes a new method that models non-linear visual process with a deep version of a linear process (Markov process).

This paper proposes a new deep generative model for sequences, particularly image sequences and video, which uses a linear structure in part of the model.

Linearizing Visual Processes with Deep Generative Models

model non-linear visual process
new
linear
deep
generative


Partial differential equations (PDEs)  play a prominent role in many disciplines such as applied mathematics, physics, chemistry, material science, computer science, etc.

PDEs are commonly derived based on physical laws or empirical observations.

However, the governing equations for many complex systems in modern applications are still not fully known.

With the rapid development of sensors, computational power, and data storage in the past decade, huge quantities of data can be easily collected and efficiently stored.

Such vast quantity of data offers new opportunities for data-driven discovery of hidden physical laws.

Inspired by the latest development of neural network designs in deep learning, we propose a new feed-forward deep network, called PDE-Net, to fulfill two objectives at the same time: to accurately predict dynamics of complex systems and to uncover the underlying hidden PDE models.

The basic idea of the proposed PDE-Net is to learn differential operators by learning convolution kernels (filters), and apply neural networks or other machine learning methods to approximate the unknown nonlinear responses.

Comparing with existing approaches, which either assume the form of the nonlinear response is known or fix certain finite difference approximations of differential operators, our approach has the most flexibility by learning both differential operators and the nonlinear responses.

A special feature of the proposed PDE-Net is that all filters are properly constrained, which enables us to easily identify the governing PDE models while still maintaining the expressive and predictive power of the network.

These constrains are carefully designed by fully exploiting the relation between the orders of differential operators and the orders of sum rules of filters (an important concept originated from wavelet theory).

We also discuss relations of the PDE-Net with some existing networks in computer vision such as Network-In-Network (NIN) and Residual Neural Network (ResNet).

Numerical experiments show that the PDE-Net has the potential to uncover the hidden PDE of the observed dynamics, and predict the dynamical behavior for a relatively long time, even in a noisy environment.

This paper proposes a new feed-forward network, call PDE-Net, to learn PDEs from data.

The paper expores the use of deep learning machinery for the purpose of identifying dynamical systems specified by PDEs.

The paper proposes a neural network based algorithm for learning from data that arises from dynamical systems with governing equations that can be written as partial differential equations.

This paper addresses complex dynamical systems modelling through nonparametric Partial Differential Equations using neural architectures, with the most important idea of the papier (PDE-net) to learn both differential operators and the function that governs the PDE.

PDE-Net: Learning PDEs from Data

pde-net learn pde
partial differential equation
datum
dynamical system
neural
govern
network


Each training step for a variational autoencoder (VAE) requires us to sample from the approximate posterior, so we usually choose simple (e.g. factorised) approximate posteriors in which sampling is an efficient computation that fully exploits GPU parallelism.

However, such simple approximate posteriors are often insufficient, as they eliminate statistical dependencies in the posterior.

While it is possible to use normalizing flow approximate posteriors for continuous latents, there is nothing analogous for discrete latents.

The most natural approach to model discrete dependencies is an autoregressive distribution, but sampling from such distributions is inherently sequential and thus slow.

We develop a fast, parallel sampling procedure for autoregressive distributions based on fixed-point iterations which enables efficient and accurate variational inference in discrete state-space models.

To optimize the variational bound, we considered two ways to evaluate probabilities: inserting the relaxed samples directly into the pmf for the discrete distribution, or converting to continuous logistic latent variables and interpreting the K-step fixed-point iterations as a normalizing flow.

We found that converting to continuous latent variables gave considerable additional scope for mismatch between the true and approximate posteriors, which resulted in biased inferences, we thus used the former approach.

We tested our approach on the neuroscience problem of inferring discrete spiking activity from noisy calcium-imaging data, and found that it gave accurate connectivity estimates in an order of magnitude less time.

We give a fast normalising-flow like sampling procedure for discrete latent variable models.

This paper uses an autoregressive filtering variational approximation for parameter estimation in discrete dynamical systems by using fixed point iterations.

The authors posit a general autoregressive posterior family for discrete variables or their continuous relaxations.

This paper has two main contributions: it extends normalizing flows to discrete settings and presents an approximate fixed-point update rule for autoregressive time-series that can exploit GPU parallelism.

Discrete flow posteriors for variational inference in discrete dynamical systems

discrete dynamical system
autoregressive
variable
variational
posterior
flow


Deep neural networks (DNNs) had great success on NLP tasks such as language modeling, machine translation and certain question answering (QA) tasks.

However, the success is limited at more knowledge intensive tasks such as QA from a big corpus.

Existing end-to-end deep QA models (Miller et al., 2016; Weston et al., 2014) need to read the entire text after observing the question, and therefore their complexity in responding a question is linear in the text size.

This is prohibitive for practical tasks such as QA from Wikipedia, a novel, or the Web.

We propose to solve this scalability issue by using symbolic meaning representations, which can be indexed and retrieved efficiently with complexity that is independent of the text size.

More specifically, we use sequence-to-sequence models to encode knowledge symbolically and generate programs to answer questions from the encoded knowledge.

We apply our approach, called the N-Gram Machine (NGM), to the bAbI tasks (Weston et al., 2015) and a special version of them (“life-long bAbI”) which has stories of up to 10 million sentences.

Our experiments show that NGM can successfully solve both of these tasks accurately and efficiently.

Unlike fully differentiable memory models, NGM’s time complexity and answering quality are not affected by the story length.

The whole system of NGM is trained end-to-end with REINFORCE (Williams, 1992).

To avoid high variance in gradient estimation, which is typical in discrete latent variable models, we use beam search instead of sampling.

To tackle the exponentially large search space, we use a stabilized auto-encoding objective and a structure tweak procedure to iteratively reduce and refine the search space.

We propose a framework that learns to encode knowledge symbolically and generate programs to reason about the encoded knowledge.

The authors propose the N-Gram machine to answer questions over long documents.

This paper presents the n-gram machine, a model that encodes sentences into simple symbolic representations which can be queried efficiently.

LEARNING TO ORGANIZE KNOWLEDGE WITH N-GRAM MACHINES


We propose to use a meta-learning objective that maximizes the speed of transfer on a modified distribution to learn how to modularize acquired knowledge.

In particular, we focus on how to factor a joint distribution into appropriate conditionals, consistent with the causal directions.

We explain when this can work, using the assumption that the changes in distributions are localized (e.g. to one of the marginals, for example due to an intervention on one of the variables).

We prove that under this assumption of localized changes in causal mechanisms, the correct causal graph will tend to have only a few of its parameters with non-zero gradient, i.e. that need to be adapted (those of the modified variables).

We argue and observe experimentally that this leads to faster adaptation, and use this property to define a meta-learning surrogate score which, in addition to a continuous parametrization of graphs, would favour correct causal graphs.

Finally, motivated by the AI agent point of view (e.g. of a robot discovering its environment autonomously), we consider how the same objective can discover the causal variables themselves, as a transformation of observed low-level variables with no causal meaning.

Experiments in the two-variable case validate the proposed ideas and theoretical results.

This paper proposes a meta-learning objective based on speed of adaptation to transfer distributions to discover a modular decomposition and causal variables.

The paper shows that a model with the correct underlying structure will adapt faster to a causal intervention than a model with the incorrect structure.

In this work, the authors proposed a general and systematic framework of meta-transfer objective incorporating the causal structure learning under unknown interventions.

A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms

meta-transfer objective
causal
intervention
learn
structure


Continual learning is a longstanding goal of artificial intelligence, but is often counfounded by catastrophic forgetting that prevents neural networks from learning tasks sequentially.

Previous methods in continual learning have demonstrated how to mitigate catastrophic forgetting, and learn new tasks while retaining performance on the previous tasks.

We analyze catastrophic forgetting from the perspective of change in classifier likelihood and propose a simple L1 minimization criterion which can be adapted to different use cases.

We further investigate two ways to minimize forgetting as quantified by this criterion and propose strategies to achieve finer control over forgetting.

Finally, we evaluate our strategies on 3 datasets of varying difficulty and demonstrate improvements over previously known L2 strategies for mitigating catastrophic forgetting.

Another perspective on catastrophic forgetting

This paper introduces a framework for combatting catastrophic forgetting based upon changing the loss term to minimise changes in classifier likelihood, obtained via a Taylor series approximation.

This paper tries to solve the continual learning prolem by focusing on regularization approaches, and it proposes a L_1 strategy to mitigate the problem.

Minimizing Change in Classifier Likelihood to Mitigate Catastrophic Forgetting

change classifier likelihood
catastrophic forgetting
mitigate


Current face modeling methods using 3DMM suffer from the lack of local control.

combining local part-based 3DMM for the eyes, nose, mouth, ears, and facial mask regions.

uses a novel method to select the best eigenvectors from the local 3DMM to ensure that the combined 3DMM is expressive

The editing controls we provide to the user are intuitive as they are extracted from

anthropometric measurements found in the literature.

Out of a large set of possible anthropometric measurements, we filter the

ones that have meaningful generative power given the face data set.

We bind the measurements to the part-based 3DMM through

mapping matrices derived from our data set of facial scans.

Our part-based 3DMM is compact yet accurate, and compared to

other 3DMM methods, it provides a new trade-off between local and global control.

We tested our approach on a data set of 135

scans used to derive the 3DMM, plus 19 scans that served for validation.

The results show that our part-based 3DMM approach

has excellent generative properties and allows intuitive local control to the user.

We propose an approach to construct realistic 3D facial morphable models (3DMM) that allows an intuitive facial attribute editing workflow by selecting the best sets of eigenvectors and anthropometric measurements.

Proposes a piecewise morphable model for human face meshes and also proposes a mapping between anthropometric measurements of the face and the parameters of the model in order to synthesize and edit faces with desired attributes.

This paper describes a method of part-based morphable facial model allowing for localized user control.

Part-Based 3D Face Morphable Model with Anthropometric Local Control

anthropometric measurement
morphable model
allow
3d
attribute
facial
edit
face
control


We review eight machine learning classification algorithms to analyze Electroencephalographic (EEG) signals in order to distinguish EEG patterns associated with five basic educational tasks.

There is a large variety of classifiers being used in this EEG-based Brain-Computer Interface (BCI) field.

While previous EEG experiments used several classifiers in the same experiments or reviewed different algorithms on datasets from different experiments, our approach focuses on review eight classifier categories on the same dataset, including linear classifiers, non-linear Bayesian classifiers, nearest neighbour classifiers, ensemble methods, adaptive classifiers, tensor classifiers, transfer learning and deep learning.

Besides, we intend to find an approach which can run smoothly on the current mainstream personal computers and smartphones.

The empirical evaluation demonstrated that Random Forest and LSTM (Long Short-Term Memory) outperform other approaches.

We used a data set which users were conducting five frequently-conduct learning-related tasks, including reading, writing, and typing.

Results showed that these best two algorithms could correctly classify different users with an accuracy increase of  5% to 9%, use each task independently.

Within each subject, the tasks could be recognized with an accuracy increase of  4% to 7%, compared with other approaches.

This work suggests that Random Forest could be a recommended approach (fast and accurate) for current mainstream hardware, while LSTM has the potential to be the first-choice approach when the mainstream computers and smartphones can process more data in a shorter time.

Two Algorithms outperformed eight others on a EEG-based BCI experiment

Ensemble methods and LSTM outperformed other eight machine learning classifiers in an EEG-based BCI experiment

eeg-based bci experiment
outperform eight


Multi-agent reinforcement learning offers a way to study how communication could emerge in communities of agents needing to solve specific problems.

In this paper, we study the emergence of communication in the negotiation environment, a semi-cooperative model of agent interaction.

We introduce two communication protocols - one grounded in the semantics of the game, and one which is a priori ungrounded.

We show that self-interested agents can use the pre-grounded communication channel to negotiate fairly, but are unable to effectively use the ungrounded, cheap talk channel to do the same.

However, prosocial agents do learn to use cheap talk to find an optimal negotiating strategy, suggesting that cooperation is necessary for language to emerge.

We also study communication behaviour in a setting where one agent interacts with agents in a community with different levels of prosociality and show how agent identifiability can aid negotiation.

We teach agents to negotiate using only reinforcement learning; selfish agents can do so, but only using a trustworthy communication channel, and prosocial agents can negotiate using cheap talk.

The authors describe a variant of the negotation game with the consideration of a secondary communication channel for cheap talk, finding that the secondary channel improves negotation outcomes.

This paper explores how agents can learn to communicate to solve a negotiation task and find that prosocial agents are able to learn to ground symbols using RL, but self-interested agents are not.

Examines problems of how agents can use communication to maximise their rewards in a simple negotiation game.

Emergent Communication through Negotiation

agent
cheap talk
communication channel
find
prosocial
game
negotiation


The goal of few-shot learning is to learn a classifier that generalizes well even when trained with a limited number of training instances per class.

The recently introduced meta-learning approaches tackle this problem by learning a generic classifier across a large number of multiclass classification tasks and generalizing the model to a new task.

Yet, even with such meta-learning, the low-data problem in the novel classification task still remains.

In this paper, we propose Transductive Propagation Network (TPN), a novel meta-learning framework for transductive inference that classifies the entire test set at once to alleviate the low-data problem.

Specifically, we propose to learn to propagate labels from labeled instances to unlabeled test instances, by learning a graph construction module that exploits the manifold structure in the data.

TPN jointly learns both the parameters of feature embedding and the graph construction in an end-to-end manner.

We validate TPN on multiple benchmark datasets, on which it largely outperforms existing few-shot learning approaches and achieves the state-of-the-art results.

We propose a novel meta-learning framework for transductive inference that classifies the entire test set at once to alleviate the low-data problem.

This paper proposes to address few-shot learning in a transductive way by learning a label propagation model in an end-to-end manner, the first to learn label propagation for transductive few-shot learning and produced effective empirical results.

This paper proposes a meta-learning framework that leverages unlabeled data by learning the graph-based label propogation in an end-to-end manner.

Studies few-host learning in a transductive setting: using meta learning to learn to propagate labels from training samples to test samples.

LEARNING TO PROPAGATE LABELS: TRANSDUCTIVE PROPAGATION NETWORK FOR FEW-SHOT LEARNING

end-to-end manner
learn propagate label
learning transductive
meta-learning framework
test
few-shot
propagation


We describe the use of an automated scheduling system for observation policy design and to schedule operations of the NASA (National Aeronautics and Space Administration) ECOSystem Spaceborne Thermal Radiometer Experiment on Space Station (ECOSTRESS).

We describe the adaptation of the Compressed Large-scale Activity Scheduler and Planner (CLASP) scheduling system to the ECOSTRESS scheduling problem, highlighting multiple use cases for automated scheduling and several challenges for the scheduling technology: handling long-term campaigns with changing information, Mass Storage Unit Ring Buffer operations challenges, and orbit uncertainty.

The described scheduling system has been used for operations of the ECOSTRESS instrument since its nominal operations start July 2018 and is expected to operate until mission end in Summer 2019.

We describe the use of an automated scheduling system for observation policy design and to schedule operations of NASA's ECOSTRESS mission.

This paper presents an adaptation of an automated scheduling system, CLASP, to target an EO experiment (ECOSTRESS) on the ISS.

Automated Science Scheduling for the ECOSTRESS Mission

automate scheduling system
ecostress mission


Adversarial examples are modified samples that preserve original image structures but deviate classifiers.

Researchers have put efforts into developing methods for generating adversarial examples and finding out origins.

Past research put much attention on decision boundary changes caused by these methods.

This paper, in contrast, discusses the origin of adversarial examples from a more underlying knowledge representation point of view.

Human beings can learn and classify prototypes as well as transformations of objects.

While neural networks store learned knowledge in a more hybrid way of combining all prototypes and transformations as a whole distribution.

Hybrid storage may lead to lower distances between different classes so that small modifications can mislead the classifier.

A one-step distribution imitation method is designed to imitate distribution of the nearest different class neighbor.

Experiments show that simply by imitating distributions from a training set without any knowledge of the classifier can still lead to obvious impacts on classification results from deep networks.

It also implies that adversarial examples can be in more forms than small perturbations.

Potential ways of alleviating adversarial examples are discussed from the representation point of view.

The first path is to change the encoding of data sent to the training step.

Training data that are more prototypical can help seize more robust and accurate structural knowledge.

The second path requires constructing learning frameworks with improved representations.

Hybird storage and representation of learned knowledge may be a reason for adversarial examples.

Explaining Adversarial Examples with Knowledge Representation

adversarial example
knowledge
representation


Differently from the popular Deep Q-Network (DQN) learning, Alternating Q-learning (AltQ) does not fully fit a target Q-function at each iteration, and is generally known to be unstable and inefficient.

Limited applications of AltQ mostly rely on substantially altering the algorithm architecture in order to improve its performance.

Although Adam appears to be a natural solution, its performance in AltQ has rarely been studied before.

In this paper, we first provide a solid exploration on how well AltQ performs with Adam.

We then take a further step to improve the implementation by adopting the technique of parameter restart.

More specifically, the proposed algorithms are tested on a batch of Atari 2600 games and exhibit superior performance than the DQN learning method.

The convergence rate of the slightly modified version of the proposed algorithms is characterized under the linear function approximation.

To the best of our knowledge, this is the first theoretical study on the Adam-type algorithms in Q-learning.

New Experiments and Theory for Adam Based Q-Learning

This paper provides a convergence result for traditional Q-learning with linear function approximation when using an Adam-like update.

This paper describes a method to improve the AltQ algorithm by using a combination of an Adam optimizer and regularly restarting the internal parameters of the Adam optimizer.

CAN ALTQ LEARN FASTER: EXPERIMENTS AND THEORY


In search for more accurate predictive models, we customize capsule networks for the learning to diagnose problem.

We also propose Spectral Capsule Networks, a novel variation of capsule networks, that converge faster than capsule network with EM routing.

Spectral capsule networks  consist of spatial coincidence filters that detect entities based on the alignment of extracted features on a one-dimensional linear subspace.

Experiments on a public benchmark learning to diagnose dataset not only shows the success of capsule networks on this task, but also confirm the faster convergence of the spectral capsule networks.

A new capsule network that converges faster on our healthcare benchmark experiments.

Presents a variant of capsule networks that instead of using EM routing employs a linear subspace spanned by the dominant eigenvector on the weighted votes matrix from the previous capsule.

The paper proposes an improved routing method, which employs tools of eigendecomposition to find capsule activation and pose.

Spectral Capsule Networks