In order to mimic the human ability of continual acquisition and transfer of knowledge across various tasks, a learning system needs the capability for life-long learning, effectively utilizing the previously acquired skills.

As such, the key challenge is to transfer and generalize the knowledge learned from one task to other tasks, avoiding interference from previous knowledge and improving the overall performance.

In this paper, within the continual learning paradigm, we introduce a method that effectively forgets the less useful data samples continuously across different tasks.

The method uses statistical leverage score information to measure the importance of the data samples in every task and adopts frequent directions approach to enable a life-long learning property.

This effectively maintains a constant training size across all tasks.

We first provide some mathematical intuition for the method and then demonstrate its effectiveness with experiments on variants of MNIST and CIFAR100 datasets.

A new method uses statistical leverage score information to measure the importance of the data samples in every task and adopts frequent directions approach to enable a life-long learning property.

Lifelong Learning via Online Leverage Score Sampling


Convolutional neural networks (CNNs) are inherently equivariant to translation.

Efforts to embed other forms of equivariance have concentrated solely on rotation.

We expand the notion of equivariance in CNNs through the Polar Transformer Network (PTN).

PTN combines ideas from the Spatial Transformer Network (STN) and canonical coordinate representations.

The result is a network invariant to translation and equivariant to both rotation and scale.

PTN is trained end-to-end and composed of three distinct stages: a polar origin predictor, the newly introduced polar transformer module and a classifier.

PTN achieves state-of-the-art on rotated MNIST and the newly introduced SIM2MNIST dataset, an MNIST variation obtained by adding clutter and perturbing digits with translation, rotation and scaling.

The ideas of PTN are extensible to 3D which we demonstrate through the Cylindrical Transformer Network.

We learn feature maps invariant to translation, and equivariant to rotation and scale.

Polar Transformer Networks


Meta-learning allows an intelligent agent to leverage prior learning episodes as a basis for quickly improving performance on a novel task.

Bayesian hierarchical modeling provides a theoretical framework for formalizing meta-learning as inference for a set of parameters that are shared across tasks.

Here, we reformulate the model-agnostic meta-learning algorithm (MAML) of Finn et al. (2017) as a method for probabilistic inference in a hierarchical Bayesian model.

In contrast to prior methods for meta-learning via hierarchical Bayes, MAML is naturally applicable to complex function approximators through its use of a scalable gradient descent procedure for posterior inference.

Furthermore, the identification of MAML as hierarchical Bayes provides a way to understand the algorithm’s operation as a meta-learning procedure, as well as an opportunity to make use of computational strategies for efficient inference.

We use this opportunity to propose an improvement to the MAML algorithm that makes use of techniques from approximate inference and curvature estimation.

A specific gradient-based meta-learning algorithm, MAML, is equivalent to an inference procedure in a hierarchical Bayesian model. We use this connection to improve MAML via methods from approximate inference and curvature estimation.

Recasting Gradient-Based Meta-Learning as Hierarchical Bayes


This work provides an automatic machine learning (AutoML) modelling architecture called Autostacker.

Autostacker improves the prediction accuracy of machine learning baselines by utilizing an innovative hierarchical stacking architecture and an efficient parameter search algorithm.

Neither prior domain knowledge about the data nor feature preprocessing is needed.

We significantly reduce the time of AutoML with a naturally inspired algorithm - Parallel Hill Climbing (PHC).

By parallelizing PHC, Autostacker can provide candidate pipelines with sufficient prediction accuracy within a short amount of time.

These pipelines can be used as is or as a starting point for human experts to build on.

By focusing on the modelling process, Autostacker breaks the tradition of following fixed order pipelines by exploring not only single model pipeline but also innovative combinations and structures.

As we will show in the experiment section, Autostacker achieves significantly better performance both in terms of test accuracy and time cost comparing with human initial trials and recent popular AutoML system.

Automate machine learning system with efficient search algorithm and innovative structure to provide better model baselines.

Autostacker: an Automatic Evolutionary Hierarchical  Machine Learning System


Surrogate models can be used to accelerate approximate Bayesian computation (ABC).

In one such framework the discrepancy between simulated and observed data is modelled with a Gaussian process.

So far principled strategies have been proposed only for sequential selection of the simulation locations.

To address this limitation, we develop Bayesian optimal design strategies to parallellise the expensive simulations.

We also touch the problem of quantifying the uncertainty of the ABC posterior due to the limited budget of simulations.

We propose principled batch Bayesian experimental design strategies and a method for uncertainty quantification of the posterior summaries in a Gaussian process surrogate-based approximate Bayesian computation framework.

Batch simulations and uncertainty quantification in Gaussian process surrogate-based approximate Bayesian computation

uncertainty quantification
gaussian process
approximate bayesian computation
batch


Randomly initialized first-order optimization algorithms are the method of choice for solving many high-dimensional nonconvex problems in machine learning, yet general theoretical guarantees cannot rule out convergence to critical points of poor objective value.

For some highly structured nonconvex problems however, the success of gradient descent can be understood by studying the geometry of the objective.

We study one such problem -- complete orthogonal dictionary learning, and provide converge guarantees for randomly initialized gradient descent to the neighborhood of a global optimum.

The resulting rates scale as low order polynomials in the dimension even though the objective possesses an exponential number of saddle points.

This efficient convergence can be viewed as a consequence of negative curvature normal to the stable manifolds associated with saddle points, and we provide evidence that this feature is shared by other nonconvex problems of importance as well.

We provide an efficient convergence rate for gradient descent on the complete orthogonal dictionary learning objective based on a geometric analysis.

Efficient Dictionary Learning with Gradient Descent

gradient descent
dictionary learn
efficient


Coding theory is a central discipline underpinning wireline and wireless modems that are the workhorses of the information age.

Progress in coding theory is largely driven by individual human ingenuity with sporadic breakthroughs over the past century.

In this paper we study whether it is possible to automate the discovery of decoding algorithms via deep learning.

We study a family of sequential codes parametrized by recurrent neural network (RNN) architectures.

We show that cre- atively designed and trained RNN architectures can decode well known sequential codes such as the convolutional and turbo codes with close to optimal performance on the additive white Gaussian noise (AWGN) channel, which itself is achieved by breakthrough algorithms of our times (Viterbi and BCJR decoders, representing dynamic programing and forward-backward algorithms).

We show strong gen- eralizations, i.e., we train at a specific signal to noise ratio and block length but test at a wide range of these quantities, as well as robustness and adaptivity to deviations from the AWGN setting.

We show that creatively designed and trained RNN architectures can decode well known sequential codes and achieve close to optimal performances.

Communication Algorithms via Deep Learning


Adam is shown not being able to converge to the optimal solution in certain cases.

Researchers recently propose several algorithms to avoid the issue of non-convergence of Adam, but their efficiency turns out to be unsatisfactory in practice.

In this paper, we provide a new insight into the non-convergence issue of Adam as well as other adaptive learning rate methods.

We argue that there exists an inappropriate correlation between gradient $g_t$ and the second moment term $v_t$ in Adam ($t$ is the timestep), which results in that a large gradient is likely to have small step size while a small gradient may have a large step size.

We demonstrate that such unbalanced step sizes are the fundamental cause of non-convergence of Adam, and we further prove that decorrelating $v_t$ and $g_t$ will lead to unbiased step size for each gradient, thus solving the non-convergence problem of Adam.

Finally, we propose AdaShift, a novel adaptive learning rate method that decorrelates $v_t$ and $g_t$ by temporal shifting, i.e., using temporally shifted gradient $g_{t-n}$ to calculate $v_t$.

The experiment results demonstrate that AdaShift is able to address the non-convergence issue of Adam, while still maintaining a competitive performance with Adam in terms of both training speed and generalization.

We analysis and solve the non-convergence issue of Adam.

AdaShift: Decorrelation and Convergence of Adaptive Learning Rate Methods


Most domain adaptation methods consider the problem of transferring knowledge to the target domain from a single source dataset.

However, in practical applications, we typically have access to multiple sources.

In this paper we propose the first approach for Multi-Source Domain Adaptation (MSDA) based on Generative Adversarial Networks.

Our method is inspired by the observation that the appearance of a given image depends on three factors: the domain, the style (characterized in terms of low-level features variations) and the content.

For this reason we propose to project the image features onto a space where only the dependence from the content is kept, and then re-project this invariant representation onto the pixel space using the target domain and style.

In this way, new labeled images can be generated which are used to train a final target classifier.

We test our  approach using common MSDA benchmarks, showing that it outperforms state-of-the-art methods.

In this paper we propose generative method for multisource domain adaptation based on decomposition of content, style and domain factors.

Generative Multi Source Domain Adaptation


Inferring the most likely configuration for a subset of variables of a joint distribution given the remaining ones – which we refer to as co-generation – is an important challenge that is computationally demanding for all but the simplest settings.

This task has received a considerable amount of attention, particularly for classical ways of modeling distributions like structured prediction.

In contrast, almost nothing is known about this task when considering recently proposed techniques for modeling high-dimensional distributions, particularly generative adversarial nets (GANs).

Therefore, in this paper, we study the occurring challenges for co-generation with GANs.

To address those challenges we develop an annealed importance sampling (AIS) based Hamiltonian Monte Carlo (HMC) co-generation algorithm.

The presented approach significantly outperforms classical gradient-based methods on synthetic data and on CelebA.

Using annealed importance sampling on the co-generation problem.

Co-Generation with GANs using AIS based HMC


Implicit probabilistic models are models defined naturally in terms of a sampling procedure and often induces a likelihood function that cannot be expressed explicitly.

We develop a simple method for estimating parameters in implicit models that does not require knowledge of the form of the likelihood function or any derived quantities, but can be shown to be equivalent to maximizing likelihood under some conditions.

Our result holds in the non-asymptotic parametric setting, where both the capacity of the model and the number of data examples are finite.

We also demonstrate encouraging experimental results.

We develop a new likelihood-free parameter estimation method that is equivalent to maximum likelihood under some conditions

Implicit Maximum Likelihood Estimation


While most approaches to the problem of Inverse Reinforcement Learning (IRL) focus on estimating a reward function that best explains an expert agent’s policy or demonstrated behavior on a control task, it is often the case that such behavior is more succinctly represented by a simple reward combined with a set of hard constraints.

In this setting, the agent is attempting to maximize cumulative rewards subject to these given constraints on their behavior.

We reformulate the problem of IRL on Markov Decision Processes (MDPs) such that, given a nominal model of the environment and a nominal reward function, we seek to estimate state, action, and feature constraints in the environment that motivate an agent’s behavior.

Our approach is based on the Maximum Entropy IRL framework, which allows us to reason about the likelihood of an expert agent’s demonstrations given our knowledge of an MDP.

Using our method, we can infer which constraints can be added to the MDP to most increase the likelihood of observing these demonstrations.

We present an algorithm which iteratively infers the Maximum Likelihood Constraint to best explain observed behavior, and we evaluate its efficacy using both simulated behavior and recorded data of humans navigating around an obstacle.

Our method infers constraints on task execution by leveraging the principle of maximum entropy to quantify how demonstrations differ from expected, un-constrained behavior.

Maximum Likelihood Constraint Inference for Inverse Reinforcement Learning


The success of reinforcement learning in the real world has been limited to instrumented laboratory scenarios, often requiring arduous human supervision to enable continuous learning.

In this work, we discuss the required elements of a robotic system that can continually and autonomously improve with data collected in the real world, and propose a particular instantiation of such a system.

Subsequently, we investigate a number of challenges of learning without instrumentation -- including the lack of episodic resets, state estimation, and hand-engineered rewards -- and propose simple, scalable solutions to these challenges.

We demonstrate the efficacy of our proposed system on dexterous robotic manipulation tasks in simulation and the real world, and also provide an insightful analysis and ablation study of the challenges associated with this learning paradigm.

System to learn robotic tasks in the real world with reinforcement learning without instrumentation

The Ingredients of Real World Robotic Reinforcement Learning

real world
reinforcement learning
robotic


We study the problem of cross-lingual voice conversion in non-parallel speech corpora and one-shot learning setting.

Most prior work require either parallel speech corpora or enough amount of training data from a target speaker.

However, we convert an arbitrary sentences of an arbitrary source speaker to target speaker's given only one target speaker training utterance.

To achieve this, we formulate the problem as learning disentangled speaker-specific and context-specific representations and follow the idea of [1] which uses Factorized Hierarchical Variational Autoencoder (FHVAE).

After training FHVAE on multi-speaker training data, given arbitrary source and target speakers' utterance, we estimate those latent representations and then reconstruct the desired utterance of converted voice to that of target speaker.

We use multi-language speech corpus to learn a universal model that works for all of the languages.

We investigate the use of a one-hot language embedding to condition the model on the language of the utterance being queried and show the effectiveness of the approach.

We conduct voice conversion experiments with varying size of training utterances and it was able to achieve reasonable performance with even just one training utterance.

We also investigate the effect of using or not using the language conditioning.

Furthermore, we visualize the embeddings of the different languages and sexes.

Finally, in the subjective tests, for one language and cross-lingual voice conversion, our approach achieved moderately better or comparable results compared to the baseline in speech quality and similarity.

We use a Variational Autoencoder to separate style and content, and achieve voice conversion by modifying style embedding and decoding. We investigate using a multi-language speech corpus and investigate its effects.

Investigation of using disentangled and interpretable representations with language conditioning for cross-lingual voice conversion


We propose an end-to-end-trainable attention module for convolutional neural network (CNN) architectures built for image classification.

The module takes as input the 2D feature vector maps which form the intermediate representations of the input image at different stages in the CNN pipeline, and outputs a 2D matrix of scores for each map.

Standard CNN architectures are modified through the incorporation of this module, and trained under the constraint that a convex combination of the intermediate 2D feature vectors, as parametrised by the score matrices, must alone be used for classification.

Incentivised to amplify the relevant and suppress the irrelevant or misleading, the scores thus assume the role of attention values.

Our experimental observations provide clear evidence to this effect: the learned attention maps neatly highlight the regions of interest while suppressing background clutter.

Consequently, the proposed function is able to bootstrap standard CNN architectures for the task of image classification, demonstrating superior generalisation over 6 unseen benchmark datasets.

When binarised, our attention maps outperform other CNN-based attention maps, traditional saliency maps, and top object proposals for weakly supervised segmentation as demonstrated on the Object Discovery dataset.

We also demonstrate improved robustness against the fast gradient sign method of adversarial attack.

The paper proposes a method for forcing CNNs to leverage spatial attention in learning more object-centric representations that perform better in various respects.

Learn to Pay Attention


Recurrent neural network(RNN) is an effective neural network in solving very complex supervised and unsupervised tasks.There has been a significant improvement in RNN field such as natural language processing, speech processing, computer vision and other multiple domains.

This paper deals with RNN application on different use cases like Incident Detection , Fraud Detection , and Android Malware Classification.

The best performing neural network architecture is chosen

by conducting different chain of experiments for different network parameters and structures.The network is run up to 1000 epochs with learning rate set in the range of 0.01 to 0.5.Obviously, RNN performed very well when compared to classical machine learning algorithms.

This is mainly possible because RNNs implicitly extracts the underlying features and also identifies the characteristics of the data.

RNNSecureNet: Recurrent neural networks for Cybersecurity use-cases

recurrent neural network cybersecurity use-case


Anatomical studies demonstrate that brain reformats input information to generate reliable responses for performing computations.

However, it remains unclear how neural circuits encode complex spatio-temporal patterns.

We show that neural dynamics are strongly influenced by the phase alignment between the input and the spontaneous chaotic activity.

Input alignment along the dominant chaotic projections causes the chaotic trajectories to become stable channels (or attractors), hence, improving the computational capability of a recurrent network.

Using mean field analysis, we derive the impact of input alignment on the overall stability of attractors formed.

Our results indicate that input alignment determines the extent of intrinsic noise suppression and hence, alters the attractor state stability, thereby controlling the network's inference ability.

Input Structuring along Chaos for Stability

Input Alignment along Chaotic directions increases Stability in Recurrent Neural Networks


Generative adversarial networks are a learning framework that rely on training a discriminator to estimate a measure of difference between a target and generated distributions.

GANs, as normally formulated, rely on the generated samples being completely differentiable w.r.t.

the generative parameters, and thus do not work for discrete data.

We introduce a method for training GANs with discrete data that uses the estimated difference measure from the discriminator to compute importance weights for generated samples, thus providing a policy gradient for training the generator.

The importance weights have a strong connection to the decision boundary of the discriminator, and we call our method boundary-seeking GANs (BGANs).

We demonstrate the effectiveness of the proposed algorithm with discrete image and character-based natural language generation.

In addition, the boundary-seeking objective extends to continuous data, which can be used to improve stability of training, and we demonstrate this on Celeba, Large-scale Scene Understanding (LSUN) bedrooms, and Imagenet without conditioning.

We address training GANs with discrete data by formulating a policy gradient that generalizes across f-divergences

Boundary Seeking GANs


Policy gradient methods have enjoyed great success in deep reinforcement learning but suffer from high variance of gradient estimates.

The high variance problem is particularly exasperated in problems with long horizons or high-dimensional action spaces.

To mitigate this issue, we derive a bias-free action-dependent baseline for variance reduction which fully exploits the structural form of the stochastic policy itself and does not make any additional assumptions about the MDP.

We demonstrate and quantify the benefit of the action-dependent baseline through both theoretical analysis as well as numerical results, including an analysis of the suboptimality of the optimal state-dependent baseline.

The result is a computationally efficient policy gradient algorithm, which scales to high-dimensional control problems, as demonstrated by a synthetic 2000-dimensional target matching task.

Our experimental results indicate that action-dependent baselines allow for faster learning on standard reinforcement learning benchmarks and high-dimensional hand manipulation and synthetic tasks.

Finally, we show that the general idea of including additional information in baselines for improved variance reduction can be extended to partially observed and multi-agent tasks.

Action-dependent baselines can be bias-free and yield greater variance reduction than state-only dependent baselines for policy gradient methods.

Variance Reduction for Policy Gradient with Action-Dependent Factorized Baselines

policy gradient
variance reduction
baseline
action-dependent


The cost of annotating training data has traditionally been a bottleneck for supervised learning approaches.

The problem is further exacerbated when supervised learning is applied to a number of correlated tasks simultaneously since the amount of labels required scales with the number of tasks.

To mitigate this concern, we propose an active multitask learning algorithm that achieves knowledge transfer between tasks.

The approach forms a so-called committee for each task that jointly makes decisions and directly shares data across similar tasks.

Our approach reduces the number of queries needed during training while maintaining high accuracy on test data.

Empirical results on benchmark datasets show significant improvements on both accuracy and number of query requests.

We propose an active multitask learning algorithm that achieves knowledge transfer between tasks.

Active Multitask Learning with Committees


Detection of photo manipulation relies on subtle statistical traces, notoriously removed by aggressive lossy compression employed online.

We demonstrate that end-to-end modeling of complex photo dissemination channels allows for codec optimization with explicit provenance objectives.

We design a lightweight trainable lossy image codec, that delivers competitive rate-distortion performance, on par with best hand-engineered alternatives, but has lower computational footprint on modern GPU-enabled platforms.

Our results show that significant improvements in manipulation detection accuracy are possible at fractional costs in bandwidth/storage.

Our codec improved the accuracy from 37% to 86% even at very low bit-rates, well below the practicality of JPEG (QF 20).

We learn an efficient lossy image codec that can be optimized to facilitate reliable photo manipulation detection at fractional cost in payload/quality and even at low bitrates.

Quantifying the Cost of Reliable Photo Authentication via High-Performance Learned Lossy Representations


Recurrent Neural Networks have long been the dominating choice for sequence modeling.

However, it severely suffers from two issues: impotent in capturing very long-term dependencies and unable to parallelize the sequential computation procedure.

Therefore, many non-recurrent sequence models that are built on convolution and attention operations have been proposed recently.

Notably, models with multi-head attention such as Transformer have demonstrated extreme effectiveness in capturing long-term dependencies in a variety of sequence modeling tasks.

Despite their success, however, these models lack necessary components to model local structures in sequences and heavily rely on position embeddings that have limited effects and require a considerable amount of design efforts.

In this paper, we propose the R-Transformer which enjoys the advantages of both RNNs and the multi-head attention mechanism while avoids their respective drawbacks.

The proposed model can effectively capture both local structures and global long-term dependencies in sequences without any use of position embeddings.

We evaluate R-Transformer through extensive experiments with data from a wide range of domains and the empirical results show that R-Transformer outperforms the state-of-the-art methods by a large margin in most of the tasks.

This paper proposes an effective generic sequence model which leverages the strengths of both RNNs and Multi-head attention.

R-TRANSFORMER: RECURRENT NEURAL NETWORK ENHANCED TRANSFORMER


Many tasks in natural language processing and related domains require high precision output that obeys dataset-specific constraints.

This level of fine-grained control can be difficult to obtain in large-scale neural network models.

In this work, we propose a structured latent-variable approach that adds discrete control states within a standard autoregressive neural paradigm.

Under this formulation, we can include a range of rich, posterior constraints to enforce task-specific knowledge that is effectively trained into the neural model.

This approach allows us to provide arbitrary grounding of internal model decisions, without sacrificing any representational power of neural models.

Experiments consider applications of this approach for text generation and part-of-speech induction.

For natural language generation, we find that this method improves over standard benchmarks, while also providing fine-grained control.

A structured latent-variable approach that adds discrete control states within a standard autoregressive neural paradigm to provide arbitrary grounding of internal model decisions, without sacrificing any representational power of neural models.

Posterior Control of Blackbox Generation


Suppose a deep classification model is trained with samples that need to be kept private for privacy or confidentiality reasons.

In this setting, can an adversary obtain the private samples if the classification model is given to the adversary?

We call this reverse engineering against the classification model the Classifier-to-Generator (C2G) Attack.

This situation arises when the classification model is embedded into mobile devices for offline prediction (e.g., object recognition for the automatic driving car and face recognition for mobile phone authentication).

For C2G attack, we introduce a novel GAN, PreImageGAN.

In PreImageGAN, the generator is designed to estimate the the sample distribution conditioned by the preimage of classification model $f$, $P(X|f(X)=y)$, where $X$ is the random variable on the sample space and $y$ is the probability vector representing the target label arbitrary specified by the adversary.

In experiments, we demonstrate PreImageGAN works successfully with hand-written character recognition and face recognition.

In character recognition, we show that, given a recognition model of hand-written digits, PreImageGAN allows the adversary to extract alphabet letter images without knowing that the model is built for alphabet letter images.

In face recognition, we show that, when an adversary obtains a face recognition model for a set of individuals, PreImageGAN allows the adversary to extract face images of specific individuals contained in the set, even when the adversary has no knowledge of the face of the individuals.

Estimation of training data distribution from trained classifier using GAN.

Classifier-to-Generator Attack: Estimation of Training Data Distribution from Classifier

estimation training datum distribution
classifier


The goal of standard compressive sensing is to estimate an unknown vector from linear measurements under the assumption of sparsity in some basis.

Recently, it has been shown that significantly fewer measurements may be required if the sparsity assumption is replaced by the assumption that the unknown vector lies near the range of a suitably-chosen generative model.

In particular, in (Bora {\em et al.}, 2017) it was shown that roughly $O(k\log L)$ random Gaussian measurements suffice for accurate recovery when the $k$-input generative model is bounded and $L$-Lipschitz, and that $O(kd \log w)$ measurements suffice for $k$-input ReLU networks with depth $d$ and width $w$.  In this paper, we establish corresponding algorithm-independent lower bounds on the sample complexity using tools from minimax statistical analysis.

In accordance with the above upper bounds, our results are summarized as follows: (i) We construct an $L$-Lipschitz generative model capable of generating group-sparse signals, and show that the resulting necessary number of measurements is $\Omega(k \log L)$;

(ii) Using similar ideas, we construct two-layer ReLU networks of high width requiring $\Omega(k \log w)$ measurements, as well as lower-width deep ReLU networks requiring $\Omega(k d)$ measurements.

As a result, we establish that the scaling laws derived in (Bora {\em et al.}, 2017) are optimal or near-optimal in the absence of further assumptions.

We establish that the scaling laws derived in (Bora et al., 2017) are optimal or near-optimal in the absence of further assumptions.

Sample Complexity Lower Bounds for Compressive Sensing with Generative Models


Discovering and exploiting the causal structure in the environment is a crucial challenge for intelligent agents.

Here we explore whether modern deep reinforcement learning can be used to train agents to perform causal reasoning.

We adopt a meta-learning approach, where the agent learns a policy for conducting experiments via causal interventions, in order to support a subsequent task which rewards making accurate causal inferences.We also found the agent could make sophisticated counterfactual predictions, as well as learn to draw causal inferences from purely observational data.

Though powerful formalisms for causal reasoning have been developed, applying them in real-world domains can be difficult because fitting to large amounts of high dimensional data often requires making idealized assumptions.

Our results suggest that causal reasoning in complex settings may benefit from powerful learning-based approaches.

More generally, this work may offer new strategies for structured exploration in reinforcement learning, by providing agents with the ability to perform—and interpret—experiments.

meta-learn a learning algorithm capable of causal reasoning

Causal Reasoning from Meta-reinforcement learning


Sentiment classification is an active research area with several applications including analysis of political opinions, classifying comments, movie reviews, news reviews and product reviews.

To employ rule based sentiment classification, we require sentiment lexicons.

However, manual construction of sentiment lexicon is time consuming and costly for resource-limited languages.

To bypass manual development time and costs, we tried to build Amharic Sentiment Lexicons relying on corpus based approach.

The intention of this approach is to handle sentiment terms specific to Amharic language from Amharic Corpus.

Small set of seed terms are manually prepared from three parts of speech such as noun, adjective and verb.

We developed algorithms for constructing Amharic sentiment lexicons automatically from Amharic news corpus.

Corpus based approach is proposed relying on the word co-occurrence distributional embedding including frequency based embedding (i.e. Positive Point-wise Mutual Information PPMI).

First we build word-context unigram frequency count matrix and transform it to point-wise mutual Information matrix.

Using this matrix, we computed the cosine distance of mean vector of seed lists and each word in the corpus vocabulary.

Based on the threshold value, the top closest words to the mean vector of seed list are added to the lexicon.

Then the mean vector of the new sentiment seed list is updated and process is repeated until we get sufficient terms in the lexicon.

Using PPMI with threshold value of 100 and 200, we got corpus based Amharic Sentiment lexicons of size 1811 and 3794 respectively by expanding 519 seeds.

Finally, the lexicon generated in corpus based approach is evaluated.

Corpus based Algorithm is developed generate Amharic Sentiment lexicon relying on corpus

Corpus Based Amharic Sentiment Lexicon Generation


Optimistic initialisation is an effective strategy for efficient exploration in reinforcement learning (RL).

In the tabular case, all provably efficient model-free algorithms rely on it.

However, model-free deep RL algorithms do not use optimistic initialisation despite taking inspiration from these provably efficient tabular algorithms.

In particular, in scenarios with only positive rewards, Q-values are initialised at their lowest possible values due to commonly used network initialisation schemes, a pessimistic initialisation.

Merely initialising the network to output optimistic Q-values is not enough, since we cannot ensure that they remain optimistic for novel state-action pairs, which is crucial for exploration.

We propose a simple count-based augmentation to pessimistically initialised Q-values that separates the source of optimism from the neural network.

We show that this scheme is provably efficient in the tabular setting and extend it to the deep RL setting.

Our algorithm, Optimistic Pessimistically Initialised Q-Learning (OPIQ), augments the Q-value estimates of a DQN-based agent with count-derived bonuses to ensure optimism during both action selection and bootstrapping.

We show that OPIQ outperforms non-optimistic DQN variants that utilise a pseudocount-based intrinsic motivation in hard exploration tasks, and that it predicts optimistic estimates for novel state-action pairs.

We augment the Q-value estimates with a count-based bonus that ensures optimism during action selection and bootstrapping, even if the Q-value estimates are pessimistic.

Optimistic Exploration even with a Pessimistic Initialisation


Model-free deep reinforcement learning (RL) algorithms have been demonstrated on a range of challenging decision making and control tasks.

However, these methods typically suffer from two major challenges: very high sample complexity and brittle convergence properties, which necessitate meticulous hyperparameter tuning.

Both of these challenges severely limit the applicability of such methods to complex, real-world domains.

In this paper, we propose soft actor-critic, an off-policy actor-critic deep RL algorithm based on the maximum entropy reinforcement learning framework.

In this framework, the actor aims to maximize expected reward while also maximizing entropy - that is, succeed at the task while acting as randomly as possible.

Prior deep RL methods based on this framework have been formulated as either off-policy Q-learning, or on-policy policy gradient methods.

By combining off-policy updates with a stable stochastic actor-critic formulation, our method achieves state-of-the-art performance on a range of continuous control benchmark tasks, outperforming prior on-policy and off-policy methods.

Furthermore, we demonstrate that, in contrast to other off-policy algorithms, our approach is very stable, achieving very similar performance across different random seeds.

We propose soft actor-critic, an off-policy actor-critic deep RL algorithm based on the maximum entropy reinforcement learning framework.

Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor

soft actor-critic
reinforcement learning
maximum entropy
off-policy
deep


In many settings, it is desirable to learn decision-making and control policies through learning or from expert demonstrations.

The most common approaches under this framework are Behaviour Cloning (BC), and Inverse Reinforcement Learning (IRL).

Recent methods for IRL have demonstrated the capacity to learn effective policies with access to a very limited set of demonstrations, a scenario in which BC methods often fail.

Unfortunately, directly comparing the algorithms for these methods does not provide adequate intuition for understanding this difference in performance.

This is the motivating factor for our work.

We begin by presenting $f$-MAX, a generalization of AIRL (Fu et al., 2018), a state-of-the-art IRL method.

$f$-MAX provides grounds for more directly comparing the objectives for LfD.

We demonstrate that $f$-MAX, and by inheritance AIRL, is a subset of the cost-regularized IRL framework laid out by Ho & Ermon (2016).

We conclude by empirically evaluating the factors of difference between various LfD objectives in the continuous control domain.

Distribution matching through divergence minimization provides a common ground for comparing adversarial Maximum-Entropy Inverse Reinforcement Learning methods to Behaviour Cloning.

Understanding the Relation Between Maximum-Entropy Inverse Reinforcement Learning and Behaviour Cloning

maximum-entropy inverse reinforcement learning


Value-based methods constitute a fundamental methodology in planning and deep reinforcement learning (RL).

In this paper, we propose to exploit the underlying structures of the state-action value function, i.e., Q function, for both planning and deep RL.

In particular, if the underlying system dynamics lead to some global structures of the Q function, one should be capable of inferring the function better by leveraging such structures.

Specifically, we investigate the low-rank structure, which widely exists for big data matrices.

We verify empirically the existence of low-rank Q functions in the context of control and deep RL tasks (Atari games).

As our key contribution, by leveraging Matrix Estimation (ME) techniques, we propose a general framework to exploit the underlying low-rank structure in Q functions, leading to a more efficient planning procedure for classical control, and additionally, a simple scheme that can be applied to any value-based RL techniques to consistently achieve better performance on ''low-rank'' tasks.

Extensive experiments on control tasks and Atari games confirm the efficacy of our approach.

We propose a generic framework that allows for exploiting the low-rank structure in both planning and deep reinforcement learning.

Harnessing Structures for Value-Based Planning and Reinforcement Learning

reinforcement learning
structure
planning


Learned representations of source code enable various software developer tools, e.g., to detect bugs or to predict program properties.

At the core of code representations often are word embeddings of identifier names in source code, because identifiers account for the majority of source code vocabulary and convey important semantic information.

Unfortunately, there currently is no generally accepted way of evaluating the quality of word embeddings of identifiers, and current evaluations are biased toward specific downstream tasks.

This paper presents IdBench, the first benchmark for evaluating to what extent word embeddings of identifiers represent semantic relatedness and similarity.

The benchmark is based on thousands of ratings gathered by surveying 500 software developers.

We use IdBench to evaluate state-of-the-art embedding techniques proposed for natural language, an embedding technique specifically designed for source code, and lexical string distance functions, as these are often used in current developer tools.

Our results show that the effectiveness of embeddings varies significantly across different embedding techniques and that the best available embeddings successfully represent semantic relatedness.

On the downside, no existing embedding provides a satisfactory representation of semantic similarities, e.g., because embeddings consider identifiers with opposing meanings as similar, which may lead to fatal mistakes in downstream developer tools.

IdBench provides a gold standard to guide the development of novel embeddings that address the current limitations.

A benchmark to evaluate neural embeddings of identifiers in source code.

Evaluating Semantic Representations of Source Code


Generative adversarial nets (GANs) are widely used to learn the data sampling process and their performance may heavily depend on the loss functions, given a limited computational budget.

This study revisits MMD-GAN that uses the maximum mean discrepancy (MMD) as the loss function for GAN and makes two contributions.

First, we argue that the existing MMD loss function may discourage the learning of fine details in data as it attempts to contract the discriminator outputs of real data.

To address this issue, we propose a repulsive loss function to actively learn the difference among the real data by simply rearranging the terms in MMD.

Second, inspired by the hinge loss, we propose a bounded Gaussian kernel to stabilize the training of MMD-GAN with the repulsive loss function.

The proposed methods are applied to the unsupervised image generation tasks on CIFAR-10, STL-10, CelebA, and LSUN bedroom datasets.

Results show that the repulsive loss function significantly improves over the MMD loss at no additional computational cost and outperforms other representative loss functions.

The proposed methods achieve an FID score of 16.21 on the CIFAR-10 dataset using a single DCGAN network and spectral normalization.

Rearranging the terms in maximum mean discrepancy yields a much better loss function for the discriminator of generative adversarial nets

Improving MMD-GAN Training with Repulsive Loss Function


Deep neural networks have shown incredible performance for inference tasks in a variety of domains.

Unfortunately, most current deep networks are enormous cloud-based structures that require significant storage space, which limits scaling of deep learning as a service (DLaaS) and use for on-device augmented intelligence.

This paper finds algorithms that directly use lossless compressed representations of deep feedforward networks (with synaptic weights drawn from discrete sets), to perform inference without full decompression.

The basic insight that allows less rate than naive approaches is the recognition that the bipartite graph layers of feedforward networks have a kind of permutation invariance to the labeling of nodes, in terms of inferential operation and that the inference operation depends locally on the edges directly connected to it.

We also provide experimental results of our approach on the MNIST dataset.

This paper finds algorithms that directly use lossless compressed representations of deep feedforward networks, to perform inference without full decompression.

Succinct Source Coding of Deep Neural Networks


Generative adversarial networks (GANs) form a generative modeling approach known for producing appealing samples, but they are notably difficult to train.

One common way to tackle this issue has been to propose new formulations of the GAN objective.

Yet, surprisingly few studies have looked at optimization methods designed for this adversarial training.

In this work, we cast GAN optimization problems in the general variational inequality framework.

Tapping into the mathematical programming literature, we counter some common misconceptions about the difficulties of saddle point optimization and propose to extend methods designed for variational inequalities to the training of GANs.

We apply averaging, extrapolation and a computationally cheaper variant that we call extrapolation from the past to the stochastic gradient method (SGD) and Adam.

We cast GANs in the variational inequality framework and import techniques from this literature to optimize GANs better; we give algorithmic extensions and empirically test their performance for training GANs.

A Variational Inequality Perspective on Generative Adversarial Networks


In order to efficiently learn with small amount of data on new tasks, meta-learning transfers knowledge learned from previous tasks to the new ones.

However, a critical challenge in meta-learning is the task heterogeneity which cannot be well handled by traditional globally shared meta-learning methods.

In addition, current task-specific meta-learning methods may either suffer from hand-crafted structure design or lack the capability to capture complex relations between tasks.

In this paper, motivated by the way of knowledge organization in knowledge bases, we propose an automated relational meta-learning (ARML) framework that automatically extracts the cross-task relations and constructs the meta-knowledge graph.

When a new task arrives, it can quickly find the most relevant structure and tailor the learned structure knowledge to the meta-learner.

As a result, the proposed framework not only addresses the challenge of task heterogeneity by a learned meta-knowledge graph, but also increases the model interpretability.

We conduct extensive experiments on 2D toy regression and few-shot image classification and the results demonstrate the superiority of ARML over state-of-the-art baselines.

Addressing task heterogeneity problem in meta-learning by introducing meta-knowledge graph

Automated Relational Meta-learning


In this paper, a deep boosting algorithm is developed to

learn more discriminative ensemble classifier by seamlessly combining a set of base deep CNNs (base experts)

with diverse capabilities, e.g., these base deep CNNs are

sequentially trained to recognize a set of

object classes in an easy-to-hard way according to their

Our experimental results have demonstrated

that our deep boosting algorithm can significantly improve the

accuracy rates on large-scale visual recognition.

A deep boosting algorithm is developed to learn more discriminative ensemble classifier by seamlessly combining a set of base deep CNNs.

Deep Boosting of Diverse Experts


We present a method for translating music across musical instruments and styles.

This method is based on unsupervised training of a multi-domain wavenet autoencoder, with a shared encoder and a domain-independent latent space that is trained end-to-end on waveforms.

Employing a diverse training dataset and large net capacity, the single encoder allows us to translate also from musical domains that were not seen during training.

We evaluate our method on a dataset collected from professional musicians, and achieve convincing translations.

We also study the properties of the obtained translation and demonstrate translating even from a whistle, potentially enabling the creation of instrumental music by untrained humans.

An automatic method for converting music between instruments and styles

A Universal Music Translation Network


Most existing defenses against adversarial attacks only consider robustness to L_p-bounded distortions.

In reality, the specific attack is rarely known in advance and adversaries are free to modify images in ways which lie outside any fixed distortion model; for example, adversarial rotations lie outside the set of L_p-bounded distortions.

In this work, we advocate measuring robustness against a much broader range of unforeseen attacks, attacks whose precise form is unknown during defense design.

We propose several new attacks and a methodology for evaluating a defense against a diverse range of unforeseen distortions.

First, we construct novel adversarial JPEG, Fog, Gabor, and Snow distortions to simulate more diverse adversaries.

We then introduce UAR, a summary metric that measures the robustness of a defense against a given distortion.

Using UAR to assess robustness against existing and novel attacks, we perform an extensive study of adversarial robustness.

We find that evaluation against existing L_p attacks yields redundant information which does not generalize to other attacks; we instead recommend evaluating against our significantly more diverse set of attacks.

We further find that adversarial training against either one or multiple distortions fails to confer robustness to attacks with other distortion types.

These results underscore the need to evaluate and study robustness against unforeseen distortions.

We propose several new attacks and a methodology to measure robustness against unforeseen adversarial attacks.

Testing Robustness Against Unforeseen Adversaries


Deep neural networks (DNNs) have witnessed as a powerful approach in this year by solving long-standing Artificial

intelligence (AI) supervised and unsupervised tasks exists in natural language processing, speech processing, computer vision and others.

In this paper, we attempt to apply DNNs on three different cyber security use cases: Android malware classification, incident detection and fraud detection.

The data set of each use case contains real known benign and malicious activities samples.

These use cases are part of Cybersecurity Data Mining Competition (CDMC) 2017.

The efficient network architecture for DNNs is chosen by conducting various trails of experiments for network parameters and network structures.

The experiments of such chosen efficient configurations of DNNs are run up to 1000 epochs with learning rate set in the range [0.01-0.5].

Experiments of DNNs performed well in comparison to the classical machine learning algorithm in all cases of experiments of cyber security use cases.

This is due to the fact that DNNs implicitly extract and build better features, identifies the characteristics of the data that lead to better accuracy.

The best accuracy obtained by DNNs and XGBoost on Android malware classification 0.940 and 0.741, incident detection 1.00 and 0.997, and fraud detection 0.972 and 0.916 respectively.

The accuracy obtained by DNNs varies -0.05%, +0.02%, -0.01% from the top scored system in CDMC 2017 tasks.

Deep-Net: Deep Neural Network for Cyber Security Use Cases

deep-net deep neural network cyber security case


In this paper, we present an approach to learn recomposable motor primitives across large-scale and diverse manipulation demonstrations.

Current approaches to decomposing demonstrations into primitives often assume manually defined primitives and bypass the difficulty of discovering these primitives.

On the other hand, approaches in primitive discovery put restrictive assumptions on the complexity of a primitive, which limit applicability to narrow tasks.

Our approach attempts to circumvent these challenges by jointly learning both the underlying motor primitives and recomposing these primitives to form the original demonstration.

Through constraints on both the parsimony of primitive decomposition and the simplicity of a given primitive, we are able to learn a diverse set of motor primitives, as well as a coherent latent representation for these primitives.

We demonstrate, both qualitatively and quantitatively, that our learned primitives capture semantically meaningful aspects of a demonstration.

This allows us to compose these primitives in a hierarchical reinforcement learning setup to efficiently solve robotic manipulation tasks like reaching and pushing.

We learn a space of motor primitives from unannotated robot demonstrations, and show these primitives are semantically meaningful and can be composed for new robot tasks.

Discovering Motor Programs by Recomposing Demonstrations


Using modern deep learning models to make predictions on time series data from wearable sensors generally requires large amounts of labeled data.

However, labeling these large datasets can be both cumbersome and costly.

In this paper, we apply weak supervision to time series data, and programmatically label a dataset from sensors worn by patients with Parkinson's.

We then built a LSTM model that predicts when these patients exhibit clinically relevant freezing behavior (inability to make effective forward stepping).

We show that (1) when our model is trained using patient-specific data (prior sensor sessions), we come within 9% AUROC of a model trained using hand-labeled data and (2) when we assume no prior observations of subjects, our weakly supervised model matched performance with hand-labeled data.

These results demonstrate that weak supervision may help reduce the need to painstakingly hand label time series training data.

We demonstrate the feasibility of a weakly supervised time series classification approach for wearable sensor data.

Weak Supervision for Time Series: Wearable Sensor Classification with Limited Labeled Data

wearable sensor
time series
classification
datum


Learning semantic correspondence between the structured data (e.g., slot-value pairs) and associated texts is a core problem for many downstream NLP applications, e.g., data-to-text generation.

Recent neural generation methods require to use large scale training data.

However, the collected data-text pairs for training are usually loosely corresponded, where texts contain additional or contradicted information compare to its paired input.

In this paper, we propose a local-to-global alignment (L2GA) framework to learn semantic correspondences from loosely related data-text pairs.

First, a local alignment model based on multi-instance learning is applied to build the semantic correspondences within a data-text pair.

Then, a global alignment model built on top of a memory guided conditional random field (CRF) layer is designed to exploit dependencies among alignments in the entire training corpus, where the memory is used to integrate the alignment clues provided by the local alignment model.

Therefore, it is capable of inducing missing alignments for text spans that are not supported by its imperfect paired input.

Experiments on recent restaurant dataset show that our proposed method can improve the alignment accuracy and as a by product, our method is also applicable to induce semantically equivalent training data-text pairs for neural generation models.

We propose a local-to-global alignment framework to learn semantic correspondences from noisy data-text pairs with weak supervision

Learning Semantic Correspondences from Noisy Data-text Pairs by Local-to-Global Alignments

learn semantic correspondence noisy data-text pair


Imitation learning algorithms provide a simple and straightforward approach for training control policies via standard supervised learning methods.

By maximizing the likelihood of good actions provided by an expert demonstrator, supervised imitation learning can produce effective policies without the algorithmic complexities and optimization challenges of reinforcement learning, at the cost of requiring an expert demonstrator -- typically a person -- to provide the demonstrations.

In this paper, we ask: can we use imitation learning to train effective policies without any expert demonstrations?

The key observation that makes this possible is that, in the multi-task setting, trajectories that are generated by a suboptimal policy can still serve as optimal examples for other tasks.

In particular, in the setting where the tasks correspond to different goals, every trajectory is a successful demonstration for the state that it actually reaches.

Informed by this observation, we propose a very simple algorithm for learning behaviors without any demonstrations, user-provided reward functions, or complex reinforcement learning methods.

Our method simply maximizes the likelihood of actions the agent actually took in its own previous rollouts, conditioned on the goal being the state that it actually reached.

Although related variants of this approach have been proposed previously in imitation learning settings with example demonstrations, we present the first instance of this approach as a method for learning goal-reaching policies entirely from scratch.

We present a theoretical result linking self-supervised imitation learning and reinforcement learning, and empirical results showing that it performs competitively with more complex reinforcement learning methods on a range of challenging goal reaching problems.

Learning how to reach goals from scratch by using imitation learning with data relabeling

Learning to Reach Goals Without Reinforcement Learning


Neural networks have recently shown excellent performance on numerous classi- fication tasks.

These networks often have a large number of parameters and thus require much data to train.

When the number of training data points is small, however, a network with high flexibility will quickly overfit the training data, resulting in a large model variance and a poor generalization performance.

To address this problem, we propose a new ensemble learning method called InterBoost for small-sample image classification.

In the training phase, InterBoost first randomly generates two complementary datasets to train two base networks of the same structure, separately, and then next two complementary datasets for further training the networks are generated through interaction (or information sharing) between the two base networks trained previously.

This interactive training process continues iteratively until a stop criterion is met.

In the testing phase, the outputs of the two networks are combined to obtain one final score for classification.

Detailed analysis of the method is provided for an in-depth understanding of its mechanism.

In the paper, we proposed an ensemble method called InterBoost for training neural networks for small-sample classification. The method has better generalization performance than other ensemble methods, and reduces variances significantly.

Interactive Boosting of Neural Networks for Small-sample Image Classification

neural network small-sample
classification


Interpreting neural networks is a crucial and challenging task in machine learning.

In this paper, we develop a novel framework for detecting statistical interactions captured by a feedforward multilayer neural network by directly interpreting its learned weights.

Depending on the desired interactions, our method can achieve significantly better or similar interaction detection performance compared to the state-of-the-art without searching an exponential solution space of possible interactions.

We obtain this accuracy and efficiency by observing that interactions between input features are created by the non-additive effect of nonlinear activation functions, and that interacting paths are encoded in weight matrices.

We demonstrate the performance of our method and the importance of discovered interactions via experimental results on both synthetic datasets and real-world application datasets.

We detect statistical interactions captured by a feedforward multilayer neural network by directly interpreting its learned weights.

Detecting Statistical Interactions from Neural Network Weights

detect statistical interaction
neural network
weight


The neural linear model is a simple adaptive Bayesian linear regression method that has recently been used in a number of problems ranging from Bayesian optimization to reinforcement learning.

Despite its apparent successes in these settings, to the best of our knowledge there has been no systematic exploration of its capabilities on simple regression tasks.

In this work we characterize these on the UCI datasets, a popular benchmark for Bayesian regression models, as well as on the recently introduced ''gap'' datasets, which are better tests of out-of-distribution uncertainty.

We demonstrate that the neural linear model is a simple method that shows competitive performance on these tasks.

We benchmark the neural linear model on the UCI and UCI "gap" datasets.

Benchmarking the Neural Linear Model for Regression


The reproducibility of reinforcement-learning research has been highlighted as a key challenge area in the field.

In this paper, we present a case study in reproducing the results of one groundbreaking algorithm, AlphaZero, a reinforcement learning system that learns how to play Go at a superhuman level given only the rules of the game.

We describe Minigo, a reproduction of the AlphaZero system using publicly available Google Cloud Platform infrastructure and Google Cloud TPUs.

The Minigo system includes both the central reinforcement learning loop as well as auxiliary monitoring and evaluation infrastructure.

With ten days of training from scratch on 800 Cloud TPUs, Minigo can play evenly against LeelaZero and ELF OpenGo, two of the strongest publicly available Go AIs.

We discuss the difficulties of scaling a reinforcement learning system and the monitoring systems required to understand the complex interplay of hyperparameter configurations.

We reproduced AlphaZero on Google Cloud Platform

Minigo: A Case Study in Reproducing Reinforcement Learning Research


Generative adversarial networks (GANs) train implicit generative models through solving minimax problems.

Such minimax problems are known as nonconvex- nonconcave, for which the dynamics of first-order methods are not well understood.

In this paper, we consider GANs in the type of the integral probability metrics (IPMs) with the generator represented by an overparametrized neural network.

When the discriminator is solved to approximate optimality in each iteration, we prove that stochastic gradient descent on a regularized IPM objective converges globally to a stationary point with a sublinear rate.

Moreover, we prove that when the width of the generator network is sufficiently large and the discriminator function class has enough discriminative ability, the obtained stationary point corresponds to a generator that yields a distribution that is close to the distribution of the observed data in terms of the total variation.

To the best of our knowledge, we seem to first establish both the global convergence and global optimality of training GANs when the generator is parametrized by a neural network.

We establish global convergence to optimality for IPM-based GANs where the generator is an overparametrized neural network.

Provable Convergence and Global Optimality  of Generative Adversarial Network


We present network embedding algorithms that capture information about a node from the local distribution over node attributes around it, as observed over random walks following an approach similar to Skip-gram.

Observations from neighborhoods of different sizes are either pooled (AE) or encoded distinctly in a multi-scale approach (MUSAE).

Capturing attribute-neighborhood relationships over multiple scales is useful for a diverse range of applications, including latent feature identification across disconnected networks with similar attributes.

We prove theoretically that matrices of node-feature pointwise mutual information are implicitly factorized by the embeddings.

Experiments show that our algorithms are robust, computationally efficient and outperform comparable models on social, web and citation network datasets.

We develop efficient multi-scale approximate attributed network embedding procedures with provable properties.

Multi-scale Attributed Node Embedding


Few-shot classiﬁcation aims to learn a classiﬁer to recognize unseen classes during training with limited labeled examples.

While signiﬁcant progress has been made, the growing complexity of network designs, meta-learning algorithms, and differences in implementation details make a fair comparison difﬁcult.

1) a consistent comparative analysis of several representative few-shot classiﬁcation algorithms, with results showing that deeper backbones signiﬁcantly reduce the gap across methods including the baseline,

2) a slightly modiﬁed baseline method that surprisingly achieves competitive performance when compared with the state-of-the-art on both the mini-ImageNet and the CUB datasets, and

3) a new experimental setting for evaluating the cross-domain generalization ability for few-shot classiﬁcation algorithms.

Our results reveal that reducing intra-class variation is an important factor when the feature backbone is shallow, but not as critical when using deeper backbones.

In a realistic, cross-domain evaluation setting, we show that a baseline method with a standard ﬁne-tuning practice compares favorably against other state-of-the-art few-shot learning algorithms.

A detailed empirical study in few-shot classification that revealing challenges in standard evaluation setting and showing a new direction.

A Closer Look at Few-shot Classification


Temporal logics are useful for describing dynamic system behavior, and have been successfully used as a language for goal definitions during task planning.

Prior works on inferring temporal logic specifications have focused on "summarizing" the input dataset -- i.e., finding specifications that are satisfied by all plan traces belonging to the given set.

In this paper, we examine the problem of inferring specifications that describe temporal differences between two sets of plan traces.

We formalize the concept of providing such contrastive explanations, then present a Bayesian probabilistic model for inferring contrastive explanations as linear temporal logic specifications.

We demonstrate the efficacy, scalability, and robustness of our model for inferring correct specifications across various benchmark planning domains and for a simulated air combat mission.

We present a Bayesian inference model to infer contrastive explanations (as LTL specifications) describing how two sets of plan traces differ.

Bayesian Inference of Temporal Specifications to Explain How Plans Differ

bayesian inference
plan
specification
differ


This work tackles the problem of characterizing and understanding the decision boundaries of neural networks with piece-wise linear non-linearity activations.

We use tropical geometry, a new development in the area of algebraic geometry, to provide a characterization of the decision boundaries of a simple neural network of the form (Affine, ReLU, Affine).

Specifically, we show that the decision boundaries are a subset of a tropical hypersurface, which is intimately related to a polytope formed by the convex hull of two zonotopes.

The generators of the zonotopes are precise functions of the neural network parameters.

We utilize this geometric characterization to shed light and new perspective on three tasks.

In doing so, we propose a new tropical perspective for the lottery ticket hypothesis, where we see the effect of different initializations on the tropical geometric representation of the decision boundaries.

Also, we leverage this characterization as a new set of tropical regularizers, which  deal directly  with the decision boundaries of a network.

We investigate the use of these regularizers  in neural network pruning (removing network parameters that do not contribute to the tropical geometric representation of the decision boundaries) and in generating adversarial input attacks (with input perturbations explicitly perturbing the decision boundaries geometry to change the network prediction of the input).

Tropical geometry can be leveraged to represent the decision boundaries of neural networks and bring to light interesting insights.

On the Decision Boundaries of Deep Neural Networks: A Tropical Geometry Perspective

tropical geometry
decision boundary
neural network


First-order methods such as stochastic gradient descent (SGD) are currently the standard algorithm for training deep neural networks.

Second-order methods, despite their better convergence rate, are rarely used in practice due to the pro- hibitive computational cost in calculating the second-order information.

In this paper, we propose a novel Gram-Gauss-Newton (GGN) algorithm to train deep neural networks for regression problems with square loss.

Our method draws inspiration from the connection between neural network optimization and kernel regression of neural tangent kernel (NTK).

Different from typical second-order methods that have heavy computational cost in each iteration, GGN only has minor overhead compared to first-order methods such as SGD.

We also give theoretical results to show that for sufficiently wide neural networks, the convergence rate of GGN is quadratic.

Furthermore, we provide convergence guarantee for mini-batch GGN algorithm, which is, to our knowledge, the first convergence result for the mini-batch version of a second-order method on overparameterized neural net- works.

Preliminary experiments on regression tasks demonstrate that for training standard networks, our GGN algorithm converges much faster and achieves better performance than SGD.

A novel Gram-Gauss-Newton method to train neural networks, inspired by neural tangent kernel and Gauss-Newton method, with fast convergence speed both theoretically and experimentally.

Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problems


Recent pretrained sentence encoders achieve state of the art results on language understanding tasks, but does this mean they have implicit knowledge of syntactic structures?

We introduce a grammatically annotated development set for the Corpus of Linguistic Acceptability (CoLA; Warstadt et al., 2018), which we use to investigate the grammatical knowledge of three pretrained encoders, including the popular OpenAI Transformer (Radford et al., 2018) and BERT (Devlin et al., 2018).

We fine-tune these encoders to do acceptability classification over CoLA and compare the models’ performance on the annotated analysis set.

Some phenomena, e.g. modification by adjuncts, are easy to learn for all models, while others, e.g. long-distance movement, are learned effectively only by models with strong overall performance, and others still, e.g. morphological agreement, are hardly learned by any model.

We investigate the implicit syntactic knowledge of sentence embeddings using a new analysis set of grammatically annotated sentences with acceptability judgments.

Grammatical Analysis of Pretrained Sentence Encoders with Acceptability Judgments

acceptability judgment
sentence
analysis


When considering simultaneously a finite number of tasks, multi-output learning enables one to account for the similarities of the tasks via appropriate regularizers.

We propose a generalization of the classical setting to a continuum of tasks by using vector-valued RKHSs.

We propose an extension of multi-output learning to a continuum of tasks using operator-valued kernels.

A Functional Extension of Multi-Output Learning


We analyze the joint probability distribution on the lengths of the

vectors of hidden variables in different layers of a fully connected

deep network, when the weights and biases are chosen randomly according to

Gaussian distributions, and the input is binary-valued.

that, if the activation function satisfies a minimal set of

assumptions, satisfied by all activation functions that we know that

are used in practice, then, as the width of the network gets large,

the ``length process'' converges in probability to a length map

that is determined as a simple function of the variances of the

random weights and biases, and the activation function.

We also show that this convergence may fail for activation functions

We prove that, for activation functions satisfying some conditions, as a deep network gets wide, the lengths of the vectors of hidden variables converge to a length map.

On the effect of the activation function on the distribution of hidden nodes in a deep network


Data augmentation is one of the most effective approaches for improving the accuracy of modern machine learning models, and it is also indispensable to train a deep model for meta-learning.

However, most current data augmentation implementations applied in meta-learning are the same as those used in the conventional image classification.

In this paper, we introduce a new data augmentation method for meta-learning, which is named as ``Task Level Data Augmentation'' (referred to Task Aug).

The basic idea of Task Aug is to increase the number of image classes rather than the number of images in each class.

In contrast, with a larger amount of classes, we can sample more diverse task instances during training.

This allows us to train a deep network by meta-learning methods with little over-fitting.

Experimental results show that our approach achieves state-of-the-art performance on miniImageNet, CIFAR-FS, and FC100 few-shot learning benchmarks.

Once paper is accepted, we will provide the link to code.

We propose a data augmentation approach for meta-learning and prove that it is valid.

Task Level Data Augmentation for Meta-Learning


In this paper, we present a general framework for distilling expectations with respect to the Bayesian posterior distribution of a deep neural network, significantly extending prior work on a method known as ``Bayesian Dark Knowledge.

"  Our generalized framework applies to the case of classification models and takes as input the architecture of a ``teacher" network, a general posterior expectation of interest, and the architecture of a ``student" network.

The distillation method performs an online compression of the selected posterior expectation using iteratively generated Monte Carlo samples from the parameter posterior of the teacher model.

We further consider the problem of optimizing the student model architecture with respect to an accuracy-speed-storage trade-off.

We present experimental results investigating multiple data sets, distillation targets,  teacher model architectures, and approaches to searching for student model architectures.

We establish the key result that distilling into a student model with an architecture that matches the teacher, as is done in Bayesian Dark Knowledge, can lead to sub-optimal performance.

Lastly, we show that student architecture search methods can identify student models with significantly improved performance.

A general framework for distilling Bayesian posterior expectations for deep neural networks.

Generalized Bayesian Posterior Expectation Distillation for Deep Neural Networks

deep neural network
bayesian posterior expectation


Variational Autoencoders (VAEs) have proven to be powerful latent variable models.

How- ever, the form of the approximate posterior can limit the expressiveness of the model.

Categorical distributions are flexible and useful building blocks for example in neural memory layers.

We introduce the Hierarchical Discrete Variational Autoencoder (HD-VAE): a hi- erarchy of variational memory layers.

The Concrete/Gumbel-Softmax relaxation allows maximizing a surrogate of the Evidence Lower Bound by stochastic gradient ascent.

We show that, when using a limited number of latent variables, HD-VAE outperforms the Gaussian baseline on modelling multiple binary image datasets.

Training very deep HD-VAE remains a challenge due to the relaxation bias that is induced by the use of a surrogate objective.

We introduce a formal definition and conduct a preliminary theoretical and empirical study of the bias.

In this paper, we introduce a discrete hierarchy of categorical latent variables that we train using the Concrete/Gumbel-Softmax relaxation and we derive an upper bound for the absolute difference between the unbiased and the biased objective.

Towards Hierarchical Discrete Variational Autoencoders


In this paper, we propose a novel technique for improving the stochastic gradient descent (SGD) method to train deep networks, which we term \emph{PowerSGD}.

The proposed PowerSGD method simply raises the stochastic gradient to a certain power $\gamma\in[0,1]$ during iterations and introduces only one additional parameter, namely, the power exponent $\gamma$ (when $\gamma=1$, PowerSGD reduces to SGD).

We further propose PowerSGD with momentum, which we term \emph{PowerSGDM}, and provide convergence rate analysis on both PowerSGD and PowerSGDM methods.

Experiments are conducted on popular deep learning models and benchmark datasets.

Empirical results show that the proposed PowerSGD and PowerSGDM obtain faster initial training speed than adaptive gradient methods,  comparable generalization ability with SGD, and improved robustness to hyper-parameter selection and vanishing gradients.

PowerSGD is essentially a gradient modifier via a nonlinear transformation.

As such, it is orthogonal and complementary to other techniques for accelerating gradient-based optimization.

We propose a new class of optimizers for accelerated non-convex optimization via a nonlinear gradient transformation.

PowerSGD: Powered Stochastic Gradient Descent Methods for Accelerated Non-Convex Optimization

accelerate non-convex optimization
gradient


We aim to build complex humanoid agents that integrate perception, motor control, and memory.

In this work, we partly factor this problem into low-level motor control from proprioception and high-level coordination of the low-level skills informed by vision.

We develop an architecture capable of surprisingly flexible, task-directed motor control of a relatively high-DoF humanoid body by combining pre-training of low-level motor controllers with a high-level, task-focused controller that switches among low-level sub-policies.

The resulting system is able to control a physically-simulated humanoid body to solve tasks that require coupling visual perception from an unstabilized egocentric RGB camera during locomotion in the environment.

Supplementary video link:  https://youtu.be/fBoir7PNxPk

Solve tasks involving vision-guided humanoid locomotion, reusing locomotion behavior from motion capture data.

Hierarchical Visuomotor Control of Humanoids


The gap between the empirical success of deep learning and the lack of strong theoretical guarantees calls for studying simpler models.

By observing that a ReLU neuron is a product of a linear function with a gate (the latter determines whether the neuron is active or not), where both share a jointly trained weight vector, we propose to decouple the two.

We introduce GaLU networks — networks in which each neuron is a product of a Linear Unit, defined by a weight vector which is being trained, with a Gate, defined by a different weight vector which is not being trained.

Generally speaking, given a base model and a simpler version of it, the two parameters that determine the quality of the simpler version are whether its practical performance is close enough to the base model and whether it is easier to analyze it theoretically.

We show that GaLU networks perform similarly to ReLU networks on standard datasets and we initiate a study of their theoretical properties, demonstrating that they are indeed easier to analyze.

We believe that further research of GaLU networks may be fruitful for the development of a theory of deep learning.

We propose Gated Linear Unit networks — a model that performs similarly to ReLU networks on real data while being much easier to analyze theoretically.

Decoupling Gating from Linearity


Machine learning systems often encounter Out-of-Distribution (OoD) errors when dealing with testing data coming from a different distribution from the one used for training.

With their growing use in critical applications, it becomes important to develop systems that are able to accurately quantify its predictive uncertainty and screen out these anomalous inputs.

However, unlike standard learning tasks, there is currently no well established guiding principle for designing architectures that can accurately quantify uncertainty.

Moreover, commonly used OoD detection approaches are prone to errors and even sometimes assign higher likelihoods to OoD samples.

To address these problems, we first seek to identify guiding principles for designing uncertainty-aware architectures, by proposing Neural Architecture Distribution Search (NADS).

Unlike standard neural architecture search methods which seek for a single best performing architecture, NADS searches for a distribution of architectures that perform well on a given task, allowing us to identify building blocks common among all uncertainty aware architectures.

With this formulation, we are able to optimize a stochastic outlier detection objective and construct an ensemble of models to perform OoD detection.

We perform multiple OoD detection experiments and observe that our NADS performs favorably compared to state-of-the-art OoD detection methods.

We propose an architecture search method to identify a distribution of architectures and use it to construct a Bayesian ensemble for outlier detection.

NADS: Neural Architecture Distribution Search for Uncertainty Awareness


Modern applications from Autonomous Vehicles to Video Surveillance generate massive amounts of image data.

In this work we propose a novel image outlier detection approach (IOD for short) that leverages the cutting-edge image classifier to discover outliers without using any labeled outlier.

We observe that although intuitively the confidence that a convolutional neural network (CNN) has that an image belongs to a particular class could serve as outlierness measure to each image, directly applying this confidence to detect outlier does not work well.

This is because CNN often has high confidence on an outlier image that does not belong to any target class due to its generalization ability that ensures the high accuracy in classification.

To solve this issue, we propose a Deep Neural Forest-based approach that harmonizes the contradictory requirements of accurately classifying images and correctly detecting the outlier images.

Our experiments using several benchmark image datasets including MNIST, CIFAR-10, CIFAR-100, and SVHN demonstrate the effectiveness of our IOD approach for outlier detection, capturing more than 90% of outliers generated by injecting one image dataset into another, while still preserving the classification accuracy of the multi-class classification problem.

A novel approach that detects outliers from image data,  while preserving the classification accuracy of image classification

Outlier Detection from Image Data


This paper introduces CloudLSTM, a new branch of recurrent neural models tailored to forecasting over data streams generated by geospatial point-cloud sources.

We design a Dynamic Point-cloud Convolution (D-Conv) operator as the core component of CloudLSTMs, which performs convolution directly over point-clouds and extracts local spatial features from sets of neighboring points that surround different elements of the input.

This operator maintains the permutation invariance of sequence-to-sequence learning frameworks, while representing neighboring correlations at each time step -- an important aspect in spatiotemporal predictive learning.

The D-Conv operator resolves the grid-structural data requirements of existing spatiotemporal forecasting models and can be easily plugged into traditional LSTM architectures with sequence-to-sequence learning and attention mechanisms.

We apply our proposed architecture to two representative, practical use cases that involve point-cloud streams, i.e. mobile service traffic forecasting and air quality indicator forecasting.

Our results, obtained with real-world datasets collected in diverse scenarios for each use case, show that CloudLSTM delivers accurate long-term predictions, outperforming a variety of neural network models.

CloudLSTM: A Recurrent Neural Model for Spatiotemporal Point-cloud Stream Forecasting

recurrent neural model
cloudlstm
stream
point-cloud


Knowledge Graphs (KG), composed of entities and relations, provide a structured representation of knowledge.

For easy access to statistical approaches on relational data, multiple methods to embed a KG as components of R^d have been introduced.

We propose TransINT, a novel and interpretable KG embedding method that isomorphically preserves the implication ordering among relations in the embedding space.

TransINT maps set of entities (tied by a relation) to continuous sets of vectors that are inclusion-ordered isomorphically to relation implications.

With a novel parameter sharing scheme, TransINT enables automatic training on missing but implied facts without rule grounding.

We achieve new state-of-the-art performances with signficant margins in Link Prediction and Triple Classification on FB122 dataset, with boosted performance even on test instances that cannot be inferred by logical rules.

The angles between the continuous sets embedded by TransINT provide an interpretable way to mine semantic relatedness and implication rules among relations.

We propose TransINT, a novel and interpretable KG embedding method that isomorphically preserves the implication ordering among relations in the embedding space in an explainable, robust, and geometrically coherent way.

TransINT: Embedding Implication Rules in Knowledge Graphs with Isomorphic Intersections of Linear Subspaces


Unsupervised domain adaptive object detection aims to learn a robust detector on the domain shift circumstance, where the training (source) domain is label-rich with bounding box annotations, while the testing (target) domain is label-agnostic and the feature distributions between training and testing domains are dissimilar or even totally different.

In this paper, we propose a gradient detach based Stacked Complementary Losses (SCL) method that uses detection objective (cross entropy and smooth l1 regression) as the primary objective, and cuts in several auxiliary losses in different network stages to utilize information from the complement data (target images) that can be effective in adapting model parameters to both source and target domains.

A gradient detach operation is applied between detection and context sub-networks during training to force networks to learn discriminative representations.

We argue that the conventional training with primary objective mainly leverages the information from the source-domain for maximizing likelihood and ignores the complement data in shallow layers of networks, which leads to an insufficient integration within different domains.

Thus, our proposed method is a more syncretic adaptation learning process.

We conduct comprehensive experiments on seven datasets, the results demonstrate that our method performs favorably better than the state-of-the-art methods by a large margin.

For instance, from Cityscapes to FoggyCityscapes, we achieve 37.9% mAP, outperforming the previous art Strong-Weak by 3.6%.

We introduce a new gradient detach based complementary objective training strategy for domain adaptive object detection.

SCL: Towards Accurate Domain Adaptive Object Detection via Gradient Detach Based Stacked Complementary Losses

domain adaptive object detection
gradient detach base
complementary


Convolutional neural networks (CNN) have become the most successful and popular approach in many vision-related domains.

While CNNs are particularly well-suited for capturing a proper hierarchy of concepts from real-world images, they are limited to domains where data is abundant.

Recent attempts have looked into mitigating this data scarcity problem by casting their original single-task problem into a new multi-task learning (MTL) problem.

The main goal of this inductive transfer mechanism is to leverage domain-specific information from related tasks, in order to improve generalization on the main task.

While recent results in the deep learning (DL) community have shown the promising potential of training task-specific CNNs in a soft parameter sharing framework, integrating the recent DL advances for improving knowledge sharing is still an open problem.

In this paper, we propose the Deep Collaboration Network (DCNet), a novel approach for connecting task-specific CNNs in a MTL framework.

We define connectivity in terms of two distinct non-linear transformation blocks.

One aggregates task-specific features into global features, while the other merges back the global features with each task-specific network.

Based on the observation that task relevance depends on depth, our transformation blocks use skip connections as suggested by residual network approaches, to more easily deactivate unrelated task-dependent features.

To validate our approach, we employed facial landmark detection (FLD) datasets as they are readily amenable to MTL, given the number of tasks they include.

Experimental results show that we can achieve up to 24.31% relative improvement in landmark failure rate over other state-of-the-art MTL approaches.

We finally perform an ablation study showing that our approach effectively allows knowledge sharing, by leveraging domain-specific features at particular depths from tasks that we know are related.

We propose a novel approach for connecting task-specific networks in a multi-task learning setting based on recent residual network advances.

Multi-Task Learning by Deep Collaboration and Application in Facial Landmark Detection


Zero-Shot Learning (ZSL) is a classification task where some classes referred as unseen classes have no labeled training images.

Instead, we only have side information (or description) about seen and unseen classes, often in the form of semantic or descriptive attributes.

Lack of training images from a set of classes restricts the use of standard classification techniques and losses, including the popular cross-entropy loss.

The key step in tackling ZSL problem is bridging visual to semantic space via learning a nonlinear embedding.

A well established approach is to obtain the semantic representation of the visual information and perform classification in the semantic space.

In this paper, we propose a novel architecture of casting ZSL as a fully connected neural-network with cross-entropy loss to embed visual space to semantic space.

During training in order to introduce unseen visual information to the network, we utilize soft-labeling based on semantic similarities between seen and unseen classes.

To the best of our knowledge, such similarity based soft-labeling is not explored for cross-modal transfer and ZSL.

We evaluate the proposed model on five benchmark datasets for zero-shot learning, AwA1, AwA2, aPY, SUN and CUB datasets, and show that, despite the simplicity, our approach achieves the state-of-the-art performance in Generalized-ZSL setting on all of these datasets and outperforms the state-of-the-art for some datasets.

How to use cross-entropy loss for zero shot learning with soft labeling on unseen classes : a simple and effective solution that achieves state-of-the-art performance on five ZSL benchmark datasets.

The Power of  Semantic Similarity based Soft-Labeling for Generalized Zero-Shot Learning


In complex tasks, such as those with large combinatorial action spaces, random exploration may be too inefficient to achieve meaningful learning progress.

In this work, we use a curriculum of progressively growing action spaces to accelerate learning.

We assume the environment is out of our control, but that the agent may set an internal curriculum by initially restricting its action space.

Our approach uses off-policy reinforcement learning to estimate optimal value functions for multiple action spaces simultaneously and efficiently transfers data,  value estimates, and state representations from restricted action spaces to the full task.

We show the efficacy of our approach in proof-of-concept control tasks and on challenging large-scale StarCraft micromanagement tasks with large, multi-agent action spaces.

Progressively growing the available action space is a great curriculum for learning agents

Growing Action Spaces


Recently, researchers have discovered that the state-of-the-art object classifiers can be fooled easily by small perturbations in the input unnoticeable to human eyes.

It is known that an attacker can generate strong adversarial examples if she knows the classifier parameters

. Conversely, a defender can robustify the classifier by retraining if she has the adversarial examples

.
The cat-and-mouse game nature of attacks and defenses raises the question of the presence of equilibria in the dynamics

.
In this paper, we present a neural-network based attack class to approximate a larger but intractable class of attacks, and

formulate the attacker-defender interaction as a zero-sum leader-follower game.

We present sensitivity-penalized optimization algorithms to find minimax solutions, which are the best worst-case defenses against whitebox attacks.

Advantages of the learning-based attacks and defenses compared to gradient-based attacks and defenses are demonstrated with MNIST and CIFAR-10.

A game-theoretic solution to adversarial attacks and defenses.

MACHINE VS MACHINE: MINIMAX-OPTIMAL DEFENSE AGAINST ADVERSARIAL EXAMPLES


Supervised learning with irregularly sampled time series have been a challenge to Machine Learning methods due to the obstacle of dealing with irregular time intervals.

Some papers introduced recently recurrent neural network models that deals with irregularity, but most of them rely on complex mechanisms to achieve a better performance.

This work propose a novel method to represent timestamps (hours or dates) as dense vectors using sinusoidal functions, called Time Embeddings.

As a data input method it and can be applied to most machine learning models.

The method was evaluated with two predictive tasks from MIMIC III, a dataset of irregularly sampled time series of electronic health records.

Our tests showed an improvement to LSTM-based and classical machine learning models, specially with very irregular data.

A novel method to create dense descriptors of time (Time Embeddings) to make simple models understand temporal structures

Improving Irregularly Sampled Time Series Learning with Dense Descriptors of Time


Community detection in graphs can be solved via spectral methods or posterior inference under certain probabilistic graphical models.

Focusing on random graph families such as the stochastic block model, recent research has unified both approaches and identified both statistical and computational detection thresholds in terms of the signal-to-noise ratio.

By recasting community detection as a node-wise classification problem on graphs, we can also study it from a learning perspective.

We present a novel family of Graph Neural Networks (GNNs) for solving community detection problems in a supervised learning setting.

We show that, in a data-driven manner and without access to the underlying generative models, they can match or even surpass the performance of the belief propagation algorithm on binary and multiclass stochastic block models, which is believed to reach the computational threshold in these cases.

In particular, we propose to augment GNNs with the non-backtracking operator defined on the line graph of edge adjacencies.

The GNNs are achieved good performance on real-world datasets.

In addition, we perform the first analysis of the optimization landscape of using (linear) GNNs to solve community detection problems, demonstrating that under certain simplifications and assumptions, the loss value at any local minimum is close to the loss value at the global minimum/minima.

We propose a novel graph neural network architecture based on the non-backtracking matrix defined over the edge adjacencies and demonstrate its effectiveness in community detection tasks on graphs.

Supervised Community Detection with Line Graph Neural Networks

graph neural network
community detection


Residual networks (Resnets) have become a prominent architecture in deep learning.

However, a comprehensive understanding of Resnets is still a topic of ongoing research.

A recent view argues that Resnets perform iterative refinement of features.

We attempt to further expose properties of this aspect.

To this end, we study Resnets both analytically and empirically.

We formalize the notion of iterative refinement in Resnets by showing that residual architectures naturally encourage features to move along the negative gradient of loss during the feedforward phase.

In addition, our empirical analysis suggests that Resnets are able to perform both representation learning and iterative refinement.

In general, a Resnet block tends to concentrate representation learning behavior in the first few layers while higher layers perform iterative refinement of features.

Finally we observe that sharing residual layers naively leads to representation explosion and hurts generalization performance, and show that simple existing strategies can help alleviating this problem.

Residual connections really perform iterative inference

Residual Connections Encourage Iterative Inference