The complex world around us is inherently multimodal and sequential (continuous).

Information is scattered across different modalities and requires multiple continuous sensors to be captured.

As machine learning leaps towards better generalization to real world, multimodal sequential learning becomes a fundamental research area.

Arguably,  modeling arbitrarily distributed spatio-temporal dynamics within and across modalities is the biggest challenge in this research area.

In this paper, we present a new transformer model, called the Factorized Multimodal Transformer (FMT) for multimodal sequential learning.

FMT inherently models the intramodal and intermodal (involving two or more modalities) dynamics within its multimodal input in a factorized manner.

The proposed factorization allows for increasing the number of self-attentions to better model the multimodal phenomena at hand; without encountering difficulties during training (e.g. overfitting) even on relatively low-resource setups.

All the attention mechanisms within FMT have a full time-domain receptive field which allows them to asynchronously capture long-range multimodal dynamics.

In our experiments we focus on datasets that contain the three commonly studied modalities of language, vision and acoustic.

We perform a wide range of experiments, spanning across 3 well-studied datasets and 21 distinct labels.

FMT shows superior performance over previously proposed models, setting new state of the art in  the studied datasets.

A multimodal transformer for multimodal sequential learning, with strong empirical results on multimodal language metrics such as multimodal sentiment analysis, emotion recognition and personality traits recognition.

Factorized Multimodal Transformer for Multimodal Sequential Learning

multimodal transformer multimodal sequential learning


We develop a novel and efficient algorithm for optimizing neural networks inspired by a recently proposed geodesic optimization algorithm.

Our algorithm, which we call Stochastic  Geodesic Optimization (SGeO), utilizes an adaptive coefficient on top of Polyak's Heavy Ball method effectively controlling the amount of weight put on the previous update to the parameters based on the change of direction in the optimization path.

Experimental results on strongly convex functions with Lipschitz gradients and deep Autoencoder benchmarks show that SGeO reaches lower errors than established first-order methods and competes well with lower or similar errors to a recent second-order method called K-FAC (Kronecker-Factored Approximate Curvature).

We also incorporate Nesterov style lookahead gradient into our algorithm (SGeO-N) and observe notable improvements.

We utilize an adaptive coefficient on top of regular momentum inspired by geodesic optimization which significantly speeds up training in both convex and non-convex functions.

Stochastic Geodesic Optimization for Neural Networks


We introduce the masked translation model (MTM) which combines encoding and decoding of sequences within the same model component.

The MTM is based on the idea of masked language modeling and supports both autoregressive and non-autoregressive decoding strategies by simply changing the order of masking.

In experiments on the WMT 2016 Romanian-English task, the MTM shows strong constant-time translation performance, beating all related approaches with comparable complexity.

We also extensively compare various decoding strategies supported by the MTM, as well as several length modeling techniques and training settings.

We use a transformer encoder to do translation by training it in the style of a masked translation model.

Masked Translation Model


We present local ensembles, a method for detecting extrapolation at test time in a pre-trained model.

We focus on underdetermination as a key component of extrapolation: we aim to detect when many possible predictions are consistent with the training data and model class.

Our method uses local second-order information to approximate the variance of predictions across an ensemble of models from the same class.

We compute this approximation by estimating the norm of the component of a test point's gradient that aligns with the low-curvature directions of the Hessian, and provide a tractable method for estimating this quantity.

Experimentally, we show that our method is capable of detecting when a pre-trained model is extrapolating on test data, with applications to out-of-distribution detection, detecting spurious correlates, and active learning.

We present local ensembles, a method for detecting extrapolation in trained models, which approximates the variance of an ensemble using local-second order information.

Detecting Extrapolation with Local Ensembles


Coalition operations are essential for responding to the increasing number of world-wide incidents that require large-scale humanitarian assistance.

Many nations and non-governmental organizations regularly coordinate to address such problems but their cooperation is often impeded by limits on what information they are able to share.

In this paper, we consider the     use of an advanced cryptographic technique called secure multi-party computation to enable coalition members to achieve joint objectives while still meeting privacy requirements

. Our particular focus is on a multi-nation aid delivery scheduling task that involves coordinating when and where various aid provider nations will deliver relief materials after the occurrence of a natural disaster

. Even with the use of secure multi-party computation technology, information about private data can leak

.  We describe how the emerging field of  quantitative information flow can be used to help data owners understand the extent to which private data might become vulnerable as the result of possible or actual scheduling operations, and to enable automated adjustments of the scheduling process to ensure privacy requirements

Privacy can be thought about in the same way as other resources in planning

Privacy-aware Adaptive Scheduling for Coalition Operations


The challenge of learning disentangled representation has recently attracted much attention and boils down to a competition.

Various methods based on variational auto-encoder have been proposed to solve this problem, by enforcing the independence between the representation and modifying the regularization term in the variational lower bound.

However recent work by Locatello et al. (2018) has demonstrated that the proposed methods are heavily influenced by randomness and the choice of the hyper-parameter.

This work is built upon the same framework in Stage 1 (Li et al., 2019), but with different settings; to make it self-contained, we provide this manuscript, which is unavoidably very similar to the report for Stage 1.

In detail, in this work, instead of designing a new regularization term, we adopt the FactorVAE but improve the reconstruction performance and increase the capacity of network and the training step.

The strategy turns out to be very effective in achieving disentanglement.

Disentanglement Challenge: From Regularization to Reconstruction


We propose a generative adversarial training approach for the problem of clarification question generation.

Our approach generates clarification questions with the goal of eliciting new information that would make the given context more complete.

We develop a Generative Adversarial Network (GAN) where the generator is a sequence-to-sequence model and the discriminator is a utility function that models the value of updating the context with the answer to the clarification question.

We evaluate on two datasets, using both automatic metrics and human judgments of usefulness, specificity and relevance, showing that our approach outperforms both a retrieval-based model and ablations that exclude the utility model and the adversarial training.

We propose an adversarial training approach to the problem of clarification question generation which uses the answer to the question to model the reward.

Answer-based Adversarial Training for Generating Clarification Questions

adversarial training
clarification question


Pattern databases are the foundation of some of the strongest admissible heuristics for optimal classical planning.

Experiments showed that the most informative way of combining information from multiple pattern databases is to use saturated cost partitioning.

Previous work selected patterns and computed saturated cost partitionings over the resulting pattern database heuristics in two separate steps.

We introduce a new method that uses saturated cost partitioning to select patterns and show that it outperforms all existing pattern selection algorithms.

Using saturated cost partitioning to select patterns is preferable to all existing pattern selection algorithms.

Pattern Selection for Optimal Classical Planning with Saturated Cost Partitioning


State-of-the-art results on neural machine translation often use attentional sequence-to-sequence models with some form of convolution or recursion.

al. (2017) propose a new architecture that avoids recurrence and convolution completely.

Instead, it uses only self-attention and feed-forward layers.

While the proposed architecture achieves state-of-the-art results on several machine translation tasks, it requires a large number of parameters and training iterations to converge.

We propose Weighted Transformer, a Transformer with modified attention layers, that not only outperforms the baseline network in BLEU score but also converges 15-40% faster.

Specifically, we replace the multi-head attention by multiple self-attention branches that the model learns to combine during the training process.

Our model improves the state-of-the-art performance by 0.5 BLEU points on the WMT 2014 English-to-German translation task and by 0.4 on the English-to-French translation task.

Using branched attention with learned combination weights outperforms the baseline transformer for machine translation tasks.

Weighted Transformer Network for Machine Translation


Imitation learning provides an appealing framework for autonomous control: in many tasks, demonstrations of preferred behavior can be readily obtained from human experts, removing the need for costly and potentially dangerous online data collection in the real world.

However, policies learned with imitation learning have limited flexibility to accommodate varied goals at test time.

Model-based reinforcement learning (MBRL) offers considerably more flexibility, since a predictive model learned from data can be used to achieve various goals at test time.

However, MBRL suffers from two shortcomings.

First, the model does not help to choose desired or safe outcomes -- its dynamics estimate only what is possible, not what is preferred.

Second, MBRL typically requires additional online data collection to ensure that the model is accurate in those situations that are actually encountered when attempting to achieve test time goals.

Collecting this data with a partially trained model can be dangerous and time-consuming.

In this paper, we aim to combine the benefits of imitation learning and MBRL, and propose imitative models: probabilistic predictive models able to plan expert-like trajectories to achieve arbitrary goals.

We find this method substantially outperforms both direct imitation and MBRL in a simulated autonomous driving task, and can be learned efficiently from a fixed set of expert demonstrations without additional online data collection.

We also show our model can flexibly incorporate user-supplied costs at test-time, can plan to sequences of goals, and can even perform well with imprecise goals, including goals on the wrong side of the road.

Hybrid Vision-Driven Imitation Learning and Model-Based Reinforcement Learning for Planning, Forecasting, and Control

Deep Imitative Models for Flexible Inference, Planning, and Control


Uncertainty estimation and ensembling methods go hand-in-hand.

Uncertainty estimation is one of the main benchmarks for assessment of ensembling performance.

At the same time, deep learning ensembles have provided state-of-the-art results in uncertainty estimation.

In this work, we focus on in-domain uncertainty for image classification.

We explore the standards for its quantification and point out pitfalls of existing metrics.

Avoiding these pitfalls, we perform a broad study of different ensembling techniques.

To provide more insight in the broad comparison, we introduce the deep ensemble equivalent (DEE) and show that many sophisticated ensembling techniques are equivalent to an ensemble of very few independently trained networks in terms of the test log-likelihood.

We highlight the problems with common metrics of in-domain uncertainty and perform a broad study of modern ensembling techniques.

Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning


Formal verification techniques that compute provable guarantees on properties of machine learning models, like robustness to norm-bounded adversarial perturbations, have yielded impressive results.

Although most techniques developed so far requires knowledge of the architecture of the machine learning model and remains hard to scale to complex prediction pipelines, the method of randomized smoothing has been shown to overcome many of these obstacles.

By requiring only black-box access to the underlying model, randomized smoothing scales to large architectures and is agnostic to the internals of the network.

However, past work on randomized smoothing has focused on restricted classes of smoothing measures or perturbations (like Gaussian or discrete) and has only been able to prove robustness with respect to simple norm bounds.

In this paper we introduce a general framework for proving robustness properties of smoothed machine learning models in the black-box setting.

Specifically, we extend randomized smoothing procedures to handle arbitrary smoothing measures and prove robustness of the smoothed classifier by using $f$-divergences.

Our methodology achieves state-of-the-art}certified robustness on MNIST, CIFAR-10 and ImageNet and also audio classification task, Librispeech, with respect to several classes of adversarial perturbations.

Develop a general framework to establish certified robustness of ML models against various classes of adversarial perturbations

A FRAMEWORK  FOR ROBUSTNESS CERTIFICATION  OF SMOOTHED CLASSIFIERS USING  F-DIVERGENCES


The ability to decompose complex multi-object scenes into meaningful abstractions like objects is fundamental to achieve higher-level cognition.

Previous approaches for unsupervised object-oriented scene representation learning are either based on spatial-attention or scene-mixture approaches and limited in scalability which is a main obstacle towards modeling real-world scenes.

In this paper, we propose a generative latent variable model, called SPACE, that provides a uniﬁed probabilistic modeling framework that combines the best of spatial-attention and scene-mixture approaches.

SPACE can explicitly provide factorized object representations for foreground objects while also decomposing background segments of complex morphology.

Previous models are good at either of these, but not both.

SPACE also resolves the scalability problems of previous methods by incorporating parallel spatial-attention and thus is applicable to scenes with a large number of objects without performance degradations.

We show through experiments on Atari and 3D-Rooms that SPACE achieves the above properties consistently in comparison to SPAIR, IODINE, and GENESIS.

Results of our experiments can be found on our project website: https://sites.google.com/view/space-project-page

We propose a generative latent variable model for unsupervised scene decomposition that provides factorized object representation per foreground object while also decomposing background segments of complex morphology.

SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition


We propose a single neural probabilistic model based on variational autoencoder that can be conditioned on an arbitrary subset of observed features and then sample the remaining features in "one shot".

The features may be both real-valued and categorical.

Training of the model is performed by stochastic variational Bayes.

The experimental evaluation on synthetic data, as well as feature imputation and image inpainting problems, shows the effectiveness of the proposed approach and diversity of the generated samples.

We propose an extension of conditional variational autoencoder that allows conditioning on an arbitrary subset of the features and sampling the remaining ones.

Variational Autoencoder with Arbitrary Conditioning

variational autoencoder
arbitrary
conditioning


We explore efficient neural architecture search methods and show that a simple yet powerful evolutionary algorithm can discover new architectures with excellent performance.

Our approach combines a novel hierarchical genetic representation scheme that imitates the modularized design pattern commonly adopted by human experts, and an expressive search space that supports complex topologies.

Our algorithm efficiently discovers architectures that outperform a large number of manually designed models for image classification, obtaining top-1 error of 3.6% on CIFAR-10 and 20.3% when transferred to ImageNet, which is competitive with the best existing neural architecture search approaches.

We also present results using random search, achieving 0.3% less top-1 accuracy on CIFAR-10 and 0.1% less on ImageNet whilst reducing the search time from 36 hours down to 1 hour.

In this paper we propose a hierarchical architecture representation in which doing random or evolutionary architecture search yields highly competitive results using fewer computational resources than the prior art.

Hierarchical Representations for Efficient Architecture Search

architecture search
representation
hierarchical


In visual planning (VP), an agent learns to plan goal-directed behavior from observations of a dynamical system obtained offline, e.g., images obtained from self-supervised robot interaction.

VP algorithms essentially combine data-driven perception and planning, and are important for robotic manipulation and navigation domains, among others.

A recent and promising approach to VP is the semi-parametric topological memory (SPTM) method, where image samples are treated as nodes in a graph, and the connectivity in the graph is learned using deep image classification.

Thus, the learned graph represents the topological connectivity of the data, and planning can be performed using conventional graph search methods.

However, training SPTM necessitates a suitable loss function for the connectivity classifier, which requires non-trivial manual tuning.

More importantly, SPTM is constricted in its ability to generalize to changes in the domain, as its graph is constructed from direct observations and thus requires collecting new samples for planning.

In this paper, we propose Hallucinative Topological Memory (HTM), which overcomes these shortcomings.

In HTM, instead of training a discriminative classifier we train an energy function using contrastive predictive coding.

In addition, we learn a conditional VAE model that generates samples given a context image of the domain, and use these hallucinated samples for building the connectivity graph, allowing for zero-shot generalization to domain changes.

In simulated domains, HTM outperforms conventional SPTM and visual foresight methods in terms of both plan quality and success in long-horizon planning.

We propose Hallucinative Topological Memory (HTM), a visual planning algorithm that can perform zero-shot long horizon planning in new environments.

Hallucinative Topological Memory for Zero-Shot Visual Planning

hallucinative topological memory
visual planning
zero-shot


Deep Neural Networks (DNNs) thrive in recent years in which Batch Normalization (BN) plays an indispensable role.

However, it has been observed that BN is costly due to the reduction operations.

In this paper, we propose alleviating the BN’s cost by using only a small fraction of data for mean & variance estimation at each iteration.

The key challenge to reach this goal is how to achieve a satisfactory balance between normalization effectiveness and execution efficiency.

We identify that the effectiveness expects less data correlation while the efficiency expects regular execution pattern.

To this end, we propose two categories of approach: sampling or creating few uncorrelated data for statistics’ estimation with certain strategy constraints.

The former includes “Batch Sampling (BS)” that randomly selects few samples from each batch and “Feature Sampling (FS)” that randomly selects a small patch from each feature map of all samples, and the latter is “Virtual Dataset Normalization (VDN)” that generates few synthetic random samples.

Accordingly, multi-way strategies are designed to reduce the data correlation for accurate estimation and optimize the execution pattern for running acceleration in the meantime.

All the proposed methods are comprehensively evaluated on various DNN models, where an overall training speedup by up to 21.7% on modern GPUs can be practically achieved without the support of any specialized libraries, and the loss of model accuracy and convergence rate are negligible.

Furthermore, our methods demonstrate powerful performance when solving the well-known “micro-batch normalization” problem in the case of tiny batch size.

We propose accelerating Batch Normalization (BN) through sampling less correlated data for reduction operations  with regular execution pattern, which achieves up to 2x and 20% speedup for BN itself and the overall training, respectively.

Effective and Efficient Batch Normalization Using Few Uncorrelated Data for Statistics' Estimation


Federated learning allows edge devices to collaboratively learn a shared model while keeping the training data on device, decoupling the ability to do model training from the need to store the data in the cloud.

We propose Federated matched averaging (FedMA) algorithm designed for federated learning of modern neural network architectures e.g. convolutional neural networks (CNNs) and LSTMs.

FedMA constructs the shared global model in a layer-wise manner by matching and averaging hidden elements (i.e. channels for convolution layers; hidden states for LSTM; neurons for fully connected layers) with similar feature extraction signatures.

Our experiments indicate that FedMA outperforms popular state-of-the-art federated learning algorithms on deep CNN and LSTM architectures trained on real world datasets, while improving the communication efficiency.

Communication efficient federated learning with layer-wise matching

Federated Learning with Matched Averaging


We present SOSELETO (SOurce SELEction for Target Optimization), a new method for exploiting a source dataset to solve a classification problem on a target dataset.

SOSELETO is based on the following simple intuition: some source examples are more informative than others for the target problem.

To capture this intuition, source samples are each given weights; these weights are solved for jointly with the source and target classification problems via a bilevel optimization scheme.

The target therefore gets to choose the source samples which are most informative for its own classification task.

Furthermore, the bilevel nature of the optimization acts as a kind of regularization on the target, mitigating overfitting.

SOSELETO may be applied to both classic transfer learning, as well as the problem of training on datasets with noisy labels; we show state of the art results on both of these problems.

Learning with limited training data by exploiting "helpful" instances from a rich data source.

SOSELETO: A Unified Approach to Transfer Learning and Training with Noisy Labels


Derivative-free optimization (DFO) using trust region methods is frequently used for machine learning applications, such as (hyper-)parameter optimization without the derivatives of objective functions known.

Inspired by the recent work in continuous-time minimizers, our work models the common trust region methods with the exploration-exploitation using a dynamical system coupling a pair of dynamical processes.

While the first exploration process searches the minimum of the blackbox function through minimizing a time-evolving surrogation function, another exploitation process updates the surrogation function time-to-time using the points traversed by the exploration process.

The efficiency of derivative-free optimization thus depends on ways the two processes couple.

In this paper, we propose a novel dynamical system, namely \ThePrev---\underline{S}tochastic \underline{H}amiltonian \underline{E}xploration and \underline{E}xploitation, that surrogates the subregions of blackbox function using a time-evolving quadratic function, then explores and tracks the minimum of the quadratic functions using a fast-converging Hamiltonian system.

The \ThePrev\ algorithm is later provided as a discrete-time numerical approximation to the system.

To further accelerate optimization, we present \TheName\ that parallelizes multiple \ThePrev\ threads for concurrent exploration and exploitation.

Experiment results based on a wide range of machine learning applications show that \TheName\ outperform a boarder range of derivative-free optimization algorithms with faster convergence speed under the same settings.

a new derivative-free optimization algorithms derived from Nesterov's accelerated gradient methods and Hamiltonian dynamics

SHE2: Stochastic Hamiltonian Exploration and Exploitation for Derivative-Free Optimization


Binarized Neural networks (BNNs) have been shown to be effective in improving network efficiency during the inference phase, after the network has been trained.

However, BNNs only binarize the model parameters and activations during propagations.

Therefore, BNNs do not offer significant efficiency improvements during training, since the gradients are still propagated and used with high precision.

We show there is no inherent difficulty in training BNNs using "Binarized BackPropagation" (BBP), in which we also binarize the gradients.

To avoid significant degradation in test accuracy, we simply increase the number of filter maps in a each convolution layer.

Using BBP on dedicated hardware can potentially significantly improve the execution efficiency (\emph{e.g.}, reduce dynamic memory footprint, memory bandwidth and computational energy) and speed up the training process with an appropriate hardware support, even after such an increase in network size.

Moreover, our method is ideal for distributed learning as it reduces the communication costs significantly (e.g., by ~32).

Using this method, we demonstrate a minimal loss in classification accuracy on several datasets and topologies.

Binarized Back-Propagation all you need for completely binarized training is to is to inflate the size of the network

Binarized Back-Propagation: Training Binarized Neural Networks with Binarized Gradients

binarize back-propagation
network
training


The weight initialization and the activation function of deep neural networks have a crucial impact on the performance of the training procedure.

An inappropriate selection can lead to the loss of information of the input during forward propagation and the exponential vanishing/exploding of gradients during back-propagation.

Understanding the theoretical properties of untrained random networks is key to identifying which deep networks may be trained successfully as recently demonstrated by Schoenholz et al. (2017) who showed that for deep feedforward neural networks only a specific choice of hyperparameters known as the `edge of chaos' can lead to good performance.

We complete this analysis by providing quantitative results showing that, for a class of ReLU-like activation functions, the information propagates indeed deeper for an initialization at the edge of chaos.

By further extending this analysis, we identify a class of activation functions that improve the information propagation over ReLU-like functions.

This class includes the Swish activation, $\phi_{swish}(x) = x \cdot \text{sigmoid}(x)$, used in Hendrycks & Gimpel (2016),

Elfwing et al. (2017) and Ramachandran et al. (2017).

This provides a theoretical grounding for the excellent empirical performance of $\phi_{swish}$ observed in these contributions.

We complement those previous results by illustrating the benefit of using a random initialization on the edge of chaos in this context.

How to effectively choose Initialization and Activation function for deep neural networks

On the Selection of Initialization and Activation Function for Deep Neural Networks

initialization activation function deep neural network


The driving force behind the recent success of LSTMs has been their ability to learn complex and non-linear relationships.

Consequently, our inability to describe these relationships has led to LSTMs being characterized as black boxes.

To this end, we introduce contextual decomposition (CD), an interpretation algorithm for analysing individual predictions made by standard LSTMs, without any changes to the underlying model.

By decomposing the output of a LSTM, CD captures the contributions of combinations of words or variables to the final prediction of an LSTM.

On the task of sentiment analysis with the Yelp and SST data sets, we show that CD is able to reliably identify words and phrases of contrasting sentiment, and how they are combined to yield the LSTM's final prediction.

Using the phrase-level labels in SST, we also demonstrate that CD is able to successfully extract positive and negative negations from an LSTM, something which has not previously been done.

We introduce contextual decompositions, an interpretation algorithm for LSTMs capable of extracting word, phrase and interaction-level importance score

Beyond Word Importance:  Contextual Decomposition to Extract Interactions from LSTMs

contextual decomposition
extract
lstms
word
importance


Most deep learning-based models for speech enhancement have mainly focused on estimating the magnitude of spectrogram while reusing the phase from noisy speech for reconstruction.

This is due to the difficulty of estimating the phase of clean speech.

To improve speech enhancement performance, we tackle the phase estimation problem in three ways.

First, we propose Deep Complex U-Net, an advanced U-Net structured model incorporating well-defined complex-valued building blocks to deal with complex-valued spectrograms.

Second, we propose a polar coordinate-wise complex-valued masking method to reflect the distribution of complex ideal ratio masks.

Third, we define a novel loss function, weighted source-to-distortion ratio (wSDR) loss, which is designed to directly correlate with a quantitative evaluation measure.

Our model was evaluated on a mixture of the Voice Bank corpus and DEMAND database, which has been widely used by many deep learning models for speech enhancement.

Ablation experiments were conducted on the mixed dataset showing that all three proposed approaches are empirically valid.

Experimental results show that the proposed method achieves state-of-the-art performance in all metrics, outperforming previous approaches by a large margin.

This paper proposes a novel complex masking method for speech enhancement along with a loss function for efficient phase estimation.

Phase-Aware Speech Enhancement with Deep Complex U-Net


All living organisms struggle against the forces of nature to carve out niches where

We propose that such a search for order amidst

chaos might offer a unifying principle for the emergence of useful behaviors in

We formalize this idea into an unsupervised reinforcement learning

method called surprise minimizing RL (SMiRL).

objective of maximizing the probability of observed states under a model trained on

The resulting agents acquire several proactive behaviors

to seek and maintain stable states such as balancing and damage avoidance, that

are closely tied to the affordances of the environment and its prevailing sources

of entropy, such as winds, earthquakes, and other agents.

our surprise minimizing agents can successfully play Tetris, Doom, and control

a  humanoid  to  avoid  falls,  without  any  task-specific  reward  supervision.

We
further show that SMiRL can be used as an unsupervised pre-training objective

that substantially accelerates subsequent reward-driven learning

Learning emergent behavior by minimizing Bayesian surprise with RL in natural environments with entropy.

SMiRL: Surprise Minimizing RL in Entropic Environments


Learning to learn is a powerful paradigm for enabling models to learn from data more effectively and efficiently.

A popular approach to meta-learning is to train a recurrent model to read in a training dataset as input and output the parameters of a learned model, or output predictions for new test inputs.

Alternatively, a more recent approach to meta-learning aims to acquire deep representations that can be effectively fine-tuned, via standard gradient descent, to new tasks.

In this paper, we consider the meta-learning problem from the perspective of universality, formalizing the notion of learning algorithm approximation and comparing the expressive power of the aforementioned recurrent models to the more recent approaches that embed gradient descent into the meta-learner.

In particular, we seek to answer the following question: does deep representation combined with standard gradient descent have sufficient capacity to approximate any learning algorithm?

We find that this is indeed true, and further find, in our experiments, that gradient-based meta-learning consistently leads to learning strategies that generalize more widely compared to those represented by recurrent models.

Deep representations combined with gradient descent can approximate any learning algorithm.

Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm

gradient descent approximate learning algorithm


Brain-Computer Interfaces (BCI) may help patients with faltering communication abilities due to neurodegenerative diseases produce text or speech by direct neural processing.

However, their practical realization has proven difficult due to limitations in speed, accuracy, and generalizability of existing interfaces.

To this end, we aim to create a BCI that decodes text directly from neural signals.

We implement a framework that initially isolates frequency bands in the input signal encapsulating differential information regarding production of various phonemic classes.

These bands form a feature set that feeds into an LSTM which discerns at each time point probability distributions across all phonemes uttered by a subject.

Finally, a particle filtering algorithm temporally smooths these probabilities incorporating prior knowledge of the English language to output text corresponding to the decoded word.

Further, in producing an output, we abstain from constraining the reconstructed word to be from a given bag-of-words, unlike previous studies.

The empirical success of our proposed approach, offers promise for the employment of such an interface by patients in unfettered, naturalistic environments.

We present an open-loop brain-machine interface whose performance is unconstrained to the traditionally used bag-of-words approach.

Translating neural signals to text using a Brain-Computer Interface


Transfer learning for feature extraction can be used to exploit deep representations in contexts where there is very few training data, where there are limited computational resources, or when tuning the hyper-parameters needed for training is not an option.

While previous contributions to feature extraction propose embeddings based on a single layer of the network, in this paper we propose a full-network embedding which successfully integrates convolutional and fully connected features, coming from all layers of a deep convolutional neural network.

To do so, the embedding normalizes features in the context of the problem, and discretizes their values to reduce noise and regularize the embedding space.

Significantly, this also reduces the computational cost of processing the resultant representations.

The proposed method is shown to outperform single layer embeddings on several image classification tasks, while also being more robust to the choice of the pre-trained model used for obtaining the initial features.

The performance gap in classification accuracy between thoroughly tuned solutions and the full-network embedding is also reduced, which makes of the proposed approach a competitive solution for a large set of applications.

We present a full-network embedding of CNN which outperforms single layer embeddings for transfer learning tasks.

An Out-of-the-box Full-network Embedding for Convolutional Neural Networks


We present a novel network pruning algorithm called Dynamic Sparse Training that can jointly ﬁnd the optimal network parameters and sparse network structure in a uniﬁed optimization process with trainable pruning thresholds.

These thresholds can have ﬁne-grained layer-wise adjustments dynamically via backpropagation.

We demonstrate that our dynamic sparse training algorithm can easily train very sparse neural network models with little performance loss using the same training epochs as dense models.

Dynamic Sparse Training achieves prior art performance compared with other sparse training algorithms on various network architectures.

Additionally, we have several surprising observations that provide strong evidence to the effectiveness and efﬁciency of our algorithm.

These observations reveal the underlying problems of traditional three-stage pruning algorithms and present the potential guidance provided by our algorithm to the design of more compact network architectures.

We present a novel network pruning method that can find the optimal sparse structure during the training process with trainable pruning threshold

Dynamic Sparse Training: Find Efficient Sparse Network From Scratch With Trainable Masked Layers


To which extent can successful machine learning inform our understanding of biological learning?

One popular avenue of inquiry in recent years has been to directly map such algorithms into a realistic circuit implementation.

Here we focus on learning in recurrent networks and investigate a range of learning algorithms.

Our approach decomposes them into their computational building blocks and discusses their abstract potential as biological operations.

This alternative strategy provides a “lazy” but principled way of evaluating ML ideas in terms of their biological plausibility

We evaluate new ML learning algorithms' biological plausibility in the abstract based on mathematical operations needed

Evaluating biological plausibility of learning algorithms the lazy way

biological plausibility
learn algorithm
evaluate


In recent years several adversarial attacks and defenses have been proposed.

Often seemingly robust models turn out to be non-robust when more sophisticated attacks are used.

One way out of this dilemma are provable robustness guarantees.

While provably robust models for specific $l_p$-perturbation models have been developed, we show that they do not come with any guarantee against other $l_q$-perturbations.

We propose a new regularization scheme, MMR-Universal, for ReLU networks which enforces robustness wrt $l_1$- \textit{and} $l_\infty$-perturbations and show how that leads to the first provably robust models wrt any $l_p$-norm for $p\geq 1$.

We introduce a method to train models with provable robustness wrt all the $l_p$-norms for $p\geq 1$ simultaneously.

Provable robustness against all adversarial $l_p$-perturbations for $p\geq 1$


In this paper, we propose a new control framework called the moving endpoint control to restore images corrupted by different degradation levels in one model.

The proposed control problem contains a restoration dynamics which is modeled by an RNN.

The moving endpoint, which is essentially the terminal time of the associated dynamics, is determined by a policy network.

We call the proposed model the dynamically unfolding recurrent restorer (DURR).

Numerical experiments show that DURR is able to achieve state-of-the-art performances on blind image denoising and JPEG image deblocking.

Furthermore, DURR can well generalize to images with higher degradation levels that are not included in the training stage.

We propose a novel method to handle image degradations of different levels by learning a diffusion terminal time. Our model can generalize to unseen degradation level and different noise statistic.

Dynamically Unfolding Recurrent Restorer: A Moving Endpoint Control Method for Image Restoration


The Wasserstein distance received a lot of attention recently in the community of machine learning, especially for its principled way of comparing distributions.

It has found numerous applications in several hard problems, such as domain adaptation, dimensionality reduction or generative models.

However, its use is still limited by a heavy computational cost.

Our goal is to alleviate this problem by providing an approximation mechanism that allows to break its inherent complexity.

It relies on the search of an embedding where the Euclidean distance mimics the Wasserstein distance.

We show that such an embedding can be found with a siamese architecture associated with a decoder network that allows to move from the embedding space back to the original input space.

Once this embedding has been found, computing optimization problems in the Wasserstein space (e.g. barycenters, principal directions or even archetypes) can be conducted extremely fast.

Numerical experiments supporting this idea are conducted on image datasets, and show the wide potential benefits of our method.

We show that it is possible to fastly approximate Wasserstein distances computation by finding an appropriate embedding where Euclidean distance emulates the Wasserstein distance

Learning Wasserstein Embeddings


Continuous Bag of Words (CBOW) is a powerful text embedding method.

Due to its strong capabilities to encode word content, CBOW embeddings perform well on a wide range of downstream tasks while being efficient to compute.

However, CBOW is not capable of capturing the word order.

The reason is that the computation of CBOW's word embeddings is commutative, i.e., embeddings of XYZ and ZYX are the same.

In order to address this shortcoming, we propose a

learning algorithm for the Continuous Matrix Space Model, which we call Continual Multiplication of Words (CMOW).

Our algorithm is an adaptation of word2vec, so that it can be trained on large quantities of unlabeled text.

We empirically show that CMOW better captures linguistic properties, but it is inferior to CBOW in memorizing word content.

Motivated by these findings, we propose a hybrid model that combines the strengths of CBOW and CMOW.

Our results show that the hybrid CBOW-CMOW-model retains CBOW's strong ability to memorize word content while at the same time substantially improving its ability to encode other linguistic information by 8%.

As a result, the hybrid also performs better on 8 out of 11 supervised downstream tasks with an average improvement of 1.2%.

We present a novel training scheme for efficiently obtaining order-aware sentence representations.

CBOW Is Not All You Need: Combining CBOW with the Compositional Matrix Space Model


This paper proposes Metagross (Meta Gated Recursive Controller), a new neural sequence modeling unit.

Our proposed unit is characterized by recursive parameterization of its gating functions, i.e., gating mechanisms of Metagross are controlled by instances of itself, which are repeatedly called in a recursive fashion.

This can be interpreted as a form of meta-gating and recursively parameterizing a recurrent model.

We postulate that our proposed inductive bias provides modeling benefits pertaining to learning with inherently hierarchically-structured sequence data (e.g., language, logical or music tasks).

To this end, we conduct extensive experiments on recursive logic tasks (sorting, tree traversal, logical inference), sequential pixel-by-pixel classification, semantic parsing, code generation, machine translation and polyphonic music modeling, demonstrating the widespread utility of the proposed approach, i.e., achieving state-of-the-art (or close) performance on all tasks.

Recursive Parameterization of Recurrent Models improve performance

Metagross: Meta Gated Recursive Controller Units for Sequence Modeling


Which generative model is the most suitable for Continual Learning?

This paper aims at evaluating and comparing generative models on disjoint sequential image generation tasks.

We investigate how several models learn and forget, considering various strategies: rehearsal, regularization, generative replay and fine-tuning.

We used two quantitative metrics to estimate the generation quality and memory ability.

We experiment with sequential tasks on three commonly used benchmarks for Continual Learning (MNIST, Fashion MNIST and CIFAR10).

We found that among all models, the original GAN performs best and among Continual Learning strategies, generative replay outperforms all other methods.

Even if we found satisfactory combinations on MNIST and Fashion MNIST, training generative models sequentially on CIFAR10 is particularly instable, and remains a challenge.

A comparative study of generative models on Continual Learning scenarios.

Generative Models from the perspective of Continual Learning


We propose a new sample-efficient methodology, called Supervised Policy Update (SPU), for deep reinforcement learning.

Starting with data generated by the current policy, SPU formulates and solves a constrained optimization problem in the non-parameterized proximal policy space.

Using supervised regression, it then converts the optimal non-parameterized policy to a parameterized policy, from which it draws new samples.

The methodology is general in that it applies to both discrete and continuous action spaces, and can handle a wide variety of proximity constraints for the non-parameterized optimization problem.

We show how the Natural Policy Gradient and Trust Region Policy Optimization (NPG/TRPO) problems, and the Proximal Policy Optimization (PPO) problem can be addressed by this methodology.

The SPU implementation is much simpler than TRPO.

In terms of sample efficiency, our extensive experiments show SPU outperforms TRPO in Mujoco simulated robotic tasks and outperforms PPO in Atari video game tasks.

first posing and solving the sample efficiency optimization problem in the non-parameterized policy space, and then solving a supervised regression problem to find a parameterized policy that is near the optimal non-parameterized policy.

Supervised Policy Update for Deep Reinforcement Learning


Recent work on modeling neural responses in the primate visual system has benefited from deep neural networks trained on large-scale object recognition, and found a hierarchical correspondence between layers of the artificial neural network and brain areas along the ventral visual stream.

However, we neither know whether such task-optimized networks enable equally good models of the rodent visual system, nor if a similar hierarchical correspondence exists.

Here, we address these questions in the mouse visual system by extracting features at several layers of a convolutional neural network (CNN) trained on ImageNet to predict the responses of thousands of neurons in four visual areas (V1, LM, AL, RL) to natural images.

We found that the CNN features outperform classical subunit energy models, but found no evidence for an order of the areas we recorded via a correspondence to the hierarchy of CNN layers.

Moreover, the same CNN but with random weights provided an equivalently useful feature space for predicting neural responses.

Our results suggest that object recognition as a high-level task does not provide more discriminative features to characterize the mouse visual system than a random network.

Unlike in the primate, training on ethologically relevant visually guided behaviors -- beyond static object recognition -- may be needed to unveil the functional organization of the mouse visual cortex.

A goal-driven approach to model four mouse visual areas (V1, LM, AL, RL) based on deep neural networks trained on static object recognition does not unveil a functional organization of visual cortex unlike in primates

How well do deep neural networks trained on object recognition characterize the mouse visual system?

deep neural network train
object recognition
mouse visual


The reparameterization trick has become one of the most useful tools in the field of variational inference.

However, the reparameterization trick is based on the standardization transformation which restricts the scope of application of this method to distributions that have tractable inverse cumulative distribution functions or are expressible as deterministic transformations of such distributions.

In this paper, we generalized the reparameterization trick by allowing a general transformation.

We discover that the proposed model is a special case of control variate indicating that the proposed model can combine the advantages of CV and generalized reparameterization.

Based on the proposed gradient model, we propose a new polynomial-based gradient estimator which has better theoretical performance than the reparameterization trick under certain condition and can be applied to a larger class of variational distributions.

In studies of synthetic and real data, we show that our proposed gradient estimator has a significantly lower gradient variance than other state-of-the-art methods thus enabling a faster inference procedure.

a generalized transformation-based gradient model for variational inference

Generalized Transformation-based Gradient


To simultaneously capture syntax and semantics from a text corpus, we propose a new larger-context language model that extracts recurrent hierarchical semantic structure via a dynamic deep topic model to guide natural language generation.

Moving beyond a conventional language model that ignores long-range word dependencies and sentence order, the proposed model captures not only intra-sentence word dependencies, but also temporal transitions between sentences and inter-sentence topic dependences.

For inference, we develop a hybrid of stochastic-gradient MCMC and recurrent autoencoding variational Bayes.

Experimental results on a variety of real-world text corpora demonstrate that the proposed model not only outperforms state-of-the-art larger-context language models, but also learns interpretable recurrent multilayer topics and generates diverse sentences and paragraphs that are syntactically correct and semantically coherent.

We introduce a novel larger-context language model to simultaneously captures syntax and semantics, making it capable of generating highly interpretable sentences and paragraphs

Recurrent Hierarchical Topic-Guided Neural Language Models


We present a novel approach to train a natural media painting using reinforcement learning.

Given a reference image,  our formulation is based on stroke-based rendering that  imitates human drawing and can be learned from scratch without supervision.

Our painting agent computes a sequence of actions that represent the primitive painting strokes.

In order to ensure that the generated policy is predictable and controllable, we use a constrained learning method and train the painting agent  using the environment model and follows the commands encoded in an observation.

We have applied our approach on many benchmarks and our results demonstrate that our constrained agent can handle different painting media and different constraints in the action space to collaborate with humans or other agents.

We train a natural media painting agent using environment model. Based on our painting agent, we present a novel approach to train a constrained painting agent that follows the command encoded in the observation.

Training a Constrained Natural Media Painting Agent using Reinforcement Learning

natural medium paint agent
train constrained


Delusional bias is a fundamental source of error in approximate Q-learning.

To date, the only techniques that explicitly address delusion require comprehensive search using tabular value estimates.

In this paper, we develop efficient methods to mitigate delusional bias by training Q-approximators with labels that are "consistent" with the underlying greedy policy class.

We introduce a simple penalization scheme that encourages Q-labels used across training batches to remain (jointly) consistent with the expressible policy class.

We also propose a search framework that allows multiple Q-approximators to be generated and tracked, thus mitigating the effect of premature (implicit) policy commitments.

Experimental results demonstrate that these methods can improve the performance of Q-learning in a variety of Atari games, sometimes dramatically.

We developed a search framework and consistency penalty to mitigate delusional bias.

ConQUR: Mitigating Delusional Bias in Deep Q-Learning


The paper proposes and demonstrates a Deep Convolutional Neural Network (DCNN) architecture to identify users with disguised face attempting a fraudulent ATM transaction.

The recent introduction of Disguised Face Identification (DFI) framework proves the applicability of deep neural networks for this very problem.

All the ATMs nowadays incorporate a hidden camera in them and capture the footage of their users.

However, it is impossible for the police to track down the impersonators with disguised faces from the ATM footage.

The proposed deep convolutional neural network is trained to identify, in real time, whether the user in the captured image is trying to cloak his identity or not.

The output of the DCNN is then reported to the ATM to take appropriate steps and prevent the swindler from completing the transaction.

The network is trained using a dataset of images captured in similar situations as of an ATM.

The comparatively low background clutter in the images enables the network to demonstrate high accuracy in feature extraction and classification for all the different disguises.

Proposed System can prevent impersonators with facial disguises from completing a fraudulent transaction using a pre-trained DCNN.

APPLICATION OF DEEP CONVOLUTIONAL NEURAL NETWORK TO PREVENT ATM FRAUD BY FACIAL DISGUISE IDENTIFICATION


Auto-encoders are commonly used for unsupervised representation learning and for pre-training deeper neural networks.

When its activation function is linear and the encoding dimension (width of hidden layer) is smaller than the input dimension, it is well known that auto-encoder is optimized to learn the principal components of the data distribution (Oja1982).

However, when the activation is nonlinear and when the width is larger than the input dimension (overcomplete), auto-encoder behaves differently from PCA, and in fact is known to perform well empirically for sparse coding problems.

We provide a theoretical explanation for this empirically observed phenomenon, when rectified-linear unit (ReLu) is adopted as the activation function and the hidden-layer width is set to be large.

In this case, we show that, with significant probability, initializing the weight matrix of an auto-encoder by sampling from a spherical Gaussian distribution followed by stochastic gradient descent (SGD) training converges towards the ground-truth representation for a class of sparse dictionary learning models.

In addition, we can show that, conditioning on convergence, the expected convergence rate is O(1/t), where t is the number of updates.

Our analysis quantifies how increasing hidden layer width helps the training performance when random initialization is used, and how the norm of network weights influence the speed of SGD convergence.

theoretical analysis of nonlinear wide autoencoder

Demystifying overcomplete nonlinear auto-encoders: fast SGD convergence towards sparse representation from random initialization


Hierarchical agents have the potential to solve sequential decision making tasks with greater sample efficiency than their non-hierarchical counterparts because hierarchical agents can break down tasks into sets of subtasks that only require short sequences of decisions.

In order to realize this potential of faster learning, hierarchical agents need to be able to learn their multiple levels of policies in parallel so these simpler subproblems can be solved simultaneously.

Yet, learning multiple levels of policies in parallel is hard because it is inherently unstable: changes in a policy at one level of the hierarchy may cause changes in the transition and reward functions at higher levels in the hierarchy, making it difficult to jointly learn multiple levels of policies.

In this paper, we introduce a new Hierarchical Reinforcement Learning (HRL) framework, Hierarchical Actor-Critic (HAC), that can overcome the instability issues that arise when agents try to jointly learn multiple levels of policies.

The main idea behind HAC is to train each level of the hierarchy independently of the lower levels by training each level as if the lower level policies are already optimal.

We demonstrate experimentally in both grid world and simulated robotics domains that our approach can significantly accelerate learning relative to other non-hierarchical and hierarchical methods.

Indeed, our framework is the first to successfully learn 3-level hierarchies in parallel in tasks with continuous state and action spaces.

We introduce the first Hierarchical RL approach to successfully learn 3-level hierarchies in parallel in tasks with continuous state and action spaces.

Learning Multi-Level Hierarchies with Hindsight


Positive-unlabeled (PU) learning addresses the problem of learning a binary classifier from positive (P) and unlabeled (U) data.

It is often applied to situations where negative (N) data are difficult to be fully labeled.

However, collecting a non-representative N set that contains only a small portion of all possible N data can be much easier in many practical situations.

This paper studies a novel classification framework which incorporates such biased N (bN) data in PU learning.

The fact that the training N data are biased also makes our work very different from those of standard semi-supervised learning.

We provide an empirical risk minimization-based method to address this PUbN classification problem.

Our approach can be regarded as a variant of traditional example-reweighting algorithms, with the weight of each example computed through a preliminary step that draws inspiration from PU learning.

We also derive an estimation error bound for the proposed method.

Experimental results demonstrate the effectiveness of our algorithm in not only PUbN learning scenarios but also ordinary PU leaning scenarios on several benchmark datasets.

This paper studied the PUbN classification problem, where we incorporate biased negative (bN) data, i.e., negative data that is not fully representative of the true underlying negative distribution, into positive-unlabeled (PU) learning.

Classification from Positive, Unlabeled and Biased Negative Data


Reinforcement learning (RL) is frequently used to increase performance in text generation tasks,

notably through the use of Minimum Risk Training (MRT) and Generative Adversarial Networks (GAN).

However, little is known about what and how these methods learn in the context of MT.

We prove that one of the most common RL methods for MT does not optimize the

expected reward, as well as show that other methods take an infeasibly long time to converge.

In fact, our results suggest that RL practices in MT are likely to improve performance

only where the pre-trained parameters are already close to yielding the correct translation.

Our findings further suggest that observed gains may be due to effects unrelated to the training signal, concretely, changes in the shape of the distribution curve.

Reinforcment practices for machine translation performance gains might not come from better predictions.

On the Weaknesses of Reinforcement Learning for Neural Machine Translation


Significant advances have been made in Natural Language Processing (NLP) modelling since the beginning of 2018.

The new approaches allow for accurate results, even when there is little labelled data, because these NLP models can benefit from training on both task-agnostic and task-specific unlabelled data.

However, these advantages come with significant size and computational costs.

This workshop paper outlines how our proposed convolutional student architecture, having been trained by a distillation process from a large-scale model, can achieve 300x inference speedup and 39x reduction in parameter count.

In some cases, the student model performance surpasses its teacher on the studied tasks.

We train a small, efficient CNN with the same performance as the OpenAI Transformer on text classification tasks

Transformer to CNN: Label-scarce distillation for efficient text classification

cnn
transformer
efficient
text classification


Combining multiple function approximators in machine learning models typically leads to better performance and robustness compared with a single function.

In reinforcement learning, ensemble algorithms such as an averaging method and a majority voting method are not always optimal, because each function can learn fundamentally different optimal trajectories from exploration.

In this paper, we propose a Temporal Difference Weighted (TDW) algorithm, an ensemble method that adjusts weights of each contribution based on accumulated temporal difference errors.

The advantage of this algorithm is that it improves ensemble performance by reducing weights of Q-functions unfamiliar with current trajectories.

We provide experimental results for Gridworld tasks and Atari tasks that show significant performance improvements compared with baseline algorithms.

Ensemble method for reinforcement learning that weights Q-functions based on accumulated TD errors.

Temporal Difference Weighted Ensemble For Reinforcement Learning


This paper introduces the Behaviour Suite for Reinforcement Learning, or bsuite for short.

bsuite is a collection of carefully-designed experiments that investigate core capabilities of reinforcement learning (RL) agents with two objectives.

First, to collect clear, informative and scalable problems that capture key issues in the design of general and efficient learning algorithms.

Second, to study agent behaviour through their performance on these shared benchmarks.

To complement this effort, we open source this http URL, which automates evaluation and analysis of any agent on bsuite.

This library facilitates reproducible and accessible research on the core issues in RL, and ultimately the design of superior learning algorithms.

Our code is Python, and easy to use within existing projects.

We include examples with OpenAI Baselines, Dopamine as well as new reference implementations.

Going forward, we hope to incorporate more excellent experiments from the research community, and commit to a periodic review of bsuite from a committee of prominent researchers.

Bsuite is a collection of carefully-designed experiments that investigate the core capabilities of RL agents.


Sequence-to-sequence attention-based models are a promising approach for end-to-end speech recognition.

The increased model power makes the training procedure more difficult, and analyzing failure modes of these models becomes harder because of the end-to-end nature.

In this work, we present various analyses to better understand training and model properties.

We investigate on pretraining variants such as growing in depth and width, and their impact on the final performance, which leads to over 8% relative improvement in word error rate.

For a better understanding of how the attention process works, we study the encoder output and the attention energies and weights.

Our experiments were performed on Switchboard, LibriSpeech and Wall Street Journal.

improved pretraining, and analysing encoder output and attention

A comprehensive analysis on attention models


Validation is a key challenge in the search for safe autonomy.

Simulations are often either too simple to provide robust validation, or too complex to tractably compute.

Therefore, approximate validation methods are needed to tractably find failures without unsafe simplifications.

This paper presents the theory behind one such black-box approach: adaptive stress testing (AST).

We also provide three examples of validation problems formulated to work with AST.

A formulation for a black-box, reinforcement learning method to find the most-likely failure of a system acting in complex scenarios.

The Adaptive Stress Testing Formulation


Multi-step greedy policies have been extensively used in model-based Reinforcement Learning (RL) and in the case when a model of the environment is available (e.g., in the game of Go).

In this work, we explore the benefits of multi-step greedy policies in model-free RL when employed in the framework of multi-step Dynamic Programming (DP): multi-step Policy and Value Iteration.

These algorithms iteratively solve short-horizon decision problems and converge to the optimal solution of the original one.

By using model-free algorithms as solvers of the short-horizon problems we derive fully model-free algorithms which are instances of the multi-step DP framework.

As model-free algorithms are prone to instabilities w.r.t. the decision problem horizon, this simple approach can help in mitigating these instabilities and results in an improved model-free algorithms.

We test this approach and show results on both discrete and continuous control problems.

Use model free algorithms like DQN/TRPO to solve short horizon problems (model free) iteratively in a Policy/Value Iteration fashion.

Multi-step Greedy Policies in Model-Free Deep Reinforcement Learning


The adversarial training procedure proposed by Madry et al. (2018) is one of the most effective methods to defend against adversarial examples in deep neural net- works (DNNs).

In our paper, we shed some lights on the practicality and the hardness of adversarial training by showing that the effectiveness (robustness on test set) of adversarial training has a strong correlation with the distance between a test point and the manifold of training data embedded by the network.

Test examples that are relatively far away from this manifold are more likely to be vulnerable to adversarial attacks.

Consequentially, an adversarial training based defense is susceptible to a new class of attacks, the “blind-spot attack”, where the input images reside in “blind-spots” (low density regions) of the empirical distri- bution of training data but is still on the ground-truth data manifold.

For MNIST, we found that these blind-spots can be easily found by simply scaling and shifting image pixel values.

Most importantly, for large datasets with high dimensional and complex data manifold (CIFAR, ImageNet, etc), the existence of blind-spots in adversarial training makes defending on any valid test examples difficult due to the curse of dimensionality and the scarcity of training data.

Additionally, we find that blind-spots also exist on provable defenses including (Kolter & Wong, 2018) and (Sinha et al., 2018) because these trainable robustness certificates can only be practically optimized on a limited set of training data.

We show that even the strongest adversarial training methods cannot defend against adversarial examples crafted on slightly scaled and shifted test images.

The Limitations of Adversarial Training and the Blind-Spot Attack


Edge intelligence especially binary neural network (BNN) has attracted considerable attention of the artificial intelligence community recently.

BNNs significantly reduce the computational cost, model size, and memory footprint.

However, there is still a performance gap between the successful full-precision neural network with ReLU activation and BNNs.

We argue that the accuracy drop of BNNs is due to their geometry.

We analyze the behaviour of the full-precision neural network with ReLU activation and compare it with its binarized counterpart.

This comparison suggests random bias initialization as a remedy to activation saturation in full-precision networks and  leads us towards an improved BNN training.

Our numerical experiments confirm our geometric intuition.

Improve saturating activations (sigmoid, tanh, htanh etc.) and Binarized Neural Network with Bias Initialization

Random Bias Initialization Improving Binary Neural Network Training

bias initialization
neural network
improve


Understanding how people represent categories is a core problem in cognitive science, with the flexibility of human learning remaining a gold standard to which modern artificial intelligence and machine learning aspire.

Decades of psychological research have yielded a variety of formal theories of categories, yet validating these theories with naturalistic stimuli remains a challenge.

The problem is that human category representations cannot be directly observed and running informative experiments with naturalistic stimuli such as images requires having a workable representation of these stimuli.

Deep neural networks have recently been successful in a range of computer vision tasks and provide a way to represent the features of images.

In this paper, we introduce a method for estimating the structure of human categories that draws on ideas from both cognitive science and machine learning, blending human-based algorithms with state-of-the-art deep representation learners.

We provide qualitative and quantitative results as a proof of concept for the feasibility of the method.

Samples drawn from human distributions rival the quality of current state-of-the-art generative models and outperform alternative methods for estimating the structure of human categories.

using deep neural networks and clever algorithms to capture human mental visual concepts

Capturing Human Category Representations by Sampling in Deep Feature Spaces


The application of deep recurrent networks to audio transcription has led to impressive gains in automatic speech recognition (ASR) systems.

Many have demonstrated that small adversarial perturbations can fool deep neural networks into incorrectly predicting a specified target with high confidence.

Current work on fooling ASR systems have focused on white-box attacks, in which the model architecture and parameters are known.

In this paper, we adopt a black-box approach to adversarial generation, combining the approaches of both genetic algorithms and gradient estimation to solve the task.

We achieve a 89.25% targeted attack similarity after 3000 generations while maintaining 94.6% audio file similarity.

We present a novel black-box targeted attack that is able to fool state of the art speech to text transcription.

Targeted Adversarial Examples for Black Box Audio Systems


Recent progress on physics-based character animation has shown impressive breakthroughs on human motion synthesis, through imitating motion capture data via deep reinforcement learning.

However, results have mostly been demonstrated on imitating a single distinct motion pattern, and do not generalize to interactive tasks that require flexible motion patterns due to varying human-object spatial configurations.

To bridge this gap, we focus on one class of interactive tasks---sitting onto a chair.

We propose a hierarchical reinforcement learning framework which relies on a collection of subtask controllers trained to imitate simple, reusable mocap motions, and a meta controller trained to execute the subtasks properly to complete the main task.

We experimentally demonstrate the strength of our approach over different single level and hierarchical baselines.

We also show that our approach can be applied to motion prediction given an image input.

A video highlight can be found at https://youtu.be/XWU3wzz1ip8/.

Synthesizing human motions on interactive tasks using mocap data and hierarchical RL.

Learning to Sit: Synthesizing Human-Chair Interactions via Hierarchical Control


We propose studying GAN training dynamics as regret minimization, which is in contrast to the popular view that there is consistent minimization of a divergence between real and generated distributions.

We analyze the convergence of GAN training from this new point of view to understand why mode collapse happens.

We hypothesize the existence of undesirable local equilibria in this non-convex game to be responsible for mode collapse.

We observe that these local equilibria often exhibit sharp gradients of the discriminator function around some real data points.

We demonstrate that these degenerate local equilibria can be avoided with a gradient penalty scheme called DRAGAN.

We show that DRAGAN enables faster training, achieves improved stability with fewer mode collapses, and leads to generator networks with better modeling performance across a variety of architectures and objective functions.

Analysis of convergence and mode collapse by studying GAN training process as regret minimization

On Convergence and Stability of GANs


Deep neural networks (DNNs) have attained surprising achievement during the last decade due to the advantages of automatic feature learning and freedom of expressiveness.

However,  their interpretability remains mysterious because DNNs are complex combinations of linear and nonlinear transformations.

Even though many models have been proposed to explore the interpretability of DNNs,  several challenges remain unsolved:

1) The lack of interpretability quantity measures for DNNs,  2) the lack of theory for stability of DNNs, and

3) the difficulty to solve nonconvex DNN problems with interpretability constraints.

To address these challenges simultaneously, this paper presents a novel intrinsic interpretability evaluation framework for DNNs.

Specifically, Four independent properties of interpretability are defined based on existing works.

Moreover, we investigate the theory for the stability of DNNs, which is an important aspect of interpretability, and prove that DNNs are generally stable given different activation functions.

Finally, an extended version of deep learning Alternating Direction Method of Multipliers (dlADMM) are proposed to solve DNN problems with interpretability constraints efficiently and accurately.

Extensive experiments on several benchmark datasets validate several DNNs by our proposed interpretability framework.

We propose a novel framework to evaluate the interpretability of neural network.

Interpretability Evaluation Framework for Deep Neural Networks

framework
neural network
interpretability


Reinforcement learning (RL) typically defines a discount factor as part of the Markov Decision Process.

The discount factor values future rewards by an exponential scheme that leads to theoretical convergence guarantees of the Bellman equation.

However, evidence from psychology, economics and neuroscience suggests that humans and animals instead have hyperbolic time-preferences.

Here we extend earlier work of Kurth-Nelson and Redish and propose an efficient deep reinforcement learning agent that acts via hyperbolic discounting and other non-exponential discount mechanisms.

We demonstrate that a simple approach approximates hyperbolic discount functions while still using familiar temporal-difference learning techniques in RL.

Additionally, and independent of hyperbolic discounting, we make a surprising discovery that simultaneously learning value functions over multiple time-horizons is an effective auxiliary task which often improves over state-of-the-art methods.

A deep RL agent that learns hyperbolic (and other non-exponential) Q-values and a new multi-horizon auxiliary task.

Hyperbolic Discounting and Learning Over Multiple Horizons


Emotion is playing a great role in our daily lives.

The necessity and importance of an automatic Emotion recognition system is getting increased.

Traditional approaches of emotion recognition are based on facial images, measurements of heart rates, blood pressure, temperatures, tones of voice/speech, etc.

However, these features can potentially be changed to fake features.

So to detect hidden and real features that is not controlled by the person are data measured from brain signals.

There are various ways of measuring brain waves: EEG, MEG, FMRI, etc.

On the bases of cost effectiveness and performance trade-offs, EEG is chosen for emotion recognition in this work.

The main aim of this study is to detect emotion based on EEG signal analysis recorded from brain in response to visual stimuli.

The approaches used were the selected visual stimuli were presented to 11 healthy target subjects and EEG signal were recorded in controlled situation to minimize artefacts (muscle or/and eye movements).

The signals were filtered and type of frequency band was computed and detected.

The proposed method predicts an emotion type (positive/negative) in response to the presented stimuli.

Finally, the performance of the proposed approach was tested.

The average accuracy of machine learning algorithms (i.e. J48, Bayes Net, Adaboost and Random Forest) are 78.86, 74.76, 77.82 and 82.46 respectively.

In this study, we also applied EEG applications in the context of neuro-marketing.

The results empirically demonstrated detection of the favourite colour preference of customers in response to the logo colour of an organization or Service.

This paper presents EEG based emotion detection of a person towards an image stimuli and its applicability on neuromarketing.

EEG based Emotion Recognition of Image Stimuli


We present a probabilistic framework for session based recommendation.

A latent variable for the user state is updated as the user views more items and we learn more about their interests.

We provide computational solutions using both the re-parameterization trick and  using the Bouchard bound for the softmax function, we further explore employing a variational auto-encoder and a variational Expectation-Maximization algorithm for tightening the variational bound.

Finally we show that the Bouchard bound causes the denominator of the softmax to decompose into a sum enabling fast noisy gradients of the bound giving a fully probabilistic algorithm reminiscent of word2vec and a fast online EM algorithm.

Fast variational approximations for approximating a user state and learning product embeddings

Latent Variable Session-Based Recommendation


An important question in task transfer learning is to determine task transferability, i.e. given a common input domain, estimating to what extent representations learned from a source task can help in learning a target task.

Typically, transferability is either measured experimentally or inferred through task relatedness, which is often defined without a clear operational meaning.

In this paper, we present a novel metric, H-score, an easily-computable evaluation function that estimates the performance of transferred representations from one task to another in classification problems.

Inspired by a principled information theoretic approach, H-score has a direct connection to the asymptotic error probability of the decision function based on the transferred feature.

This formulation of transferability can further be used to select a suitable set of source tasks in task transfer learning problems or to devise efficient transfer learning policies.

Experiments using both synthetic and real image data show that not only our formulation of transferability is meaningful in practice, but also it can generalize to inference problems beyond classification, such as recognition tasks for 3D indoor-scene understanding.

We present a provable and easily-computable evaluation function that estimates the performance of transferred representations from one learning task to another in task transfer learning.

An Information-Theoretic Metric of Transferability for Task Transfer Learning


This paper presents a generic framework to tackle the crucial class mismatch problem in unsupervised domain adaptation (UDA) for multi-class distributions.

Previous adversarial learning methods condition domain alignment only on pseudo labels, but noisy and inaccurate pseudo labels may perturb the multi-class distribution embedded in probabilistic predictions, hence bringing insufficient alleviation to the latent mismatch problem.

Compared with pseudo labels, class prototypes are more accurate and reliable since they summarize over all the instances and are  able  to  represent  the  inherent  semantic  distribution  shared  across  domains.

Therefore, we propose a novel Prototype-Assisted Adversarial Learning (PAAL) scheme, which incorporates instance probabilistic predictions and class prototypes together  to  provide  reliable  indicators  for  adversarial  domain  alignment.

With the PAAL scheme,  we align both the instance feature representations and class prototype  representations  to  alleviate  the  mismatch  among  semantically  different classes.

Also,  we exploit the class prototypes as proxy to minimize the within-class variance in the target domain to mitigate the mismatch among semantically similar classes.

With these novelties, we constitute a Prototype-Assisted Conditional Domain Adaptation (PACDA) framework which well tackles the class mismatch problem.

We demonstrate the good performance and generalization ability of the PAAL scheme and also PACDA framework on two UDA tasks, i.e., object recognition (Office-Home,ImageCLEF-DA, andOffice) and synthetic-to-real semantic segmentation (GTA5→CityscapesandSynthia→Cityscapes).

We propose a reliable conditional adversarial learning scheme along with a simple, generic yet effective framework for UDA tasks.

PROTOTYPE-ASSISTED ADVERSARIAL LEARNING FOR UNSUPERVISED DOMAIN ADAPTATION


We present a new methodology that constructs a family of \emph{positive definite kernels} from any given dissimilarity measure on structured inputs whose elements are either real-valued time series or discrete structures such as strings, histograms, and graphs.

Our approach, which we call D2KE (from Distance to Kernel and Embedding), draws from the literature of Random Features.

However, instead of deriving random feature maps from a user-defined kernel to approximate kernel machines, we build a kernel from a random feature map, that we specify given the distance measure.

We further propose use of a finite number of random objects to produce a random feature embedding of each instance.

We provide a theoretical analysis showing that D2KE enjoys better generalizability than universal Nearest-Neighbor estimates.

On one hand, D2KE subsumes the widely-used \emph{representative-set method} as a special case, and relates to the well-known \emph{distance substitution kernel} in a limiting case.

On the other hand, D2KE generalizes existing \emph{Random Features methods} applicable only to vector input representations to complex structured inputs of variable sizes.

We conduct classification experiments over such disparate domains as time series, strings, and histograms (for texts and images), for which our proposed framework compares favorably to existing distance-based learning methods in terms of both testing accuracy and computational time.

D2KE: From Distance to Kernel and Embedding via Random Features For Structured Inputs

kernel embed via random feature structured input


Deep Convolution Neural Networks (CNNs), rooted by the pioneer work of  \cite{Hinton1986,LeCun1985,Alex2012}, and summarized in \cite{LeCunBengioHinton2015},   have been shown to be very useful in a variety of fields.

The  state-of-the art CNN machines such as image rest net \cite{He_2016_CVPR} are described by real value inputs and kernel convolutions followed by the local and  non-linear rectified linear outputs.

Understanding the role of these layers, the accuracy and limitations of them,  as well as making them more efficient (fewer parameters)  are all ongoing research questions.

Inspired in quantum theory, we propose the use of complex value kernel functions, followed by the local non-linear  absolute (modulus) operator square.

We argue that an advantage of quantum inspired complex kernels is robustness to realistic unpredictable scenarios (such as clutter noise,  data deformations).

We study a concrete problem of shape detection and show that when multiple overlapping shapes are deformed and/or clutter noise is added, a convolution layer with quantum inspired complex kernels outperforms the statistical/classical kernel counterpart and a "Bayesian shape estimator" .

The superior performance is due to the quantum phenomena of interference, not present in classical CNNs.

A quantum inspired kernel for convolution network, exhibiting interference phenomena,  can be very useful (and compared it with real value  counterpart).

Towards Quantum Inspired Convolution Networks


We present an artificial intelligence research platform inspired by the human game genre of MMORPGs (Massively Multiplayer Online Role-Playing Games, a.k.a. MMOs).

We demonstrate how this platform can be used to study behavior and learning in large populations of neural agents.

Unlike currently popular game environments, our platform supports persistent environments, with variable number of agents, and open-ended task descriptions.

The emergence of complex life on Earth is often attributed to the arms race that ensued from a huge number of organisms all competing for finite resources.

Our platform aims to simulate this setting in microcosm: we conduct a series of experiments to test how large-scale multiagent competition can incentivize the development of skillful behavior.

We find that population size magnifies the complexity of the behaviors that emerge and results in agents that out-compete agents trained in smaller populations.

An MMO-inspired research game platform for studying emergent behaviors of large populations in a complex environment

Neural MMO: A massively multiplayer game environment for intelligent agents


In this paper, we propose an improved quantitative evaluation framework for Generative Adversarial Networks (GANs) on generating domain-specific images, where we improve conventional evaluation methods on two levels: the feature representation and the evaluation metric.

Unlike most existing evaluation frameworks which transfer the representation of ImageNet inception model to map images onto the feature space, our framework uses a specialized encoder to acquire fine-grained domain-specific representation.

Moreover, for datasets with multiple classes, we propose Class-Aware Frechet Distance (CAFD), which employs a Gaussian mixture model on the feature space to better fit the multi-manifold feature distribution.

Experiments and analysis on both the feature level and the image level were conducted to demonstrate improvements of our proposed framework over the recently proposed state-of-the-art FID method.

To our best knowledge, we are the first to provide counter examples where FID gives inconsistent results with human judgments.

It is shown in the experiments that our framework is able to overcome the shortness of FID and improves robustness.

This paper improves existing sample-based evaluation for GANs and contains some insightful experiments.

Improving Sample-based Evaluation for Generative Adversarial Networks


Recent efforts on training light-weight binary neural networks offer promising execution/memory efficiency.

This paper introduces ResBinNet, which is a composition of two interlinked methodologies aiming to address the slow convergence speed and limited accuracy of binary convolutional neural networks.

The first method, called residual binarization, learns a multi-level binary representation for the features within a certain neural network layer.

The second method, called temperature adjustment, gradually binarizes the weights of a particular layer.

The two methods jointly learn a set of soft-binarized parameters that improve the convergence rate and accuracy of binary neural networks.

We corroborate the applicability and scalability of ResBinNet by implementing a prototype hardware accelerator.

The accelerator is reconfigurable in terms of the numerical precision of the binarized features, offering a trade-off between runtime and inference accuracy.

Residual Binary Neural Networks significantly improve the convergence rate and inference accuracy of the binary neural networks.

ResBinNet: Residual Binary Neural Network


In real-world machine learning applications, large outliers and pervasive noise are commonplace, and access to clean training data as required by standard deep autoencoders is unlikely.

Reliably detecting anomalies in a given set of images is a task of high practical relevance for visual quality inspection, surveillance, or medical image analysis.

Autoencoder neural networks learn to reconstruct normal images, and hence can classify those images as anomalous if the reconstruction error exceeds some threshold.

In this paper, we proposed an unsupervised method based on subset scanning over autoencoder activations.

The contributions of our work are threefold.

First, we propose a novel method combining detection with reconstruction error and subset scanning scores to improve the anomaly score of current autoencoders without requiring any retraining.

Second, we provide the ability to inspect and visualize the set of anomalous nodes in the reconstruction error space that make a sample noised.

Third, we show that subset scanning can be used for anomaly detection in the inner layers of the autoencoder.

We provide detection power results for several untargeted adversarial noise models under standard datasets.

Unsupervised method to detect adversarial samples in autoencoder's activations and reconstruction error space

Anomalous Pattern Detection in Activations and Reconstruction Error of Autoencoders

activation reconstruction error
autoencoder


Learning knowledge graph embeddings (KGEs) is an efficient approach to knowledge graph completion.

Conventional KGEs often suffer from limited knowledge representation, which causes less accuracy especially when training on sparse knowledge graphs.

To remedy this, we present Pretrain-KGEs, a training framework for learning better knowledgeable entity and relation embeddings, leveraging the abundant linguistic knowledge from pretrained language models.

Specifically, we propose a unified approach in which we first learn entity and relation representations via pretrained language models and use the representations to initialize entity and relation embeddings for training KGE models.

Our proposed method is model agnostic in the sense that it can be applied to any variant of KGE models.

Experimental results show that our method can consistently improve results and achieve state-of-the-art performance using different KGE models such as TransE and QuatE, across four benchmark KG datasets in link prediction and triplet classification tasks.

We propose to learn knowledgeable entity and relation representations from Bert for knowledge graph embeddings.

Pretrain-KGEs: Learning Knowledge Representation from Pretrained Models for Knowledge Graph Embeddings

knowledge graph embedding
learn
representation


We describe a novel way of representing a symbolic knowledge base (KB) called a sparse-matrix reified KB.

This representation enables neural modules that are fully differentiable, faithful to the original semantics of the KB, expressive enough to model multi-hop inferences, and scalable enough to use with realistically large KBs.

The sparse-matrix reified KB can be distributed across multiple GPUs, can scale to tens of millions of entities and facts, and is orders of magnitude faster than naive sparse-matrix implementations.

The reified KB enables very simple end-to-end architectures to obtain competitive performance on several benchmarks representing two families of tasks: KB completion, and learning semantic parsers from denotations.

A scalable differentiable neural module that implements reasoning on symbolic KBs.

Scalable Neural Methods for Reasoning With a Symbolic Knowledge   Base


We study the Cross-Entropy Method (CEM) for the non-convex optimization of a continuous and parameterized objective function and introduce a differentiable variant (DCEM) that enables us to differentiate the output of CEM with respect to the objective function's parameters.

In the machine learning setting this brings CEM inside of the end-to-end learning pipeline in cases this has otherwise been impossible.

We show applications in a synthetic energy-based structured prediction task and in non-convex continuous control.

In the control setting we show on the simulated cheetah and walker tasks that we can embed their optimal action sequences with DCEM and then use policy optimization to fine-tune components of the controller as a step towards combining model-based and model-free RL.

DCEM learns latent domains for optimization problems and helps bridge the gap between model-based and model-free RL --- we create a differentiable controller and fine-tune parts of it with PPO

The Differentiable Cross-Entropy Method


We propose a new framework for entity and event extraction based on generative adversarial imitation learning -- an inverse reinforcement learning method using generative adversarial network (GAN).

We assume that instances and labels yield to various extents of difficulty and the gains and penalties (rewards) are expected to be diverse.

We utilize discriminators to estimate proper rewards according to the difference between the labels committed by ground-truth (expert) and the extractor (agent).

Experiments also demonstrate that the proposed framework outperforms state-of-the-art methods.

We use dynamic rewards to train event extractors.

End-to-End Entity and Event Extraction with Generative Adversarial Imitation Learning


Despite recent progress in generative image modeling, successfully generating high-resolution, diverse samples from complex datasets such as ImageNet remains an elusive goal.

To this end, we train Generative Adversarial Networks at the largest scale yet attempted, and study the instabilities specific to such scale.

We find that applying orthogonal regularization to the generator renders it amenable to a simple "truncation trick", allowing fine control over the trade-off between sample fidelity and variety by reducing the variance of the Generator's input.

Our modifications lead to models which set the new state of the art in class-conditional image synthesis.

When trained on ImageNet at 128x128 resolution, our models (BigGANs) achieve an Inception Score (IS) of 166.3 and Frechet Inception Distance (FID) of 9.6, improving over the previous best IS of 52.52 and FID of 18.65.

Large Scale GAN Training for High Fidelity Natural Image Synthesis


In this work, we present a novel upper bound of target error to address the problem for unsupervised domain adaptation.

Recent studies reveal that a deep neural network can learn transferable features which generalize well to novel tasks.

Furthermore,  Ben-David et al. (2010) provide an upper bound for target error when transferring the knowledge, which can be summarized as minimizing the source error and  distance between marginal distributions simultaneously.

However, common methods based on the theory usually ignore the joint error such that samples from different classes might be mixed together when matching marginal distribution.

And in such case, no matter how we minimize the marginal discrepancy, the target error is not bounded due to an increasing joint error.

To address this problem, we propose a general upper bound taking joint error into account, such that the undesirable case can be properly penalized.

In addition, we utilize constrained hypothesis space to further formalize a tighter bound as well as a novel cross margin discrepancy to measure the dissimilarity between hypotheses which alleviates instability during adversarial learning.

Extensive empirical evidence shows that our proposal outperforms related approaches in image classification error rates on standard domain adaptation benchmarks.

joint error matters for unsupervised domain adaptation especially when the domain shift is huge

A General Upper Bound for Unsupervised Domain Adaptation


A zoo of deep nets is available these days for almost any given task, and it is increasingly unclear which net to start with when addressing a new task, or which net to use as an initialization for fine-tuning a new model.

To address this issue, in this paper, we develop knowledge flow which moves ‘knowledge’ from multiple deep nets, referred to as teachers, to a new deep net model, called the student.

The structure of the teachers and the student can differ arbitrarily and they can be trained on entirely different tasks with different output spaces too.

Upon training with knowledge flow the student is independent of the teachers.

We demonstrate our approach on a variety of supervised and reinforcement learning tasks, outperforming fine-tuning and other ‘knowledge exchange’ methods.

‘Knowledge Flow’ trains a deep net (student) by injecting information from multiple nets (teachers). The student is independent upon training and performs very well on learned tasks irrespective of the setting (reinforcement or supervised learning).

Knowledge Flow: Improve Upon Your Teachers