One of the big challenges in machine learning applications is that training data can be different from the real-world data faced by the algorithm.

In language modeling, users’ language (e.g. in private messaging) could change in a year and be completely different from what we observe in publicly available data.

At the same time, public data can be used for obtaining general knowledge (i.e. general model of English).

We study approaches to distributed fine-tuning of a general model on user private data with the additional requirements of maintaining the quality on the general data and minimization of communication costs.

We propose a novel technique that significantly improves prediction quality on users’ language compared to a general model and outperforms gradient compression methods in terms of communication efficiency.

The proposed procedure is fast and leads to an almost 70% perplexity reduction and 8.7 percentage point improvement in keystroke saving rate on informal English texts.

Finally, we propose an experimental framework for evaluating differential privacy of distributed training of language models and show that our approach has good privacy guarantees.

We propose a method of distributed fine-tuning of language models on user devices without collection of private data

This paper deals with improving language models on mobile equipments based on small portion of text that the user has inputted by employing a linearly interpolated objectives between user specific text and general English.

Distributed Fine-tuning of Language Models on Private Data

distribute fine-tuning language model
private datum
user


We propose that approximate Bayesian algorithms should optimize a new criterion, directly derived from the loss, to calculate their approximate posterior which we refer to as pseudo-posterior.

Unlike standard variational inference which optimizes a lower bound on the log marginal likelihood, the new algorithms can be analyzed to provide loss guarantees on the predictions with the pseudo-posterior.

Our criterion can be used to derive new sparse Gaussian process algorithms that have error guarantees applicable to various likelihoods.

This paper utilizes the analysis of Lipschitz loss on a bounded hypothesis space to derive new ERM-type algorithms with strong performance guarantees that can be applied to the non-conjugate sparse GP model.

Pseudo-Bayesian Learning via Direct Loss Minimization with Applications to Sparse Gaussian Process Models


In this paper, we propose a novel regularization method, RotationOut, for neural networks.

Different from Dropout that handles each neuron/channel independently, RotationOut regards its input layer as an entire vector and introduces regularization by randomly rotating the vector.

RotationOut can also be used in convolutional layers and recurrent layers with a small modification.

We further use a noise analysis method to interpret the difference between RotationOut and Dropout in co-adaptation reduction.

Using this method, we also show how to use RotationOut/Dropout together with Batch Normalization.

Extensive experiments in vision and language tasks are conducted to show the effectiveness of the proposed method.

We propose a regularization method for neural network and a noise analysis method

This paper proposes a new regularization method to mitigate the overfitting issue of deep neural networks by rotating features with a random rotation matrix to reduce co-adaptation.

This paper proposes a novel regularization method for training neural networks, which adds noise neurons in an inter-dependent fashion.

RotationOut as a Regularization Method for Neural Network

regularization method neural network
noise


Formulating the reinforcement learning (RL) problem in the framework of probabilistic inference not only offers a new perspective about RL, but also yields practical algorithms that are more robust and easier to train.

While this connection between RL and probabilistic inference has been extensively studied in the single-agent setting, it has not yet been fully understood in the multi-agent setup.

In this paper, we pose the problem of multi-agent reinforcement learning as the problem of performing inference in a particular graphical model.

We model the environment, as seen by each of the agents, using separate but related Markov decision processes.

We derive a practical off-policy maximum-entropy actor-critic algorithm that we call Multi-agent Soft Actor-Critic (MA-SAC) for performing approximate inference in the proposed model using variational inference.

MA-SAC can be employed in both cooperative and competitive settings.

Through experiments, we demonstrate that MA-SAC outperforms a strong baseline on several multi-agent scenarios.

While MA-SAC is one resultant multi-agent RL algorithm that can be derived from the proposed probabilistic framework, our work provides a unified view of maximum-entropy algorithms in the multi-agent setting.

A probabilistic framework for multi-agent reinforcement learning

This paper proposes a new algorithm named Multi-Agent Soft Actor-Critic (MA-SAC) based on the off-policy maximum-entropy actor critic algorithm Soft Actor-Critic (SAC)

Probabilistic View of Multi-agent Reinforcement Learning: A Unified Approach

multi-agent reinforcement learning
probabilistic


Sorting input objects is an important step in many machine learning pipelines.

However, the sorting operator is non-differentiable with respect to its inputs, which prohibits end-to-end gradient-based optimization.

In this work, we propose NeuralSort, a general-purpose continuous relaxation of the output of the sorting operator from permutation matrices to the set of unimodal row-stochastic matrices, where every row sums to one and has a distinct argmax.

This relaxation permits straight-through optimization of any computational graph involve a sorting operation.

Further, we use this relaxation to enable gradient-based stochastic optimization over the combinatorially large space of permutations by deriving a reparameterized gradient estimator for the Plackett-Luce family of distributions over permutations.

We demonstrate the usefulness of our framework on three tasks that require learning semantic orderings of high-dimensional objects, including a fully differentiable, parameterized extension of the k-nearest neighbors algorithm

We provide a continuous relaxation to the sorting operator, enabling end-to-end, gradient-based stochastic optimization.

The paper considers how to sort a number of items without explicitly necessarily learning their actual meanings or values and proposes a method to perform the optimization via a continuous relaxation.

This work builds on a sum(top k) identity to derive a pathwise differentiable sampler of 'unimodal row stochastic' matrices.

Introduces a continuous relaxation of the sorting operator in order to construct an end-to-end gradient-based optimization and introduces a stochastic extension of its method using Placket-Luce distributions and Monte Carlo.

Stochastic Optimization of Sorting Networks via Continuous Relaxations

continuous relaxation sort operator
method
end-to-end gradient-based
stochastic optimization
via


Transferring knowledge across tasks to improve data-efficiency is one of

the open key challenges in the area of global optimization algorithms.

available algorithms are typically designed to be universal optimizers and, thus,

We propose a novel transfer learning method to

obtain customized optimizers within the well-established framework of Bayesian

optimization, allowing our algorithm to utilize the proven generalization

Using reinforcement learning to meta-train an

acquisition function (AF) on a set of related tasks, the proposed method learns to

extract implicit structural information and to exploit it for improved data-efficiency.

We present experiments on a sim-to-real transfer task as well as on several simulated

functions and two hyperparameter search problems.

algorithm (1) automatically identifies structural properties of objective functions

from available source tasks or simulations, (2) performs favourably in settings with

both scarse and abundant source data, and (3) falls back to the performance level

of general AFs if no structure is present.

We perform efficient and flexible transfer learning in the framework of Bayesian optimization through meta-learned neural acquisition functions.

The authors present MetaBO which uses reinforcement learning to meta-learn the acquisition function for Bayesian Optimization, showing increasing sample efficiency on new tasks.

The authors propose a meta-learning based alternative to standard acquisition functions (AFs), whereby a pretrained neural network outputs acquisition values as a function of hand-chosen features.

Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimization

bayesian optimization
acquisition function
author
learning
neural
transfer
meta-learning


We study the evolution of internal representations during deep neural network (DNN) training, aiming to demystify the compression aspect of the information bottleneck theory.

The theory suggests that DNN training comprises a rapid fitting phase followed by a slower compression phase, in which the mutual information I(X;T) between the input X and internal representations T decreases.

Several papers observe compression of estimated mutual information on different DNN models, but the true I(X;T) over these networks is provably either constant (discrete X) or infinite (continuous X).

This work explains the discrepancy between theory and experiments, and clarifies what was actually measured by these past works.

To this end, we introduce an auxiliary (noisy) DNN framework for which I(X;T) is a meaningful quantity that depends on the network's parameters.

This noisy framework is shown to be a good proxy for the original (deterministic) DNN both in terms of performance and the learned representations.

We then develop a rigorous estimator for I(X;T) in noisy DNNs and observe compression in various models.

By relating I(X;T) in the noisy DNN to an information-theoretic communication problem, we show that compression is driven by the progressive clustering of hidden representations of inputs from the same class.

Several methods to directly monitor clustering of hidden representations, both in noisy and deterministic DNNs, are used to show that meaningful clusters form in the T space.

Finally, we return to the estimator of I(X;T) employed in past works, and demonstrate that while it fails to capture the true (vacuous) mutual information, it does serve as a measure for clustering.

This clarifies the past observations of compression and isolates the geometric clustering of hidden representations as the true phenomenon of interest.

Deterministic deep neural networks do not discard information, but they do cluster their inputs.

This paper provides a principled way to examine the compression phrase in deep neural networks by providing an theoretical sounding entropy estimator to estimate mutual information.

Estimating Information Flow in DNNs

deep neural network
information
estimate


A central challenge in multi-agent reinforcement learning is the induction of coordination between agents of a team.

In this work, we investigate how to promote inter-agent coordination using policy regularization and discuss two possible avenues respectively based on inter-agent modelling and synchronized sub-policy selection.

We test each approach in four challenging continuous control tasks with sparse rewards and compare them against three baselines including MADDPG, a state-of-the-art multi-agent reinforcement learning algorithm.

To ensure a fair comparison, we rely on a thorough hyper-parameter selection and training methodology that allows a fixed hyper-parameter search budget for each algorithm and environment.

We consequently assess both the hyper-parameter sensitivity, sample-efficiency and asymptotic performance of each learning method.

Our experiments show that the proposed methods lead to significant improvements on cooperative problems.

We further analyse the effects of the proposed regularizations on the behaviors learned by the agents.

We propose regularization objectives for multi-agent RL algorithms that foster coordination on cooperative tasks.

This paper proposes two methods of biasing agents towards learning coordinated behaviours and evaluates both rigorously across multi-agent domains of suitable complexity.

This paper proposes two methods building upon MADDPG to encourage collaboration amongst decentralized MARL agents.

Promoting Coordination through Policy Regularization in Multi-Agent Deep Reinforcement Learning

two method
coordination
regularization
multi-agent
agent


Multimodal sentiment analysis is a core research area that studies speaker sentiment expressed from the language, visual, and acoustic modalities.

The central challenge in multimodal learning involves inferring joint representations that can process and relate information from these modalities.

However, existing work learns joint representations using multiple modalities as input and may be sensitive to noisy or missing modalities at test time.

With the recent success of sequence to sequence models in machine translation, there is an opportunity to explore new ways of learning joint representations that may not require all input modalities at test time.

In this paper, we propose a method to learn robust joint representations by translating between modalities.

Our method is based on the key insight that translation from a source to a target modality provides a method of learning joint representations using only the source modality as input.

We augment modality translations with a cycle consistency loss to ensure that our joint representations retain maximal information from all modalities.

Once our translation model is trained with paired multimodal data, we only need data from the source modality at test-time for prediction.

This ensures that our model remains robust from perturbations or missing target modalities.

We train our model with a coupled translation-prediction objective and it achieves new state-of-the-art results on multimodal sentiment analysis datasets: CMU-MOSI, ICT-MMMO, and YouTube.

Additional experiments show that our model learns increasingly discriminative joint representations with more input modalities while maintaining robustness to perturbations of all other modalities.

We present a model that learns robust joint representations by performing hierarchical cyclic translations between multiple modalities.

This paper presents the Multimodal Cyclic Translation Network (MCTN) and evaluates it for multimodal sentiment analysis.

Learning Robust Joint Representations for Multimodal Sentiment Analysis

learn robust joint representation
multimodal sentiment analysis
present
cyclic translation


The geometric properties of loss surfaces, such as the local flatness of a solution, are associated with generalization in deep learning.

The Hessian is often used to understand these geometric properties.

We investigate the differences between the eigenvalues of the neural network Hessian evaluated over the empirical dataset, the Empirical Hessian, and the eigenvalues of the Hessian under the data generating distribution, which we term the True Hessian.

Under mild assumptions, we use random matrix theory to show that the True Hessian has eigenvalues of smaller absolute value than the Empirical Hessian.

We support these results for different SGD schedules on both a 110-Layer ResNet and VGG-16.

To perform these experiments we propose a framework for spectral visualization, based on GPU accelerated stochastic Lanczos quadrature.

This approach is an order of magnitude faster than state-of-the-art methods for spectral visualization, and can be generically used to investigate the spectral properties of matrices in deep learning.

Understanding the neural network Hessian eigenvalues under the data generating distribution.

This paper analyzes the spectrum of the Hessian matrix of large neural networks, with an analysis of max/min eigenvalues and visualization of spectra using a Lanczos quadrature approach.

This paper uses the random matrix theory to study the spectrum distribution of the empirical Hessian and true Hessian for deep learning, and proposes an efficient spectrum visualization methods.

Towards understanding the true loss surface of deep neural networks using random matrix theory and iterative spectral methods

random matrix theory
neural network
eigenvalue
understand
distribution
hessian
spectrum
deep
true
visualization
method


Summarization of long sequences into a concise statement is a core problem in natural language processing, requiring non-trivial understanding of the input.

Based on the promising results of graph neural networks on highly structured data, we develop a framework to extend existing sequence encoders with a graph component that can reason about long-distance relationships in weakly structured data such as text.

In an extensive evaluation, we show that the resulting hybrid sequence-graph models outperform both pure sequence models as well as pure graph models on a range of summarization tasks.

One simple trick to improve sequence models: Compose them with a graph model

This paper presents a structural summarization model with a graph-based encoder extended from RNN.

This work combines Graph Neural Networks with a sequential approach to abstractive summarization, effective across all datasets in comparison to external baselines.

Structured Neural Summarization


In probabilistic classification, a discriminative model based on Gaussian mixture exhibits flexible fitting capability.

Nevertheless, it is difficult to determine the number of components.

We propose a sparse classifier based on a discriminative Gaussian mixture model (GMM), which is named sparse discriminative Gaussian mixture (SDGM).

In the SDGM, a GMM-based discriminative model is trained by sparse Bayesian learning.

This learning algorithm improves the generalization capability by obtaining a sparse solution and automatically determines the number of components by removing redundant components.

The SDGM can be embedded into neural networks (NNs) such as convolutional NNs and can be trained in an end-to-end manner.

Experimental results indicated that the proposed method prevented overfitting by obtaining sparsity.

Furthermore, we demonstrated that the proposed method outperformed a fully connected layer with the softmax function in certain cases when it was used as the last layer of a deep NN.

A sparse classifier based on a discriminative Gaussian mixture model, which can also be embedded into a neural network.

The paper presents a Gaussian mixture model trained via gradient descent arguments which allows for inducing sparsity and reducing the trainable model layer parameters.

This paper proposes a classifier, called SDGM, based on discriminative Gaussian mixture and its sparse parameter estimation.

SDGM: Sparse Bayesian Classifier Based on a Discriminative Gaussian Mixture Model

classifier base discriminative gaussian mixture model


We recently observed that convolutional filters initialized

farthest apart from each other using offthe-

shelf pre-computed Grassmannian subspace

packing codebooks performed surprisingly well

we’d like to disseminate some initial results in this

regard in the hope that we stimulate the curiosity

of the deep-learning community towards considering

results as a source of new ideas for more efficient

Initialize weights using off-the-shelf Grassmannian codebooks, get  faster training and better accuracy

Grassmannian initialization: Neural network initialization using sub-space packing


Domain adaptation is critical for success in new, unseen environments.

Adversarial adaptation models applied in feature spaces discover domain invariant representations, but are difficult to visualize and sometimes fail to capture pixel-level and low-level domain shifts.

Recent work has shown that generative adversarial networks combined with cycle-consistency constraints are surprisingly effective at  mapping images between domains, even without the use of aligned image pairs.

We propose a novel discriminatively-trained Cycle-Consistent Adversarial Domain Adaptation model.

CyCADA adapts representations at both the pixel-level and feature-level, enforces cycle-consistency while leveraging a task loss, and does not require aligned pairs.

Our model can be applied in a variety of visual recognition and prediction settings.

We show new state-of-the-art results across multiple adaptation tasks, including digit classification and semantic segmentation of road scenes demonstrating transfer from synthetic to real world domains.

An unsupervised domain adaptation approach which adapts at both the pixel and feature levels

This paper proposes a domain adaptation approach by extending the CycleGAN with task specific loss functions and loss imposed over both pixels and features.

This paper proposes the use of CycleGANs for Domain Adaptation

This paper makes a novel extension to the previous work on CycleGAN by coupling it with adversarial adaptation approaches, including a new feature and semantic loss in the overall objective of the CycleGAN, with clear benefits.

CyCADA: Cycle-Consistent Adversarial Domain Adaptation

pixel feature
cyclegan
domain adaptation approach
loss
adversarial


Stemming is the process of removing affixes( i.e. prefixes, infixes and suffixes) that improve the accuracy and performance of information retrieval systems.This paper presents the reduction of Amharic words to corresponding stem where with the intention that it preserves semantic information.

The proposed approach efficiently removes affixes from an Amharic word.

The process of removing such affixes (prefixes, infixes and suffixes) from a word to its base form is called stemming.

While many stemmers exist for dominant languages such as English, under resourced languages such as Amharic which lacks such powerful tool support.

In this paper, we design a light Amharic stemmer relying on the rules that receives an Amharic word and then it finds a match to the beginning of a word to the possible prefixes and to its ending with the possible suffixes and finally it checks whether it has infix.

The final result is the stem if there is any prefix, infix or/and suffix, otherwise it remains in one of the earlier states.

The technique does not rely on any additional resource (e.g. dictionary) to verify the generated stem.

The performance of the generated stemmer is evaluated using manually annotated Amharic words.

The result is compared with current state-of-the-art stemmer for Amharic showing an increase of 7% in stemmer correctness.

Amharic Light Stemmer is designed for improving performance of  Amharic Sentiment Classification.

This paper studies the stemming for morphologically rich languages with a light stemmer that only removes affixes to the extent that the original semantic information in the word is kept.

This paper proposes a technique for Amharic light stemming using a cascade of transformations that standardize the form, remove suffixes, prefixes, and infixes.

Amharic Light Stemmer


Place and grid-cells are known to aid navigation in animals and humans.

Together with concept cells, they allow humans to form an internal representation of the external world, namely the concept space.

We investigate the presence of such a space in deep neural networks by plotting the activation profile of its hidden layer neurons.

Although place cell and concept-cell like properties are found, grid-cell like firing patterns are absent thereby indicating a lack of path integration or feature transformation functionality in trained networks.

Overall, we present a plausible inadequacy in current deep learning practices that restrict deep networks from performing analogical reasoning and memory retrieval tasks.

We investigated if simple deep networks possess grid cell-like artificial neurons while memory retrieval in the learned concept space.

Do deep neural networks possess concept space grid cells?


We develop a comprehensive description of the active inference framework, as proposed by Friston (2010), under a machine-learning compliant perspective.

Stemming from a biological inspiration and the auto-encoding principles, a sketch of a cognitive architecture is proposed that should provide ways to implement estimation-oriented control policies.

Computer simulations illustrate the effectiveness of the approach through a foveated inspection of the input data.

The pros and cons of the control policy are analyzed in detail, showing interesting promises in terms of processing compression.

Though optimizing future posterior entropy over the actions set is shown enough to attain locally optimal action selection, offline calculation using class-specific saliency maps is shown better for it saves processing costs through saccades pathways pre-processing, with a negligible effect on the recognition/compression rates.

Pros and cons of saccade-based computer vision under a predictive coding perspective

Presents a computational framework for the active vision problem and explains how the control policy can be learned to reduce the entropy of the posterior belief.

Toward predictive machine learning for active vision


Graphs possess exotic features like variable size and absence of natural ordering of the nodes that make them difficult to analyze and compare.

To circumvent this problem and learn on graphs, graph feature representation is required.

Main difficulties with feature extraction lie in the trade-off between expressiveness, consistency and efficiency, i.e. the capacity to extract features that represent the structural information of the graph while being deformation-consistent and isomorphism-invariant.

While state-of-the-art methods enhance expressiveness with powerful graph neural-networks, we propose to leverage natural spectral properties of graphs to study a simple graph feature: the graph Laplacian spectrum (GLS).

We analyze the representational power of this object that satisfies both isomorphism-invariance, expressiveness and deformation-consistency.

In particular, we propose a theoretical analysis based on graph perturbation to understand what kind of comparison between graphs we do when comparing GLS.

To do so, we derive bounds for the distance between GLS that are related to the divergence to isomorphism, a standard computationally expensive graph divergence.

Finally, we experiment GLS as graph representation through consistency tests and classification tasks, and show that it is a strong graph feature representation baseline.

We study theoretically the consistency the Laplacian spectrum and use it as whole-graph embeddding

This paper forcuses on the laplacian spectrum of a graph as means to generate a representation to be used to compare graphs and classify them.

This work proposed to use Graph Laplacian spectrum to learn graph representation.

Study of a Simple, Expressive and Consistent Graph Feature Representation

laplacian spectrum
graph
study
representation


Adversarial training, a method for learning robust deep networks, is typically assumed to be more expensive than traditional training due to the necessity of constructing adversarial examples via a first-order method like projected gradient decent (PGD).

In this paper, we make the surprising discovery that it is possible to train empirically robust models using a much weaker and cheaper adversary, an approach that was previously believed to be ineffective, rendering the method no more costly than standard training in practice.

Specifically, we show that adversarial training with the fast gradient sign method (FGSM), when combined with random initialization, is as effective as PGD-based training but has significantly lower cost.

Furthermore we show that FGSM adversarial training can be further accelerated by using standard techniques for efficient training of deep networks, allowing us to learn a robust CIFAR10 classifier with 45% robust accuracy at epsilon=8/255 in 6 minutes, and a robust ImageNet classifier with 43% robust accuracy at epsilon=2/255 in 12 hours, in comparison to past work based on ``free'' adversarial training which took 10 and 50 hours to reach the same respective thresholds.

FGSM-based adversarial training, with randomization, works just as well as PGD-based adversarial training: we can use this to train a robust classifier in 6 minutes on CIFAR10, and 12 hours on ImageNet, on a single machine.

This paper revisits Random+FGSM method to train robust models against strong PGD evasion attacks faster than previous methods.

The main claim of this paper is that a simple strategy of randomization plus fast gradient sign method (FGSM) adversarial training yields robust neural networks.

Fast is better than free: Revisiting adversarial training

well
adversarial training
train robust
randomization
method
revisit
fast


In seeking for sparse and efficient neural network models, many previous works investigated on enforcing L1 or L0 regularizers to encourage weight sparsity during training.

The L0 regularizer measures the parameter sparsity directly and is invariant to the scaling of parameter values.

But it cannot provide useful gradients and therefore requires complex optimization techniques.

The L1 regularizer is almost everywhere differentiable and can be easily optimized with gradient descent.

Yet it is not scale-invariant and causes the same shrinking rate to all parameters, which is inefficient in increasing sparsity.

Inspired by the Hoyer measure (the ratio between L1 and L2 norms) used in traditional compressed sensing problems, we present DeepHoyer, a set of sparsity-inducing regularizers that are both differentiable almost everywhere and scale-invariant.

Our experiments show that enforcing DeepHoyer regularizers can produce even sparser neural network models than previous works, under the same accuracy level.

We also show that DeepHoyer can be applied to both element-wise and structural pruning.

We propose almost everywhere differentiable and scale invariant regularizers for DNN pruning, which can lead to supremum sparsity through standard SGD training.

The paper proposes a scale-invariant regularizer (DeepHoyer) inspired by the Hoyer measure to enforce sparsity in neural networks.

DeepHoyer: Learning Sparser Neural Network with Differentiable Scale-Invariant Sparsity Measures

neural network
sparsity
regularizer
differentiable
deephoyer
scale-invariant
measure


Self-supervision, in which a target task is improved without external supervision, has primarily been explored in settings that assume the availability of additional data.

However, in many cases, particularly in healthcare, one may not have access to additional data (labeled or otherwise).

In such settings, we hypothesize that self-supervision based solely on the structure of the data at-hand can help.

We explore a novel self-supervision framework for time-series data, in which multiple auxiliary tasks (e.g., forecasting) are included to improve overall performance on a sequence-level target task without additional training data.

We call this approach limited self-supervision, as we limit ourselves to only the data at-hand.

We demonstrate the utility of limited self-supervision on three sequence-level classification tasks, two pertaining to real clinical data and one using synthetic data.

Within this framework, we introduce novel forms of self-supervision and demonstrate their utility in improving performance on the target task.

Our results indicate that limited self-supervision leads to a consistent improvement over a supervised baseline, across a range of domains.

In particular, for the task of identifying atrial fibrillation from small amounts of electrocardiogram data, we observe a nearly 13% improvement in the area under the receiver operating characteristics curve (AUC-ROC) relative to the baseline (AUC-ROC=0.55 vs. AUC-ROC=0.62).

Limited self-supervision applied to sequential data can aid in learning intermediate representations, making it particularly applicable in settings where data collection is difficult.

We show that extra unlabeled data is not required for self-supervised auxiliary tasks to be useful for time series classification, and present new and effective auxiliary tasks.

This paper proposes a self-supervised method for learning from time series data in healthcare settings via designing auxilliary tasks based on data's internal structure to create more labeled auxilliary training tasks.

This paper propose an approach for self-supervised learning on time series.

Learning Through Limited Self-Supervision: Improving Time-Series Classification Without Additional Data via Auxiliary Tasks

time series
task
datum
self-supervised
learn
classification
auxiliary
via


Are neural networks biased toward simple functions?

Does depth always help learn more complex features?

Is training the last layer of a network as good as training all layers?

These questions seem unrelated at face value, but in this work we give all of them a common treatment from the spectral perspective.

We will study the spectra of the *Conjugate Kernel, CK,* (also called the *Neural Network-Gaussian Process Kernel*), and the *Neural Tangent Kernel, NTK*.

Roughly, the CK and the NTK tell us respectively ``"what a network looks like at initialization" and "``what a network looks like during and after training."

Their spectra then encode valuable information about the initial distribution and the training and generalization properties of neural networks.

By analyzing the eigenvalues, we lend novel insights into the questions put forth at the beginning, and we verify these insights by extensive experiments of neural networks.

We believe the computational tools we develop here for analyzing the spectra of CK and NTK serve as a solid foundation for future studies of deep neural networks.

We have open-sourced the code for it and for generating the plots in this paper at github.com/jxVmnLgedVwv6mNcGCBy/NNspectra.

Eigenvalues of Conjugate (aka NNGP) and Neural Tangent Kernel can be computed in closed form over the Boolean cube and reveal the effects of hyperparameters on neural network inductive bias, training, and generalization.

This paper gives a spectral analysis on neural networks' conjugate kernel and neural tangent kernel on boolean cube to resolve why deep networks are biased towards simple functions.

A Fine-Grained Spectral Perspective on Neural Networks

neural tangent kernel
boolean cube
conjugate
bias
network
spectral


To communicate, to ground hypotheses, to analyse data, neuroscientists often refer to divisions of the brain.

Here we consider atlases used to parcellate the brain when studying brain function.

We discuss the meaning and the validity of these parcellations, from a conceptual point of view as well as by running various analytical tasks on popular functional brain parcellations.

All functional brain parcellations are wrong, but some are useful

What’s in a functional brain parcellation?


High-dimensional sparse reward tasks present major challenges for reinforcement learning agents.

In this work we use imitation learning to address two of these challenges:  how to learn a useful representation of the world e.g.  from pixels, and how to explore efficiently given the rarity of a reward signal?

We show that adversarial imitation can work well even in this high dimensional observation space.

Surprisingly the adversary itself, acting as the learned reward function, can be tiny, comprising as few as 128 parameters, and can be easily trained using the most basic GAN formulation.

Our approach removes limitations present in most contemporary imitation approaches: requiring no demonstrator actions (only video), no special initial conditions or warm starts, and no explicit tracking of any single demo.

The proposed agent can solve a challenging robot manipulation task of block stacking from only video demonstrations and sparse reward, in which the non-imitating agents fail to learn completely.

Furthermore, our agent learns much faster than competing approaches that depend on hand-crafted, staged dense reward functions, and also better compared to standard GAIL baselines.

Finally, we develop a new adversarial goal recognizer that in some cases allows the agent to learn stacking without any task reward, purely from imitation.

Imitation from pixels, with sparse or no reward, using off-policy RL and a tiny adversarially-learned reward function.

The paper proposes to use a "minimal adversary" in generative adversarial imitation learning under high-dimensional visual spaces.

This paper aims at solving the problem of estimating sparse rewards in a high-dimensional input setting.

Visual Imitation with a Minimal Adversary

sparse reward
imitation
minimal adversary
high-dimensional
visual


In this paper we show strategies to easily identify fake samples generated with the Generative Adversarial Network framework.

One strategy is based on the statistical analysis and comparison of raw pixel values and features extracted from them.

The other strategy learns formal specifications from the real data and shows that fake samples violate the specifications of the real data.

We show that fake samples produced with GANs have a universal signature that can be used to identify fake samples.

We provide results on MNIST, CIFAR10, music and speech data.

We show strategies to easily identify fake samples generated with the Generative Adversarial Network framework.

Show that fake samples created with common generative adversarial network (GAN) implementations are easily identified using various statistical techniques.

The paper proposes statistics to identify fake data generated using GANs based on simple marginal statistics or formal specifications automatically generated from real data.

TequilaGAN: How To Easily Identify GAN Samples

generative adversarial network
easily identify
generate
fake sample
show
gan


Efforts to reduce the numerical precision of computations in deep learning training have yielded systems that aggressively quantize weights and activations, yet employ wide high-precision accumulators for partial sums in inner-product operations to preserve the quality of convergence.

The absence of any framework to analyze the precision requirements of partial sum accumulations results in conservative design choices.

This imposes an upper-bound on the reduction of complexity of multiply-accumulate units.

We present a statistical approach to analyze the impact of reduced accumulation precision on deep learning training.

Observing that a bad choice for accumulation precision results in loss of information that manifests itself as a reduction in variance in an ensemble of partial sums, we derive a set of equations that relate this variance to the length of accumulation and the minimum number of bits needed for accumulation.

We apply our analysis to three benchmark networks: CIFAR-10 ResNet 32, ImageNet ResNet 18 and ImageNet AlexNet.

In each case, with accumulation precision set in accordance with our proposed equations, the networks successfully converge to the single precision floating-point baseline.

We also show that reducing accumulation precision further degrades the quality of the trained network, proving that our equations produce tight bounds.

Overall this analysis enables precise tailoring of computation hardware to the application, yielding area- and power-optimal systems.

We present an analytical framework to determine accumulation bit-width requirements in all three deep learning training GEMMs and verify the validity and tightness of our method via benchmarking experiments.

The authors propose an analytical method to predict the number of mantissa bits needed for partial summations for convolutional and fully connected layers

The authors conduct a thorough analysis of the numeric precision required for the accumulation operations in neural network training and show the theoretical impact of reducing number of bits in the floating point accumulator.

Accumulation Bit-Width Scaling For Ultra-Low Precision Training Of Deep Networks

accumulation bit-width
analytical
method
author
number
training
deep
bit
network
precision


Unsupervised domain adaptation is a promising avenue to enhance the performance of deep neural networks on a target domain, using labels only from a source domain.

However, the two predominant methods, domain discrepancy reduction learning and semi-supervised learning, are not readily applicable when source and target domains do not share a common label space.

This paper addresses the above scenario by learning a representation space that retains discriminative power on both the (labeled) source and (unlabeled) target domains while keeping representations for the two domains well-separated.

Inspired by a theoretical analysis, we first reformulate the disjoint classification task, where the source and target domains correspond to non-overlapping class labels, to a verification one.

To handle both within and cross domain verifications, we propose a Feature Transfer Network (FTN) to separate the target feature space from the original source space while aligned with a transformed source space.

Moreover, we present a non-parametric multi-class entropy minimization loss to further boost the discriminative power of FTNs on the target domain.

In experiments, we first illustrate how FTN works in a controlled setting of adapting from MNIST-M to MNIST with disjoint digit classes between the two domains and then demonstrate the effectiveness of FTNs through state-of-the-art performances on a cross-ethnicity face recognition problem.

A new theory of unsupervised domain adaptation for distance metric learning and its application to face recognition across diverse ethnicity variations.

Proposes a novel feature transfer network that optimizes domain adversarial loss and domain separation loss.

Unsupervised Domain Adaptation for Distance Metric Learning

domain adaptation distance metric learning


In this paper, we consider the problem of training neural networks (NN).

To promote a NN with specific structures, we explicitly take into consideration the nonsmooth regularization (such as L1-norm) and constraints (such as interval constraint).

This is formulated as a constrained nonsmooth nonconvex optimization problem, and we propose a convergent proximal-type stochastic gradient descent (Prox-SGD) algorithm.

We show that under properly selected learning rates, momentum eventually resembles the unknown real gradient and thus is crucial in analyzing the convergence.

We establish that with probability 1, every limit point of the sequence generated by the proposed Prox-SGD is a stationary point.

Then the Prox-SGD is tailored to train a sparse neural network and a binary neural network, and the theoretical analysis is also supported by extensive numerical tests.

We propose a convergent proximal-type stochastic gradient descent algorithm for constrained nonsmooth nonconvex optimization problems

This paper proposes Prox-SGD, a theoretical framework for stochastic optimization algorithms shown to converge asymptotically to stationarity for smooth non-convvex loss + convex constraint/regularizer.

The paper proposes a new gradient-based stochastic optimization algorithm with gradient averaging by adapting theory for proximal algorithms to the non-convex setting.

ProxSGD: Training Structured Neural Networks under Regularization and Constraints

stochastic optimization algorithm
gradient
constraint


The loss of a few neurons in a brain rarely results in any visible loss of function.

However, the insight into what “few” means in this context is unclear.

How many random neuron failures will it take to lead to a visible loss of function?

In this paper, we address the fundamental question of the impact of the crash of a random subset of neurons on the overall computation of a neural network and the error in the output it produces.

We study fault tolerance of neural networks subject to small random neuron/weight crash failures in a probabilistic setting.

We give provable guarantees on the robustness of the network to these crashes.

Our main contribution is a bound on the error in the output of a network under small random Bernoulli crashes proved by using a Taylor expansion in the continuous limit, where close-by neurons at a layer are similar.

The failure mode we adopt in our model is characteristic of neuromorphic hardware, a promising technology to speed up artificial neural networks, as well as of biological networks.

We show that our theoretical bounds can be used to compare the fault tolerance of different architectures and to design a regularizer improving the fault tolerance of a given architecture.

We design an algorithm achieving fault tolerance using a reasonable number of neurons.

In addition to the theoretical proof, we also provide experimental validation of our results and suggest a connection to the generalization capacity problem.

We give a bound for NNs on the output error in case of random weight failures using a Taylor expansion in the continuous limit where nearby neurons are similar

This paper considers the problem of dropping neurons from a neural network, showing that if the goal is to become robust to randomly dropped neurons during evaluation, then it is sufficient to just train with dropout.

This contribution studies the impact of deletions of random neurons on prediction accuracy of trained architecture, with the application to failure analysis and the specific context of neuromorphic hardware.

The Probabilistic Fault Tolerance of Neural Networks in the Continuous Limit

neural network
continuous limit
failure
neuron
random
train


Truly intelligent agents need to capture the interplay of all their senses to build a rich physical understanding of their world.

In robotics, we have seen tremendous progress in using visual and tactile perception; however we have often ignored a key sense: sound.

This is primarily due to lack of data that captures the interplay of action and sound.

In this work, we perform the first large-scale study of the interactions between sound and robotic action.

To do this, we create the largest available sound-action-vision dataset with 15,000 interactions on 60 objects using our robotic platform Tilt-Bot.

By tilting objects and allowing them to crash into the walls of a robotic tray, we collect rich four-channel audio information.

Using this data, we explore the synergies between sound and action, and present three key insights.

First, sound is indicative of fine-grained object class information, e.g., sound can differentiate a metal screwdriver from a metal wrench.

Second, sound also contains information about the causal effects of an action, i.e. given the sound produced, we can predict what action was applied on the object.

Finally, object representations derived from audio embeddings are indicative of implicit physical properties.

We demonstrate that on previously unseen objects, audio embeddings generated through interactions can predict forward models 24% better than passive visual embeddings.

We explore and study the synergies between sound and action.

This paper explores the connections between action and sound by building a sound-action-vision dataset with a tilt-bot.

This paper studies the role of audio in object and action perception, as well as how auditory information can help learning forward and inverse dynamics models.

Swoosh! Rattle! Thump! - Actions that Sound


Hierarchical label structures widely exist in many machine learning tasks, ranging from those with explicit label hierarchies such as image classification to the ones that have latent label hierarchies such as semantic segmentation.

Unfortunately, state-of-the-art methods often utilize cross-entropy loss which in-explicitly assumes the independence among class labels.

Motivated by the fact that class members from the same hierarchy need to be similar to each others, we design a new training diagram called Hierarchical Complement Objective Training (HCOT).

In HCOT, in addition to maximizing the probability of the ground truth class, we also neutralize the probabilities of rest of the classes in a hierarchical fashion, making the model take advantage of the label hierarchy explicitly.

We conduct our method on both image classification and semantic segmentation.

Results show that HCOT outperforms state-of-the-art models in CIFAR100, Imagenet, and PASCAL-context.

Our experiments also demonstrate that HCOT can be applied on tasks with latent label hierarchies, which is a common characteristic in many machine learning tasks.

We propose Hierarchical Complement Objective Training, a novel training paradigm to effectively leverage category hierarchy in the labeling space on both image classification and semantic segmentation.

A method that regularizes the entropy of the posterior distribution over classes which can be useful for image classsification and segmentation tasks

hierarchical complement objective training


There is a growing interest in automated neural architecture search (NAS).

To improve the efficiency of NAS, previous approaches adopt  weight sharing method to force all models share the same set of weights.

However, it has been observed that a model performing better with shared weights does not necessarily perform  better when trained alone.

In this paper, we analyse existing weight sharing one-shot NAS approaches from a Bayesian point of view and identify the posterior fading problem, which compromises the effectiveness of shared weights.

To alleviate this problem, we present a practical approach to guide the parameter posterior towards its true distribution.

Moreover, a hard latency constraint is introduced during the search so that the desired latency can be achieved.

The resulted method, namely Posterior Convergent NAS (PC-NAS), achieves state-of-the-art performance under standard GPU latency constraint on ImageNet.

In our small search space, our model PC-NAS-S attains76.8% top-1 accuracy, 2.1% higher than MobileNetV2 (1.4x) with the same latency.

When adopted to our large search space, PC-NAS-L achieves 78.1% top-1 accuracy within 11ms.

The discovered architecture also transfers well to other computer vision applications such as object detection and person re-identification.

Our paper identifies the issue of existing weight sharing approach in neural architecture search and propose a practical method, achieving strong results.

Author identifies an issue with NAS called posterior fading and introduces Posterior Convergent NAS to mitigate this effect

Improving One-Shot NAS By Suppressing The Posterior Fading


Noisy labels are very common in real-world training data, which lead to poor generalization on test data because of overfitting to the noisy labels.

In this paper, we claim that such overfitting can be avoided by "early stopping" training a deep neural network before the noisy labels are severely memorized.

Then, we resume training the early stopped network using a "maximal safe set," which maintains a collection of almost certainly true-labeled samples at each epoch since the early stop point.

Putting them all together, our novel two-phase training method, called Prestopping, realizes noise-free training under any type of label noise for practical use.

Extensive experiments using four image benchmark data sets verify that our method significantly outperforms four state-of-the-art methods in test error by 0.4–8.2 percent points under existence of real-world noise.

We propose a novel two-phase training approach based on "early stopping" for robust training on noisy labels.

Paper proposes to study how early stopping in optimization helps find confident examples

This paper proposes a two-phase training method for learning with label noise.

Prestopping: How Does Early Stopping Help Generalization Against Label Noise?

early stop
label noise
two-phase training
help


Learning when to communicate and doing that effectively is essential in multi-agent tasks.

Recent works show that continuous communication allows efficient training with back-propagation in multi-agent scenarios, but have been restricted to fully-cooperative tasks.

In this paper, we present Individualized Controlled Continuous Communication Model (IC3Net) which has better training efficiency than simple continuous communication model, and can be applied to semi-cooperative and competitive settings along with the cooperative settings.

IC3Net controls continuous communication with a gating mechanism and uses individualized rewards foreach agent to gain better performance and scalability while fixing credit assignment issues.

Using variety of tasks including StarCraft BroodWars explore and combat scenarios, we show that our network yields improved performance and convergence rates than the baselines as the scale increases.

Our results convey that IC3Net agents learn when to communicate based on the scenario and profitability.

We introduce IC3Net, a single network which can be used to train agents in cooperative, competitive and mixed scenarios. We also show that agents can learn when to communicate using our model.

Author proposes a new architecture for multi-agent reinforcement learning that uses several LSTM controllers with tied weights that transmit a continuous vector to each other

The authors propose an interesting gating scheme allowing agents to communicate in an multi-agent RL setting.

Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks

learn communicate
cooperative competitive
author
agent
multi-agent


Neural sequence-to-sequence models are a recently proposed family of approaches used in abstractive summarization of text documents, useful for producing condensed versions of source text narratives without being restricted to using only words from the original text.

Despite the advances in abstractive summarization, custom generation of summaries (e.g. towards a user's preference) remains unexplored.

In this paper, we present CATS, an abstractive neural summarization model, that summarizes content in a sequence-to-sequence fashion but also introduces a new mechanism to control the underlying latent topic distribution of the produced summaries.

Our experimental results on the well-known CNN/DailyMail dataset show that our model achieves state-of-the-art performance.

We present the first neural abstractive summarization model capable of customization of generated summaries.

CATS: Customizable Abstractive Topic-based Summarization


We propose a software framework based on ideas of the Learning-Compression algorithm , that allows one to compress any neural network by different compression mechanisms (pruning, quantization, low-rank, etc.).

By design, the learning of the neural net (handled by SGD) is decoupled from the compression of its parameters (handled by a signal compression function), so that the framework can be easily extended to handle different combinations of neural net and compression type.

In addition, it has other advantages, such as easy integration with deep learning frameworks, efficient training time, competitive practical performance in the loss-compression tradeoff, and reasonable convergence guarantees.

Our toolkit is written in Python and Pytorch and we plan to make it available by the workshop time, and eventually open it for contributions from the community.

This paper presents the design of a software library that makes it easier for the user to compress their networks by hiding away the details of the compression methods.

A Flexible, Extensible Software Framework for Neural Net Compression

compress
software framework
neural
network
compression


This work seeks the possibility of generating the human face from voice solely based on the audio-visual data without any human-labeled annotations.

To this end, we propose a multi-modal learning framework that links the inference stage and generation stage.

First, the inference networks are trained to match the speaker identity between the two different modalities.

Then the pre-trained inference networks cooperate with the generation network by giving conditional information about the voice.

This paper proposes a method of end-to-end multi-modal generation of human face from speech based on a self-supervised learning framework.

This paper presents a multi-modal learning framework that links the inference stage and generation stage for seeking the possibility of generating the human face from voice solely.

This work aims to build one conditional face image generation framework from the audio signal.

From Inference to Generation: End-to-end Fully Self-supervised Generation of Human Face from Speech

generation human face speech
learning framework
end-to-end
inference
self-supervised
multi-modal


We present a simple neural model that given a formula and a property tries to answer the question whether the formula has the given property, for example whether a propositional formula is always true.

The structure of the formula is captured by a feedforward neural network recursively built for the given formula in a top-down manner.

The results of this network are then processed by two recurrent neural networks.

One of the interesting aspects of our model is how propositional atoms are treated.

For example, the model is insensitive to their names, it only matters whether they are the same or distinct.

A top-down approach how to recursively represent propositional formulae by neural networks is presented.

This paper provides a new neural-net model of logical formulae that gathers information about a given formula by traversing its parse tree top-down.

The paper pursues the path of a tree-structured network isomorphic to the parse tree of a propositional-calculus formula, but by passing information top-down rather than bottom-up.

Top-Down Neural Model For Formulae

parse tree
top-down
formulae
formula
model
neural
network
information


Despite significant advances in the field of deep Reinforcement Learning (RL), today's algorithms still fail to learn human-level policies consistently over a set of diverse tasks such as Atari 2600 games.

We identify three key challenges that any algorithm needs to master in order to perform well on all games:  processing diverse reward distributions, reasoning over long time horizons, and exploring efficiently.

In this paper, we propose an algorithm that addresses each of these challenges and is able to learn human-level policies on nearly all Atari games.

A new transformed Bellman operator allows our algorithm to process rewards of varying densities and scales; an auxiliary temporal consistency loss allows us to train stably using a discount factor of 0.999 (instead of 0.99) extending the effective planning horizon by an order of magnitude; and we ease the exploration problem by using human demonstrations that guide the agent towards rewarding states.

When tested on a set of 42 Atari games, our algorithm exceeds the performance  of an average human on 40 games using a common set of hyper parameters.

Ape-X DQfD = Distributed (many actors + one learner + prioritized replay) DQN with demonstrations optimizing the unclipped 0.999-discounted return on Atari.

The paper proposes three extensions (Bellman update, temporal consistency loss, and expert demonstration) to DQN to improve the learning performance on Atari games, achieving outperformance over the state-of-the-art results for Atari games.

This paper proposes a transformed Bellman operator that aims to solve sensitivity to unclipped reward, robustness to the value of the discount factor, and the exploration problem.

Towards Consistent Performance on Atari using Expert Demonstrations

dqn
atari
expert demonstration
unclipped
performance
bellman


The knowledge that humans hold about a problem often extends far beyond a set of training data and output labels.

While the success of deep learning mostly relies on supervised training, important properties cannot be inferred efficiently from end-to-end annotations alone, for example causal relations or domain-specific invariances.

We present a general technique to supplement supervised training with prior knowledge expressed as relations between training instances.

We illustrate the method on the task of visual question answering to exploit various auxiliary annotations, including relations of equivalence and of logical entailment between questions.

Existing methods to use these annotations, including auxiliary losses and data augmentation, cannot guarantee the strict inclusion of these relations into the model since they require a careful balancing against the end-to-end objective.

Our method uses these relations to shape the embedding space of the model, and treats them as strict constraints on its learned representations.

%The resulting model encodes relations that better generalize across instances.

In the context of VQA, this approach brings significant improvements in accuracy and robustness, in particular over the common practice of incorporating the constraints as a soft regularizer.

We also show that incorporating this type of prior knowledge with our method brings consistent improvements, independently from the amount of supervised data used.

It demonstrates the value of an additional training signal that is otherwise difficult to extract from end-to-end annotations alone.

Training method to enforce strict constraints on learned embeddings during supervised training. Applied to visual question answering.

The authors propose a framework to incorporate additional semantic prior knowledge into the traditional training of deep learning models to regularize the embedding space instead of the parameter space.

The paper argues for encoding external knowledge in the linguistic embedding layer of a multimodal neural network, as a set of hard constraints.

On Incorporating Semantic Prior Knowlegde in Deep Learning Through Embedding-Space Constraints

deep learning
knowledge
constraint
training
incorporate
semantic prior
embed


Artificial neural networks revolutionized many areas of computer science in recent years since they provide solutions to a number of previously unsolved problems.

On the other hand, for many problems, classic algorithms exist, which typically exceed the accuracy and stability of neural networks.

To combine these two concepts, we present a new kind of neural networks—algorithmic neural networks (AlgoNets).

These networks integrate smooth versions of classic algorithms into the topology of neural networks.

Our novel reconstructive adversarial network (RAN) enables solving inverse problems without or with only weak supervision.

Solving inverse problems by using smooth approximations of the forward algorithms to train the inverse models.

AlgoNet: $C^\infty$ Smooth Algorithmic Neural Networks for Solving Inverse Problems


Pointwise localization allows more precise localization and accurate interpretability, compared to bounding box, in applications where objects are highly unstructured such as in medical domain.

In this work, we focus on  weakly supervised localization (WSL) where a model is trained to classify an image and localize regions of interest at pixel-level using only global image annotation.

Typical convolutional attentions maps are prune to high false positive regions.

To alleviate this issue, we propose a new deep learning method for WSL, composed of a localizer and a classifier, where the localizer is constrained to determine relevant and irrelevant regions using conditional entropy (CE) with the aim to reduce false positive regions.

Experimental results on a public medical dataset and two natural datasets, using Dice index, show that, compared to state of the art WSL methods, our proposal can provide significant improvements in terms of image-level classification and pixel-level localization (low false positive) with robustness to overfitting.

A public reproducible PyTorch implementation is provided.

A deep learning method for weakly-supervised pointwise localization that learns using image-level label only. It relies on conditional entropy to localize relevant and irrelevant regions aiming to minimize false positive regions.

This work explores the problem of WSL using a novel design of regularization terms and a recursive erasing algorithm.

This paper presents a new weakly supervised approach for learning object segmentation with image-level class labels.

Min-max Entropy for Weakly Supervised Pointwise Localization

learn
label
entropy
pointwise localization
image-level
weakly


Model-based reinforcement learning has been empirically demonstrated as a successful strategy to improve sample efficiency.

Particularly, Dyna architecture, as an elegant model-based architecture integrating learning and planning, provides huge flexibility of using a model.

One of the most important components in Dyna is called search-control, which refers to the process of generating state or state-action pairs from which we query the model to acquire simulated experiences.

Search-control is critical to improve learning efficiency.

In this work, we propose a simple and novel search-control strategy by searching high frequency region on value function.

Our main intuition is built on Shannon sampling theorem from signal processing, which indicates that a high frequency signal requires more samples to reconstruct.

We empirically show that a high frequency function is more difficult to approximate.

This suggests a search-control strategy: we should use states in high frequency region of the value function to query the model to acquire more samples.

We develop a simple strategy to locally measure the frequency of a function by gradient norm, and provide theoretical justification for this approach.

We then apply our strategy to search-control in Dyna, and conduct experiments to show its property and effectiveness on benchmark domains.

Acquire states from high frequency region for search-control in Dyna.

The authors propose to do sampling in the high-frequency domain to increase the sample efficiency

This paper proposes a new way to select states from which do do transitions in dyna algorithm.

Frequency-based Search-control in Dyna


We propose a new architecture for distributed image compression from a group of distributed data sources.

The work is motivated by practical needs of data-driven codec design, low power consumption, robustness, and data privacy.

The proposed architecture, which we refer to as Distributed Recurrent Autoencoder for Scalable Image Compression (DRASIC), is able to train distributed encoders and one joint decoder on correlated data sources.

Its compression capability is much better than the method of training codecs separately.

Meanwhile, for 10 distributed sources, our distributed system remarkably performs within 2 dB peak signal-to-noise ratio (PSNR) of that of a single codec trained with all data sources.

We experiment distributed sources with different correlations and show how our methodology well matches the Slepian-Wolf Theorem in Distributed Source Coding (DSC).

Our method is also shown to be robust to the lack of presence of encoded data from a number of distributed sources.

Moreover, it is scalable in the sense that codes can be decoded simultaneously at more than one compression quality level.

To the best of our knowledge, this is the first data-driven DSC framework for general distributed code design with deep learning.

We introduce a data-driven Distributed Source Coding framework based on Distributed Recurrent Autoencoder for Scalable Image Compression (DRASIC).

The paper proposed a distributed recurrent auto-encoder for image compression that uses a ConvLSTM to learn binary codes that are constructed progressively from residuals of previously encoded information

The authors propose a method to train image compression models on multiple sources, with a separate encoder on each source, and a shared decoder.

DRASIC: Distributed Recurrent Autoencoder for Scalable Image Compression

distribute recurrent autoencoder scalable image compression


Long short-term memory networks (LSTMs) were introduced to combat vanishing gradients in simple recurrent neural networks (S-RNNs) by augmenting them with additive recurrent connections controlled by gates.

We present an alternate view to explain the success of LSTMs: the gates themselves are powerful recurrent models that provide more representational power than previously appreciated.

We do this by showing that the LSTM's gates can be decoupled from the embedded S-RNN, producing a restricted class of RNNs where the main recurrence computes an element-wise weighted sum of context-independent functions of the inputs.

Experiments on a range of challenging NLP problems demonstrate that the simplified gate-based models work substantially better than S-RNNs, and often just as well as the original LSTMs, strongly suggesting that the gates are doing much more in practice than just alleviating vanishing gradients.

Gates do all the heavy lifting in LSTMs by computing element-wise weighted sums, and removing the internal simple RNN does not degrade model performance.

This paper proposes a simplified LSTM variants by removing the non-linearity of content item and output gate

This paper presents an analysis of LSTMS, showing that they have a form where the memory cell contents at each step is a weighted combination of the “content update” values computed at each time step and offers a simplification of LSTMs that compute the value by which the memory cell at each time step in terms of a deterministic function of the input rather than a function of the input and the current context.

The paper proposes a new insight to LSTM in which the core is an element-wise weighted sum and argues that LSTM is redundant by keeping only input and forget gates to compute the weights

Long Short-Term Memory as a Dynamically Computed Element-wise Weighted Sum

element-wise weight sum
lstms compute
remove
memory
input
gate
content
lstm


Machine learning algorithms designed to characterize, monitor, and intervene on human health (ML4H) are expected to perform safely and reliably when operating at scale, potentially outside strict human supervision.

This requirement warrants a stricter attention to issues of reproducibility than other fields of machine learning.

In this work, we conduct a systematic evaluation of over 100 recently published ML4H research papers along several dimensions related to reproducibility we identified.

We find that the field of ML4H compares poorly to more established machine learning fields, particularly concerning data accessibility and code accessibility.

Finally, drawing from success in other fields of science, we propose recommendations to data providers, academic publishers, and the ML4H research community in order to promote reproducible research moving forward.

By analyzing more than 300 papers in recent machine learning conferences, we found that Machine Learning for Health (ML4H) applications lag behind other machine learning fields in terms of reproducibility metrics.

This paper conducts a quantitative and qualitative review of the state of the reproducibility for ML healthcare applications and proposes reccomendations to make research more reproducible.

Reproducibility in Machine Learning for Health

machine learning
health
application
reproducibility


We propose a solution for evaluation of mathematical expression.

However, instead of designing a single end-to-end model we propose a Lego bricks style architecture.

In this architecture instead of training a complex end-to-end neural network, many small networks can be trained independently each accomplishing one specific operation and acting a single lego brick.

More difficult or complex task can then be solved using a combination of these smaller network.

In this work we first identify 8 fundamental operations that are commonly used to solve arithmetic operations (such as 1 digit multiplication, addition, subtraction, sign calculator etc).

These fundamental operations are then learned using simple feed forward neural networks.

We then shows that different operations can be designed simply by reusing these smaller networks.

As an example we reuse these smaller networks to develop larger and a more complex network to solve n-digit multiplication, n-digit division, and cross product.

This bottom-up strategy not only introduces reusability, we also show that it allows to generalize for computations involving n-digits and we show results for up to 7 digit numbers.

Unlike existing methods, our solution also generalizes for both positive as well as negative numbers.

We train many small networks each for a specific operation, these are then combined to perform complex operations

This paper proposes to use neural networks to evaluate the mathematical expressions by designing 8 small building blocks for 8 fundamental operations, e.g., addition, subtraction, etc and then designing multi-digit multiplication and division using these small blocks.

The paper proposes a method to design a NN based mathematical expression evaluation engine.

Neural Arithmetic Unit by reusing many small pre-trained networks

design
network
operation
many small
mathematical expression
neural


In standard generative adversarial network (SGAN), the discriminator estimates the probability that the input data is real.

The generator is trained to increase the probability that fake data is real.

We argue that it should also simultaneously decrease the probability that real data is real because

1) this would account for a priori knowledge that half of the data in the mini-batch is fake,

2) this would be observed with divergence minimization, and

3) in optimal settings, SGAN would be equivalent to integral probability metric (IPM) GANs.

We show that this property can be induced by using a relativistic discriminator which estimate the probability that the given real data is more realistic than a randomly sampled fake data.

We also present a variant in which the discriminator estimate the probability that the given real data is more realistic than fake data, on average.

We generalize both approaches to non-standard GAN loss functions and we refer to them respectively as Relativistic GANs (RGANs) and Relativistic average GANs (RaGANs).

We show that IPM-based GANs are a subset of RGANs which use the identity function.

1) RGANs and RaGANs are significantly more stable and generate higher quality data samples than their non-relativistic counterparts,

2) Standard RaGAN with gradient penalty generate data of better quality than WGAN-GP while only requiring a single discriminator update per generator update (reducing the time taken for reaching the state-of-the-art by 400%), and

3) RaGANs are able to generate plausible high resolutions images (256x256) from a very small sample (N=2011), while GAN and LSGAN cannot; these images are of significantly better quality than the ones generated by WGAN-GP and SGAN with spectral normalization.

The code is freely available on https://github.com/AlexiaJM/RelativisticGAN.

Improving the quality and stability of GANs using a relativistic discriminator; IPM GANs (such as WGAN-GP) are a special case.

The paper proposes a “relativistic discriminator”, whic helps in some settings, although a bit sensitive to hyperparameters, architectures, and datasets.

In this work, the authors considers a variation of GAN by simultaneously decreasing the probability that real data is real for the generator.

The relativistic discriminator: a key element missing from standard GAN


Some of the most successful applications of deep reinforcement learning to challenging domains in discrete and continuous control have used policy gradient methods in the on-policy setting.

However, policy gradients can suffer from large variance that may limit performance, and in practice require carefully tuned entropy regularization to prevent policy collapse.

As an alternative to policy gradient algorithms, we introduce V-MPO, an on-policy adaptation of Maximum a Posteriori Policy Optimization (MPO) that performs policy iteration based on a learned state-value function.

We show that V-MPO surpasses previously reported scores for both the Atari-57 and DMLab-30 benchmark suites in the multi-task setting, and does so reliably without importance weighting, entropy regularization, or population-based tuning of hyperparameters.

On individual DMLab and Atari levels, the proposed algorithm can achieve scores that are substantially higher than has previously been reported.

V-MPO is also applicable to problems with high-dimensional, continuous action spaces, which we demonstrate in the context of learning to control simulated humanoids with 22 degrees of freedom from full state observations and 56 degrees of freedom from pixel observations, as well as example OpenAI Gym tasks where V-MPO achieves substantially higher asymptotic scores than previously reported.

A state-value function-based version of MPO that achieves good results in a wide range of tasks in discrete and continuous control.

This paper presents an algorithm for on-policy reinforcement learning that can handle both continuous/discrete control, single/multi-task learning and use both low dimensional states and pixels.

The paper proposes an online variant of MPO, V-MPO, which learns the V-function and updates the non-parametric distribution towards the advantages.

V-MPO: On-Policy Maximum a Posteriori Policy Optimization for Discrete and Continuous Control

discrete continuous control
mpo
on-policy
v-mpo


Turing complete computation and reasoning are often regarded as necessary pre- cursors to general intelligence.

There has been a significant body of work studying neural networks that mimic general computation, but these networks fail to generalize to data distributions that are outside of their training set.

We study this problem through the lens of fundamental computer science problems: sorting and graph processing.

We modify the masking mechanism of a transformer in order to allow them to implement rudimentary functions with strong generalization.

We call this model the Neural Execution Engine, and show that it learns, through supervision, to numerically compute the basic subroutines comprising these algorithms with near perfect accuracy.

Moreover, it retains this level of accuracy while generalizing to unseen data and long sequences outside of the training distribution.

We propose neural execution engines (NEEs), which leverage a learned mask and supervised execution traces to mimic the functionality of subroutines and demonstrate strong generalization.

This paper investigates a problem of building a program execution engine with neural networks and proposes a transformer-based model to learn basic subroutines and applies them in several standard algorithms.

This paper deals with the problem of designing neural network architectures that can learn and implement general programs.

NEURAL EXECUTION ENGINES

execution engine
problem
neural
subroutine
learn
program
network


Meta-learning is a promising strategy for learning to efficiently learn within new tasks, using data gathered from a distribution of tasks.

However, the meta-learning literature thus far has focused on the task segmented setting, where at train-time, offline data is assumed to be split according to the underlying task, and at test-time, the algorithms are optimized to learn in a single task.

In this work, we enable the application of generic meta-learning algorithms to settings where this task segmentation is unavailable, such as continual online learning with a time-varying task.

We present meta-learning via online changepoint analysis (MOCA), an approach which augments a meta-learning algorithm with a differentiable Bayesian changepoint detection scheme.

The framework allows both training and testing directly on time series data without segmenting it into discrete tasks.

We demonstrate the utility of this approach on a nonlinear meta-regression benchmark as well as two meta-image-classification benchmarks.

Bayesian changepoint detection enables meta-learning directly from time series data.

The paper considers the meta-learning in the task un-segmented setting and apply Bayesian online change point detection with meta-learning.

This paper pushes meta-learning towards task-unsegmented settings, where the MOCA framework adopts a Bayesian changepoint estimation scheme for task change detection.

Continuous Meta-Learning without Tasks

bayesian changepoint
task
detection
meta-learne
change
meta-learning


People with high-frequency hearing loss rely on hearing aids that employ frequency lowering algorithms.

These algorithms shift some of the sounds from the high frequency band to the lower frequency band where the sounds become more perceptible for the people with the condition.

Fricative phonemes have an important part of their content concentrated in high frequency bands.

It is important that the frequency lowering algorithm is activated exactly for the duration of a fricative phoneme, and kept off at all other times.

Therefore, timely (with zero delay) and accurate fricative phoneme detection is a key problem for high quality hearing aids.

In this paper we present a deep learning based fricative phoneme detection algorithm that has zero detection delay and achieves state-of-the-art fricative phoneme detection accuracy on the TIMIT Speech Corpus.

All reported results are reproducible and come with easy to use code that could serve as a baseline for future research.

A deep learning based approach for zero delay fricative phoneme detection

This paper apples supervised deep learning methods to detect exact duration of a fricative phoneme in order to improve practical frequency lowering algorithm.

FRICATIVE PHONEME DETECTION WITH ZERO DELAY

fricative phoneme detection
zero delay
deep


Sequence-to-sequence models with soft attention have been successfully applied to a wide variety of problems, but their decoding process incurs a quadratic time and space cost and is inapplicable to real-time sequence transduction.

To address these issues, we propose Monotonic Chunkwise Attention (MoChA), which adaptively splits the input sequence into small chunks over which soft attention is computed.

We show that models utilizing MoChA can be trained efficiently with standard backpropagation while allowing online and linear-time decoding at test time.

When applied to online speech recognition, we obtain state-of-the-art results and match the performance of a model using an offline soft attention mechanism.

In document summarization experiments where we do not expect monotonic alignments, we show significantly improved performance compared to a baseline monotonic attention-based model.

An online and linear-time attention mechanism that performs soft attention over adaptively-located chunks of the input sequence.

This paper proposes a small modification to the monotonic attention in [1] by adding a soft attention to the segment predicted by the monotonic attention.

The paper proposes an extension to a previous monotonic attention model (Raffel et al 2017) to attend to a fixed-sized window up to the alignment position.

Monotonic Chunkwise Attention


We present a framework for automatically ordering image patches that enables in-depth analysis of dataset relationship to learnability of a classification task using convolutional neural network.

An image patch is a group of pixels residing in a continuous area contained in the sample.

Our preliminary experimental results show that an informed smart shuffling of patches at a sample level can expedite training by exposing important features at early stages of training.

In addition, we conduct systematic experiments and provide evidence that CNN’s generalization capabilities do not correlate with human recognizable features present in training samples.

We utilized the framework not only to show that spatial locality of features within samples do not correlate with generalization, but also to expedite convergence while achieving similar generalization performance.

Using multiple network architectures and datasets, we show that ordering image regions using mutual information measure between adjacent patches, enables CNNs to converge in a third of the total steps required to train the same network without patch ordering.

Develop new techniques that rely on patch reordering to enable detailed analysis of data-set relationship to training and generalization performances.

EXPLORING DEEP LEARNING USING INFORMATION THEORY TOOLS AND PATCH ORDERING


Producing agents that can generalize to a wide range of environments is a significant challenge in reinforcement learning.

One method for overcoming this issue is domain randomization, whereby at the start of each training episode some parameters of the environment are randomized so that the agent is exposed to many possible variations.

However, domain randomization is highly inefficient and may lead to policies with high variance across domains.

In this work, we formalize the domain randomization problem, and show that minimizing the policy's Lipschitz constant with respect to the randomization parameters leads to low variance in the learned policies.

We propose a method where the agent only needs to be trained on one variation of the environment, and its learned state representations are regularized during training to minimize this constant.

We conduct experiments that demonstrate that our technique leads to more efficient and robust learning than standard domain randomization, while achieving equal generalization scores.

We produce reinforcement learning agents that generalize well to a wide range of environments using a novel regularization technique.

The paper introduces the high variance policies challenge in domain randomization for reinforcement learning and mainly focuses on the problem of visual randomization, where the different randomized domains differ only in state space and the underlying rewards and dynamics are the same.

To improve the generalization ability of deep RL agents across the tasks with different visual patterns, this paper proposed a simple regularization technique for domain randomization.

Robust Domain Randomization for Reinforcement Learning

domain randomization reinforcement learning


Claims from the fields of network neuroscience and connectomics suggest that topological models of the brain involving complex networks are of particular use and interest.

The field of deep neural networks has mostly left inspiration from these claims out.

In this paper, we propose three architectures and use each of them to explore the intersection of network neuroscience and deep learning in an attempt to bridge the gap between the two fields.

Using the teachings from network neuroscience and connectomics, we show improvements over the ResNet architecture, we show a possible connection between early training and the spectral properties of the network, and we show the trainability of a DNN based on the neuronal network of C.Elegans.

We explore the intersection of network neurosciences and deep learning.

Deep connectomics networks: Results from neural network architectures inspired from network neuroscience


Creating a knowledge base that is accurate, up-to-date and complete remains a significant challenge despite substantial efforts in automated knowledge base construction.

In this paper, we present Alexandria -- a system for unsupervised, high-precision knowledge base construction.

Alexandria uses a probabilistic program to define a process of converting knowledge base facts into unstructured text.

Using probabilistic inference, we can invert this program and so retrieve facts, schemas and entities from web text.

The use of a probabilistic program allows uncertainty in the text to be propagated through to the retrieved facts, which increases accuracy and helps merge facts from multiple sources.

Because Alexandria does not require labelled training data, knowledge bases can be constructed with the minimum of manual input.

We demonstrate this by constructing a high precision (typically 97\%+) knowledge base for people from a single seed fact.

This paper presents a system for unsupervised, high-precision knowledge base construction using a probabilistic program to define a process of converting knowledge base facts into unstructured text.

Overview about existing knowledge base that is constructed with a probabilistic model, with the knowledge base construction approach evaluated against other knowledge base approaches YAGO2, NELL, Knowledge Vault, and DeepDive.

This paper uses a probabilistic program describing the process by which facts describing entities can be realised in text and large number of web pages, to learn to perform fact extraction about people using a single seed fact.

Alexandria: Unsupervised High-Precision Knowledge Base Construction using a Probabilistic Program

high-precision knowledge base construction probabilistic program


Recent advances have made it possible to create deep complex-valued neural networks.

Despite this progress, many challenging learning tasks have yet to leverage the power of complex representations.

Building on recent advances, we propose a new deep complex-valued method for signal retrieval and extraction in the frequency domain.

As a case study, we perform audio source separation in the Fourier domain.

Our new method takes advantage of the convolution theorem which states that the Fourier transform of two convolved signals is the elementwise product of their Fourier transforms.

Our novel method is based on a complex-valued version of Feature-Wise Linear Modulation (FiLM) and serves as the keystone of our proposed signal extraction method.

We also introduce a new and explicit amplitude and phase-aware loss, which is scale and time invariant, taking into account the complex-valued components of the spectrogram.

Using the Wall Street Journal Dataset, we compared our phase-aware loss to several others that operate both in the time and frequency domains and demonstrate the effectiveness of our proposed signal extraction method and proposed loss.

New Signal Extraction Method in the Fourier Domain

Retrieving Signals with Deep Complex Extractors


We propose an implementation of GNN that predicts and imitates the motion be- haviors from observed swarm trajectory data.

The network’s ability to capture interaction dynamics in swarms is demonstrated through transfer learning.

We finally discuss the inherent availability and challenges in the scalability of GNN, and proposed a method to improve it with layer-wise tuning and mixing of data enabled by padding.

Improve the scalability of graph neural networks on imitation learning and prediction of swarm motion

The paper proposes a new time series model for learning a sequence of graphs.

This work considers sequence prediction problems in a multi-agent system.

Towards Scalable Imitation Learning for Multi-Agent Systems with Graph Neural Networks

graph neural network
multi-agent system
learn
prediction
imitation
sequence


Embedding layers are commonly used to map discrete symbols into continuous embedding vectors that reflect their semantic meanings.

Despite their effectiveness, the number of parameters in an embedding layer increases linearly with the number of symbols and poses a critical challenge on memory and storage constraints.

In this work, we propose a generic and end-to-end learnable compression framework termed differentiable product quantization (DPQ).

We present two instantiations of DPQ that leverage different approximation techniques to enable differentiability in end-to-end learning.

Our method can readily serve as a drop-in alternative for any existing embedding layer.

Empirically, DPQ offers significant compression ratios (14-238x) at negligible or no performance cost on 10 datasets across three different language tasks.

We propose a differentiable product quantization framework that can reduce the size of embedding layer in an end-to-end training at no performance cost.

This paper works on methods for compressing embedding layers for low memory inference, where compressed embeddings are learned together with the task-specific models in a differentiable end-to-end fashion.

Learning Compact Embedding Layers via Differentiable Product Quantization

differentiable
embed layer
product quantization
end-to-end
learn


For multi-valued functions---such as when the conditional distribution on targets given the inputs is multi-modal---standard regression approaches are not always desirable because they provide the conditional mean.

Modal regression approaches aim to instead find the conditional mode, but are restricted to nonparametric approaches.

Such approaches can be difficult to scale, and make it difficult to benefit from parametric function approximation, like neural networks, which can learn complex relationships between inputs and targets.

In this work, we propose a parametric modal regression algorithm, by using the implicit function theorem to develop an objective for learning a joint parameterized function over inputs and targets.

We empirically demonstrate on several synthetic problems that our method

(i) can learn multi-valued functions and produce the conditional modes,

(ii) scales well to high-dimensional inputs and

(iii) is even more effective for certain unimodal problems, particularly for high frequency data where the joint function over inputs and targets can better capture the complex relationship between them.

We conclude by showing that our method provides small improvements on two regression datasets that have asymmetric distributions over the targets.

We introduce a simple and novel modal regression algorithm which is easy to scale to large problems.

The paper proposes an implicit function approach to learning the modes of multimodal regression.

The present work proposes a parametric approach to estimate the conditional mode using the Implicit Function Theorem for multi-modal distributions.

An implicit function learning approach for parametric modal regression

implicit function
mode
modal regression
approach
parametric
learn


Deep reinforcement learning algorithms require large amounts of experience to learn an individual task.

While in principle meta-reinforcement learning (meta-RL) algorithms enable agents to learn new skills from small amounts of experience, several major challenges preclude their practicality.

Current methods rely heavily on on-policy experience, limiting their sample efficiency.

They also lack mechanisms to reason about task uncertainty when adapting to new tasks, limiting their effectiveness in sparse reward problems.

In this paper, we address these challenges by developing an off-policy meta-RL algorithm that disentangles task inference and control.

In our approach, we perform online probabilistic filtering of latent task variables to infer how to solve a new task from small amounts of experience.

This probabilistic interpretation enables posterior sampling for structured and efficient exploration.

We demonstrate how to integrate these task variables with off-policy RL algorithms to achieve both meta-training and adaptation efficiency.

Our method outperforms prior algorithms in sample efficiency by 20-100X as well as in asymptotic performance on several meta-RL benchmarks.

Sample efficient meta-RL by combining variational inference of probabilistic task variables with off-policy RL

This paper proposes using off-policy RL during the meta-training time to greatly improve sample efficiency of Meta-RL methods.

Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables

off-policy rl
meta-rl
efficient
probabilistic
variable
sample


Knowledge bases, massive collections of facts (RDF triples) on diverse topics, support vital modern applications.

However, existing knowledge bases contain very little data compared to the wealth of information on the Web.

This is because the industry standard in knowledge base creation and augmentation suffers from a serious bottleneck: they rely on domain experts to identify appropriate web sources to extract data from.

Efforts to fully automate knowledge extraction have failed to improve this standard: these automated systems are able to retrieve much more data and from a broader range of sources, but they suffer from very low precision and recall.

As a result, these large-scale extractions remain unexploited.

In this paper, we present MIDAS, a system that harnesses the results of automated knowledge extraction pipelines to repair the bottleneck in industrial knowledge creation and augmentation processes.

MIDAS automates the suggestion of good-quality web sources and describes what to extract with respect to augmenting an existing knowledge base.

First, we introduce a novel concept, web source slices, to describe the contents of a web source.

Second, we define a profit function to quantify the value of a web source slice with respect to augmenting an existing knowledge base.

Third, we develop effective and highly-scalable algorithms to derive high-profit web source slices.

We demonstrate that MIDAS produces high-profit results and outperforms the baselines significantly on both real-word and synthetic datasets.

This paper focuses on identifying high quality web sources for industrial knowledge base augmentation pipeline.

MIDAS: Finding the Right Web Sources to Fill Knowledge Gaps