The training of deep neural networks with Stochastic Gradient Descent (SGD) with a large learning rate or a small batch-size typically ends in flat regions of the weight space, as indicated by small eigenvalues of the Hessian of the training loss.

This was found to correlate with a good final generalization performance.

In this paper we extend previous work by investigating the curvature of the loss surface along the whole training trajectory, rather than only at the endpoint.

We find that initially SGD visits increasingly sharp regions, reaching a maximum sharpness determined by both the learning rate and the batch-size of SGD.

At this peak value SGD starts to fail to minimize the loss along directions in the loss surface corresponding to the largest curvature (sharpest directions).

To further investigate the effect of these dynamics in the training process, we study a variant of SGD using a reduced learning rate along the sharpest directions which we show can improve training speed while finding both sharper and better generalizing solution, compared to vanilla SGD.

Overall, our results show that the SGD dynamics in the subspace of the sharpest directions influence the regions that SGD steers to (where larger learning rate or smaller batch size result in wider regions visited), the overall training speed, and the generalization ability of the final model.

SGD is steered early on in training towards a region in which its step is too large compared to curvature, which impacts the rest of training.

Analyzes the relationship between the convergence/generalization and the update on largest eigenvectors of Hessian of the empirical losses of DNNs.

This work studies the relationship between the SGD step size and the curvature of the loss surface

On the Relation Between the Sharpest Directions of DNN Loss and the SGD Step Length

curvature
relationship
sgd step
loss
large
dnn


We introduce a new approach to estimate continuous actions using actor-critic algorithms for reinforcement learning problems.

Policy gradient methods usually predict one continuous action estimate or parameters of a presumed distribution (most commonly Gaussian) for any given state which might not be optimal as it may not capture the complete description of the target distribution.

Our approach instead predicts M actions with the policy network (actor) and then uniformly sample one action during training as well as testing at each state.

This allows the agent to learn a simple stochastic policy that has an easy to compute expected return.

In all experiments, this facilitates better exploration of the state space during training and converges to a better policy.

We introduce a novel reinforcement learning algorithm, that predicts multiple actions and samples from them.

This work introduces a uniform mixture of deterministic policies, and find that this parametrization of stochastic policies outperforms DDPG on several OpenAI gym benchmarks.

The authors investigate a method for improving the performance of networks trained with DDPG, and show improved performance on a large number of standard continuous control environment.

Predicting Multiple Actions for Stochastic Continuous Control

predict multiple action
introduce
ddpg
stochastic
continuous control


Recently convolutional neural networks (CNNs) achieve great accuracy in visual recognition tasks.

DenseNet becomes one of the most popular CNN models due to its effectiveness in feature-reuse.

However, like other CNN models, DenseNets also face overfitting problem if not severer.

Existing dropout method can be applied but not as effective due to the introduced nonlinear connections.

In particular, the property of feature-reuse in DenseNet will be impeded, and the dropout effect will be weakened by the spatial correlation inside feature maps.

To address these problems, we craft the design of a specialized dropout method from three aspects, dropout location, dropout granularity, and dropout probability.

The insights attained here could potentially be applied as a general approach for boosting the accuracy of other CNN models with similar nonlinear connections.

Experimental results show that DenseNets with our specialized dropout method yield better accuracy compared to vanilla DenseNet and state-of-the-art CNN models, and such accuracy boost increases with the model depth.

Realizing the drawbacks when applying original dropout on DenseNet, we craft the design of dropout method from three aspects, the idea of which could also be applied on other CNN models.

Application of different binary dropout structures and schedules with the specific aim to regularise the DenseNet architecture.

Proposes a pre-dropout technique for densenet which implements the dropout before the non-linear activation function.

Reconciling Feature-Reuse and Overfitting in DenseNet with Specialized Dropout


While extremely successful in several applications, especially with low-level representations; sparse, noisy samples and structured domains (with multiple objects and interactions) are some of the open challenges in most deep models.

Column Networks, a deep architecture, can succinctly capture such domain structure and interactions, but may still be prone to sub-optimal learning from sparse and noisy samples.

Inspired by the success of human-advice guided learning in AI, especially in data-scarce domains, we propose Knowledge-augmented Column Networks that leverage human advice/knowledge for better learning with noisy/sparse samples.

Our experiments demonstrate how our approach leads to either superior overall performance or faster convergence.

Guiding relation-aware deep models towards better learning with human knowledge.

This work proposes a variant of the column network based on the injection of human guidance by modifying calculations in the network.

A method to incorporate human advices to deep learning by extending Column Network, a graph neural network for collective classification.

Human-Guided Column Networks: Augmenting Deep Learning with Advice

column network
deep learning
human
advice


Recent research has shown that one can train a neural network with binary weights and activations at train time by augmenting the weights with a high-precision continuous latent variable that accumulates small changes from stochastic gradient descent.

However, there is a dearth of work to explain why one can effectively capture the features in data with binary weights and activations.

Our main result is that the neural networks with binary weights and activations trained using the method of Courbariaux, Hubara et al. (2016) work because of the high-dimensional geometry of binary vectors.

In particular, the ideal continuous vectors that extract out features in the intermediate representations of these BNNs are well-approximated by binary vectors in the sense that dot products are approximately preserved.

Compared to previous research that demonstrated good classification performance with BNNs, our work explains why these BNNs work in terms of HD geometry.

Furthermore, the results and analysis used on BNNs are shown to generalize to neural networks with ternary weights and activations.

Our theory serves as a foundation for understanding not only BNNs but a variety of methods that seek to compress traditional neural networks.

Furthermore, a better understanding of multilayer binary neural networks serves as a starting point for generalizing BNNs to other neural network architectures such as recurrent neural networks.

Recent successes of Binary Neural Networks can be understood based on the geometry of high-dimensional binary vectors

Investigates numerically and theoretically the reasons behind the empirical success of binarized neural networks.

This paper analyzes the effectiveness of binary neural networks and why binarization is able to preserve model performance.

The High-Dimensional Geometry of Binary Neural Networks

binary neural network
high-dimensional
geometry
success


In recent years Convolutional Neural Networks (CNN) have been used extensively for Superresolution (SR).

In this paper, we use inverse problem and sparse representation solutions to form a mathematical basis for CNN operations.

We show how a single neuron is able to provide the optimum solution for inverse problem, given a low resolution image dictionary as an operator.

Introducing a new concept called Representation Dictionary Duality, we show that CNN elements (filters) are trained to be representation vectors and then, during reconstruction, used as dictionaries.

In the light of theoretical work, we propose a new algorithm which uses two networks with different structures that are separately trained with low and high coherency image patches and show that it performs faster compared to the state-of-the-art algorithms while not sacrificing from performance.

After proving that a neuron acts as an inverse problem solver for superresolution and a network of neurons is guarantied to provide a solution, we proposed a double network architecture that performs faster than state-of-the-art.

Discusses using neural networks for super-resolution

A new architecture for solving image super-resolution tasks, and an analysis aiming to establish a connection between CNNs for solving super resolution and solving sparse regularized inverse problems.

CNNs as Inverse Problem Solvers and Double Network Superresolution

inverse problem solver
double network
architecture
superresolution
super-resolution


We consider the learning of algorithmic tasks by mere observation of input-output

Rather than studying this as a black-box discrete regression problem with

no assumption whatsoever on the input-output mapping, we concentrate on tasks

that are amenable to the principle of divide and conquer, and study what are its

This principle creates a powerful inductive bias that we leverage with neural

architectures that are defined recursively and dynamically, by learning two scale-

invariant atomic operations: how to split a given input into smaller sets, and how

to merge two partially solved tasks into a larger partial solution.

trained in weakly supervised environments, namely by just observing input-output

pairs, and in even weaker environments, using a non-differentiable reward signal.

Moreover, thanks to the dynamic aspect of our architecture, we can incorporate

the computational complexity as a regularization term that can be optimized by

We demonstrate the flexibility and efficiency of the Divide-

and-Conquer Network on several combinatorial and geometric tasks: convex hull,

nature of our model, we show significant improvements in terms of generalization

Dynamic model that learns divide and conquer strategies by weak supervision.

Proposes to add new inductive bias to neural network architecture by using a divide and conquer strategy.

This paper studies problems that can be solved using a dynamic programming approach, and proposes a neural network architecture to solve such problems that beats sequence to sequence baselines.

The paper proposes a unique network architecture that can learn divide-and-conquer strategies to solve algorithmic tasks.

Divide and Conquer Networks

neural network architecture
divide conquer strategy
learn
solve
dynamic


Within many machine learning algorithms, a fundamental problem concerns efficient calculation of an unbiased gradient wrt parameters $\boldsymbol{\gamma}$ for expectation-based objectives $\mathbb{E}_{q_{\boldsymbol{\gamma}} (\boldsymbol{y})} [f (\boldsymbol{y}) ]$.

Most existing methods either ($i$) suffer from high variance, seeking help from (often) complicated variance-reduction techniques; or ($ii$) they only apply to reparameterizable continuous random variables and employ a reparameterization trick.

To address these limitations, we propose a General and One-sample (GO) gradient that ($i$) applies to many distributions associated with non-reparameterizable continuous {\em or} discrete random variables, and ($ii$) has the same low-variance as the reparameterization trick.

We find that the GO gradient often works well in practice based on only one Monte Carlo sample (although one can of course use more samples if desired).

Alongside the GO gradient, we develop a means of propagating the chain rule through distributions, yielding statistical back-propagation, coupling neural networks to common random variables.

a Rep-like gradient for non-reparameterizable continuous/discrete distributions; further generalized to deep probabilistic models, yielding statistical back-propagation

Presents a gradient estimator for expectation-based objectives that is unbiased, has low variance, and applies to either continuous and discrete random variables.

An improved method for computing derivates of the expectation, and a new gradient estimator of low variance that allows training of generative models in which observations or latent variables are discrete.

Designs a low variance gradient for distributions associated with continuous or discrete random variables.

GO Gradient for Expectation-Based Objectives

continuous discrete random variable
low variance
expectation-based objective
gradient estimator
model
distribution


Quantum computers promise significant advantages over classical computers for a number of different applications.

We show that the complete loss function landscape of a neural network can be represented as the quantum state output by a quantum computer.

We demonstrate this explicitly for a binary neural network and, further, show how a quantum computer can train the network by manipulating this state using a well-known algorithm known as quantum amplitude amplification.

We further show that with minor adaptation, this method can also represent the meta-loss landscape of a number of neural network architectures simultaneously.

We search this meta-loss landscape with the same method to simultaneously train and design a binary neural network.

We show that NN parameter and hyperparameter cost landscapes can be generated as quantum states using a single quantum circuit and that these can be used for training and meta-training.

Describes a method where a deep learning framework can be quantised by considering the two state form of a Bloch sphere/qubit and creating a quantum binary neural network.

This paper proposes quantum amplitude amplification, a new algorithm for training and model selection in binary neural networks.

Proposes a novel idea of outputting a quantum state that represents a complete cost landscape of all parameters for a given binary neural network, by constructing a quantum binary neural network (QBNN).

Neural Network Cost Landscapes as Quantum States

quantum binary neural network
cost landscape
training
state
parameter


Several recent works have developed methods for training classifiers that are certifiably robust against norm-bounded adversarial perturbations.

These methods assume that all the adversarial transformations are equally important, which is seldom the case in real-world applications.

We advocate for cost-sensitive robustness as the criteria for measuring the classifier's performance for tasks where some adversarial transformation are more important than others.

We encode the potential harm of each adversarial transformation in a cost matrix, and propose a general objective function to adapt the robust training method of Wong & Kolter (2018) to optimize for cost-sensitive robustness.

Our experiments on simple MNIST and CIFAR10 models with a variety of cost matrices show that the proposed approach can produce models with substantially reduced cost-sensitive robust error, while maintaining classification accuracy.

A general method for training certified cost-sensitive robust classifier against adversarial perturbations

Calculates and plugs in the costs of adversarial attack into the objective of optimization to get a model that is cost-sensitively robust against adversarial attacks.

Build on semnial work by Dalvi et al. and extends approach to certifiable robustness with a cost matrix that specifies for each pair of source-target classes whether the model should be robust to adversarial examples.

Cost-Sensitive Robustness against Adversarial Examples

model
cost
robust adversarial
robustness
cost-sensitive
example


Retinal prostheses for treating incurable blindness are designed to electrically stimulate surviving retinal neurons,  causing them to send artificial visual signals to the brain.

However, electrical stimulation generally cannot precisely reproduce  normal patterns of neural activity in the retina.

Therefore, an electrical stimulus must be selected that produces a neural response as close as possible to the desired response.

This requires a technique for computing a distance between the desired response and the achievable response that is meaningful in terms of the visual signal being conveyed.

Here we propose a method to learn such a metric on neural responses, directly from recorded light responses of a population of retinal ganglion cells (RGCs) in the primate retina.

The learned metric produces a measure of similarity of RGC population responses that accurately reflects the similarity of the visual input.

Using data from electrical stimulation experiments, we demonstrate that this metric may improve the performance of a prosthesis.

Using triplets to learn a metric for comparing neural responses and improve the performance of a prosthesis.

Authors develop new spike train distance metrics, including neural networks and quadratic metrics. These metrics are shown to outperform the naive Hamming distance metric, and implicitly captures some structure in neural code.

With the application of improving neural prosthesis in mind, the authors propose to learn a metric between neural responses by either optimizing a quadratic form or a deep neural network .

Learning a neural response metric for retinal prosthesis

learn metric
neural response
improve
prosthesis
quadratic
network
author


We introduce a novel workflow, QCue, for providing textual stimulation during mind-mapping.

Mind-mapping is a powerful tool whose intent is to allow one to externalize ideas and their relationships surrounding a central problem.

The key challenge in mind-mapping is the difficulty in balancing the exploration of different aspects of the problem (breadth) with a detailed exploration of each of those aspects (depth).

Our idea behind QCue is based on two mechanisms: (1) computer-generated automatic cues to stimulate the user to explore the breadth of topics based on the temporal and topological evolution of a mind-map and (2) user-elicited queries for helping the user explore the depth for a given topic.

We present a two-phase study wherein the first phase provided insights that led to the development of our work-flow for stimulating the user through cues and queries.

In the second phase, we present a between-subjects evaluation comparing QCue with a digital mind-mapping work-flow without computer intervention.

Finally, we present an expert rater evaluation of the mind-maps created by users in conjunction with user feedback.

This paper introduces a method to generate questions (cues) and queries (suggestions) to help users perform mind-mapping.

Presents a tool to assist mind-mapping through suggested context related to existing nodes and through questions that expand on less developed branches.

This paper presents an approach for assisting people with mindmapping tasks, designing an interface and algorithmic features to suppport mindmapping, and contributes a evaluative study.

QCue: Queries and Cues for Computer-Facilitated Mind-Mapping

query cue question assist present mind-mapping


The ability to detect when an input sample was not drawn from the training distribution is an important  desirable property of deep neural networks.

In this paper, we show that a simple ensembling of first and second order deep feature statistics can be exploited to effectively differentiate in-distribution and out-of-distribution samples.

Specifically, we observe that  the mean and standard deviation within feature maps  differs greatly between in-distribution and out-of-distribution samples.

Based on this observation, we propose a simple and  efficient plug-and-play detection procedure that does not require re-training, pre-processing or changes to the model.

The proposed method outperforms the state-of-the-art by a large margin in all standard benchmarking tasks, while being much simpler to implement and execute.

Notably, our method improves the true negative rate from 39.6% to 95.3% when 95% of in-distribution (CIFAR-100) are correctly detected using a DenseNet and the out-of-distribution dataset is TinyImageNet resize.

The source code of our method will be made publicly available.

Detecting out-of-distribution samples by using low-order feature statistics without requiring any change in underlying DNN.

Presents an algorithm to detect out-of-distribution samples by using the running estimate of mean and variance within BatchNorm layers to construct feature representations later fed into a linear classifier.

An approach for detecting out-of-distribution samples in which the authors propose to use logistic regression over simple statistics of each batch normalization layer of CNN.

The paper suggests using Z-scores for comparing ID and OOD samples to evaluate what deep nets are trying to do.

Detecting Out-Of-Distribution Samples Using Low-Order Deep Features Statistics

detect out-of-distribution sample
low-order
feature statistic
layer
deep


Due to the sharp increase in the severity of the threat imposed by software vulnerabilities, the detection of vulnerabilities in binary code has become an important concern in the software industry, such as the embedded systems industry, and in the field of computer security.

However, most of the work in binary code vulnerability detection has relied on handcrafted features which are manually chosen by a select few, knowledgeable domain experts.

In this paper, we attempt to alleviate this severe binary vulnerability detection bottleneck by leveraging recent advances in deep learning representations and propose the Maximal Divergence Sequential Auto-Encoder.

In particular, latent codes representing vulnerable and non-vulnerable binaries are encouraged to be maximally divergent, while still being able to maintain crucial information from the original binaries.

We conducted extensive experiments to compare and contrast our proposed methods with the baselines, and the results show that our proposed methods outperform the baselines in all performance measures of interest.

We propose a novel method named Maximal Divergence Sequential Auto-Encoder that leverages Variational AutoEncoder representation for binary code vulnerability detection.

This paper proposes a variational autoencoder-based architecture for code embeddings for binary software vulnerability detection, with learned embeddings more effective at distinguishing between vulnerable and non-vulnerable binary code than baselines.

This paper proposes a model to automatically extract features for vulnerability detection using deep learning technique.

Maximal Divergence Sequential Autoencoder for Binary Software Vulnerability Detection

binary software vulnerability detection
maximal divergence sequential
code
autoencoder
variational


Modern neural architectures critically rely on attention for mapping structured inputs to sequences.

In this paper we show that prevalent attention architectures do not adequately model the dependence among the attention and output tokens across a predicted sequence.

We present an alternative architecture called  Posterior Attention Models that after a principled factorization of the full joint distribution of the attention and output variables, proposes two major changes.

First, the position where attention is marginalized is changed from the input to the output.

Second, the attention propagated to the next decoding stage is a posterior attention distribution conditioned on the output.

Empirically on five translation and two morphological inflection tasks the proposed posterior attention models yield better BLEU score and alignment accuracy than existing attention models.

Computing attention based on posterior distribution leads to more meaningful attention and better performance

This paper proposes a sequence to sequence model where attention is treated as a latent variable, and derives novel inference procedures for this model, obtaining improvements in machine translation and morphological inflection generation tasks.

This paper presents a novel posterior attention model for seq2seq problems

Posterior Attention Models for Sequence to Sequence Learning

sequence sequence
posterior attention model
novel


The growing interest to implement Deep Neural Networks (DNNs) on resource-bound hardware has motivated innovation of compression algorithms.

Using these algorithms, DNN model sizes can be substantially reduced, with little to no accuracy degradation.

This is achieved by either eliminating components from the model, or penalizing complexity during training.

While both approaches demonstrate considerable compressions, the former often ignores the loss function during compression while the later produces unpredictable compressions.

In this paper, we propose a technique that directly minimizes both the model complexity and the changes in the loss function.

In this technique, we formulate compression as a constrained optimization problem, and then present a solution for it.

We will show that using this technique, we can achieve competitive results.

Compressing trained DNN models by minimizing their complexity while constraining their loss.

This paper proposes a method for deep neural network compression under accuracy constraints.

This paper presents a loss value constrained k-means encoding method for network compression and develops an iterative algorithm for model optimization.

DNN Model Compression Under Accuracy Constraints

compression accuracy constraint
model
loss
method
dnn
constrain
network


Deep neural networks are able to solve tasks across a variety of domains and modalities of data.

Despite many empirical successes, we lack the ability to clearly understand and interpret the learned mechanisms that contribute to such effective behaviors and more critically, failure modes.

In this work, we present a general method for visualizing an arbitrary neural network's inner mechanisms and their power and limitations.

Our dataset-centric method produces visualizations of how a trained network attends to components of its inputs.

The computed "attention masks" support improved interpretability by highlighting which input attributes are critical in determining output.

We demonstrate the effectiveness of our framework on a variety of deep neural network architectures in domains from computer vision and natural language processing.

The primary contribution of our approach is an interpretable visualization of attention that provides unique insights into the network's underlying decision-making process irrespective of the data modality.

We develop a technique to visualize attention mechanisms in arbitrary neural networks.

Proposes to learn a Latent Attention Network that can help to visualize the inner structure of a deep neural network.

The authors of this paper propose a data-driven black-box visualization scheme.

Modeling Latent Attention Within Neural Networks

neural network
visualize
latent attention


The design of small molecules with bespoke properties is of central importance to drug discovery.

However significant challenges yet remain for computational methods, despite recent advances such as deep recurrent networks and reinforcement learning strategies for sequence generation, and it can be difficult to compare results across different works.

This work proposes 19 benchmarks selected by subject experts, expands smaller datasets previously used to approximately 1.1 million training molecules, and explores how to apply new reinforcement learning techniques effectively for molecular design.

The benchmarks here, built as OpenAI Gym environments, will be open-sourced to encourage innovation in molecular design algorithms and to enable usage by those without a background in chemistry.

Finally, this work explores recent development in reinforcement-learning methods with excellent sample complexity (the A2C and PPO algorithms) and investigates their behavior in molecular generation, demonstrating significant performance gains compared to standard reinforcement learning techniques.

We investigate a variety of RL algorithms for molecular generation and define new benchmarks (to be released as an OpenAI Gym), finding PPO and a hill-climbing MLE algorithm work best.

Considers model evaluation for molecule generation by proposing 19 benchmarks, expanding small data sets to a large, standardized dataset, and exploring how to apply RL techniques for molecular design.

This paper shows that the most sophisticated RL methods are less effective than the simple hill-climbing technique, with PPO as the exception, when modeling and synthesizing molecules.

Exploring Deep Recurrent Models with Reinforcement Learning for Molecule Design

ppo molecular generation benchmark molecule technique model hill-climbing rl design explore


Analogical reasoning has been a principal focus of various waves of AI research.

Analogy is particularly challenging for machines because it requires relational structures to be represented such that they can be flexibly applied across diverse domains of experience.

Here, we study how analogical reasoning can be induced in neural networks that learn to perceive and reason about raw visual data.

We find that the critical factor for inducing such a capacity is not an elaborate architecture, but rather, careful attention to the choice of data and the manner in which it is presented to the model.

The most robust capacity for analogical reasoning is induced when networks learn analogies by contrasting abstract relational structures in their input domains, a training method that uses only the input data to force models to learn about important abstract features.

Using this technique we demonstrate capacities for complex, visual and symbolic analogy making and generalisation in even the simplest neural network architectures.

The most robust capacity for analogical reasoning is induced when networks learn analogies by contrasting abstract relational structures in their input domains.

The paper investigates the ability of a neural network to learn analogy, showing that a simple neural network is able to solve certain analogy problems

This paper describes an approach to train neural networks for analogical reasoning tasks, specifically considering visual analogy and symbolic analogies.

Learning to Make Analogies by Contrasting Abstract Relational Structure

analogy contrast abstract relational structure


Building chatbots that can accomplish goals such as booking a flight ticket is an unsolved problem in natural language understanding.

Much progress has been made to build conversation models using techniques such as sequence2sequence modeling.

One challenge in applying such techniques to building goal-oriented conversation models is that maximum likelihood-based models are not optimized toward accomplishing goals.

Recently, many methods have been proposed to address this issue by optimizing a reward that contains task status or outcome.

However, adding the reward optimization on the fly usually provides little guidance for language construction and the conversation model soon becomes decoupled from the language model.

In this paper, we propose a new setting in goal-oriented dialogue system to tighten the gap between these two aspects by enforcing model level information isolation on individual models between two agents.

Language construction now becomes an important part in reward optimization since it is the only way information can be exchanged.

We experimented our models using self-play and results showed that our method not only beat the baseline sequence2sequence model in rewards but can also generate human-readable meaningful conversations of comparable quality.

A Goal-oriented Neural Conversation Model by Self-Play

A self-play model for goal oriented dialog generation, aiming to enforce a stronger coupling between the task reward and the language model.

This paper describes a method for improving a goal oriented dialogue system using selfplay.

goal-oriented neural conversation model self-play


Search engine users nowadays heavily depend on query completion and correction to shape their queries.

Typically, the completion is done by database lookup which does not understand the context and cannot generalize to prefixes not in the database

. In the paper, we propose to use unsupervised deep language models to complete and correct the queries given an arbitrary prefix

.  We show how to address two main challenges that renders this method practical for large-scale deployment

: 1) we propose a method for integrating error correction into the language model completion via a edit-distance potential and a variant of beam search that can exploit these potential functions; and

2) we show how to efficiently perform CPU-based computation to complete the queries, with error correction, in real time (generating top 10 completions within 16 ms).

Experiments show that the method substantially increases hit rate over standard approaches, and is capable of handling tail queries.

realtime search query completion using character-level LSTM language models

This paper presents methods for query completion that includes prefix correction, and some engineering details to meet particular latency requirements on a CPU.

The authors propose an algorithm for solving the query completion problem with error correction, and adopt character-level RNN-based modeling and optimize the inference part to achieve targets in real time.

Realtime query completion via deep language models

query completion
correction
language model
realtime
character-level


RMSProp and ADAM continue to be extremely popular algorithms for training neural nets but their theoretical convergence properties have remained unclear.

Further, recent work has seemed to suggest that these algorithms have worse generalization properties when compared to carefully tuned stochastic gradient descent or its momentum variants.

In this work, we make progress towards a deeper understanding of ADAM and RMSProp in two ways.

First, we provide proofs that these adaptive gradient algorithms are guaranteed to reach criticality for smooth non-convex objectives, and we give bounds on the running time.

Next we design experiments to empirically study the convergence and generalization properties of RMSProp and ADAM against Nesterov's Accelerated Gradient method on a variety of common autoencoder setups and on VGG-9 with CIFAR-10.

Through these experiments we demonstrate the interesting sensitivity that ADAM has to its momentum parameter \beta_1.

We show that at very high values of the momentum parameter (\beta_1 = 0.99) ADAM outperforms a carefully tuned NAG on most of our experiments, in terms of getting lower training and test losses.

On the other hand, NAG can sometimes do better when ADAM's \beta_1 is set to the most commonly used value: \beta_1 = 0.9, indicating the importance of tuning the hyperparameters of ADAM to get better generalization performance.

We also report experiments on different autoencoders to demonstrate that NAG has better abilities in terms of reducing the gradient norms, and it also produces iterates which exhibit an increasing trend for the minimum eigenvalue of the Hessian of the loss function at the iterates.

In this paper we prove convergence to criticality of (stochastic and deterministic) RMSProp and deterministic ADAM for smooth non-convex objectives and we demonstrate an interesting beta_1 sensitivity for ADAM on autoencoders.

This paper presents a convergence analysis of RMSProp and ADAM in the case of smooth non-convex functions

Convergence Guarantees for RMSProp and ADAM in Non-Convex Optimization and an Empirical Comparison to Nesterov Acceleration

rmsprop adam
smooth non-convex
convergence


Recent advances in adversarial Deep Learning (DL) have opened up a new and largely unexplored surface for malicious attacks jeopardizing the integrity of autonomous DL systems.

We introduce a novel automated countermeasure called Parallel Checkpointing Learners (PCL) to thwart the potential adversarial attacks and significantly improve the reliability (safety) of a victim DL model.

The proposed PCL methodology is unsupervised, meaning that no adversarial sample is leveraged to build/train parallel checkpointing learners.

We formalize the goal of preventing adversarial attacks as an optimization problem to minimize the rarely observed regions in the latent feature space spanned by a DL network.

To solve the aforementioned minimization problem, a set of complementary but disjoint checkpointing modules are trained and leveraged to validate the victim model execution in parallel.

Each checkpointing learner explicitly characterizes the geometry of the input data and the corresponding high-level data abstractions within a particular DL layer.

As such, the adversary is required to simultaneously deceive all the defender modules in order to succeed.

We extensively evaluate the performance of the PCL methodology against the state-of-the-art attack scenarios, including Fast-Gradient-Sign (FGS), Jacobian Saliency Map Attack (JSMA), Deepfool, and Carlini&WagnerL2 algorithm.

Extensive proof-of-concept evaluations for analyzing various data collections including MNIST, CIFAR10, and ImageNet corroborate the effectiveness of our proposed defense mechanism against adversarial samples.

Devising unsupervised defense mechanisms against adversarial attacks is crucial to ensure the generalizability of the defense.

This paper presents a method for detecting adversarial examples in a deep learning classification setting

This paper presents an unsupervised method for detecting adversarial examples of neural networks.

Towards Safe Deep Learning: Unsupervised Defense Against Generic Adversarial Attacks

method detect adversarial example
present
defense
attack
unsupervised
deep


Neural architecture search (NAS) has a great impact by automatically designing effective neural network architectures.

However, the prohibitive computational demand of conventional NAS algorithms (e.g. 10 4 GPU hours) makes it difficult to directly search the architectures on large-scale tasks (e.g. ImageNet).

Differentiable NAS can reduce the cost of GPU hours via a continuous representation of network architecture but suffers from the high GPU memory consumption issue (grow linearly w.r.t. candidate set size).

As a result, they need to utilize proxy tasks, such as training on a smaller dataset, or learning with only a few blocks, or training just for a few epochs.

These architectures optimized on proxy tasks are not guaranteed to be optimal on the target task.

In this paper, we present ProxylessNAS that can directly learn the architectures for large-scale target tasks and target hardware platforms.

We address the high memory consumption issue of differentiable NAS and reduce the computational cost (GPU hours and GPU memory) to the same level of regular training while still allowing a large candidate set.

Experiments on CIFAR-10 and ImageNet demonstrate the effectiveness of directness and specialization.

On CIFAR-10, our model achieves 2.08% test error with only 5.7M parameters, better than the previous state-of-the-art architecture AmoebaNet-B, while using 6× fewer parameters.

On ImageNet, our model achieves 3.1% better top-1 accuracy than MobileNetV2, while being 1.2× faster with measured GPU latency.

We also apply ProxylessNAS to specialize neural architectures for hardware with direct hardware metrics (e.g. latency) and provide insights for efficient CNN architecture design.

Proxy-less neural architecture search for directly learning architectures on large-scale target task (ImageNet) while reducing the cost to the same level of normal training.

This paper addresses the problem of architecture search, and specifically seeks to do this without having to train on "proxy" tasks where the problem is simplified through more limited optimization, architectural complexity, or dataset size.

ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware


With the recently rapid development in deep learning, deep neural networks have been widely adopted in many real-life applications.

However, deep neural networks are also known to have very little control over its uncertainty for test examples, which potentially causes very harmful and annoying consequences in practical scenarios.

In this paper, we are particularly interested in designing a higher-order uncertainty metric for deep neural networks and investigate its performance on the out-of-distribution detection task proposed by~\cite{hendrycks2016baseline}.

Our method first assumes there exists a underlying higher-order distribution $\mathcal{P}(z)$

, which generated label-wise distribution $\mathcal{P}(y)$

over classes on the K-dimension simplex, and then approximate such higher-order distribution via parameterized posterior function $p_{\theta}(z|x)$ under variational inference framework, finally we use the entropy of learned posterior distribution $p_{\theta}(z|x)$ as uncertainty measure to detect out-of-distribution examples. However

, we identify the overwhelming over-concentration issue in such a framework, which greatly hinders the detection performance. Therefore

, we further design a log-smoothing function to alleviate such issue to greatly increase the robustness of the proposed entropy-based uncertainty measure. Through

comprehensive experiments on various datasets and architectures, our proposed variational Dirichlet framework with entropy-based uncertainty measure is consistently observed to yield significant improvements over many baseline systems.

A new framework based variational inference for out-of-distribution detection

Describes a probabilistic approach to quantifying uncertainty in DNN classification tasks that outperforms other SOTA methods in the task of out-of-distribution detection.

A new framework for out-of-distribution detection, based on variaitonal inference and a prior Dirichlet distribution, that reports state of the art results on several datasets.

An out-of distribution detection via a new method to approximate the confidence distribution of classification probability using variational inference of Dirichlet distribution.

A Variational Dirichlet Framework for Out-of-Distribution Detection

out-of-distribution detection
dirichlet distribution
variational inference
new framework
method
base
classification


Intelligent agents can learn to represent the action spaces of other agents simply by observing them act.

Such representations help agents quickly learn to predict the effects of their own actions on the environment and to plan complex action sequences.

In this work, we address the problem of learning an agent’s action space purely from visual observation.

We use stochastic video prediction to learn a latent variable that captures the scene's dynamics while being minimally sensitive to the scene's static content.

We introduce a loss term that encourages the network to capture the composability of visual sequences and show that it leads to representations that disentangle the structure of actions.

We call the full model with composable action representations Composable Learned Action Space Predictor (CLASP).

We show the applicability of our method to synthetic settings and its potential to capture action spaces in complex, realistic visual settings.

When used in a semi-supervised setting, our learned representations perform comparably to existing fully supervised methods on tasks such as action-conditioned video prediction and planning in the learned action space, while requiring orders of magnitude fewer action labels.

We learn a representation of an agent's action space from pure visual observations. We use a recurrent latent variable approach with a novel composability loss.

Proposes a compositional latent-variable model to learn models that predict what will happen next in scenarios where action-labels are not available in abundance.

A variational IB based approach to learn action representations directly from videos of actions being taken, achieving better efficiency of subsequent learning methods while requiring lesser amount of action label videos.

This paper proposes an approach to video prediction which autonomously finds an action space encoding differences between subsequent frames

Learning what you can do before doing anything

learn
approach
action space
subsequent
video
representation


When autonomous agents interact in the same environment, they must often cooperate to achieve their goals.

One way for agents to cooperate effectively is to form a team, make a binding agreement on a joint plan, and execute it.

However, when agents are self-interested, the gains from team formation must be allocated appropriately to incentivize agreement.

Various approaches for multi-agent negotiation have been proposed, but typically only work for particular negotiation protocols.

More general methods usually require human input or domain-specific data, and so do not scale.

To address this, we propose a framework for training agents to negotiate and form teams using deep reinforcement learning.

Importantly, our method makes no assumptions about the specific negotiation protocol, and is instead completely experience driven.

We evaluate our approach on both non-spatial and spatially extended team-formation negotiation environments, demonstrating that our agents beat hand-crafted bots and reach negotiation outcomes consistent with fair solutions predicted by cooperative game theory.

Additionally, we investigate how the physical location of agents influences negotiation outcomes.

Reinforcement learning can be used to train agents to negotiate team formation across many negotiation protocols

This paper studies deep multi-agent RL in settings where all of the agents must cooperate to accomplish a task (e.g., search and rescue, multi-player video games), and uses simple cooperative weighted voting games to study the efficacy of deep RL and to compare solutions found by deep RL to a fair solution.

A reinforcement learning approach for negotiating coalitions in cooperative game theory settings that can be used in cases where unlimited training simulations are available.

Negotiating Team Formation Using Deep Reinforcement Learning

setting
negotiate team formation
reinforcement learning
agent
deep
cooperative
game


Neural machine translation (NMT) models learn representations containing substantial linguistic information.

However, it is not clear if such information is fully distributed or if some of it can be attributed to individual neurons.

We develop unsupervised methods for discovering important neurons in NMT models.

Our methods rely on the intuition that different models learn similar properties, and do not require any costly external supervision.

We show experimentally that translation quality depends on the discovered neurons, and find that many of them capture common linguistic phenomena.

Finally, we show how to control NMT translations in predictable ways, by modifying activations of individual neurons.

Unsupervised methods for finding, analyzing, and controlling important neurons in NMT

This paper presents unsupervised approaches to discovering important neurons in neural machine translation systems and analyzes linguistic properties controlled by those neurons.

Unsupervised methods for ranking neurons in machine translation where important neurons are thus identified and used to control the MT output.

Identifying and Controlling Important Neurons in Neural Machine Translation

important neuron neural machine translation


Recent state-of-the-art reinforcement learning algorithms are trained under the goal of excelling in one specific task.

Hence, both environment and task specific knowledge are entangled into one framework.

However, there are often scenarios where the environment (e.g. the physical world) is fixed while only the target task changes.

Hence, borrowing the idea from hierarchical reinforcement learning, we propose a framework that disentangles task and environment specific knowledge by separating them into two units.

The environment-specific unit handles how to move from one state to the target state; and the task-specific unit plans for the next target state given a specific task.

The extensive results in simulators indicate that our method can efficiently separate and learn two independent units, and also adapt to a new task more efficiently than the state-of-the-art methods.

We propose a DRL framework that disentangles task and environment specific knowledge.

The authors propose to decompose reinforcement learning into a PATH function and a GOAL function

A modular architecture with the aim of separating environment specific knowledge and task-specific knowledge into different modules, on par with standard A3C across a wide range of tasks.

Universal Agent for Disentangling Environments and Tasks

environment specific knowledge
disentangle
task


Modelling 3D scenes from 2D images is a long-standing problem in computer vision with implications in, e.g., simulation and robotics.

We propose pix2scene, a deep generative-based approach that implicitly models the geometric properties of a scene from images.

Our method learns the depth and orientation of scene points visible in images.

Our model can then predict the structure of a scene from various, previously unseen view points.

It relies on a bi-directional adversarial learning mechanism to generate scene representations from a latent code, inferring the 3D representation of the underlying scene geometry.

We showcase a novel differentiable renderer to train the 3D model in an end-to-end fashion, using only images.

We demonstrate the generative ability of our model qualitatively on both a custom dataset and on ShapeNet.

Finally, we evaluate the effectiveness of the learned 3D scene representation in supporting a 3D spatial reasoning.

pix2scene: a deep generative based approach for implicitly modelling the geometrical properties of a 3D scene from images

Explores explaining scenes with surfels in a neural recognition model, and demonstrate results on image reconstruction, synthesis, and mental shape rotation.

Authors introduce a method to create a 3D scene model given a 2D image and a camera pose using a self-superfised model

Pix2Scene: Learning Implicit 3D Representations from Images


Identifying the relations that connect words is an important step towards understanding human languages and is useful for various NLP tasks such as knowledge base completion and analogical reasoning.

Simple unsupervised operators such as vector offset between two-word embeddings have shown to recover some specific relationships between those words, if any.

Despite this, how to accurately learn generic relation representations from word representations remains unclear.

We propose a method for learning relation representations using a feed-forward neural network that performs relation prediction.

Our evaluations on two benchmark datasets reveal that the penultimate layer of the trained neural network-based relational predictor acts as a good representation for the relations between words.

Identifying the relations that connect words is important for various NLP tasks. We model relation representation as a supervised learning problem and learn parametrised operators that map pre-trained word embeddings to relation representations.

This paper presents a novel method for representing lexical relations as vectors using just pre-trained word embeddings and a novel loss function operating over pairs of word pairs.

A novel solution to the relation compositon problem when you already have pre trained word/entity embeddings and are interested only in learning to compose relation representations.

Learning Relation Representations from Word Representations

pre-trained word embedding
relation representation
learn
problem
novel


Recurrent neural networks (RNNs) are important class of architectures among neural networks useful for language modeling and sequential prediction.

However, optimizing RNNs is known to be harder compared to feed-forward neural networks.

A number of techniques have been proposed in literature to address this problem.

In this paper we propose a simple technique called fraternal dropout that takes advantage of dropout to achieve this goal.

Specifically, we propose to train two identical copies of an RNN (that share parameters) with different dropout masks while minimizing the difference between their (pre-softmax) predictions.

In this way our regularization encourages the representations of RNNs to be invariant to dropout mask, thus being robust.

We show that our regularization term is upper bounded by the expectation-linear dropout objective which has been shown to address the gap due to the difference between the train and inference phases of dropout.

We evaluate our model and achieve state-of-the-art results in sequence modeling tasks on two benchmark datasets - Penn Treebank and Wikitext-2.

We also show that our approach leads to performance improvement by a significant margin in image captioning (Microsoft COCO) and semi-supervised (CIFAR-10) tasks.

We propose to train two identical copies of an recurrent neural network (that share parameters) with different dropout masks while minimizing the difference between their (pre-softmax) predictions.

Presents Fraternal dropout as an improvement over Expectation-linear dropout in terms of convergence, and demonstrates the utility of Fraternal dropout on a number of tasks and datasets.

Fraternal Dropout


We propose a novel approach for deformation-aware neural networks that learn the weighting and synthesis of dense volumetric deformation fields.

Our method specifically targets the space-time representation of physical surfaces from liquid simulations.

Liquids exhibit highly complex, non-linear behavior under changing simulation conditions such as different initial conditions.

Our algorithm captures these complex phenomena in two stages: a first neural network computes a weighting function for a set of pre-computed deformations, while a second network directly generates a deformation field for refining the surface.

Key for successful training runs in this setting is a suitable loss function that encodes the effect of the deformations, and a robust calculation of the corresponding gradients.

To demonstrate the effectiveness of our approach, we showcase our method with several complex examples of flowing liquids with topology changes.

Our representation makes it possible to rapidly generate the desired implicit surfaces.

We have implemented a mobile application to demonstrate that real-time interactions with complex liquid effects are possible with our approach.

Learning weighting and deformations of space-time data sets for highly efficient approximations of liquid behavior.

A neural-network based model is used to interpolate simulations for novel scene conditions from densely registered 4D implicit surfaces for a structured scene.

This paper presents a coupled deep learning approach for generating realistic liquid simulation data that can be useful for real-time decision support applications.

This paper introduces a deep learning approach for physical simulation that combines two networks for synthesizing 4D data that represents 3D physical simulations

Generating Liquid Simulations with Deformation-aware Neural Networks

deep learning approach
simulation
datum
liquid
4d
generate
network


This is an empirical paper which constructs color invariant networks and evaluates their performances on a realistic data set.

The paper studies the simplest possible case of color invariance: invariance under pixel-wise permutation of the color channels.

Thus the network is aware not of the specific color object, but its colorfulness.

The data set introduced in the paper consists of images showing crashed cars from which ten classes were extracted.

An additional annotation was done which labeled whether the car shown was red or non-red.

The networks were evaluated by their performance on the classification task.

With the color annotation we altered the color ratios  in the training data and analyzed the generalization capabilities of the networks on the unaltered test data.

We further split the test data in red and non-red cars and did a similar evaluation.

It is shown in the paper that an pixel-wise ordering of the rgb-values of the images performs better or at least similarly for small deviations from the true color ratios.

The limits of these networks are also discussed.

We construct and evaluate color invariant neural nets on a novel realistic data set

Proposes a method to make neural networks for image recognition color invariant and evaluates it on the cifar 10 dataset.

The authors investigate a modified input layer that results in color invariant networks, and show that certain color invariant input layers can improve accuracy for test-images from a different color distribution than the training images.

The authors test a CNN on images with color channels modified to be invariant to permutations, with performance not degraded by too much.

On the Construction and Evaluation of Color Invariant Networks

color invariant network
evaluate
image
author
modify
neural


Expressive efficiency refers to the relation between two architectures A and B, whereby any function realized by B could be replicated by A, but there exists functions realized by A, which cannot be replicated by B unless its size grows significantly larger.

For example, it is known that deep networks are exponentially efficient with respect to shallow networks, in the sense that a shallow network must grow exponentially large in order to approximate the functions represented by a deep network of polynomial size.

In this work, we extend the study of expressive efficiency to the attribute of network connectivity and in particular to the effect of "overlaps" in the convolutional process, i.e., when the stride of the convolution is smaller than its filter size (receptive field).

To theoretically analyze this aspect of network's design, we focus on a well-established surrogate for ConvNets called Convolutional Arithmetic Circuits (ConvACs), and then demonstrate empirically that our results hold for standard ConvNets as well.

Specifically, our analysis shows that having overlapping local receptive fields, and more broadly denser connectivity, results in an exponential increase in the expressive capacity of neural networks.

Moreover, while denser connectivity can increase the expressive capacity, we show that the most common types of modern architectures already exhibit exponential increase in expressivity, without relying on fully-connected layers.

We analyze how the degree of overlaps between the receptive fields of a convolutional network affects its expressive power.

The paper studies the expressive power provided by "overlap" in convolution layers of DNNs by considering linear activations with product pooling.

This paper analyzes the expressivity of convolutional arithmetic circuits and shows that an exponentialy large number of non-overlapping ConvACs are required to approximate the grid tensor of an overlapping ConvACs.

On the Expressive Power of Overlapping Architectures of Deep Learning

expressive power
analyze
overlap
convolutional


We provide a theoretical algorithm for checking local optimality and escaping saddles at nondifferentiable points of empirical risks of two-layer ReLU networks.

Our algorithm receives any parameter value and returns: local minimum, second-order stationary point, or a strict descent direction.

The presence of M data points on the nondifferentiability of the ReLU divides the parameter space into at most 2^M regions, which makes analysis difficult.

By exploiting polyhedral geometry, we reduce the total computation down to one convex quadratic program (QP) for each hidden node, O(M) (in)equality tests, and one (or a few) nonconvex QP.

For the last QP, we show that our specific problem can be solved efficiently, in spite of nonconvexity.

In the benign case, we solve one equality constrained QP, and we prove that projected gradient descent solves it exponentially fast.

In the bad case, we have to solve a few more inequality constrained QPs, but we prove that the time complexity is exponential only in the number of inequality constraints.

Our experiments show that either benign case or bad case with very few inequality constraints occurs, implying that our algorithm is efficient in most cases.

A theoretical algorithm for testing local optimality and extracting descent directions at nondifferentiable points of empirical risks of one-hidden-layer ReLU networks.

Proposes an algorithm to check whether a given point is a generalized second-order stationary point.

A theoretical algorithm, involving solving convex and non-convex quadratic programs, for checking local optimality and escaping saddles when training two-layer ReLU networks.

Author proposes a method to check if a point is a stationary point or not and then classify stationary points as either local min or second-order stationary

Efficiently testing local optimality and escaping saddles for ReLU networks

local optimality escape saddle
theoretical algorithm
relu network
point
check
second-order stationary
test


We present a new technique for learning visual-semantic embeddings for cross-modal retrieval.

Inspired by the use of hard negatives in structured prediction, and ranking loss functions used in retrieval, we introduce a simple change to common loss functions used to learn multi-modal embeddings.

That, combined with fine-tuning and the use of augmented data, yields significant gains in retrieval performance.

We showcase our approach, dubbed VSE++, on the MS-COCO and Flickr30K datasets, using ablation studies and comparisons with existing methods.

On MS-COCO our approach outperforms state-of-the-art methods by 8.8% in caption retrieval, and 11.3% in image retrieval (based on R@1).

A new loss based on relatively hard negatives that achieves state-of-the-art performance in image-caption retrieval.

Learning joint embedding of sentences and images using triplet loss that is applied to hardest negatives instead of averaging over all triplets

VSE++: Improving Visual-Semantic Embeddings with Hard Negatives


We present DANTE, a novel method for training neural networks, in particular autoencoders, using the alternating minimization principle.

DANTE provides a distinct perspective in lieu of traditional gradient-based backpropagation techniques commonly used to train deep networks.

It utilizes an adaptation of quasi-convex optimization techniques to cast autoencoder training as a bi-quasi-convex optimization problem.

We show that for autoencoder configurations with both differentiable (e.g. sigmoid) and non-differentiable (e.g. ReLU) activation functions, we can perform the alternations very effectively.

DANTE effortlessly extends to networks with multiple hidden layers and varying network configurations.

In experiments on standard datasets, autoencoders trained using the proposed method were found to be very promising when compared to those trained using traditional backpropagation techniques, both in terms of training speed, as well as feature extraction and reconstruction performance.

We utilize the alternating minimization principle to provide an effective novel technique to train deep autoencoders.

Alternating minimization framework for training autoencoder and encoder-decoder networks

The authors explore an alternating optimization approach for training Auto Encoders, treating each layer as a generalized linear model, and suggest using the stochastic normalized GD as the minimization algorithm in each phase.

Training Autoencoders by Alternating Minimization

alternate minimization
training autoencoder
train


We develop new algorithms for estimating heterogeneous treatment effects, combining recent developments in transfer learning for neural networks with insights from the causal inference literature.

By taking advantage of transfer learning, we are able to efficiently use different data sources that are related to the same underlying causal mechanisms.

We compare our algorithms with those in the extant literature using extensive simulation studies based on large-scale voter persuasion experiments and the MNIST database.

Our methods can perform an order of magnitude better than existing benchmarks while using a fraction of the data.

Transfer learning for estimating causal effects using neural networks.

Develops algorithms to estimate conditional average treatment effect by auxiliary dataset in different environments, both with and without base learner.

The authors propose methods to address a novel task of transfer learning for estimating the CATE function, and evaluate them using a synthetic setting and a real-world experimental dataset.

Using neural network regression and comparing transfer learning frameworks to estimate a conditional average treatment effect under string ignorability assumptions

Transfer Learning for Estimating Causal Effects Using Neural Networks

transfer learn estimate causal effect neural network


Neuronal assemblies, loosely defined as subsets of neurons with reoccurring spatio-temporally coordinated activation patterns, or "motifs", are thought to be building blocks of neural representations and information processing.

We here propose LeMoNADe, a new exploratory data analysis method that facilitates hunting for motifs in calcium imaging videos, the dominant microscopic functional imaging modality in neurophysiology.

Our nonparametric method extracts motifs directly from videos, bypassing the difficult intermediate step of spike extraction.

Our technique augments variational autoencoders with a discrete stochastic node, and we show in detail how a differentiable reparametrization and relaxation can be used.

An evaluation on simulated data, with available ground truth, reveals excellent quantitative performance.

In real video data acquired from brain slices, with no ground truth available, LeMoNADe uncovers nontrivial candidate motifs that can help generate hypotheses for more focused biological investigations.

We present LeMoNADe, an end-to-end learned motif detection method directly operating on calcium imaging videos.

This paper proposes a VAE-style model for identifying motifs from calcium imaging videos, relying on Bernouli variables and requires Gumbel-softmax trick for inference.

LeMoNADe: Learned Motif and Neuronal Assembly Detection in calcium imaging videos

calcium imaging video
learn motif
lemonade
detection


A noisy and diverse demonstration set may hinder the performances of an agent aiming to acquire certain skills via imitation learning.

However, state-of-the-art imitation learning algorithms often assume the optimality of the given demonstration set.

In this paper, we address such optimal assumption by learning only from the most suitable demonstrations in a given set.

Suitability of a demonstration is estimated by whether imitating it produce desirable outcomes for achieving the goals of the tasks.

For more efficient demonstration suitability assessments, the learning agent should be capable of imitating a demonstration as quick as possible, which shares similar spirit with fast adaptation in the meta-learning regime.

Our framework, thus built on top of Model-Agnostic Meta-Learning, evaluates how desirable the imitated outcomes are, after adaptation to each demonstration in the set.

The resulting assessments hence enable us to select suitable demonstration subsets for acquiring better imitated skills.

The videos related to our experiments are available at: https://sites.google.com/view/deepdj

We propose a framework to learn a good policy through imitation learning from a noisy demonstration set via meta-training a demonstration suitability assessor.

Contributes a MAML based algorithm to imitation learning which automatically determines if provided demonstrations are "suitable".

A method for doing imitation learning from a set of demonstrations that includes useless behavior, which selects the useful demonstrations by their provided performance gains at the meta-training time.

Learning from Noisy Demonstration Sets via Meta-Learned Suitability Assessor

noisy demonstration set via
imitation learn
suitability assessor
provide


We introduce causal implicit generative models (CiGMs): models that allow sampling from not only the true observational but also the true interventional distributions.

We show that adversarial training can be used to learn a CiGM, if the generator architecture is structured based on a given causal graph.

We consider the application of conditional and interventional sampling of face images with binary feature labels, such as mustache, young.

We preserve the dependency structure between the labels with a given causal graph.

We devise a two-stage procedure for learning a CiGM over the labels and the image.

First we train a CiGM over the binary labels using a  Wasserstein GAN where the generator neural network is consistent with the causal graph between the labels.

Later, we combine this with a conditional GAN to generate images conditioned on the binary labels.

We propose two new conditional GAN architectures: CausalGAN and CausalBEGAN.

We show that the optimal generator of the CausalGAN, given the labels, samples from the image distributions conditioned on these labels.

The conditional GAN combined with a trained CiGM for the labels is then a CiGM over the labels and the generated image.

We show that the proposed architectures can be used to sample from observational and interventional image distributions, even for interventions which do not naturally occur in the dataset.

We introduce causal implicit generative models, which can sample from conditional and interventional distributions and also propose two new conditional GANs which we use for training them.

A method of combining a casual graph, describing the dependency structure of labels with two conditional GAN architechtures that generate images conditioning on the binary label

The authors address the issue of learning a causal model between image variables and the image itself from observational data, when given a causal structure between image labels.

CausalGAN: Learning Causal Implicit Generative Models with Adversarial Training

causal implicit generative model
conditional gan
two
label
structure
learn
image


Self-normalizing discriminative models approximate the normalized probability of a class without having to compute the partition function.

This property is useful to computationally-intensive neural network classifiers, as the cost of computing the partition function grows linearly with the number of classes and may become prohibitive.

In particular, since neural language models may deal with up to millions of classes, their self-normalization properties received notable attention.

recent studies empirically found that language models, trained using Noise Contrastive Estimation (NCE), exhibit self-normalization, but could not explain why.

In this study, we provide a theoretical justification to this property by viewing

Our empirical investigation compares NCE to the alternative explicit approach for self-normalizing language models.

It also uncovers a surprising negative correlation between self-normalization and

perplexity, as well as some regularity in the observed errors that may potentially be used for improving self-normalization algorithms in the future.

We prove that NCE is self-normalized and demonstrate it on datasets

Presents a proof of the self normalization of NCE as a result of being a low-rank matrix approximation of low-rank approximation of the normalized conditional probabilities matrix.

This paper considers the problem of self-normalizing models and explains the self-normalizing mechanism by interpreting NCE in terms of matrix factorization.

A Matrix Approximation View of NCE that Justifies Self-Normalization


Learning word representations from large available corpora relies on the distributional hypothesis that words present in similar contexts tend to have similar meanings.

Recent work has shown that word representations learnt in this manner lack sentiment information which, fortunately, can be leveraged using external knowledge.

Our work addresses the question: can affect lexica improve the word representations learnt from a corpus?

In this work, we propose techniques to incorporate affect lexica, which capture fine-grained information about a word's psycholinguistic and emotional orientation, into the training process of Word2Vec SkipGram, Word2Vec CBOW and GloVe methods using a joint learning approach.

We use affect scores from Warriner's affect lexicon to regularize the vector representations learnt from an unlabelled corpus.

Our proposed method outperforms previously proposed methods on standard tasks for word similarity detection, outlier detection and sentiment detection.

We also demonstrate the usefulness of our approach for a new task related to the prediction of formality, frustration and politeness in corporate communication.

Enriching word embeddings with affect information improves their performance on sentiment prediction tasks.

Proposes to use affect lexica to improve word embeddings to outperform the standard Word2vec and Glove.

This paper proposes integrating information from a semantic resource quantifying the affect of words into a text-based word embedding algorithm to make language models more reflective of semantic and pragmatic phenomena.

This paper introduces modifications the word2vec and GloVe loss functions to incorporate affect lexica to facilitate the learning of affect-sensitive word embeddings.

Towards Building Affect sensitive Word Distributions

word2vec glove
word embedding
affect lexica
improve
information


Different kinds of representation learning techniques on graph have shown significant effect in downstream machine learning tasks.

Recently, in order to inductively learn representations for graph structures that is unobservable during training, a general framework with sampling and aggregating (GraphSAGE) was proposed by Hamilton and Ying and had been proved more efficient than transductive methods on fileds like transfer learning or evolving dataset.

However, GraphSAGE is uncapable of selective neighbor sampling and lack of memory of known nodes that've been trained.

To address these problems, we present an unsupervised method that samples neighborhood information attended by co-occurring structures and optimizes a trainable global bias as a representation expectation for each node in the given graph.

Experiments show that our approach outperforms the state-of-the-art inductive and unsupervised methods for representation learning on graphs.

For unsupervised and inductive network embedding, we propose a novel approach to explore most relevant neighbors and preserve previously learnt knowledge of nodes by utilizing bi-attention architecture and introducing global bias, respectively

This proposes an extension to GraphSAGE using a global embedding bias matrix in the local aggregating functions and a method to sample interesting nodes.

BIGSAGE: unsupervised inductive representation learning of graph via bi-attended sampling and global-biased aggregating

node embed inductive bias global aggregating


Learning distributed representations for nodes in graphs is a crucial primitive in network analysis with a wide spectrum of applications.

Linear graph embedding methods learn such representations by optimizing the likelihood of both positive and negative edges while constraining the dimension of the embedding vectors.

We argue that the generalization performance of these methods is not due to the dimensionality constraint as commonly believed, but rather the small norm of embedding vectors.

Both theoretical and empirical evidence are provided to support this argument:

(a) we prove that the generalization error of these methods can be bounded by limiting the norm of vectors, regardless of the embedding dimension;

(b) we show that the generalization performance of linear graph embedding methods is correlated with the norm of embedding vectors, which is small due to the early stopping of SGD and the vanishing gradients.

We performed extensive experiments to validate our analysis and showcased the importance of proper norm regularization in practice.

We argue that the generalization of linear graph embedding is not due to the dimensionality constraint but rather the small norm of embedding vectors.

The authors show that the generalization error of linear graph embedding methods is bounded by the norm of embedding vectors rather than dimensionality constraints

The authors propose a theoretical bound on the generalization performance of learning graph embeddings and argue that the norm of the coordinates determines the success of the learnt representation.

The Importance of Norm Regularization in Linear Graph Embedding: Theoretical Analysis and Empirical Demonstration

norm embed vector
argue
generalization
linear graph
dimensionality constraint
rather
author
theoretical


Momentum-based acceleration of stochastic gradient descent (SGD) is widely used in deep learning.

We propose the quasi-hyperbolic momentum algorithm (QHM) as an extremely simple alteration of momentum SGD, averaging a plain SGD step with a momentum step.

We describe numerous connections to and identities with other algorithms, and we characterize the set of two-state optimization algorithms that QHM can recover.

Finally, we propose a QH variant of Adam called QHAdam, and we empirically demonstrate that our algorithms lead to significantly improved training in a variety of settings, including a new state-of-the-art result on WMT16 EN-DE.

We hope that these empirical results, combined with the conceptual and practical simplicity of QHM and QHAdam, will spur interest from both practitioners and researchers.

Mix plain SGD and momentum (or do something similar with Adam) for great profit.

The paper proposes simple modifications to SGD and Adam, called QH-variants, that can recover the “parent” method and a host of other optimization tricks.

A variant of classical momentum which takes a weighted average of momentum and gradient update, and an evaluation of its relationships between other momentum based optimization schemes.

Quasi-hyperbolic momentum and Adam for deep learning


Reinforcement Learning (RL) can model complex behavior policies for goal-directed sequential decision making tasks.

A hallmark of RL algorithms is Temporal Difference (TD) learning: value function for the current state is moved towards a bootstrapped target that is estimated using the next state's value function.

lambda-returns define the target of the RL agent as a weighted combination of rewards estimated by using multiple many-step look-aheads.

Although mathematically tractable, the use of  exponentially decaying weighting of n-step returns based targets in lambda-returns is a rather ad-hoc design choice.

Our major contribution  is that we propose a generalization of lambda-returns called Confidence-based Autodidactic Returns (CAR), wherein the RL agent learns the weighting of the n-step returns in an end-to-end manner.

In contrast to lambda-returns wherein the RL agent is restricted to use an exponentially decaying weighting scheme, CAR allows the agent to learn to decide how much it wants to weigh the n-step returns based targets.

Our experiments, in addition to showing the efficacy of CAR, also empirically demonstrate that using sophisticated weighted mixtures of multi-step returns (like CAR and lambda-returns) considerably outperforms the use of n-step returns.

We perform our experiments on the  Asynchronous Advantage Actor Critic (A3C) algorithm in the Atari 2600 domain.

A novel way to generalize lambda-returns by allowing the RL agent to decide how much it wants to weigh each of the n-step returns.

Extends the A3C algorithm with lambda returns, and proposes an approach for learning the weights of the returns.

The authors present confidence-based autodidactic returns, a Deep learning RL method to adjust the weights of an eligibility vector in TD(lambda)-like value estimation to favour more stable estimates of the state.

Learning to Mix n-Step Returns: Generalizing Lambda-Returns for Deep Reinforcement Learning

generalize lambda-return
weight
n-step return
learn
deep
rl
learning


Current end-to-end deep learning driving models have two problems: (1) Poor

generalization ability of unobserved driving environment when diversity of train-

ing driving dataset is limited (2) Lack of accident explanation ability when driving

To tackle these two problems, rooted on the be-

lieve that knowledge of associated easy task is benificial for addressing difficult

task, we proposed a new driving model which is composed of perception module

for see and think and driving module for behave, and trained it with multi-task

perception-related basic knowledge and driving knowledge stepwisely.

cally segmentation map and depth map (pixel level understanding of images) were

considered as what & where and how far knowledge for tackling easier driving-

related perception problems before generating final control commands for difficult

The results of experiments demonstrated the effectiveness of multi-

task perception knowledge for better generalization and accident explanation abil-

With our method the average sucess rate of finishing most difficult navigation

tasks in untrained city of CoRL test surpassed current benchmark method for 15

percent in trained weather and 20 percent in untrained weathers.

we proposed a new self-driving model which is composed of perception module for see and think and driving module for behave to acquire better generalization  and accident explanation ability.

Presents a multitask learning architecture for depth and segmentation map estimation and the driving prediction using a perception module and a driving decision module.

A method for a modified end-to-end architecture that has better generalization and explanation ability, is more robust to a different testing setting, and has decoder output that can help with debugging the model.

The authors present a multi-task convolutional neural network for end-to-end driving and provide evaluations with the CARLA open source simulator showing better generalization performance in new driving conditions than baselines

RETHINKING SELF-DRIVING : MULTI -TASK KNOWLEDGE FOR BETTER GENERALIZATION AND ACCIDENT EXPLANATION ABILITY

perception module
accident explanation ability
well generalization
model
drive
new
architecture
present
self-driving
end-to-end


Recently there has been a surge of interest in designing graph embedding methods.

Few, if any, can scale to a large-sized graph with millions of nodes due to both computational complexity and memory requirements.

In this paper, we relax this limitation by introducing the MultI-Level Embedding (MILE) framework – a generic methodology allowing contemporary graph embedding methods to scale to large graphs.

MILE repeatedly coarsens the graph into smaller ones using a hybrid matching technique to maintain the backbone structure of the graph.

It then applies existing embedding methods on the coarsest graph and refines the embeddings to the original graph through a novel graph convolution neural network that it learns.

The proposed MILE framework is agnostic to the underlying graph embedding techniques and can be applied to many existing graph embedding methods without modifying them.

We employ our framework on several popular graph embedding techniques and conduct embedding for real-world graphs.

Experimental results on five large-scale datasets demonstrate that MILE significantly boosts the speed (order of magnitude) of graph embedding while also often generating embeddings of better quality for the task of node classification.

MILE can comfortably scale to a graph with 9 million nodes and 40 million edges, on which existing methods run out of memory or take too long to compute on a modern workstation.

A generic framework to scale existing graph embedding techniques to large graphs.

This paper proposes a multi-level embedding framework to be applied on top of existing network embedding methods in order to scale to large scale networks with faster speed.

The authors propose a three-stage framework for large-scale graph embedding with improved embedding quality.

MILE: A Multi-Level Framework for Scalable Graph Embedding

multi-level
framework
scale
large
graph embed
exist


Anomaly detection discovers regular patterns in unlabeled data and identifies the non-conforming data points, which in some cases are the result of malicious attacks by adversaries.

Learners such as One-Class Support Vector Machines (OCSVMs) have been successfully in anomaly detection, yet their performance may degrade significantly in the presence of sophisticated adversaries, who target the algorithm itself by compromising the integrity of the training data.

With the rise in the use of machine learning in mission critical day-to-day activities where errors may have significant consequences, it is imperative that machine learning systems are made secure.

To address this, we propose a defense mechanism that is based on a contraction of the data, and we test its effectiveness using OCSVMs.

The proposed approach introduces a layer of uncertainty on top of the OCSVM learner, making it infeasible for the adversary to guess the specific configuration of the learner.

We theoretically analyze the effects of adversarial perturbations on the separating margin of OCSVMs and provide empirical evidence on several benchmark datasets, which show that by carefully contracting the data in low dimensional spaces, we can successfully identify adversarial samples that would not have been identifiable in the original dimensional space.

The numerical results show that the proposed method improves OCSVMs performance significantly (2-7%)

A novel method to increase the resistance of OCSVMs against targeted, integrity attacks by selective nonlinear transformations of data to lower dimensions.

The authors propose a defense against attacks on the security of one-class SVM based anomaly detectors

This paper explores how random projections can be used to make OCSVM robust to adversarially perturbed training data.

Unsupervised Adversarial Anomaly  Detection using One-Class Support Vector Machines


In this paper, we present a layer-wise learning of stochastic neural networks (SNNs) in an information-theoretic perspective.

In each layer of an SNN, the compression and the relevance are defined to quantify the amount of information that the layer contains about the input space and the target space, respectively.

We jointly optimize the compression and the relevance of all parameters in an SNN to better exploit the neural network's representation.

Previously, the Information Bottleneck (IB) framework (\cite{Tishby99}) extracts relevant information for a target variable.

Here, we propose Parametric Information Bottleneck (PIB) for a neural network by utilizing (only) its model parameters explicitly to approximate the compression and the relevance.

We show that, as compared to the maximum likelihood estimate (MLE) principle, PIBs : (i) improve the generalization of neural networks in classification tasks, (ii) push the representation of neural networks closer to the optimal information-theoretical representation in a faster manner.

Learning a better neural networks' representation with Information Bottleneck principle

Proposes a learning method based on the information bottleneck framework, where hidden layers of deep nets compress the input X while maintaining sufficient information to predict the output Y.

This paper presents a new way of training stochastic neural network following an information relevance/compression framework similar to the Information Bottleneck.

Parametric Information Bottleneck to Optimize Stochastic Neural Networks

information bottleneck
stochastic neural network
framework


The maximum mean discrepancy (MMD) between two probability measures P

and Q is a metric that is zero if and only if all moments of the two measures

are equal, making it an appealing statistic for two-sample tests.

from P and Q, Gretton et al. (2012) show that we can construct an unbiased

estimator for the square of the MMD between the two distributions.

distribution of interest and Q is the distribution implied by a generative neural

network with stochastic inputs, we can use this estimator to train our neural network.

However, in practice we do not always have i.i.d. samples from our target

Data sets often exhibit biases—for example, under-representation of

certain demographics—and if we ignore this fact our machine learning algorithms

Alternatively, it may be useful to assume our data has

been gathered via a biased sample selection mechanism in order to manipulate

properties of the estimating distribution Q.

In this paper, we construct an estimator for the MMD between P and Q when we

only have access to P via some biased sample selection mechanism, and suggest

methods for estimating this sample selection mechanism when it is not already

We show that this estimator can be used to train generative neural networks

on a biased data sample, to give a simulator that reverses the effect of that

We propose an estimator for the maximum mean discrepancy, appropriate when a target distribution is only accessible via a biased sample selection procedure, and show that it can be used in a generative network to correct for this bias.

Proposes an importance-weighted estimator of the MMD to estimate the MMD between distributions based on samples biased according to a known or estimated unknown scheme.

The authors address the problem of sample selection bias in MMD-GANs and propose an estimate of the MMD between two distributions using weighted maximum mean discrepancy.

This paper presents a modification of the objective used to train generative networks with an MMD adversary

Directing Generative Networks with Weighted Maximum Mean Discrepancy

estimate mmd
maximum mean discrepancy
generative network
estimator
distribution
sample selection
bias


We propose Bayesian Deep Q-Network  (BDQN), a  practical Thompson sampling based Reinforcement Learning (RL) Algorithm.

Thompson sampling allows for targeted exploration in high dimensions through posterior sampling but is usually computationally expensive.

We address this limitation by introducing uncertainty only at the output layer of the network through a Bayesian Linear Regression (BLR) model, which can be trained with fast closed-form updates and its samples can be drawn efficiently through the Gaussian distribution.

We apply our method to a wide range of Atari Arcade Learning Environments.

Since BDQN carries out more efficient exploration, it is able to reach higher rewards substantially faster than a key baseline, DDQN.

Using Bayesian regression to estimate the posterior over Q-functions and deploy Thompson Sampling as a targeted exploration strategy with efficient trade-off the exploration and exploitation

The authors propose a new algorithm for exploration in Deep RL where they apply Bayesian linear regression with features from the last layer of a DQN network to estimate the Q function for each action.

The authors describe how to use Bayesian neural networks with Thompson sampling for efficient exploration in q-learning and propose an approach that outperforms epsilon-greedy exploration approaches.

Efficient Exploration through Bayesian   Deep Q-Networks

exploration estimate regression bayesian efficient author network thompson deep


In this work, we propose the polynomial convolutional neural network (PolyCNN), as a new design of a weight-learning efficient variant of the traditional CNN.

The biggest advantage of the PolyCNN is that at each convolutional layer, only one convolutional filter is needed for learning the weights, which we call the seed filter, and all the other convolutional filters are the polynomial transformations of the seed filter, which is termed as an early fan-out.

Alternatively, we can also perform late fan-out on the seed filter response to create the number of response maps needed to be input into the next layer.

Both early and late fan-out allow the PolyCNN to learn only one convolutional filter at each layer, which can dramatically reduce the model complexity by saving 10x to 50x parameters during learning.

While being efficient during both training and testing, the PolyCNN does not suffer performance due to the non-linear polynomial expansion which translates to richer representational power within the convolutional layers.

By allowing direct control over model complexity, PolyCNN provides a flexible trade-off between performance and efficiency.

We have verified the on-par performance between the proposed PolyCNN and the standard CNN on several visual datasets, such as MNIST, CIFAR-10, SVHN, and ImageNet.

PolyCNN only needs to learn one seed convolutional filter at each layer. This is an efficient variant of traditional CNN, with on-par performance.

Attempts at reducing the number of CNN model parameters by using the polynomial transformation of filters to create blow-up the filter responses.

The authors propose a weight sharing architecture for reducing the number of convolutional neural network parameters with seed filters

PolyCNN: Learning Seed Convolutional Filters

reduce number
seed convolutional filter
polycnn
learn
cnn
parameter


Detecting the emergence of abrupt property changes in time series is a challenging problem.

Kernel two-sample test has been studied for this task which makes fewer assumptions on the distributions than traditional parametric approaches.

However, selecting kernels is non-trivial in practice.

Although kernel selection for the two-sample test has been studied, the insufficient samples in change point detection problem hinder the success of those developed kernel selection algorithms.

In this paper, we propose KL-CPD, a novel kernel learning framework for time series CPD that optimizes a lower bound of test power via an auxiliary generative model.

With deep kernel parameterization, KL-CPD endows kernel two-sample test with the data-driven kernel to detect different types of change-points in real-world applications.

The proposed approach significantly outperformed other state-of-the-art methods in our comparative evaluation of benchmark datasets and simulation studies.

In this paper, we propose KL-CPD, a novel kernel learning framework for time series CPD that optimizes a lower bound of test power via an auxiliary generative model as a surrogate to the abnormal distribution.

Describes a novel approach to optimising the choice of kernel towards increased testing power and shown to offer improvements over alternatives.

Kernel Change-point Detection with Auxiliary Deep Generative Models

novel
auxiliary
generative model
power
kernel


Theories in cognitive psychology postulate that humans use similarity as a basis

However, work in image classification generally as-

sumes disjoint and equally dissimilar classes to achieve super-human levels of

In our work, we adapt notions of similarity using

weak labels over multiple hierarchical levels to boost classification performance.

Instead of pitting clustering directly against classification, we use a warm-start

based evaluation to explicitly provide value to a clustering representation by its

We evaluate on CIFAR10 and a fine-grained classifi-

cation dataset to show improvements in performance with the procedural addition

of intermediate losses and weak labels based on multiple hierarchy levels.

more, we show that pretraining AlexNet on hierarchical weak labels in conjunc-

tion with intermediate losses outperforms a classification baseline by over 17% on

Finally, we show improvement over AlexNet trained

using ImageNet pre-trained weights as initializations which further supports our

Cluster before you classify; using weak labels to improve classification

Proposes using a clustering based loss function at multiple levels of a deepnet as well as using hierarchical structure of the label space to train better representations.

This paper uses hierarchical label information to impose additional losses on intermediate representations in neural network training.

Cluster-based Warm-Start Nets


Deep reinforcement learning algorithms that estimate state and state-action value functions have been shown to be effective in a variety of challenging domains, including learning control strategies from raw image pixels.

However, algorithms that estimate state and state-action value functions typically assume a fully observed state and must compensate for partial or non-Markovian observations by using finite-length frame-history observations or recurrent networks.

In this work, we propose a new deep reinforcement learning algorithm based on counterfactual regret minimization that iteratively updates an approximation to a cumulative clipped advantage function and is robust to partially observed state.

We demonstrate that on several partially observed reinforcement learning tasks, this new class of algorithms can substantially outperform strong baseline methods: on Pong with single-frame observations, and on the challenging Doom (ViZDoom) and Minecraft (Malmö) first-person navigation benchmarks.

Advantage-based regret minimization is a new deep reinforcement learning algorithm that is particularly effective on partially observable tasks, such as 1st person navigation in Doom and Minecraft.

This paper introduces the concepts of counterfactual regret minimization in the field of Deep RL and an algorithm called ARM which can deal with partial observability better.

The paper provides a game-theoretic inspired variant of policy-gradient algorithm based on the idea of counter-factual regret minimization and claims that the approach can deal with the partial observable domain better than standard methods.

Regret Minimization for Partially Observable Deep Reinforcement Learning

deal partial
regret minimization
deep reinforcement learning
algorithm
partially observable
well