Since deep neural networks are over-parameterized, they can memorize noisy examples.

We address such memorizing issue in the presence of annotation noise.

From the fact that deep neural networks cannot generalize neighborhoods of the features acquired via memorization, we hypothesize that noisy examples do not consistently incur small losses on the network under a certain perturbation.

Based on this, we propose a novel training method called Learning with Ensemble Consensus (LEC) that prevents overfitting noisy examples by eliminating them using the consensus of an ensemble of perturbed networks.

One of the proposed LECs, LTEC outperforms the current state-of-the-art methods on noisy MNIST, CIFAR-10, and CIFAR-100 in an efficient manner.

This work presents a method of generating and using ensembles effectively to identify noisy examples in the presence of annotation noise.

Robust training with ensemble consensus


Recovering sparse conditional independence graphs from data is a fundamental problem in machine learning with wide applications.

A popular formulation of the problem is an $\ell_1$ regularized maximum likelihood estimation.

Many convex optimization algorithms have been designed to solve this formulation to recover the graph structure.

Recently, there is a surge of interest to learn algorithms directly based on data, and in this case, learn to map empirical covariance to the sparse precision matrix.

However, it is a challenging task in this case, since the symmetric positive definiteness (SPD) and sparsity of the matrix are not easy to enforce in learned algorithms, and a direct mapping from data to precision matrix may contain many parameters.

We propose a deep learning architecture, GLAD, which uses an Alternating Minimization (AM) algorithm as our model inductive bias, and learns the model parameters via supervised learning.

We show that GLAD learns a very compact and effective model for recovering sparse graphs from data.

A data-driven learning algorithm based on unrolling the Alternating Minimization optimization for sparse graph recovery.

GLAD: Learning Sparse Graph Recovery


Counterfactual regret minimization (CFR) is a fundamental and effective technique for solving Imperfect Information Games (IIG).

However, the original CFR algorithm only works for discrete states and action spaces, and the resulting strategy is maintained as a tabular representation.

Such tabular representation limits the method from being directly applied to large games.

In this paper, we propose a double neural representation for the IIGs, where one neural network represents the cumulative regret, and the other represents the average strategy.

Such neural representations allow us to avoid manual game abstraction and carry out end-to-end optimization.

To make the learning efficient, we also developed several novel techniques including a robust sampling method and a mini-batch Monte Carlo Counterfactual Regret Minimization (MCCFR) method, which may be of independent interests.

Empirically, on games tractable to tabular approaches, neural strategies trained with our algorithm converge comparably to their tabular counterparts, and significantly outperform those based on deep reinforcement learning.

On extremely large games with billions of decision nodes, our approach achieved strong performance while using hundreds of times less memory than the tabular CFR.

On head-to-head matches of hands-up no-limit texas hold'em, our neural agent beat the strong agent ABS-CFR by $9.8\pm4.1$ chips per game.

It's a successful application of neural CFR in large games.

We proposed a double neural framework to solve large-scale imperfect information game.

Double Neural Counterfactual Regret Minimization


correctness relative to a specification which identifies 1) a state space consisting of

all relevant states of the world and 2) an observation process that produces neural

network inputs from the states of the world.

a finite number of tiles, obtaining ground truth bounds from the state tiles and

network output bounds from the input tiles, then comparing the ground truth and

network output bounds delivers an upper bound on the network output error for

Results from two case studies highlight the ability of our

technique to deliver tight error bounds for all inputs of interest and show how the

error bounds vary over the state and input spaces.

We present the first verification that a neural network for perception tasks produces a correct output within a specified tolerance for every input of interest.

Correctness Verification of Neural Network


Deep generative models have achieved remarkable progress in recent years.

Despite this progress, quantitative evaluation and comparison of generative models remains as one of the important challenges.

One of the most popular metrics for evaluating generative models is the log-likelihood.

While the direct computation of log-likelihood can be intractable, it has been recently shown that the log-likelihood of some of the most interesting generative models such as variational autoencoders (VAE) or generative adversarial networks (GAN) can be efficiently estimated using annealed importance sampling (AIS).

In this work, we argue that the log-likelihood metric by itself cannot represent all the different performance characteristics of generative models, and propose to use rate distortion curves to evaluate and compare deep generative models.

We show that we can approximate the entire rate distortion curve using one single run of AIS for roughly the same computational cost as a single log-likelihood estimate.

We evaluate lossy compression rates of different deep generative models such as VAEs, GANs (and its variants) and adversarial autoencoders (AAE) on MNIST and CIFAR10, and arrive at a number of insights not obtainable from log-likelihoods alone.

We study rate distortion approximations for evaluating deep generative models, and show that rate distortion curves provide more insights about the model than the log-likelihood alone while requiring roughly the same computational cost.

Evaluating Lossy Compression Rates of Deep Generative Models


Although reinforcement learning methods can achieve impressive results in simulation, the real world presents two major challenges: generating samples is exceedingly expensive, and unexpected perturbations or unseen situations cause proficient but specialized policies to fail at test time.

Given that it is impractical to train separate policies to accommodate all situations the agent may see in the real world, this work proposes to learn how to quickly and effectively adapt online to new tasks.

To enable sample-efficient learning, we consider learning online adaptation in the context of model-based reinforcement learning.

Our approach uses meta-learning to train a dynamics model prior such that, when combined with recent data, this prior can be rapidly adapted to the local context.

Our experiments demonstrate online adaptation for continuous control tasks on both simulated and real-world agents.

We first show simulated agents adapting their behavior online to novel terrains, crippled body parts, and highly-dynamic environments.

We also illustrate the importance of incorporating online adaptation into autonomous agents that operate in the real world by applying our method to a real dynamic legged millirobot: We demonstrate the agent's learned ability to quickly adapt online to a missing leg, adjust to novel terrains and slopes, account for miscalibration or errors in pose estimation, and compensate for pulling payloads.

A model-based meta-RL algorithm that enables a real robot to adapt online in dynamic environments

Learning to Adapt in Dynamic, Real-World Environments through Meta-Reinforcement Learning


Model-free deep reinforcement learning approaches have shown superhuman performance in simulated environments (e.g., Atari games, Go, etc).

During training, these approaches often implicitly construct a latent space that contains key information for decision making.

In this paper, we learn a forward model on this latent space and apply it to model-based planning in miniature Real-time Strategy game with incomplete information (MiniRTS).

We first show that the latent space constructed from existing actor-critic models contains relevant information of the game, and design training procedure to learn forward models.

We also show that our learned forward model can predict meaningful future state and is usable for latent space Monte-Carlo Tree Search (MCTS), in terms of win rates against rule-based agents.

The paper analyzes the latent space learned by model-free approaches in a miniature incomplete information game, trains a forward model in the latent space and apply it to Monte-Carlo Tree Search, yielding positive performance.

Latent forward model for Real-time Strategy game planning with incomplete information

forward model
incomplete information
game
latent


Several state of the art convolutional networks rely on inter-connecting different layers to ease the flow of information and gradient between their input and output layers.

These techniques have enabled practitioners to successfully train deep convolutional networks with hundreds of layers.

Particularly, a novel way of interconnecting layers was introduced as the Dense Convolutional Network (DenseNet) and has achieved state of the art performance on relevant image recognition tasks.

Despite their notable empirical success, their theoretical understanding is still limited.

In this work, we address this problem by analyzing the effect of layer interconnection on the overall expressive power of a convolutional network.

In particular, the  connections used in DenseNet are compared with other types of inter-layer connectivity.

We carry out a tensor analysis on the expressive power inter-connections on convolutional arithmetic circuits (ConvACs) and relate our results to standard convolutional networks.

The analysis leads to performance bounds and practical guidelines for design of ConvACs.

The generalization of these results are discussed for other kinds of convolutional networks via generalized tensor decompositions.

We analyze the expressive power of the connections used in DenseNets via tensor decompositions.

A Tensor Analysis on Dense Connectivity via Convolutional Arithmetic Circuits


We consider the following central question in the field of Deep Reinforcement Learning (DRL):

How can we use implicit human feedback to accelerate and optimize the training of a DRL algorithm?

State-of-the-art methods rely on any human feedback to be provided explicitly, requiring the active participation of humans (e.g., expert labeling, demonstrations, etc.).

In this work, we investigate an alternative paradigm, where non-expert humans are silently observing (and assessing) the agent interacting with the environment.

The human's intrinsic reactions to the agent's behavior is sensed as implicit feedback by placing electrodes on the human scalp and monitoring what are known as event-related electric potentials.

The implicit feedback is then used to augment the agent's learning in the RL tasks.

We develop a system to obtain and accurately decode the implicit human feedback (specifically error-related event potentials) for state-action pairs in an Atari-type environment.

As a baseline contribution, we demonstrate the feasibility of capturing error-potentials  of a human observer watching an agent learning to play several different Atari-games using an electroencephalogram (EEG) cap, and then decoding the signals appropriately and using them as an auxiliary reward function to a DRL algorithm with the intent of accelerating its learning of the game.

Building atop the baseline, we then make the following novel contributions in our work:
(i) We argue that the definition of error-potentials is generalizable across different environments; specifically we show that error-potentials of an observer can be learned for a specific game, and the definition used as-is for another game without requiring re-learning of the error-potentials.

(ii) We propose two different frameworks to combine recent advances in DRL into the error-potential based feedback system in a sample-efficient manner, allowing humans to provide implicit feedback while training in the loop, or prior to the training of the RL agent.

(iii) Finally, we scale the implicit human feedback (via ErrP) based RL to reasonably complex environments (games) and demonstrate the significance of our approach through synthetic and real user experiments.

We use implicit human feedback (via error-potentials, EEG) to accelerate and optimize the training of a DRL algorithm, in a practical manner.

Deep Reinforcement Learning with Implicit Human Feedback


Deep learning has demonstrated abilities to learn complex structures, but they can be restricted by available data.

Recently, Consensus Networks (CNs) were proposed to alleviate data sparsity by utilizing features from multiple modalities, but they too have been limited by the size of labeled data.

In this paper, we extend CN to Transductive Consensus Networks (TCNs), suitable for semi-supervised learning.

In TCNs, different modalities of input are compressed into latent representations, which we encourage to become indistinguishable during iterative adversarial training.

To understand TCNs two mechanisms, consensus and classification, we put forward its three variants in ablation studies on these mechanisms.

To further investigate TCN models, we treat the latent representations as probability distributions and measure their similarities as the negative relative Jensen-Shannon divergences.

We show that a consensus state beneficial for classification desires a stable but imperfect similarity between the representations.

Overall, TCNs outperform or align with the best benchmark algorithms given 20 to 200 labeled samples on the Bank Marketing and the DementiaBank datasets.

TCN for multimodal semi-supervised learning + ablation study of its mechanisms + interpretations of latent representations

Semi-supervised classification by reaching consensus among modalities


Several first order stochastic optimization methods commonly used in the Euclidean domain such as stochastic gradient descent (SGD), accelerated gradient descent or variance reduced methods have already been adapted to certain Riemannian settings.

However, some of the most popular of these optimization tools - namely Adam, Adagrad and the more recent Amsgrad - remain to be generalized to Riemannian manifolds.

We discuss the difficulty of generalizing such adaptive schemes to the most agnostic Riemannian setting, and then provide algorithms and convergence proofs for geodesically convex objectives in the particular case of a product of Riemannian manifolds, in which adaptivity is implemented across manifolds in the cartesian product.

Our generalization is tight in the sense that choosing the Euclidean space as Riemannian manifold yields the same algorithms and regret bounds as those that were already known for the standard algorithms.

Experimentally, we show faster convergence and to a lower train loss value for Riemannian adaptive methods over their corresponding baselines on the realistic task of embedding the WordNet taxonomy in the Poincare ball.

Adapting Adam, Amsgrad, Adagrad to Riemannian manifolds.

Riemannian Adaptive Optimization Methods


We study the problem of defending deep neural network approaches for image classification from physically realizable attacks.

First, we demonstrate that the two most scalable and effective methods for learning robust models, adversarial training with PGD attacks and randomized smoothing, exhibit very limited effectiveness against three of the highest profile physical attacks.

Next, we propose a new abstract adversarial model, rectangular occlusion attacks, in which an adversary places a small adversarially crafted rectangle in an image, and develop two approaches for efficiently computing the resulting adversarial examples.

Finally, we demonstrate that adversarial training using our new attack yields image classification models that exhibit high robustness against the physically realizable attacks we study, offering the first effective generic defense against such attacks.

Defending Against Physically Realizable Attacks on Image Classification

defend physically realizable attack image classification


Continual learning is the problem of sequentially learning new tasks or knowledge while protecting previously acquired knowledge.

However, catastrophic forgetting poses a grand challenge for neural networks performing such learning process.

Thus, neural networks that are deployed in the real world often struggle in scenarios where the data distribution is non-stationary (concept drift), imbalanced, or not always fully available, i.e., rare edge cases.

We propose a Differentiable Hebbian Consolidation model which is composed of a Differentiable Hebbian Plasticity (DHP) Softmax layer that adds a rapid learning plastic component (compressed episodic memory) to the fixed (slow changing) parameters of the softmax output layer; enabling learned representations to be retained for a longer timescale.

We demonstrate the flexibility of our method by integrating well-known task-specific synaptic consolidation methods to penalize changes in the slow weights that are important for each target task.

We evaluate our approach on the Permuted MNIST, Split MNIST and Vision Datasets Mixture benchmarks, and introduce an imbalanced variant of Permuted MNIST --- a dataset that combines the challenges of class imbalance and concept drift.

Our proposed model requires no additional hyperparameters and outperforms comparable baselines by reducing forgetting.

Hebbian plastic weights can behave as a compressed episodic memory storage in neural networks and with the combination of task-specific synaptic consolidation can improve the ability to alleviate catastrophic forgetting in continual learning.

Differentiable Hebbian Consolidation for Continual Learning

continual learning
consolidation
hebbian


In order to choose a neural network architecture that will be effective for a particular modeling problem, one must understand the limitations imposed by each of the potential options.

These limitations are typically described in terms of information theoretic bounds, or by comparing the relative complexity needed to approximate example functions between different architectures.

In this paper, we examine the topological constraints that the architecture of a neural network imposes on the level sets of all the functions that it is able to approximate.

This approach is novel for both the nature of the limitations and the fact that they are independent of network depth for a broad family of activation functions.

This paper proves that skinny neural networks cannot approximate certain functions, no matter how deep they are.

Deep, Skinny Neural Networks are not Universal Approximators


Program verification offers a framework for ensuring program correctness and therefore systematically eliminating different classes of bugs.

Inferring loop invariants is one of the main challenges behind automated verification of real-world programs which often contain many loops.

In this paper, we present Continuous Logic Network (CLN), a novel neural architecture for automatically learning loop invariants directly from program execution traces.

Unlike existing neural networks, CLNs can learn precise and explicit representations of formulas in Satisfiability Modulo Theories (SMT)  for loop invariants from program execution traces.

We develop a new sound and complete semantic mapping for assigning SMT formulas to continuous truth values that allows CLNs to be trained efficiently.

We use CLNs to implement a new inference system for loop invariants, CLN2INV, that significantly outperforms existing approaches on the popular Code2Inv dataset.

CLN2INV is the first tool to solve all 124 theoretically solvable problems in the Code2Inv dataset.

Moreover, CLN2INV takes only 1.1 second on average for each problem, which is 40 times faster than existing approaches.

We further demonstrate that CLN2INV can even learn 12 significantly more complex loop invariants than the ones required for the Code2Inv dataset.

We introduce the Continuous Logic Network (CLN), a novel neural architecture for automatically learning loop invariants and general SMT formulas.

CLN2INV: Learning Loop Invariants with Continuous Logic Networks

learn loop invariant
continuous logic network


Single cell RNA sequencing (scRNAseq) technology enables quantifying gene expression profiles by individual cells within cancer.

Dimension reduction methods have been commonly used for cell clustering analysis and visualization of the data.

Current dimension reduction methods tend overly eliminate the expression variations correspond to less dominating characteristics, such we fail to find the homogenious properties of cancer development.

In this paper, we proposed a new and clustering analysis method for scRNAseq data, namely BBSC, via implementing a binarization of the gene expression profile into on/off frequency changes with a Boolean matrix factorization.

The low rank representation of expression matrix recovered by BBSC increase the resolution in identifying distinct cell types or functions.

Application of BBSC on two cancer scRNAseq data successfully discovered both homogeneous and heterogeneous cancer cell clusters.

Further finding showed potential in preventing cancer progression.

Our finding shed lights in preventing cancer progression

Cancer homogeneity in single cell revealed by Bi-state model and Binary matrix factorization


While normalizing flows have led to significant advances in modeling high-dimensional continuous distributions, their applicability to discrete distributions remains unknown.

In this paper, we show that flows can in fact be extended to discrete events---and under a simple change-of-variables formula not requiring log-determinant-Jacobian computations.

Discrete flows have numerous applications.

We display proofs of concept under 2 flow architectures: discrete autoregressive flows enable bidirectionality, allowing for example tokens in text to depend on both left-to-right and right-to-left contexts in an exact language model; and discrete bipartite flows (i.e., with layer structure from RealNVP) enable parallel generation such as exact nonautoregressive text modeling.

We extend autoregressive flows and RealNVP to discrete data.

Discrete Flows: Invertible Generative Models of Discrete Data


We present a Deep Neural Network with Spike Assisted Feature Extraction (SAFE-DNN) to improve robustness of classification under stochastic perturbation of inputs.

The proposed network augments a DNN with unsupervised learning of low-level features using spiking neuron network (SNN) with Spike-Time-Dependent-Plasticity (STDP).

The complete network learns to ignore local perturbation while performing global feature detection and classification.

The experimental results on CIFAR-10 and ImageNet subset demonstrate improved noise robustness for multiple DNN architectures without sacrificing accuracy on clean images.

A noise robust deep learning architecture.

SAFE-DNN: A Deep Neural Network with Spike Assisted Feature Extraction for Noise Robust Inference


Neural embeddings have been used with great success in Natural Language Processing (NLP) where they provide compact representations that encapsulate word similarity and attain state-of-the-art performance in a range of linguistic tasks.

The success of neural embeddings has prompted significant amounts of research into applications in domains other than language.

One such domain is graph-structured data, where embeddings of vertices can be learned that encapsulate vertex similarity and improve performance on tasks including edge prediction and vertex labelling.

For both NLP and graph-based tasks, embeddings in high-dimensional Euclidean spaces have been learned.

However, recent work has shown that the appropriate isometric space for embedding complex networks is not the flat Euclidean space, but a negatively curved hyperbolic space.

We present a new concept that exploits these recent insights and propose learning neural embeddings of graphs in hyperbolic space.

We provide experimental evidence that hyperbolic embeddings significantly outperform Euclidean embeddings on vertex classification tasks for several real-world public datasets.

We learn neural embeddings of graphs in hyperbolic instead of Euclidean space

Hybed: Hyperbolic Neural Graph Embedding


The International Competition on Knowledge Engineering for Planning and Scheduling (ICKEPS) plays a pivotal role in fostering the development of new Knowledge Engineering (KE) tools, and in emphasising the importance of principled approaches for all the different KE aspects that are needed for the successful long-term use of planning in real-world applications.

In this paper, as an exercise in synthesis and for the sake of stimulating thoughts and discussion, we review the format of previous ICKEPS, to suggest alternative formats for future competitions, ideally to motivate someone to step up and organise the next ones.

The International Competition on Knowledge Engineering for Planning and Scheduling: Food for Thoughts (and Call to Action)


We show that generating English Wikipedia articles can be approached as a multi-

document summarization of source documents.

to coarsely identify salient information and a neural abstractive model to generate

For the abstractive model, we introduce a decoder-only architecture

that can scalably attend to very long sequences, much longer than typical encoder-

decoder architectures used in sequence transduction.

generate fluent, coherent multi-sentence paragraphs and even whole Wikipedia

When given reference documents, we show it can extract relevant factual

information as reflected in perplexity, ROUGE scores and human evaluations.

We generate Wikipedia articles abstractively conditioned on source document text.

Generating Wikipedia by Summarizing Long Sequences


Abstract Stochastic gradient descent (SGD) and Adam are commonly used to optimize deep neural networks, but choosing one usually means making tradeoffs between speed, accuracy and stability.

Here we present an intuition for why the tradeoffs exist as well as a method for unifying the two in a continuous way.

This makes it possible to control the way models are trained in much greater detail.

We show that for default parameters, the new algorithm equals or outperforms SGD and Adam across a range of models for image classification tasks and outperforms SGD for language modeling tasks.

An algorithm for unifying SGD and Adam and empirical study of its performance

SoftAdam: Unifying SGD and Adam for better stochastic gradient descent


The use of imitation learning to learn a single policy for a complex task that has multiple modes or hierarchical structure can be challenging.

In fact, previous work has shown that when the modes are known, learning separate policies for each mode or sub-task can greatly improve the performance of imitation learning.

In this work, we discover the interaction between sub-tasks from their resulting state-action trajectory sequences using a directed graphical model.

We propose a new algorithm based on the generative adversarial imitation learning framework which automatically learns sub-task policies from unsegmented demonstrations.

Our approach maximizes the directed information flow in the graphical model between sub-task latent variables and their generated trajectories.

We also show how our approach connects with the existing Options framework, which is commonly used to learn hierarchical policies.

Directed-Info GAIL: Learning Hierarchical Policies from Unsegmented Demonstrations using Directed Information

learn hierarchical policy unsegmented demonstration directed information


The Convolutional Neural Network (CNN) has been successfully applied in many fields during recent decades; however it lacks the ability to utilize prior domain knowledge when dealing with many realistic problems.

We present a framework called Geometric Operator Convolutional Neural Network (GO-CNN) that uses domain knowledge, wherein the kernel of the first convolutional layer is replaced with a kernel generated by a geometric operator function.

This framework integrates many conventional geometric operators, which allows it to adapt to a diverse range of problems.

Under certain conditions, we theoretically analyze the convergence and the bound of the generalization errors between GO-CNNs and common CNNs.

Although the geometric operator convolution kernels have fewer trainable parameters than common convolution kernels, the experimental results indicate that GO-CNN performs more accurately than common CNN on CIFAR-10/100.

Furthermore, GO-CNN reduces dependence on the amount of training examples and enhances adversarial stability.

Traditional image processing algorithms are combined with Convolutional Neural Networks，a new neural network.


Determinantal point processes (DPPs) is an effective tool to deliver diversity on multiple machine learning and computer vision tasks.

Under deep learning framework, DPP is typically optimized via approximation, which is not straightforward and has some conflict with diversity requirement.

We note, however, there has been no deep learning paradigms to optimize DPP directly since it involves matrix inversion which may result in highly computational instability.

This fact greatly hinders the wide use of DPP on some specific objectives where DPP serves as a term to measure the feature diversity.

In this paper, we devise a simple but effective algorithm to address this issue to optimize DPP term directly expressed with L-ensemble in spectral domain over gram matrix, which is more flexible than learning on parametric kernels.

By further taking into account some geometric constraints, our algorithm seeks to generate valid sub-gradients of DPP term in case when the DPP gram matrix is not invertible (no gradients exist in this case).

In this sense, our algorithm can be easily incorporated with multiple deep learning tasks.

Experiments show the effectiveness of our algorithm, indicating promising performance for practical learning problems.

We proposed a specific back-propagation method via proper spectral sub-gradient to integrate determinantal point process to deep learning framework.

Deep Learning of Determinantal Point Processes via Proper Spectral Sub-gradient

via proper spectral sub-gradient
determinantal point process
deep learning


The quality of a machine translation system depends largely on the availability of sizable parallel corpora.

For the recently popular Neural Machine Translation (NMT) framework, data sparsity problem can become even more severe.

With large amount of tunable parameters, the NMT model may overfit to the existing language pairs while failing to understand the general diversity in language.

In this paper, we advocate to broadcast every sentence pair as two groups of similar sentences to incorporate more diversity in language expressions, which we name as parallel cluster.

Then we define a more general cluster-to-cluster correspondence score and train our model to maximize this score.

Since direct maximization is difficult, we derive its lower-bound as our surrogate objective, which is found to generalize point-point Maximum Likelihood Estimation (MLE) and point-to-cluster Reward Augmented Maximum Likelihood (RAML) algorithms as special cases.

Based on this novel objective function, we delineate four potential systems to realize our cluster-to-cluster framework and test their performances in three recognized translation tasks, each task with forward and reverse translation directions.

In each of the six experiments, our proposed four parallel systems have consistently proved to outperform the MLE baseline, RL (Reinforcement Learning) and RAML systems significantly.

Finally, we have performed case study to empirically analyze the strength of the cluster-to-cluster NMT framework.

We invent a novel cluster-to-cluster framework for NMT training, which can better understand the both source and target language diversity.

A cluster-to-cluster framework for neural machine translation


The capability of making interpretable and self-explanatory decisions is essential for developing responsible machine learning systems.

In this work, we study the learning to explain the problem in the scope of inductive logic programming (ILP).

We propose Neural Logic Inductive Learning (NLIL), an efficient differentiable ILP framework that learns first-order logic rules that can explain the patterns in the data.

In experiments, compared with the state-of-the-art models, we find NLIL is able to search for rules that are x10 times longer while remaining x3 times faster.

We also show that NLIL can scale to large image datasets, i.e. Visual Genome, with 1M entities.

An efficient differentiable ILP model that learns first-order logic rules that can explain the data.

Learn to Explain Efficiently via Neural Logic Inductive Learning


Neural network-based classifiers parallel or exceed human-level accuracy on many common tasks and are used in practical systems.

Yet, neural networks are susceptible to adversarial examples, carefully perturbed inputs that cause networks to misbehave in arbitrarily chosen ways.

When generated with standard methods, these examples do not consistently fool a classifier in the physical world due to a combination of viewpoint shifts, camera noise, and other natural transformations.

Adversarial examples generated using standard techniques require complete control over direct input to the classifier, which is impossible in many real-world systems.

We introduce the first method for constructing real-world 3D objects that consistently fool a neural network across a wide distribution of angles and viewpoints.

We present a general-purpose algorithm for generating adversarial examples that are robust across any chosen distribution of transformations.

We demonstrate its application in two dimensions, producing adversarial images that are robust to noise, distortion, and affine transformation.

Finally, we apply the algorithm to produce arbitrary physical 3D-printed adversarial objects, demonstrating that our approach works end-to-end in the real world.

Our results show that adversarial examples are a practical concern for real-world systems.

We introduce a new method for synthesizing adversarial examples robust in the physical world and use it to fabricate the first 3D adversarial objects.

Synthesizing Robust Adversarial Examples


Representations of sets are challenging to learn because operations on sets should be permutation-invariant.

To this end, we propose a Permutation-Optimisation module that learns how to permute a set end-to-end.

The permuted set can be further processed to learn a permutation-invariant representation of that set, avoiding a bottleneck in traditional set models.

We demonstrate our model's ability to learn permutations and set representations with either explicit or implicit supervision on four datasets, on which we achieve state-of-the-art results: number sorting, image mosaics, classification from image mosaics, and visual question answering.

Learn how to permute a set, then encode permuted set with RNN to obtain a set representation.

Learning Representations of Sets through Optimized Permutations


The physical design of a robot and the policy that controls its motion are inherently coupled.

However, existing approaches largely ignore this coupling, instead choosing to alternate between separate design and control phases, which requires expert intuition throughout and risks convergence to suboptimal designs.

In this work, we propose a method that jointly optimizes over the physical design of a robot and the corresponding control policy in a model-free fashion, without any need for expert supervision.

Given an arbitrary robot morphology, our method maintains a distribution over the design parameters and uses reinforcement learning to train a neural network controller.

Throughout training, we refine the robot distribution to maximize the expected reward.

This results in an assignment to the robot parameters and neural network policy that are jointly optimal.

We evaluate our approach in the context of legged locomotion, and demonstrate that it discovers novel robot designs and walking gaits for several different morphologies, achieving performance comparable to or better than that of hand-crafted designs.

Use deep reinforcement learning to design the physical attributes of a robot jointly with a control policy.

Jointly Learning to Construct and Control Agents using Deep Reinforcement Learning

deep reinforcement learning
control
jointly


Deep learning enables training of large and flexible function approximators from scratch at the cost of large amounts of data.

Applications of neural networks often consider learning in the context of a single task.

However, in many scenarios what we hope to learn is not just a single task, but a model that can be used to solve multiple different tasks.

Such multi-task learning settings have the potential to improve data efficiency and generalization by sharing data and representations across tasks.

However, in some challenging multi-task learning settings, particularly in reinforcement learning, it is very difficult to learn a single model that can solve all the tasks while realizing data efficiency and performance benefits.

Learning each of the tasks independently from scratch can actually perform better in such settings, but it does not benefit from the representation sharing that multi-task learning can potentially provide.

In this work, we develop an approach that endows a single model with the ability to represent both extremes: joint training and independent training.

To this end, we introduce matrix-interleaving (Mint), a modification to standard neural network models that projects the activations for each task into a different learned subspace, represented by a per-task and per-layer matrix.

By learning these matrices jointly with the other model parameters, the optimizer itself can decide how much to share representations between tasks.

On three challenging multi-task supervised learning and reinforcement learning problems with varying degrees of shared task structure, we find that this model consistently matches or outperforms joint training and independent training, combining the best elements of both.

We propose an approach that endows a single model with the ability to represent both extremes: joint training and independent training, which leads to effective multi-task learning.

Mint: Matrix-Interleaving for Multi-Task Learning


Training agents to operate in one environment often yields overfitted models that are unable to generalize to the changes in that environment.

However, due to the numerous variations that can occur in the real-world, the agent is often required to be robust in order to be useful.

This has not been the case for agents trained with reinforcement learning (RL) algorithms.

In this paper, we investigate the overfitting of RL agents to the training environments in visual navigation tasks.

Our experiments show that deep RL agents can overfit even when trained on multiple environments simultaneously.

We propose a regularization method which combines RL with supervised learning methods by adding a term to the RL objective that would encourage the invariance of a policy to variations in the observations that ought not to affect the action taken.

The results of this method, called invariance regularization, show an improvement in the generalization of policies to environments not seen during training.

We propose a regularization term that, when added to the reinforcement learning objective, allows the policy to maximize the reward and simultaneously learn to be invariant to the irrelevant changes within the input..

Improving the Generalization of Visual Navigation Policies using Invariance Regularization


Though visual information has been introduced for enhancing neural machine translation (NMT), its effectiveness strongly relies on the availability of large amounts of bilingual parallel sentence pairs with manual image annotations.

In this paper, we present a universal visual representation learned over the monolingual corpora with image annotations, which overcomes the lack of large-scale bilingual sentence-image pairs, thereby extending image applicability in NMT.

In detail, a group of images with similar topics to the source sentence will be retrieved from a light topic-image lookup table learned over the existing sentence-image pairs, and then is encoded as image representations by a pre-trained ResNet.

An attention layer with a gated weighting is to fuse the visual information and text information as input to the decoder for predicting target translations.

In particular, the proposed method enables the visual information to be integrated into large-scale text-only NMT in addition to the multimodel NMT.

Experiments on four widely used translation datasets, including the WMT'16 English-to-Romanian, WMT'14 English-to-German, WMT'14 English-to-French, and Multi30K, show that the proposed approach achieves significant improvements over strong baselines.

This work proposed a universal visual representation for neural machine translation (NMT) using retrieved images with similar topics to source sentence,  extending image applicability in NMT.

Neural Machine Translation with Universal Visual Representation

neural machine translation
universal visual representation


This paper introduces a novel framework for learning algorithms to solve online combinatorial optimization problems.

Towards this goal, we introduce a number of key ideas from traditional algorithms and complexity theory.

First, we draw a new connection between primal-dual methods and reinforcement learning.

Next, we introduce the concept of adversarial distributions (universal and high-entropy training sets), which are distributions that encourage the learner to find algorithms that work well in the worst case.

We test our new ideas on a number of optimization problem such as the AdWords problem, the online knapsack problem, and the secretary problem.

Our results indicate that the models have learned behaviours that are consistent with the traditional optimal algorithms for these problems.

By combining ideas from traditional algorithms design and reinforcement learning, we introduce a novel framework for learning algorithms that solve online combinatorial optimization problems.

A new dog learns old tricks:  RL finds classic optimization algorithms


Despite their popularity and successes, deep neural networks are poorly understood theoretically and treated as 'black box' systems.

Using a functional view of these networks gives us a useful new lens with which to understand them.

This allows us us to theoretically or experimentally probe properties of these networks, including the effect of standard initializations, the value of depth, the underlying loss surface, and the origins of generalization.

One key result is that generalization results from smoothness of the functional approximation, combined with a flat initial approximation.

This smoothness increases with number of units, explaining why massively overparamaterized networks continue to generalize well.

A functional approach reveals that flat initialization, preserved by gradient descent, leads to generalization ability.

A Functional Characterization of Randomly Initialized Gradient Descent in Deep ReLU Networks


It is well-known that deeper neural networks are harder to train than shallower ones.

In this short paper, we use the (full) eigenvalue spectrum of the Hessian to explore how the loss landscape changes as the network gets deeper, and as residual connections are added to the architecture.

Computing a series of quantitative measures on the Hessian spectrum, we show that the Hessian eigenvalue distribution in deeper networks has substantially heavier tails (equivalently, more outlier eigenvalues), which makes the network harder to optimize with first-order methods.

We show that adding residual connections mitigates this effect substantially, suggesting a mechanism by which residual connections improve training.

Network depth increases outlier eigenvalues in the Hessian. Residual connections mitigate this.

The Effect of Network Depth on the Optimization Landscape


In the context of optimization, a gradient of a neural network indicates the amount a specific weight should change with respect to the loss.

Therefore, small gradients indicate a good value of the weight that requires no change and can be kept frozen during training.

This paper provides an experimental study on the importance of a neural network weights, and to which extent do they need to be updated.

We wish to show that starting from the third epoch, freezing weights which have no informative gradient and are less likely to be changed during training, results in a very slight drop in the overall accuracy (and in sometimes better).

We experiment on the MNIST, CIFAR10 and Flickr8k datasets using several architectures (VGG19,

On CIFAR10, we show that freezing 80% of the VGG19 network parameters from the third epoch onwards results in 0.24% drop in accuracy, while freezing 50% of Resnet-110 parameters results in 0.9% drop in accuracy and finally freezing 70% of Densnet-121 parameters results in 0.57% drop in accuracy.

Furthermore, to experiemnt with real-life applications, we train an image captioning model with attention mechanism on the Flickr8k dataset using LSTM networks, freezing 60% of the parameters from the third epoch onwards, resulting in a better BLEU-4 score than the fully trained model.

Our source code can be found in the appendix.

An experimental paper that proves the amount of redundant weights that can be freezed from the third epoch only, with only a very slight drop in accuracy.

HOW IMPORTANT ARE NETWORK WEIGHTS? TO WHAT EXTENT DO THEY NEED AN UPDATE?


Like humans, deep networks learn better when samples are organized and introduced in a meaningful order or curriculum.

While conventional approaches to curriculum learning emphasize the difficulty of samples as the core incremental strategy, it forces networks to learn from small subsets of data while introducing pre-computation overheads.

In this work, we propose Learning with Incremental Labels and Adaptive Compensation (LILAC), which introduces a novel approach to curriculum learning.

LILAC emphasizes incrementally learning labels instead of incrementally learning difficult samples.

It works in two distinct phases: first, in the incremental label introduction phase, we unmask ground-truth labels in fixed increments during training, to improve the starting point from which networks learn.

In the adaptive compensation phase, we compensate for failed predictions by adaptively altering the target vector to a smoother distribution.

We evaluate LILAC against the closest comparable methods in batch and curriculum learning and label smoothing, across three standard image benchmarks, CIFAR-10, CIFAR-100, and STL-10.

We show that our method outperforms batch learning with higher mean recognition accuracy as well as lower standard deviation in performance consistently across all benchmarks.

We further extend LILAC to state-of-the-art performance across CIFAR-10 using simple data augmentation while exhibiting label order invariance among other important properties.

A novel approach to curriculum learning by incrementally learning labels and adaptively smoothing labels for mis-classified samples which boost average performance and decreases standard deviation.

Rethinking Curriculum Learning With Incremental Labels And Adaptive Compensation


Words in natural language follow a Zipfian distribution whereby some words are frequent but most are rare.

Learning representations for words in the ``long tail'' of this distribution requires enormous amounts of data.

Representations of rare words trained directly on end tasks are usually poor, requiring us to pre-train embeddings on external data, or treat all rare words as out-of-vocabulary words with a unique representation.

We provide a method for predicting embeddings of rare words on the fly from small amounts of auxiliary data with a network trained end-to-end for the downstream task.

We show that this improves results against baselines where embeddings are trained on the end task for reading comprehension, recognizing textual entailment and language modeling.

We propose a method to deal with rare words by computing their embedding from definitions.

Learning to Compute Word Embeddings On the Fly


The capability of reliably detecting out-of-distribution samples is one of the key factors in deploying a good classifier, as the test distribution always does not match with the training distribution in most real-world applications.

In this work, we propose a deep generative classifier which is effective to detect out-of-distribution samples as well as classify in-distribution samples, by integrating the concept of Gaussian discriminant analysis into deep neural networks.

Unlike the discriminative (or softmax) classifier that only focuses on the decision boundary partitioning its latent space into multiple regions, our generative classifier aims to explicitly model class-conditional distributions as separable Gaussian distributions.

Thereby, we can define the confidence score by the distance between a test sample and the center of each distribution.

Our empirical evaluation on multi-class images and tabular data demonstrate that the generative classifier achieves the best performances in distinguishing out-of-distribution samples, and also it can be generalized well for various types of deep neural networks.

This paper proposes a deep generative classifier which is effective to detect out-of-distribution samples as well as classify in-distribution samples, by integrating the concept of Gaussian discriminant analysis into deep neural networks.

Deep Generative Classifier for Out-of-distribution Sample Detection

deep generative classifier
out-of-distribution sample


One of the most prevalent symptoms among the elderly population, dementia, can be detected by classifiers trained on linguistic features extracted from narrative transcripts.

However, these linguistic features are impacted in a similar but different fashion by the normal aging process.

Aging is therefore a confounding factor, whose effects have been hard for machine learning classifiers to isolate.

In this paper, we show that deep neural network (DNN) classifiers can infer ages from linguistic features, which is an entanglement that could lead to unfairness across age groups.

We show this problem is caused by undesired activations of v-structures in causality diagrams, and it could be addressed with fair representation learning.

We build neural network classifiers that learn low-dimensional representations reflecting the impacts of dementia yet discarding the effects of age.

To evaluate these classifiers, we specify a model-agnostic score $\Delta_{eo}^{(N)}$ measuring how classifier results are disentangled from age.

Our best models outperform baseline neural network classifiers in disentanglement, while compromising accuracy by as little as 2.56\% and 2.25\% on DementiaBank and the Famous People dataset respectively.

Show that age confounds cognitive impairment detection + solve with fair representation learning + propose metrics and models.

Isolating effects of age with fair representation learning when assessing dementia


Much of the focus in the design of deep neural networks had been on improving accuracy, leading to more powerful yet highly complex network architectures that are difficult to deploy in practical scenarios.

As a result, there has been a recent interest in the design of quantitative metrics for evaluating deep neural networks that accounts for more than just model accuracy as the sole indicator of network performance.

In this study, we continue the conversation towards universal metrics for evaluating the performance of deep neural networks for practical on-device edge usage by introducing NetScore, a new metric designed specifically to provide a quantitative assessment of the balance between accuracy, computational complexity, and network architecture complexity of a deep neural network.

In what is one of the largest comparative analysis between deep neural networks in literature, the NetScore metric, the top-1 accuracy metric, and the popular information density metric were compared across a diverse set of 60 different deep convolutional neural networks for image classification on the ImageNet Large Scale Visual Recognition Challenge (ILSVRC 2012) dataset.

The evaluation results across these three metrics for this diverse set of networks are presented in this study to act as a reference guide for practitioners in the field.

We introduce NetScore, new metric designed to provide a quantitative assessment of the balance between accuracy, computational complexity, and network architecture complexity of a deep neural network.

NetScore: Towards Universal Metrics for Large-scale Performance Analysis of Deep Neural Networks for Practical On-Device Edge Usage


Deep Neural Networks (DNNs) are vulnerable to adversarial attacks, especially white-box targeted attacks.

This paper studies the problem of how aggressive white-box targeted attacks can be to go beyond widely used Top-1 attacks.

We propose to learn ordered Top-k attacks (k>=1), which enforce the Top-k predicted labels of an adversarial example to be the k (randomly) selected and ordered labels (the ground-truth label is exclusive).

First, we extend the vanilla Carlini-Wagner (C&W) method and use it as a strong baseline.

Second, we present an adversarial distillation framework consisting of two components:

(i) Computing an adversarial probability distribution for any given ordered Top-$k$ targeted labels.

(ii) Learning adversarial examples by minimizing the Kullback-Leibler (KL) divergence between the adversarial distribution and the predicted distribution, together with the perturbation energy penalty.

In computing adversarial distributions, we explore how to leverage label semantic similarities, leading to knowledge-oriented attacks.

In experiments, we test Top-k (k=1,2,5,10) attacks in the ImageNet-1000 val dataset using two popular DNNs trained with the clean ImageNet-1000  train dataset, ResNet-50 and DenseNet-121.

Overall, the adversarial distillation approach obtains the best results, especially by large margin when computation budget is limited..

It reduces the perturbation energy consistently with the same attack success rate on all the four k's, and improve the attack success rate by large margin against the modified C&W method for k=10.

How Aggressive Can Adversarial Attacks Be: Learning Ordered Top-k Attacks


Neural message passing algorithms for semi-supervised classification on graphs have recently achieved great success.

However, for classifying a node these methods only consider nodes that are a few propagation steps away and the size of this utilized neighborhood is hard to extend.

In this paper, we use the relationship between graph convolutional networks (GCN) and PageRank to derive an improved propagation scheme based on personalized PageRank.

We utilize this propagation procedure to construct a simple model, personalized propagation of neural predictions (PPNP), and its fast approximation, APPNP.

Our model's training time is on par or faster and its number of parameters on par or lower than previous models.

It leverages a large, adjustable neighborhood for classification and can be easily combined with any neural network.

We show that this model outperforms several recently proposed methods for semi-supervised classification in the most thorough study done so far for GCN-like models.

Personalized propagation of neural predictions (PPNP) improves graph neural networks by separating them into prediction and propagation via personalized PageRank.

Predict then Propagate: Graph Neural Networks meet Personalized PageRank

graph neural network
personalized pagerank


Most of recent work in cross-lingual word embeddings is severely Anglocentric.

The vast majority of lexicon induction evaluation dictionaries are between English and another language, and the English embedding space is selected by default as the hub when learning in a multilingual setting.

With this work, however, we challenge these practices.

First, we show that the choice of hub language can significantly impact downstream lexicon induction performance.

Second, we both expand the current evaluation dictionary collection to include all language pairs using triangulation, and also create new dictionaries for under-represented languages.

Evaluating established methods over all these language pairs sheds light into their suitability and presents new challenges for the field.

Finally, in our analysis we identify general guidelines for strong cross-lingual embeddings baselines, based on more than just Anglocentric experiments.

The choice of the hub (target) language affects the quality of cross-lingual embeddings, which shouldn't be evaluated only  on English-centric dictionaries.

Should All Cross-Lingual Embeddings Speak English?


Interpreting generative adversarial network (GAN) training as approximate divergence minimization has been theoretically insightful, has spurred discussion, and has lead to theoretically and practically interesting extensions such as f-GANs and Wasserstein GANs.

For both classic GANs and f-GANs, there is an original variant of training and a "non-saturating" variant which uses an alternative form of generator update.

The original variant is theoretically easier to study, but the alternative variant frequently performs better and is recommended for use in practice.

The alternative generator update is often regarded as a simple modification to deal with optimization issues, and it appears to be a common misconception that the two variants minimize the same divergence.

In this short note we derive the divergences approximately minimized by the original and alternative variants of GAN and f-GAN training.

This highlights important differences between the two variants.

For example, we show that the alternative variant of KL-GAN training actually minimizes the reverse KL divergence, and that the alternative variant of conventional GAN training minimizes a "softened" version of the reverse KL.

We hope these results may help to clarify some of the theoretical discussion surrounding the divergence minimization view of GAN training.

Typical GAN training doesn't optimize Jensen-Shannon, but something like a reverse KL divergence.

The divergences minimized by non-saturating GANs


REINFORCE can be used to train models in structured prediction settings to directly optimize the test-time objective.

However, the common case of sampling one prediction per datapoint (input) is data-inefficient.

We show that by drawing multiple samples (predictions) per datapoint, we can learn with significantly less data, as we freely obtain a REINFORCE baseline to reduce variance.

Additionally we derive a REINFORCE estimator with baseline, based on sampling without replacement.

Combined with a recent technique to sample sequences without replacement using Stochastic Beam Search, this improves the training procedure for a sequence model that predicts the solution to the Travelling Salesman Problem.

We show that by drawing multiple samples (predictions) per input (datapoint), we can learn with less data as we freely obtain a REINFORCE baseline.

Buy 4 REINFORCE Samples, Get a Baseline for Free!


Reinforcement learning (RL) is a powerful technique to train an agent to perform a task.

However, an agent that is trained using RL is only capable of achieving the single task that is specified via its reward function.

Such an approach does not scale well to settings in which an agent needs to perform a diverse set of tasks, such as navigating to varying positions in a room or moving objects to varying locations.

Instead, we propose a method that allows an agent to automatically discover the range of tasks that it is capable of performing in its environment.

We use a generator network to propose tasks for the agent to try to achieve, each task being specified as reaching a certain parametrized subset of the state-space.

The generator network is optimized using adversarial training to produce tasks that are always at the appropriate level of difficulty for the agent.

Our method thus automatically produces a curriculum of tasks for the agent to learn.

We show that, by using this framework, an agent can efficiently and automatically learn to perform a wide set of tasks without requiring any prior knowledge of its environment (Videos and code available at: https://sites.google.com/view/goalgeneration4rl).

Our method can also learn to achieve tasks with sparse rewards, which pose significant challenges for traditional RL methods.

We efficiently solve multi-task problems with an automatic curriculum generation algorithm based on a generative model that tracks the learning agent's performance.

Automatic Goal Generation for Reinforcement Learning Agents


A wide range of defenses have been proposed to harden neural networks against adversarial attacks.

However, a pattern has emerged in which the majority of adversarial defenses are quickly broken by new attacks.

Given the lack of success at generating robust defenses, we are led to ask a fundamental question:  Are adversarial attacks inevitable?

This paper analyzes adversarial examples from a theoretical perspective, and identifies fundamental bounds on the susceptibility of a classifier to adversarial attacks.

We show that, for certain classes of problems, adversarial examples are inescapable.

Using experiments, we explore the implications of theoretical guarantees for real-world problems and discuss how factors such as dimensionality and image complexity limit a classifier's robustness against adversarial examples.

This paper identifies classes of problems for which adversarial examples are inescapable, and derives fundamental bounds on the susceptibility of any classifier to adversarial examples.

Are adversarial examples inevitable?


For computer vision applications, prior works have shown the efficacy of reducing numeric precision of model parameters (network weights) in deep neural networks.

Activation maps, however, occupy a large memory footprint during both the training and inference step when using mini-batches of inputs.

One way to reduce this large memory footprint is to reduce the precision of activations.

However, past works have shown that reducing the precision of activations hurts model accuracy.

We study schemes to train networks from scratch using reduced-precision activations without hurting accuracy.

We reduce the precision of activation maps (along with model parameters) and increase the number of filter maps in a layer, and find that this scheme matches or surpasses the accuracy of the baseline full-precision network.

As a result, one can significantly improve the execution efficiency (e.g. reduce dynamic memory footprint, memory band- width and computational energy) and speed up the training and inference process with appropriate hardware support.

We call our scheme WRPN -- wide reduced-precision networks.

We report results and show that WRPN scheme is better than previously reported accuracies on ILSVRC-12 dataset while being computationally less expensive compared to previously reported reduced-precision networks.

Lowering precision (to 4-bits, 2-bits and even binary) and widening the filter banks gives as accurate network as those obtained with FP32 weights and activations.

WRPN: Wide Reduced-Precision Networks


We investigate methods for semi-supervised learning (SSL) of a neural linear-chain conditional random field (CRF) for Named Entity Recognition (NER) by treating the tagger as the amortized variational posterior in a generative model of text given tags.

We first illustrate how to incorporate a CRF in a VAE, enabling end-to-end training on semi-supervised data.

We then investigate a series of increasingly complex deep generative models of tokens given tags enabled by end-to-end optimization, comparing the proposed models against supervised and strong CRF SSL baselines on the Ontonotes5 NER dataset.

We find that our best proposed model consistently improves performance by $\approx 1\%$ F1 in low- and moderate-resource regimes and easily addresses degenerate model behavior in a more difficult, partially supervised setting.

We embed a CRF in a VAE of tokens and NER tags for semi-supervised learning and show improvements in low-resource settings.

Semi-Supervised Named Entity Recognition with CRF-VAEs


To make deep neural networks feasible in resource-constrained environments (such as mobile devices), it is beneficial to quantize models by using low-precision weights.

One common technique for quantizing neural networks is the straight-through gradient method, which enables back-propagation through the quantization mapping.

Despite its empirical success, little is understood about why the straight-through gradient method works.

Building upon a novel observation that the straight-through gradient method is in fact identical to the well-known Nesterov’s dual-averaging algorithm on a quantization constrained optimization problem, we propose a more principled alternative approach, called ProxQuant , that formulates quantized network training as a regularized learning problem instead and optimizes it via the prox-gradient method.

ProxQuant does back-propagation on the underlying full-precision vector and applies an efficient prox-operator in between stochastic gradient steps to encourage quantizedness.

For quantizing ResNets and LSTMs, ProxQuant outperforms state-of-the-art results on binary quantization and is on par with state-of-the-art on multi-bit quantization.

For binary quantization, our analysis shows both theoretically and experimentally that ProxQuant is more stable than the straight-through gradient method (i.e. BinaryConnect), challenging the indispensability of the straight-through gradient method and providing a powerful alternative.

A principled framework for model quantization using the proximal gradient method.

ProxQuant: Quantized Neural Networks via Proximal Operators


The vulnerabilities of deep neural networks against adversarial examples have become a significant concern for deploying these models in sensitive domains.

Devising a definitive defense against such attacks is proven to be challenging, and the methods relying on detecting adversarial samples are only valid when the attacker is oblivious to the detection mechanism.

In this paper, we consider the adversarial detection problem under the robust optimization framework.

We partition the input space into subspaces and train adversarial robust subspace detectors using asymmetrical adversarial training (AAT).

The integration of the classifier and detectors presents a detection mechanism that provides a performance guarantee to the adversary it considered.

We demonstrate that AAT promotes the learning of class-conditional distributions, which further gives rise to generative detection/classification approaches that are both robust and more interpretable.

We provide comprehensive evaluations of the above methods, and demonstrate their competitive performances and compelling properties on adversarial detection and robust classification problems.

A new generative modeling technique based on asymmetrical adversarial training, and its applications to adversarial example detection and robust classification

Adversarial Example Detection and Classification with Asymmetrical Adversarial Training

adversarial example detection
asymmetrical
classification
training


Exploration is a key component of successful reinforcement learning, but optimal approaches are computationally intractable, so researchers have focused on hand-designing mechanisms based on exploration bonuses and intrinsic reward, some inspired by curious behavior in natural systems.

In this work, we propose a strategy for encoding curiosity algorithms as programs in a domain-specific language and searching, during a meta-learning phase, for algorithms that enable RL agents to perform well in new domains.

Our rich language of programs, which can combine neural networks with other building blocks including nearest-neighbor modules and can choose its own loss functions, enables the expression of highly generalizable programs that perform well in domains as disparate as grid navigation with image input, acrobot, lunar lander, ant and hopper.

To  make this approach feasible, we develop several pruning techniques, including learning to predict a program's success based on its syntactic properties.

We demonstrate the effectiveness of the approach empirically, finding curiosity strategies that are similar to those in published literature, as well as novel strategies that are competitive with them and generalize well.

Meta-learning curiosity algorithms by searching through a rich space of programs yields novel mechanisms that generalize across very different reinforcement-learning domains.

Meta-learning curiosity algorithms


Many machine learning algorithms represent input data with vector embeddings or discrete codes.

When inputs exhibit compositional structure (e.g. objects built from parts or procedures from subroutines), it is natural to ask whether this compositional structure is reflected in the the inputs’ learned representations.

While the assessment of compositionality in languages has received significant attention in linguistics and adjacent fields, the machine learning literature lacks general-purpose tools for producing graded measurements of compositional structure in more general (e.g. vector-valued) representation spaces.

We describe a procedure for evaluating compositionality by measuring how well the true representation-producing model can be approximated by a model that explicitly composes a collection of inferred representational primitives.

We use the procedure to provide formal and empirical characterizations of compositional structure in a variety of settings, exploring the relationship between compositionality and learning dynamics, human judgments, representational similarity, and generalization.

This paper proposes a simple procedure for evaluating compositional structure in learned representations, and uses the procedure to explore the role of compositionality in four learning problems.

Measuring Compositionality in Representation Learning


In this paper, we propose an end-to-end deep learning model, called E2Efold, for RNA secondary structure prediction which can effectively take into account the inherent constraints in the problem.

The key idea of E2Efold is to directly predict the RNA base-pairing matrix, and use an unrolled constrained programming algorithm as a building block in the architecture to enforce constraints.

With comprehensive experiments on benchmark datasets, we demonstrate the superior performance of E2Efold: it predicts significantly better structures compared to previous SOTA (29.7% improvement in some cases in F1 scores and even larger improvement for pseudoknotted structures) and runs as efficient as the fastest algorithms in terms of inference time.

A DL model for RNA secondary structure prediction, which uses an unrolled algorithm in the architecture to enforce constraints.

RNA Secondary Structure Prediction By Learning Unrolled Algorithms

rna secondary structure prediction
unrolled algorithm


Learning in recurrent neural networks (RNNs) is most often implemented by gradient descent using backpropagation through time (BPTT), but BPTT does not model accurately how the brain learns.

Instead, many experimental results on synaptic plasticity can be summarized as three-factor learning rules involving eligibility traces of the local neural activity and a third factor.

We present here eligibility propagation (e-prop), a new factorization of the loss gradients in RNNs that fits the framework of three factor learning rules when derived for biophysical spiking neuron models.

When tested on the TIMIT speech recognition benchmark, it is competitive with BPTT both for training artificial LSTM networks and spiking RNNs.

Further analysis suggests that the diversity of learning signals and the consideration of slow internal neural dynamics are decisive to the learning efficiency of e-prop.

We present eligibility propagation an alternative to BPTT that is compatible with experimental data on synaptic plasticity and competes with BPTT on machine learning benchmarks.

Eligibility traces provide a data-inspired alternative to backpropagation through time


Recurrent neural networks (RNNs) are an effective representation of control policies for a wide range of reinforcement and imitation learning problems.

RNN policies, however, are particularly difficult to explain, understand, and analyze due to their use of continuous-valued memory vectors and observation features.

In this paper, we introduce a new technique, Quantized Bottleneck Insertion, to learn finite representations of these vectors and features.

The result is a quantized representation of the RNN that can be analyzed to improve our understanding of memory use and general behavior.

We present results of this approach on synthetic environments and six Atari games.

The resulting finite representations are surprisingly small in some cases, using as few as 3 discrete memory states and 10 observations for a perfect Pong policy.

We also show that these finite policy representations lead to improved interpretability.

Extracting a finite state machine from a recurrent neural network via quantization for the purpose of interpretability with experiments on Atari.

Learning Finite State Representations of Recurrent Policy Networks


Building upon the recent success of deep reinforcement learning methods, we investigate the possibility of on-policy reinforcement learning improvement by reusing the data from several consecutive policies.

On-policy methods bring many benefits, such as ability to evaluate each resulting policy.

However, they usually discard all the information about the policies which existed before.

In this work, we propose adaptation of the replay buffer concept, borrowed from the off-policy learning setting, to the on-policy algorithms.

To achieve this, the proposed algorithm generalises the Q-, value and advantage functions for data from multiple policies.

The method uses trust region optimisation, while avoiding some of the common problems of the algorithms such as TRPO or ACKTR: it uses hyperparameters to replace the trust region selection heuristics, as well as  the trainable covariance matrix instead of the fixed one.

In many cases, the method not only improves the results comparing to the state-of-the-art trust region on-policy learning algorithms such as ACKTR and TRPO, but also with respect to their off-policy counterpart DDPG.

We investigate the theoretical and practical evidence of on-policy reinforcement learning improvement by reusing the data from several consecutive policies.

On-Policy Trust Region Policy Optimisation with Replay Buffers


Convolutional Neural Networks (CNN) have been successful in processing data signals that are uniformly sampled in the spatial domain (e.g., images).

However, most data signals do not natively exist on a grid, and in the process of being sampled onto a uniform physical grid suffer significant aliasing error and information loss.

Moreover, signals can exist in different topological structures as, for example, points, lines, surfaces and volumes.

It has been challenging to analyze signals with mixed topologies (for example, point cloud with surface mesh).

To this end, we develop mathematical formulations for Non-Uniform Fourier Transforms (NUFT) to directly, and optimally, sample nonuniform data signals of different topologies defined on a simplex mesh into the spectral domain with no spatial sampling error.

The spectral transform is performed in the Euclidean space, which removes the translation ambiguity from works on the graph spectrum.

Our representation has four distinct advantages: (1) the process causes no spatial sampling error during initial sampling, (2) the generality of this approach provides a unified framework for using CNNs to analyze signals of mixed topologies, (3) it allows us to leverage state-of-the-art backbone CNN architectures for effective learning without having to design a particular architecture for a particular data structure in an ad-hoc fashion, and (4) the representation allows weighted meshes where each element has a different weight (i.e., texture) indicating local properties.

We achieve good results on-par with state-of-the-art for 3D shape retrieval task, and new state-of-the-art for point cloud to surface reconstruction task.

We use non-Euclidean Fourier Transformation of shapes defined by a simplicial complex for deep learning, achieving significantly better results than point-based sampling techiques used in current 3D learning literature.

Convolutional Neural Networks on Non-uniform Geometrical Signals Using Euclidean Spectral Transformation


Discovering causal structure among a set of variables is a fundamental problem in many empirical sciences.

Traditional score-based casual discovery methods rely on various local heuristics to search for a Directed Acyclic Graph (DAG) according to a predefined score function.

While these methods, e.g., greedy equivalence search, may have attractive results with infinite samples and certain model assumptions, they are less satisfactory in practice due to finite data and possible violation of assumptions.

Motivated by recent advances in neural combinatorial optimization, we propose to use Reinforcement Learning (RL) to search for the DAG with the best scoring.

Our encoder-decoder model takes observable data as input and generates graph adjacency matrices that are used to compute rewards.

The reward incorporates both the predefined score function and two penalty terms for enforcing acyclicity.

In contrast with typical RL applications where the goal is to learn a policy, we use RL as a search strategy and our final output would be the graph, among all graphs generated during training, that achieves the best reward.

We conduct experiments on both synthetic and real datasets, and show that the proposed approach not only has an improved search ability but also allows for a flexible score function under the acyclicity constraint.

We apply reinforcement learning to score-based causal discovery and achieve promising results on both synthetic and real datasets

Causal Discovery with Reinforcement Learning


The topic modeling discovers the latent topic probability of given the text documents.

To generate the more meaningful topic that better represents the given document, we proposed a universal method which can be used in the data preprocessing stage.

First, it generates the word/word-pair from every single document.

Second, it applies a two way parallel TF-IDF algorithm to word/word-pair for semantic filtering.

Third, it uses the k-means algorithm to merge the word pairs that have the similar semantic meaning.

Experiments are carried out on the Open Movie Database (OMDb), Reuters Dataset and 20NewsGroup Dataset and use the mean Average Precision score as the evaluation metric.

Comparing our results with other state-of-the-art topic models, such as Latent Dirichlet allocation and traditional Restricted Boltzmann Machines.

Our proposed data preprocessing can improve the generated topic accuracy by up to 12.99\%.

How the number of clusters and the number of word pairs should be adjusted for different type of text document is also discussed.

We proposed a universal method which can be used in the data preprocessing stage to generate the more meaningful topic that better represents the given document

Learning Topics using Semantic Locality


Multi-task learning has been successful in modeling multiple related tasks with large, carefully curated labeled datasets.

By leveraging the relationships among different tasks, multi-task learning framework can improve the performance significantly.

However, most of the existing works are under the assumption that the predefined tasks are related to each other.

Thus, their applications on real-world are limited, because rare real-world problems are closely related.

Besides, the understanding of relationships among tasks has been ignored by most of the current methods.

Along this line, we propose a novel multi-task learning framework - Learning To Transfer Via Modelling Multi-level Task Dependency, which constructed attention based dependency relationships among different tasks.

At the same time, the dependency relationship can be used to guide what knowledge should be transferred, thus the performance of our model also be improved.

To show the effectiveness of our model and the importance of considering multi-level dependency relationship, we conduct experiments on several public datasets, on which we obtain significant improvements over current methods.

We propose a novel multi-task learning framework which extracts multi-view dependency relationship automatically and use it to guide the knowledge transfer among different tasks.

Learning to Transfer via Modelling Multi-level Task Dependency


We design simple and quantifiable testing of global translation-invariance in deep learning models trained on the MNIST dataset.

Experiments on convolutional and capsules neural networks show that both models have poor performance in dealing with global translation-invariance; however, the performance improved by using data augmentation.

Although the capsule network is better on the MNIST testing dataset, the convolutional neural network generally has better performance on the translation-invariance.

Testing of global translational invariance in Convolutional and Capsule Networks

A quantifiable testing of global translational invariance in Convolutional and Capsule Networks

testing global translational invariance convolutional capsule network


Gaussian processes are ubiquitous in nature and engineering.

A case in point is a class of neural networks in the infinite-width limit, whose priors correspond to Gaussian processes.

Here we perturbatively extend this correspondence to finite-width neural networks, yielding non-Gaussian processes as priors.

The methodology developed herein allows us to track the flow of preactivation distributions by progressively integrating out random variables from lower to higher layers, reminiscent of renormalization-group flow.

We further develop a perturbative prescription to perform Bayesian inference with weakly non-Gaussian priors.

We develop an analytical method to study Bayesian inference of finite-width neural networks and find that the renormalization-group flow picture naturally emerges.

Non-Gaussian processes and neural networks at finite widths


Distillation is a method to transfer knowledge from one model to another and often achieves higher accuracy with the same capacity.

In this paper, we aim to provide a theoretical understanding on what mainly helps with the distillation.

Assuming that the teacher network is overparameterized, we argue that the teacher network is essentially harvesting dark knowledge from the data via early stopping.

This can be justified by a new concept, Anisotropic In- formation Retrieval (AIR), which means that the neural network tends to fit the informative information first and the non-informative information (including noise) later.

Motivated by the recent development on theoretically analyzing overparame- terized neural networks, we can characterize AIR by the eigenspace of the Neural Tangent Kernel(NTK).

AIR facilities a new understanding of distillation.

With that, we further utilize distillation to refine noisy labels.

We propose a self-distillation al- gorithm to sequentially distill knowledge from the network in the previous training epoch to avoid memorizing the wrong labels.

We also demonstrate, both theoret- ically and empirically, that self-distillation can benefit from more than just early stopping.

Theoretically, we prove convergence of the proposed algorithm to the ground truth labels for randomly initialized overparameterized neural networks in terms of l2 distance, while the previous result was on convergence in 0-1 loss.

The theoretical result ensures the learned neural network enjoy a margin on the training data which leads to better generalization.

Empirically, we achieve better testing accuracy and entirely avoid early stopping which makes the algorithm more user-friendly.

theoretically understand the regularization effect of distillation. We show that early stopping is essential in this process. From this perspective, we developed a distillation method for learning with corrupted Label with theoretical guarantees.

Distillation $\approx$ Early Stopping? Harvesting Dark Knowledge Utilizing Anisotropic Information Retrieval For Overparameterized NN


Lifelong learning poses considerable challenges in terms of effectiveness (minimizing prediction errors for all tasks) and overall computational tractability for real-time performance.

This paper addresses continuous lifelong multitask learning by jointly re-estimating the inter-task relations (\textit{output} kernel) and the per-task model parameters at each round, assuming data arrives in a streaming fashion.

We propose a novel algorithm called  \textit{Online Output Kernel Learning Algorithm} (OOKLA) for lifelong learning setting.

To avoid the memory explosion, we propose a robust budget-limited versions of the proposed algorithm that efficiently utilize the relationship between the tasks to bound the total number of representative examples in the support set.

In addition, we propose a two-stage budgeted scheme for efficiently tackling the task-specific budget constraints in lifelong learning.

Our empirical results over three datasets indicate superior AUC performance for OOKLA and its budget-limited cousins over strong baselines.

a novel approach for online lifelong learning using output kernels.

Lifelong Learning with Output Kernels


Minecraft is a videogame that offers many interesting challenges for AI systems.

In this paper, we focus in construction scenarios where an agent must build a complex structure made of individual blocks.

As higher-level objects are formed of lower-level objects, the construction can naturally be modelled as a hierarchical task network.

We model a house-construction scenario in classical and HTN planning and compare the advantages and disadvantages of both kinds of models.

We model a house-construction scenario in Minecraft in classical and HTN planning and compare the advantages and disadvantages of both kinds of models.

Construction-Planning Models in Minecraft


Attacks on natural language models are difficult to compare due to their different definitions of what constitutes a successful attack.

We present a taxonomy of constraints to categorize these attacks.

For each constraint, we present a real-world use case and a way to measure how well generated samples enforce the constraint.

We then employ our framework to evaluate two state-of-the art attacks which fool models with synonym substitution.

These attacks claim their adversarial perturbations preserve the semantics and syntactical correctness of the inputs, but our analysis shows these constraints are not strongly enforced.

For a significant portion of these adversarial examples, a grammar checker detects an increase in errors.

Additionally, human studies indicate that many of these adversarial examples diverge in semantic meaning from the input or do not appear to be human-written.

Finally, we highlight the need for standardized evaluation of attacks that share constraints.

Without shared evaluation metrics, it is up to researchers to set thresholds that determine the trade-off between attack quality and attack success.

We recommend well-designed human studies to determine the best threshold to approximate human judgement.

We present a framework for evaluating adversarial examples in natural language processing and demonstrate that generated adversarial examples are often not semantics-preserving, syntactically correct, or non-suspicious.

Pragmatic Evaluation of Adversarial Examples in Natural Language


In order for machine learning to be deployed and trusted in many applications, it is crucial to be able to reliably explain why the machine learning algorithm makes certain predictions.

For example, if an algorithm classifies a given pathology image to be a malignant tumor, then the doctor may need to know which parts of the image led the algorithm to this classification.

How to interpret black-box predictors is thus an important and active area of research.

A fundamental question is: how much can we trust the interpretation itself?

In this paper, we show that interpretation of deep learning predictions is extremely fragile in the following sense:  two perceptively indistinguishable inputs with the same predicted label can be assigned very different}interpretations.

We systematically characterize the fragility of the interpretations generated by several widely-used feature-importance interpretation methods (saliency maps, integrated gradient, and DeepLIFT) on ImageNet and CIFAR-10.

Our experiments show that even small random perturbation can change the feature importance and new systematic perturbations can lead to dramatically different interpretations without changing the label.

We extend these results to show that interpretations based on exemplars (e.g. influence functions) are similarly fragile.

Our analysis of the geometry of the Hessian matrix gives insight on why fragility could be a fundamental challenge to the current interpretation approaches.

Can we trust a neural network's explanation for its prediction? We examine the robustness of several popular notions of interpretability of neural networks including saliency maps and influence functions and design adversarial examples against them.

INTERPRETATION OF NEURAL NETWORK IS FRAGILE


Stochastic AUC maximization has garnered an increasing interest due to better fit to imbalanced data classification.

However, existing works are limited to stochastic AUC maximization with a linear predictive model, which restricts its predictive power when dealing with extremely complex data.

In this paper, we consider stochastic AUC maximization problem with a deep neural network as the predictive model.

Building on the saddle point reformulation of a surrogated loss of AUC, the problem can be cast into a {\it non-convex concave} min-max problem.

The main contribution made in this paper is to make stochastic AUC maximization more practical for deep neural networks and big data with theoretical insights as well.

In particular, we propose to explore Polyak-\L{}ojasiewicz (PL) condition that has been proved and observed in deep learning, which enables us to develop new stochastic algorithms with even faster convergence rate and more practical step size scheme.

An AdaGrad-style algorithm is also analyzed under the PL condition with adaptive convergence rate.

Our experimental results demonstrate the effectiveness of the proposed algorithms.

The paper designs two algorithms for the stochastic AUC maximization problem with state-of-the-art complexities when using deep neural network as predictive model, which are also verified by empirical studies.

Stochastic AUC Maximization with Deep Neural Networks

deep neural network
stochastic auc maximization


Designing rewards for Reinforcement Learning (RL) is challenging because it needs to convey the desired task, be efficient to optimize, and be easy to compute.

The latter is particularly problematic when applying RL to robotics, where detecting whether the desired configuration is reached might require considerable supervision and instrumentation.

Furthermore, we are often interested in being able to reach a wide range of configurations, hence setting up a different reward every time might be unpractical.

Methods like Hindsight Experience Replay (HER) have recently shown promise to learn policies able to reach many goals, without the need of a reward.

Unfortunately, without tricks like resetting to points along the trajectory, HER might take a very long time to discover how to reach certain areas of the state-space.

In this work we investigate different approaches to incorporate demonstrations to drastically speed up the convergence to a policy able to reach any goal, also surpassing the performance of an agent trained with other Imitation Learning algorithms.

Furthermore, our method can be used when only trajectories without expert actions are available, which can leverage kinestetic or third person demonstration.

We tackle goal-conditioned tasks by combining Hindsight Experience Replay and Imitation Learning algorithms, showing faster convergence than the first and higher final performance than the second.

Goal-conditioned Imitation Learning


Bayesian neural networks, which both use the negative log-likelihood loss function and average their predictions using a learned posterior over the parameters, have been used successfully across many scientific fields, partly due to their ability to `effortlessly' extract desired representations from many large-scale datasets.

However, generalization bounds for this setting is still missing.

In this paper, we present a new PAC-Bayesian generalization bound for the negative log-likelihood loss which utilizes the \emph{Herbst Argument} for the log-Sobolev inequality to bound the moment generating function of the learners risk.

We derive a new PAC-Bayesian Bound for unbounded loss functions (e.g. Negative Log-Likelihood).

PAC-Bayesian Neural Network Bounds


Data augmentation techniques, e.g., flipping or cropping, which systematically enlarge the training dataset by explicitly generating more training samples, are effective in improving the generalization performance of deep neural networks.

In the supervised setting, a common practice for data augmentation is to assign the same label to all augmented samples of the same source.

However, if the augmentation results in large distributional discrepancy among them (e.g., rotations), forcing their label invariance may be too difficult to solve and often hurts the performance.

To tackle this challenge, we suggest a simple yet effective idea of learning the joint distribution of the original and self-supervised labels of augmented samples.

The joint learning framework is easier to train, and enables an aggregated inference combining the predictions from different augmented samples for improving the performance.

Further, to speed up the aggregation process, we also propose a knowledge transfer technique, self-distillation, which transfers the knowledge of augmentation into the model itself.

We demonstrate the effectiveness of our data augmentation framework on various fully-supervised settings including the few-shot and imbalanced classification scenarios.

We propose a simple self-supervised data augmentation technique which improves performance of fully-supervised scenarios including few-shot learning and imbalanced classification.

Rethinking Data Augmentation: Self-Supervision and Self-Distillation


Long short-term memory (LSTM) networks allow to exhibit temporal dynamic behavior with feedback connections and seem a natural choice for learning sequences of 3D meshes.

We introduce an approach for dynamic mesh representations as used for numerical simulations of car crashes.

To bypass the complication of using 3D meshes, we transform the surface mesh sequences into spectral descriptors that efficiently encode the shape.

A two branch LSTM based network architecture is chosen to learn the representations and dynamics of the crash during the simulation.

The architecture is based on unsupervised video prediction by an LSTM without any convolutional layer.

It uses an encoder LSTM to map an input sequence into a fixed length vector representation.

On this representation one decoder LSTM performs the reconstruction of the input sequence, while the other decoder LSTM predicts the future behavior by receiving initial steps of the sequence as seed.

The spatio-temporal error behavior of the model is analysed to study how well the model can extrapolate the learned spectral descriptors into the future, that is, how well it has learned to represent the underlying dynamical structural mechanics.

Considering that only a few training examples are available, which is the typical case for numerical simulations, the network performs very well.

A two branch LSTM based network architecture learns the representation and dynamics of 3D meshes of numerical crash simulations.

Unsupervised Learning of Automotive 3D Crash Simulations using LSTMs