Hierarchical Task Networks (HTN) generate plans using a decomposition process guided by extra domain knowledge to guide search towards a planning task.

While many HTN planners can make calls to external processes (e.g. to a simulator interface) during the decomposition process, this is a computationally expensive process, so planner implementations often use such calls in an ad-hoc way using very specialized domain knowledge to limit the number of calls.

Conversely, the few classical planners that are capable of using external calls (often called semantic attachments) during planning do so in much more limited ways by generating a fixed number of ground operators at problem grounding time.

In this paper we develop the notion of semantic attachments for HTN planning using semi co-routines, allowing such procedurally defined predicates to link the planning process to custom unifications outside of the planner.

The resulting planner can then use such co-routines as part of its backtracking mechanism to search through parallel dimensions of the state-space (e.g. through numeric variables).

We show empirically that our planner outperforms the state-of-the-art numeric planners in a number of domains using minimal extra domain knowledge.

An approach to perform HTN planning using external procedures to evaluate predicates at runtime (semantic attachments).

HTN Planning with Semantic Attachments


Recent literature suggests that averaged word vectors followed by simple post-processing outperform many deep learning methods on semantic textual similarity tasks.

Furthermore, when averaged word vectors are trained supervised on large corpora of paraphrases, they achieve state-of-the-art results on standard STS benchmarks.

Inspired by these insights, we push the limits of word embeddings even further.

We propose a novel fuzzy bag-of-words (FBoW) representation for text that contains all the words in the vocabulary simultaneously but with different degrees of membership, which are derived from similarities between word vectors.

We show that max-pooled word vectors are only a special case of fuzzy BoW and should be compared via fuzzy Jaccard index rather than cosine similarity.

Finally, we propose DynaMax, a completely unsupervised and non-parametric similarity measure that dynamically extracts and max-pools good features depending on the sentence pair.

This method is both efficient and easy to implement, yet outperforms current baselines on STS tasks by a large margin and is even competitive with supervised word vectors trained to directly optimise cosine similarity.

Max-pooled word vectors with fuzzy Jaccard set similarity are an extremely competitive baseline for semantic similarity; we propose a simple dynamic variant that performs even better.

Don't Settle for Average, Go for the Max: Fuzzy Sets and Max-Pooled Word Vectors


State-of-the-art results in imitation learning are currently held by adversarial methods that iteratively estimate the divergence between student and expert policies and then minimize this divergence to bring the imitation policy closer to expert behavior.

Analogous techniques for imitation learning from observations alone (without expert action labels), however, have not enjoyed the same ubiquitous successes.

Recent work in adversarial methods for generative models has shown that the measure used to judge the discrepancy between real and synthetic samples is an algorithmic design choice, and that different choices can result in significant differences in model performance.

Choices including Wasserstein distance and various $f$-divergences have already been explored in the adversarial networks literature, while more recently the latter class has been investigated for imitation learning.

Unfortunately, we find that in practice this existing imitation-learning framework for using $f$-divergences suffers from numerical instabilities stemming from the combination of function approximation and policy-gradient reinforcement learning.

In this work, we alleviate these challenges and offer a reparameterization of adversarial imitation learning as $f$-divergence minimization before further extending the framework to handle the problem of imitation from observations only.

Empirically, we demonstrate that our design choices for coupling imitation learning and $f$-divergences are critical to recovering successful imitation policies.

Moreover, we find that with the appropriate choice of $f$-divergence, we can obtain imitation-from-observation algorithms that outperform baseline approaches and more closely match expert performance in continous-control tasks with low-dimensional observation spaces.

With high-dimensional observations, we still observe a significant gap with and without action labels, offering an interesting avenue for future work.

The overall goal of this work is to enable sample-efficient imitation from expert demonstrations, both with and without the provision of expert action labels, through the use of f-divergences.

Reparameterized Variational Divergence Minimization for Stable Imitation


Momentary fluctuations in attention (perceptual accuracy) correlate with neural activity fluctuations in primate visual areas.

Yet, the link between such momentary neural fluctuations and attention state remains to be shown in the human brain.

We investigate this link using a real-time cognitive brain machine interface (cBMI) based on steady state visually evoked potentials (SSVEPs): occipital EEG potentials evoked by rhythmically flashing stimuli.

Tracking momentary fluctuations in SSVEP power, in real-time, we presented stimuli time-locked to when this power reached (predetermined) high or low thresholds.

We observed a significant increase in discrimination accuracy (d') when stimuli were triggered during high (versus low) SSVEP power epochs, at the location cued for attention.

Our results indicate a direct link between attention’s effects on perceptual accuracy and and neural gain in EEG-SSVEP power, in the human brain.

With a cognitive brain-machine interface, we show a direct link between attentional effects on perceptual accuracy and neural gain in EEG-SSVEP power, in the human brain.

Tracking momentary attention fluctuations with an EEG-based cognitive brain-machine interface


Recent studies have demonstrated the vulnerability of deep convolutional neural networks against adversarial examples.

Inspired by the observation that the intrinsic dimension of image data is much smaller than its pixel space dimension and the vulnerability of neural networks grows with the input dimension, we propose to embed high-dimensional input images into a low-dimensional space to perform classification.

However, arbitrarily projecting the input images to a low-dimensional space without regularization will not improve the robustness of deep neural networks.

We propose a new framework, Embedding Regularized Classifier (ER-Classifier), which improves the adversarial robustness of the classifier through embedding regularization.

Experimental results on several benchmark datasets show that, our proposed framework achieves state-of-the-art performance against strong adversarial attack methods.

A general and easy-to-use framework that improves the adversarial robustness of deep classification models through embedding regularization.

Defending Against Adversarial Examples by Regularized Deep Embedding


We investigate task clustering for deep learning-based multi-task and few-shot learning in the settings with large numbers of diverse tasks.

Our method measures task similarities using cross-task transfer performance matrix.

Although this matrix provides us critical information regarding similarities between tasks, the uncertain task-pairs, i.e., the ones with extremely asymmetric transfer scores, may collectively mislead clustering algorithms to output an inaccurate task-partition.

Moreover, when the number of tasks is large, generating the full transfer performance matrix can be very time consuming.

To overcome these limitations, we propose a novel task clustering algorithm to estimate the similarity matrix based on the theory of matrix completion.

The proposed algorithm can work on partially-observed similarity matrices based on only sampled task-pairs with reliable scores, ensuring its efficiency and robustness.

Our theoretical analysis shows that under mild assumptions, the reconstructed matrix perfectly matches the underlying “true” similarity matrix with an overwhelming probability.

The final task partition is computed by applying an efficient spectral clustering algorithm to the recovered matrix.

Our results show that the new task clustering method can discover task clusters that benefit both multi-task learning and few-shot learning setups for sentiment classification and dialog intent classification tasks.

We propose a matrix-completion based task clustering algorithm for deep multi-task and few-shot learning in the settings with large numbers of diverse tasks.

Robust Task Clustering for Deep and Diverse Multi-Task and Few-Shot Learning

multi-task few-shot learning
diverse
task cluster
deep


The resemblance between the methods used in studying quantum-many body physics and in machine learning has drawn considerable attention.

In particular, tensor networks (TNs) and deep learning architectures bear striking similarities to the extent that TNs can be used for machine learning.

Previous results used one-dimensional TNs in image recognition, showing limited scalability and a request of high bond dimension.

In this work, we train two-dimensional hierarchical TNs to solve image recognition problems, using a training algorithm derived from the multipartite entanglement renormalization ansatz (MERA).

This approach overcomes scalability issues and implies novel mathematical connections among quantum many-body physics, quantum information theory, and machine learning.

While keeping the TN unitary in the training phase, TN states can be defined, which optimally encodes each class of the images into a quantum many-body state.

We study the quantum features of the TN states, including quantum entanglement and fidelity.

We suggest these quantities could be novel properties that characterize the image classes, as well as the machine learning tasks.

Our work could be further applied to identifying possible quantum properties of certain artificial intelligence methods.

Machine Learning by Two-Dimensional Hierarchical Tensor Networks: A Quantum Information Theoretic Perspective on Deep Architectures


Predicting outcomes and planning interactions with the physical world are long-standing goals for machine learning.

A variety of such tasks involves continuous physical systems, which can be described by partial differential equations (PDEs) with many degrees of freedom.

Existing methods that aim to control the dynamics of such systems are typically limited to relatively short time frames or a small number of interaction parameters.

We present a novel hierarchical predictor-corrector scheme which enables neural networks to learn to understand and control complex nonlinear physical systems over long time frames.

We propose to split the problem into two distinct tasks: planning and control.

To this end, we introduce a predictor network that plans optimal trajectories and a control network that infers the corresponding control parameters.

Both stages are trained end-to-end using a differentiable PDE solver.

We demonstrate that our method successfully develops an understanding of complex physical systems and learns to control them for tasks involving PDEs such as the incompressible Navier-Stokes equations.

We train a combination of neural networks to predict optimal trajectories for complex physical systems.

Learning to Control PDEs with Differentiable Physics


The ability of overparameterized deep networks to generalize well has been linked to the fact that stochastic gradient descent (SGD) finds solutions that lie in flat, wide minima in the training loss -- minima where the output of the network is resilient to small random noise added to its parameters.

So far this observation has been used to provide generalization guarantees only for neural networks whose parameters are either \textit{stochastic} or \textit{compressed}.

In this work, we present a general PAC-Bayesian framework that leverages this observation to provide a bound on the original network learned -- a network that is deterministic and uncompressed.

What enables us to do this is a key novelty in our approach: our framework allows us to show that if on training data, the interactions between the weight matrices satisfy certain conditions that imply a wide training loss minimum, these conditions themselves {\em generalize} to the interactions between the matrices on test data, thereby implying a wide test loss minimum.

We then apply our general framework in a setup where we assume that the pre-activation values of the network are not too small (although we assume this only on the training data).

In this setup, we provide a generalization guarantee for the original (deterministic, uncompressed) network, that does not scale with product of the spectral norms of the weight matrices -- a guarantee that would not have been possible with prior approaches.

We provide a PAC-Bayes based generalization guarantee for uncompressed, deterministic deep networks by generalizing noise-resilience of the network on the training data to the test data.

Deterministic PAC-Bayesian generalization bounds for deep networks via generalizing noise-resilience

deep network
generalize noise-resilience
deterministic
generalization


Training neural networks to be certifiably robust is critical to ensure their safety against adversarial attacks.

However, it is currently very difficult to train a neural network that is both accurate and certifiably robust.

In this work we take a step towards addressing this challenge.

We prove that for every continuous function $f$, there exists a network $n$ such that:

(i) $n$ approximates $f$ arbitrarily close, and

(ii) simple interval bound propagation of a region $B$ through $n$ yields a result that is arbitrarily close to the optimal output of $f$ on $B$.

Our result can be seen as a Universal Approximation Theorem for interval-certified ReLU networks.

To the best of our knowledge, this is the first work to prove the existence of accurate, interval-certified networks.

We prove that for a large class of functions f there exists an interval certified robust network approximating f up to arbitrary precision.

Universal Approximation with Certified Networks


In this paper, we propose an efficient framework to accelerate convolutional neural networks.

We utilize two types of acceleration methods: pruning and hints.

Pruning can reduce model size by removing channels of layers.

Hints can improve the performance of student model by transferring knowledge from teacher model.

We demonstrate that pruning and hints are complementary to each other.

On one hand, hints can benefit pruning by maintaining similar feature representations.

On the other hand, the model pruned from teacher networks is a good initialization for student model, which increases the transferability between two networks.

Our approach performs pruning stage and hints stage iteratively to further improve the

Furthermore, we propose an algorithm to reconstruct the parameters of hints layer and make the pruned model more suitable for hints.

Experiments were conducted on various tasks including classification and pose estimation.

Results on CIFAR-10, ImageNet and COCO demonstrate the generalization and superiority of our framework.

This is a work aiming for boosting all the existing pruning and mimic method.

PRUNING WITH HINTS: AN EFFICIENT FRAMEWORK FOR MODEL ACCELERATION


For typical sequence prediction problems such as language generation, maximum likelihood estimation (MLE) has commonly been adopted as it encourages the predicted sequence most consistent with the ground-truth sequence to have the highest probability of occurring.

However, MLE focuses on once-to-all matching between the predicted sequence and gold-standard, consequently treating all incorrect predictions as being equally incorrect.

We refer to this drawback as {\it negative diversity ignorance} in this paper.

Treating all incorrect predictions as equal unfairly downplays the nuance of these sequences' detailed token-wise structure.

To counteract this, we augment the MLE loss by introducing an extra Kullback--Leibler divergence term derived by comparing a data-dependent Gaussian prior and the detailed training prediction.

The proposed data-dependent Gaussian prior objective (D2GPo) is defined over a prior topological order of tokens and is poles apart from the data-independent Gaussian prior (L2 regularization) commonly adopted in smoothing the training of MLE.

Experimental results show that the proposed method makes effective use of a more detailed prior in the data and has improved performance in typical language generation tasks, including supervised and unsupervised machine translation, text summarization, storytelling, and image captioning.

We introduce an extra data-dependent Gaussian prior objective to augment the current MLE training, which is designed to capture the prior knowledge in the ground-truth data.

Data-dependent Gaussian Prior Objective for Language Generation


We  propose an  interactive  classification approach  for natural  language  queries.

Instead of classifying given the natural language query only, we ask the user for additional information using a sequence of binary and multiple-choice questions.

At each turn, we use a policy controller to decide if to present a question or pro-vide the user the final answer, and select the best question to ask by maximizing the system information gain.

Our formulation enables bootstrapping the system without any interaction data, instead relying on non-interactive crowdsourcing an-notation tasks.

Our evaluation shows the interaction helps the system increase its accuracy and handle ambiguous queries, while our approach effectively balances the number of questions and the final accuracy.

We propose an interactive approach for classifying natural language queries by asking users for additional information using information gain and a reinforcement learning policy controller.

Interactive Classification by Asking Informative Questions


Convolutional neural networks (CNNs) have achieved state of the art performance on recognizing and representing audio, images, videos and 3D volumes; that is, domains where the input can be characterized by a regular graph structure.

However, generalizing CNNs to irregular domains like 3D meshes is challenging.

Additionally, training data for 3D meshes is often limited.

In this work, we generalize convolutional autoencoders to mesh surfaces.

We perform spectral decomposition of meshes and apply convolutions directly in frequency space.

In addition, we use max pooling and introduce upsampling within the network to represent meshes in a low dimensional space.

We construct a complex dataset of 20,466 high resolution meshes with extreme facial expressions and encode it using our Convolutional Mesh Autoencoder.

Despite limited training data, our method outperforms state-of-the-art PCA models of faces with 50% lower error,  while using 75% fewer parameters.

Convolutional autoencoders generalized to mesh surfaces for encoding and reconstructing extreme 3D facial expressions.

Convolutional Mesh Autoencoders for 3D Face Representation


Computing distances between examples is at the core of many learning algorithms for time series.

Consequently, a great deal of work has gone into designing effective time series distance measures.

We present Jiffy, a simple and scalable distance metric for multivariate time series.

Our approach is to reframe the task as a representation learning problem---rather than design an elaborate distance function, we use a CNN to learn an embedding such that the Euclidean distance is effective.

By aggressively max-pooling and downsampling, we are able to construct this embedding using a highly compact neural network.

Experiments on a diverse set of multivariate time series datasets show that our approach consistently outperforms existing methods.

Jiffy is a convolutional approach to learning a distance metric  for multivariate time series that outperforms existing methods in terms of nearest-neighbor classification accuracy.

Jiffy: A Convolutional Approach to Learning Time Series Similarity

jiffy convolutional approach learn
time series


Prefrontal cortex (PFC) is a part of the brain which is responsible for behavior repertoire.

Inspired by PFC functionality and connectivity,  as well as human behavior formation process, we propose a novel modular architecture of neural networks with a Behavioral Module (BM) and corresponding end-to-end training strategy.

This approach allows the efficient learning of behaviors and preferences representation.

This property is particularly useful for user modeling (as for dialog agents) and recommendation tasks, as allows learning personalized representations of different user states.

In the experiment with video games playing, the resultsshow that the proposed method allows separation of main task’s objectives andbehaviors between different BMs.

The experiments also show network extendability through independent learning of new behavior patterns.

Moreover, we demonstrate a strategy for an efficient transfer of newly learned BMs to unseen tasks.

Extendable Modular Architecture is proposed for developing of variety of Agent Behaviors in DQN.

BEHAVIOR MODULE IN NEURAL NETWORKS


A major component of overfitting in model-free reinforcement learning (RL) involves the case where the agent may mistakenly correlate reward with certain spurious features from the observations generated by the Markov Decision Process (MDP).

We provide a general framework for analyzing this scenario, which we use to design multiple synthetic benchmarks from only modifying the observation space of an MDP.

When an agent overfits to different observation spaces even if the underlying MDP dynamics is fixed, we term this observational overfitting.

Our experiments expose intriguing properties especially with regards to implicit regularization, and also corroborate results from previous works in RL generalization and supervised learning (SL).

We isolate one factor of RL generalization by analyzing the case when the agent only overfits to the observations. We show that architectural implicit regularizations occur in this regime.

Observational Overfitting in Reinforcement Learning


We propose a neural language model capable of unsupervised syntactic structure induction.

The model leverages the structure information to form better semantic representations and better language modeling.

Standard recurrent neural networks are limited by their structure and fail to efficiently use syntactic information.

On the other hand, tree-structured recursive networks usually require additional structural supervision at the cost of human expert annotation.

In this paper, We propose a novel neural language model, called the Parsing-Reading-Predict Networks (PRPN), that can simultaneously induce the syntactic structure from unannotated sentences and leverage the inferred structure to learn a better language model.

In our model, the gradient can be directly back-propagated from the language model loss into the neural parsing network.

Experiments show that the proposed model can discover the underlying syntactic structure and achieve state-of-the-art performance on word/character-level language model tasks.

Neural Language Modeling by Jointly Learning Syntax and Lexicon


Unsupervised embedding learning aims to extract good representations from data without the use of human-annotated labels.

Such techniques are apparently in the limelight because of the challenges in collecting massive-scale labels required for supervised learning.

This paper proposes a comprehensive approach, called Super-AND, which is based on the Anchor Neighbourhood Discovery model.

Multiple losses defined in Super-AND make similar samples gather even within a low-density space and keep features invariant against augmentation.

As a result, our model outperforms existing approaches in various benchmark datasets and achieves an accuracy of 89.2% in CIFAR-10 with the Resnet18 backbone network, a 2.9% gain over the state-of-the-art.

We proposed a comprehensive approach for unsupervised embedding learning on the basis of AND algorithm.

Super-AND: A Holistic Approach to Unsupervised Embedding Learning


Analysis of histopathology slides is a critical step for many diagnoses, and in particular in oncology where it defines the gold standard.

In the case of digital histopathological analysis, highly trained pathologists must review vast whole-slide-images of extreme digital resolution (100,000^2 pixels) across multiple zoom levels in order to locate abnormal regions of cells, or in some cases single cells, out of millions.

The application of deep learning to this problem is hampered not only by small sample sizes, as typical datasets contain only a few hundred samples, but also by the generation of ground-truth localized annotations for training interpretable classification and segmentation models.

We propose a method for disease available during training.

Even without pixel-level annotations, we are able to demonstrate performance comparable with models trained with strong annotations on the Camelyon-16 lymph node metastases detection challenge.

We accomplish this through the use of pre-trained deep convolutional networks, feature embedding, as well as learning via top instances and negative evidence, a multiple instance learning technique fromatp the field of semantic segmentation and object detection.

We propose a weakly supervised learning method for the classification and localization of cancers in extremely high resolution histopathology whole slide images using only image-wide labels.

Classification and Disease Localization in Histopathology Using Only Global Labels: A Weakly-Supervised Approach

label classification localization histopathology


Massively multi-label prediction/classification problems arise in environments like health-care or biology where it is useful to make very precise predictions.

One challenge with massively multi-label problems is that there is often a long-tailed frequency distribution for the labels, resulting in few positive examples for the rare labels.

We propose a solution to this problem by modifying the output layer of a neural network to create a Bayesian network of sigmoids which takes advantage of ontology relationships between the labels to help share information between the rare and the more common labels.

We apply this method to the two massively multi-label tasks of disease prediction (ICD-9 codes) and protein function prediction (Gene Ontology terms) and obtain significant improvements in per-label AUROC and average precision.

We propose a new method for using ontology information to improve performance on massively multi-label prediction/classification problems.

Using Ontologies To Improve Performance In Massively Multi-label Prediction

improve performance massively multi-label prediction


Generative Adversarial Networks have made data generation possible in various use cases, but in case of complex, high-dimensional distributions it can be difficult to train them, because of convergence problems and the appearance of mode collapse.

Sliced Wasserstein GANs and especially the application of the Max-Sliced Wasserstein distance made it possible to approximate Wasserstein distance during training in an efficient and stable way and helped ease convergence problems of these architectures.

This method transforms sample assignment and distance calculation into sorting the one-dimensional projection of the samples, which results a sufficient approximation of the high-dimensional Wasserstein distance.

In this paper we will demonstrate that the approximation of the Wasserstein distance by sorting the samples is not always the optimal approach and the greedy assignment of the real and fake samples can result faster convergence and better approximation of the original distribution.

We apply a greedy assignment on the projected samples instead of sorting to approximate Wasserstein distance

A Greedy Approach to Max-Sliced Wasserstein GANs


Lifelong machine learning focuses on adapting to novel tasks without forgetting the old tasks, whereas few-shot learning strives to learn a single task given a small amount of data.

These two different research areas are crucial for artificial general intelligence, however, their existing studies have somehow assumed some impractical settings when training the models.

For lifelong learning, the nature (or the quantity) of incoming tasks during inference time is assumed to be known at training time.

As for few-shot learning, it is commonly assumed that a large number of tasks is available during training.

Humans, on the other hand, can perform these learning tasks without regard to the aforementioned assumptions.

Inspired by how the human brain works, we propose a novel model, called the Slow Thinking to Learn (STL), that makes sophisticated (and slightly slower) predictions by iteratively considering interactions between current and previously seen tasks at runtime.

Having conducted experiments, the results empirically demonstrate the effectiveness of STL for more realistic lifelong and few-shot learning settings.

This paper studies the interactions between the fast-learning and slow-prediction models and demonstrate how such interactions can improve machine capability to solve the joint lifelong and few-shot learning problems.

Slow Thinking Enables Task-Uncertain Lifelong and Sequential Few-Shot Learning


Hypernetworks are meta neural networks that generate weights for a main neural network in an end-to-end differentiable manner.

Despite extensive applications ranging from multi-task learning to Bayesian deep learning, the problem of optimizing hypernetworks has not been studied to date.

We observe that classical weight initialization methods like Glorot & Bengio (2010) and He et al. (2015), when applied directly on a hypernet, fail to produce weights for the mainnet in the correct scale.

We develop principled techniques for weight initialization in hypernets, and show that they lead to more stable mainnet weights, lower training loss, and faster convergence.

The first principled weight initialization method for hypernetworks

Principled Weight Initialization for Hypernetworks


For bidirectional joint image-text modeling, we develop variational hetero-encoder (VHE) randomized generative adversarial network (GAN), a versatile deep generative model that integrates a probabilistic text decoder, probabilistic image encoder, and GAN into a coherent end-to-end multi-modality learning framework.

VHE randomized GAN (VHE-GAN) encodes an image to decode its associated text, and feeds the variational posterior as the source of randomness into the GAN image generator.

We plug three off-the-shelf modules, including a deep topic model, a ladder-structured image encoder, and StackGAN++, into VHE-GAN, which already achieves competitive performance.

This further motivates the development of VHE-raster-scan-GAN that generates photo-realistic images in not only a multi-scale low-to-high-resolution manner, but also a hierarchical-semantic coarse-to-fine fashion.

By capturing and relating hierarchical semantic and visual concepts with end-to-end training, VHE-raster-scan-GAN achieves state-of-the-art performance in a wide variety of image-text multi-modality learning and generation tasks.

A novel Bayesian deep learning framework that captures and relates hierarchical semantic and visual concepts, performing well on a variety of image and text modeling and generation tasks.

Variational Hetero-Encoder Randomized GANs for Joint Image-Text Modeling


Current classical planners are very successful in finding (non-optimal) plans, even for large planning instances.

To do so, most planners rely on a preprocessing stage that computes a grounded representation of the task.

Whenever the grounded task is too big to be generated (i.e., whenever this preprocess fails) the instance cannot even be tackled by the actual planner.

To address this issue, we introduce a partial grounding approach that grounds only a projection of the task, when complete grounding is not feasible.

We propose a guiding mechanism that, for a given domain, identifies the parts of a task that are relevant to find a plan by using off-the-shelf machine learning methods.

Our empirical evaluation attests that the approach is capable of solving planning instances that are too big to be fully grounded.

This paper introduces partial grounding to tackle the problem that arises when the full grounding process, i.e., the translation of a PDDL input task into a ground representation like STRIPS, is infeasible due to memory or time constraints.

Learning How to Ground a Plan - Partial Grounding in Classical Planning


In this paper, we introduce a novel method to interpret recurrent neural networks (RNNs), particularly long short-term memory networks (LSTMs) at the cellular level.

We propose a systematic pipeline for interpreting individual hidden state dynamics within the network using response characterization methods.

The ranked contribution of individual cells to the network's output is computed by analyzing a set of interpretable metrics of their decoupled step and sinusoidal responses.

As a result, our method is able to uniquely identify neurons with insightful dynamics, quantify relationships between dynamical properties and test accuracy through ablation analysis, and interpret the impact of network capacity on a network's dynamical distribution.

Finally, we demonstrate generalizability and scalability of our method by evaluating a series of different benchmark sequential datasets.

Introducing the response charactrization method for interpreting cell dynamics in learned long short-term memory (LSTM) networks.

Response Characterization for Auditing Cell Dynamics in Long Short-term Memory Networks

long short-term memory
cell dynamic
response
network


Decades of research on the neural code underlying spatial navigation have revealed a diverse set of neural response properties.

The Entorhinal Cortex (EC) of the mammalian brain contains a rich set of spatial correlates, including grid cells which encode space using tessellating patterns.

However, the mechanisms and functional significance of these spatial representations remain largely mysterious.

As a new way to understand these neural representations, we trained recurrent neural networks (RNNs) to perform navigation tasks in 2D arenas based on velocity inputs.

Surprisingly, we find that grid-like spatial response patterns emerge in trained networks, along with units that exhibit other spatial correlates, including border cells and band-like cells.

All these different functional types of neurons have been observed experimentally.

The order of the emergence of grid-like and border cells is also consistent with observations from developmental studies.

Together, our results suggest that grid cells, border cells and others as observed in EC may be a natural solution for representing space efficiently given the predominant recurrent connections in the neural circuits.

To our knowledge, this is the first study to show how neural representations of space, including grid-like cells and border cells as observed in the brain, could emerge from training a recurrent neural network to perform navigation tasks.

Emergence of grid-like representations by training recurrent neural networks to perform spatial localization

train recurrent neural network perform
representation
grid-like


Source separation for music is the task of isolating contributions, or stems, from different instruments recorded individually and arranged together to form a song.

Such components include voice, bass, drums and any other accompaniments.

While end-to-end models that directly generate the waveform are state-of-the-art in many audio synthesis problems, the best multi-instrument source separation models generate masks on the magnitude spectrum and achieve performances far above current end-to-end, waveform-to-waveform models.

We present an in-depth analysis of a new architecture, which we will refer to as Demucs, based on a (transposed) convolutional autoencoder, with a bidirectional LSTM at the bottleneck layer and skip-connections as in U-Networks (Ronneberger et al., 2015).

Compared to the state-of-the-art waveform-to-waveform model, Wave-U-Net (Stoller et al., 2018), the main features of our approach in addition of the bi-LSTM are the use of trans-posed convolution layers instead of upsampling-convolution blocks, the use of gated linear units, exponentially growing the number of channels with depth and a new careful initialization of the weights.

Results on the MusDB dataset show that our architecture achieves a signal-to-distortion ratio (SDR) nearly 2.2 points higher than the best waveform-to-waveform competitor (from 3.2 to 5.4 SDR).

This makes our model match the state-of-the-art performances on this dataset, bridging the performance gap between models that operate on the spectrogram and end-to-end approaches.

We match the performance of spectrogram based model with a model trained end-to-end in the waveform domain

Music Source Separation in the Waveform Domain


Although challenging, strategy profile evaluation in large connected learner networks is crucial for enabling the next wave of machine learning applications.

Recently, $\alpha$-Rank, an evolutionary algorithm, has been proposed as a solution for ranking joint policy profiles in multi-agent systems.

$\alpha$-Rank claimed scalability through a polynomial time implementation with respect to the total number of pure strategy profiles.

In this paper, we formally prove that such a claim is not grounded.

In fact, we show that $\alpha$-Rank exhibits an exponential complexity in number of agents, hindering its application beyond a small finite number of joint profiles.

Realizing such a limitation, we contribute by proposing a scalable evaluation protocol that we title  $\alpha^{\alpha}$-Rank.

Our method combines evolutionary dynamics with stochastic optimization and double oracles for \emph{truly} scalable ranking with linear (in number of agents) time and memory complexities.

Our contributions allow us, for the first time, to conduct large-scale evaluation experiments of multi-agent systems, where we show successful results on large joint strategy profiles with sizes in the  order of $\mathcal{O}(2^{25})$ (i.e., $\approx \text{$33$ million strategies}$) -- a setting not evaluable using current techniques.

We provide a scalable solution to multi-agent evaluation with linear rate complexity in both time and memory in terms of number of agents

$\alpha^{\alpha}$-Rank: Scalable Multi-agent Evaluation through Evolution


Deep neural networks are known to be annotation-hungry.

Numerous efforts have been devoted to reducing the annotation cost when learning with deep networks.

Two prominent directions include learning with noisy labels and semi-supervised learning by exploiting unlabeled data.

In this work, we propose DivideMix, a novel framework for learning with noisy labels by leveraging semi-supervised learning techniques.

In particular, DivideMix models the per-sample loss distribution with a mixture model to dynamically divide the training data into a labeled set with clean samples and an unlabeled set with noisy samples, and trains the model on both the labeled and unlabeled data in a semi-supervised manner.

To avoid confirmation bias, we simultaneously train two diverged networks where each network uses the dataset division from the other network.

During the semi-supervised training phase, we improve the MixMatch strategy by performing label co-refinement and label co-guessing on labeled and unlabeled samples, respectively.

Experiments on multiple benchmark datasets demonstrate substantial improvements over state-of-the-art methods.

Code is available at https://github.com/LiJunnan1992/DivideMix .

We propose a novel semi-supervised learning approach with SOTA performance on combating learning with noisy labels.

DivideMix: Learning with Noisy Labels as Semi-supervised Learning

learn noisy label
semi-supervised learning


We present a new algorithm to train a robust neural network against adversarial attacks.

Our algorithm is motivated by the following two ideas.

First, although recent work has demonstrated that fusing randomness can improve the robustness of neural networks (Liu 2017), we noticed that adding noise blindly to all the layers is not the optimal way to incorporate randomness.

Instead, we model randomness under the framework of Bayesian Neural Network (BNN) to formally learn the posterior distribution of models in a scalable way.

Second, we formulate the mini-max problem in BNN to learn the best model distribution under adversarial attacks, leading to an adversarial-trained Bayesian neural net.

Experiment results demonstrate that the proposed algorithm achieves state-of-the-art performance under strong attacks.

On CIFAR-10 with VGG network, our model leads to 14% accuracy improvement compared with adversarial training (Madry 2017) and random self-ensemble (Liu, 2017) under PGD attack with 0.035 distortion, and the gap becomes even larger on a subset of ImageNet.

We design an adversarial training method to Bayesian neural networks, showing a much stronger defense to white-box adversarial attacks

Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network

bayesian neural network
defense
adversarial


Federated learning distributes model training among a multitude of agents, who, guided by privacy concerns, perform training using their local data but share only model parameter updates, for iterative aggregation at the server.

In this work, we explore the threat of model poisoning attacks on federated learning initiated by a single, non-colluding malicious agent where the adversarial objective is to cause the model to misclassify a set of chosen inputs with high confidence.

We explore a number of strategies to carry out this attack, starting with simple boosting of the malicious agent's update to overcome the effects of other agents' updates.

To increase attack stealth, we propose an alternating minimization strategy, which alternately optimizes for the training loss and the adversarial objective.

We follow up by using parameter estimation for the benign agents' updates to improve on attack success.

Finally, we use a suite of interpretability techniques to generate visual explanations of model decisions for both benign and malicious models and show that the explanations are nearly visually indistinguishable.

Our results indicate that even a highly constrained adversary can carry out model poisoning attacks while simultaneously maintaining stealth, thus highlighting the vulnerability of the federated learning setting and the need to develop effective defense strategies.

Effective model poisoning attacks on federated learning able to cause high-confidence targeted misclassification of desired inputs

Analyzing Federated Learning through an Adversarial Lens


Despite rapid advances in speech recognition, current models remain brittle to superficial perturbations to their inputs.

Small amounts of noise can destroy the performance of an otherwise state-of-the-art model.

To harden models against background noise, practitioners often perform data augmentation, adding artificially-noised examples to the training set, carrying over the original label.

In this paper, we hypothesize that a clean example and its superficially perturbed counterparts shouldn't merely map to the same class--- they should map to the same representation.

We propose invariant-representation-learning (IRL): At each training iteration, for each training example, we sample a noisy counterpart.

We then apply a penalty term to coerce matched representations at each layer (above some chosen layer).

Our key results, demonstrated on the LibriSpeech dataset are the following:

(i) IRL significantly reduces character error rates (CER)on both `clean' (3.3% vs 6.5%) and `other' (11.0% vs 18.1%) test sets;

(ii) on several out-of-domain noise settings (different from those seen during training), IRL's benefits are even more pronounced.

Careful ablations confirm that our results are not simply due to shrinking activations at the chosen layers.

In this paper, we hypothesize that superficially perturbed data points shouldn’t merely map to the same class---they should map to the same representation.

Learning Noise-Invariant Representations for Robust Speech Recognition


In this paper we design a harmonic acoustic model for pitch detection.

This model arranges conventional convolution and sparse convolution in a way such that the global harmonic patterns captured by sparse convolution are composed of the enough number of local patterns captured by layers of conventional convolution.

When trained on the MAPS dataset, the harmonic model outperforms all existing pitch detection systems trained on the same dataset.

Most impressively, when trained on MAPS with simple data augmentation, the harmonic model with an LSTM layer on top surpasses an up-to-date, more complex pitch detection system trained on the MAESTRO dataset to which complicated data augmentation is applied and whose training split is an order-of-magnitude larger than the training split of MAPS.

The harmonic model has demonstrated potential to be used for advanced automatic music transcription (AMT) systems.

A Harmonic Structure-Based Neural Network Model for Musical Pitch Detection


Learning domain-invariant representation is a dominant approach for domain generalization.

However, previous methods based on domain invariance overlooked the underlying dependency of classes on domains, which is responsible for the trade-off between classification accuracy and the invariance.

This study proposes a novel method {\em adversarial feature learning under accuracy constraint (AFLAC)}, which maximizes domain invariance within a range that does not interfere with accuracy.

Empirical validations show that the performance of AFLAC is superior to that of baseline methods, supporting the importance of considering the dependency and the efficacy of the proposed method to overcome the problem.

Address the trade-off caused by the dependency of classes on domains by improving domain adversarial nets

Adversarial Feature Learning under Accuracy Constraint for Domain Generalization


Recent advances in deep generative models have lead to remarkable progress in synthesizing high quality images.

Following their successful application in image processing and representation learning, an important next step is to consider videos.

Learning generative models of video is a much harder task, requiring a model to capture the temporal dynamics of a scene, in addition to the visual presentation of objects.

While recent generative models of video have had some success, current progress is hampered by the lack of qualitative metrics that consider visual quality, temporal coherence, and diversity of samples.

To this extent we propose Fréchet Video Distance (FVD), a new metric for generative models of video based on FID.

We contribute a large-scale human study, which confirms that FVD correlates well with qualitative human judgment of generated videos.

We propose FVD: a new metric for generative models of video based on FID. A large-scale human study confirms that FVD correlates well with qualitative human judgment of generated videos.

FVD: A new Metric for Video Generation


Despite advances in deep learning, artificial neural networks do not learn the same way as humans do.

Today, neural networks can learn multiple tasks when trained on them jointly, but cannot maintain performance on learnt tasks when tasks are presented one at a time -- this phenomenon called catastrophic forgetting is a fundamental challenge to overcome before neural networks can learn continually from incoming data.

In this work, we derive inspiration from human memory to develop an architecture capable of learning continuously from sequentially incoming tasks, while averting catastrophic forgetting.

Specifically, our model consists of a dual memory architecture to emulate the complementary learning systems (hippocampus and the neocortex) in the human brain and maintains a consolidated long-term memory via generative replay of past experiences.

(i) substantiate our claim that replay should be generative,

(ii) show the benefits of generative replay and dual memory via experiments, and

(iii) demonstrate improved performance retention even for small models with low capacity.

Our architecture displays many important characteristics of the human memory and provides insights on the connection between sleep and learning in humans.

A dual memory architecture inspired from human brain to learn sequentially incoming tasks, while averting catastrophic forgetting.

Deep Generative Dual Memory Network for Continual Learning


Most research on lifelong learning applies to images or games, but not language.

We present LAMOL, a simple yet effective method for lifelong language learning (LLL) based on language modeling.

LAMOL replays pseudo-samples of previous tasks while requiring no extra memory or model capacity.

Specifically, LAMOL is a language model that simultaneously learns to solve the tasks and generate training samples.

When the model is trained for a new task, it generates pseudo-samples of previous tasks for training alongside data for the new task.

The results show that LAMOL prevents catastrophic forgetting without any sign of intransigence and can perform five very different language tasks sequentially with only one model.

Overall, LAMOL outperforms previous methods by a considerable margin and is only 2-3% worse than multitasking, which is usually considered the LLL upper bound.

The source code is available at https://github.com/jojotenya/LAMOL.

Language modeling for lifelong language learning.

LAMOL: LAnguage MOdeling for Lifelong Language Learning


Deep learning natural language processing models often use vector word embeddings, such as word2vec or GloVe, to represent words.

A discrete sequence of words can be much more easily integrated with downstream neural layers if it is represented as a  sequence of continuous vectors.

Also, semantic relationships between words, learned from a text corpus, can be encoded in the relative configurations of the embedding vectors.

However, storing and accessing embedding vectors for all words in a dictionary requires large amount of space, and may stain systems with limited GPU memory.

Here, we used approaches inspired by quantum computing to propose two related methods, word2ket and word2ketXS, for storing word embedding matrix during training and inference in a highly efficient way.

Our approach achieves a hundred-fold or more reduction in the space required to store the embeddings with almost no relative drop in accuracy in practical natural language processing tasks.

We use ideas from quantum computing to proposed word embeddings that utilize much fewer trainable parameters.

word2ket: Space-efficient Word Embeddings inspired by Quantum Entanglement


One of the distinguishing aspects of human language is its compositionality, which allows us to describe complex environments with limited vocabulary.

Previously, it has been shown that neural network agents can learn to communicate in a highly structured, possibly compositional language based on disentangled input (e.g. hand- engineered features).

Humans, however, do not learn to communicate based on well-summarized features.

In this work, we train neural agents to simultaneously develop visual perception from raw image pixels, and learn to communicate with a sequence of discrete symbols.

The agents play an image description game where the image contains factors such as colors and shapes.

We train the agents using the obverter technique where an agent introspects to generate messages that maximize its own understanding.

Through qualitative analysis, visualization and a zero-shot test, we show that the agents can develop, out of raw image pixels, a language with compositional properties, given a proper pressure from the environment.

We train neural network agents to develop a language with compositional properties from raw pixel input.

Compositional Obverter Communication Learning from Raw Visual Input


The ability to forecast a set of likely yet diverse possible future behaviors of an agent (e.g., future trajectories of a pedestrian) is essential for safety-critical perception systems (e.g., autonomous vehicles).

In particular, a set of possible future behaviors generated by the system must be diverse to account for all possible outcomes in order to take necessary safety precautions.

It is not sufficient to maintain a set of the most likely future outcomes because the set may only contain perturbations of a dominating single outcome (major mode).

While generative models such as variational autoencoders (VAEs) have been shown to be a powerful tool for learning a distribution over future trajectories, randomly drawn samples from the learned implicit likelihood model may not be diverse -- the likelihood model is derived from the training data distribution and the samples will concentrate around the major mode of the data.

In this work, we propose to learn a diversity sampling function (DSF) that generates a diverse yet likely set of future trajectories.

The DSF maps forecasting context features to a set of latent codes which can be decoded by a generative model (e.g., VAE) into a set of diverse trajectory samples.

Concretely, the process of identifying the diverse set of samples is posed as DSF parameter estimation.

To learn the parameters of the DSF, the diversity of the trajectory samples is evaluated by a diversity loss based on a determinantal point process (DPP).

Gradient descent is performed over the DSF parameters, which in turn moves the latent codes of the sample set to find an optimal set of diverse yet likely trajectories.

Our method is a novel application of DPPs to optimize a set of items (forecasted trajectories) in continuous space.

We demonstrate the diversity of the trajectories produced by our approach on both low-dimensional 2D trajectory data and high-dimensional human motion data.

We learn a diversity sampling function with DPPs to obtain a diverse set of samples from a generative model.

Diverse Trajectory Forecasting with Determinantal Point Processes


There is mounting evidence that pretraining can be valuable for neural network language understanding models, but we do not yet have a clear understanding of how the choice of pretraining objective affects the type of linguistic information that models learn.

With this in mind, we compare four objectives---language modeling, translation, skip-thought, and autoencoding---on their ability to induce syntactic and part-of-speech information, holding constant the genre and quantity of training data.

We find that representations from language models consistently perform best on our syntactic auxiliary prediction tasks, even when trained on relatively small amounts of data, which suggests that language modeling may be the best data-rich pretraining task for transfer learning applications requiring syntactic information.

We also find that a randomly-initialized, frozen model can perform strikingly well on our auxiliary tasks, but that this effect disappears when the amount of training data for the auxiliary tasks is reduced.

Representations from language models consistently perform better than translation encoders on syntactic auxiliary prediction tasks.

Language Modeling Teaches You More than Translation Does: Lessons Learned Through Auxiliary Task Analysis


We consider the problem of generating configurations that satisfy physical constraints for optimal material nano-pattern design, where multiple (and often conflicting) properties need to be simultaneously satisfied.

Consider, for example, the trade-off between thermal resistance, electrical conductivity, and mechanical stability needed to design a nano-porous template with optimal thermoelectric efficiency.

To that end, we leverage the posterior regularization framework andshow that this constraint satisfaction problem can be formulated as sampling froma Gibbs distribution.

The main challenges come from the black-box nature ofthose physical constraints, since they are obtained via solving highly non-linearPDEs.

To overcome those difficulties, we introduce Surrogate-based Constrained Langevin dynamics for black-box sampling.

The first approach exploits zero-order approximation of gradients in the Langevin Sampling and we refer to it as Zero-Order Langevin.

In practice, this approach can be prohibitive since we still need to often query the expensive PDE solvers.

The second approach approximates the gradients in the Langevin dynamics with deep neural networks, allowing us an efficient sampling strategy using the surrogate model.

We prove the convergence of those two approaches when the target distribution is log-concave and smooth.

We show the effectiveness of both approaches in designing optimal nano-porous material configurations, where the goal is to produce nano-pattern templates with low thermal conductivity and reasonable mechanical stability.

We propose surrogate based Constrained Langevin sampling with application in nano-porous material configuration design.

Surrogate-Based Constrained Langevin Sampling With Applications to Optimal Material Configuration Design

constrained langevin sample application
material configuration design


There is growing interest in geometrically-inspired embeddings for learning hierarchies, partial orders, and lattice structures, with natural applications to transitive relational data such as entailment graphs.

Recent work has extended these ideas beyond deterministic hierarchies to probabilistically calibrated models, which enable learning from uncertain supervision and inferring soft-inclusions among concepts, while maintaining the geometric inductive bias of hierarchical embedding models.

We build on the Box Lattice model of Vilnis et al. (2018), which showed promising results in modeling soft-inclusions through an overlapping hierarchy of sets, parameterized as high-dimensional hyperrectangles (boxes).

However, the hard edges of the boxes present difficulties for standard gradient based optimization; that work employed a special surrogate function for the disjoint case, but we find this method to be fragile.

In this work, we present a novel hierarchical embedding model, inspired by a relaxation of box embeddings into parameterized density functions using Gaussian convolutions over the boxes.

Our approach provides an alternative surrogate to the original lattice measure that improves the robustness of optimization in the disjoint case, while also preserving the desirable properties with respect to the original lattice.

We demonstrate increased or matching performance on WordNet hypernymy prediction, Flickr caption entailment, and a MovieLens-based market basket dataset.

We show especially marked improvements in the case of sparse data, where many conditional probabilities should be low, and thus boxes should be nearly disjoint.

Improve hierarchical embedding models using kernel smoothing

Smoothing the Geometry of Probabilistic Box Embeddings


We present a weakly-supervised data augmentation approach to improve Named Entity Recognition (NER) in a challenging domain: extracting biomedical entities (e.g., proteins) from the scientific literature.

First, we train a neural NER (NNER) model over a small seed of fully-labeled examples.

Second, we use a reference set of entity names (e.g., proteins in UniProt) to identify entity mentions with high precision, but low recall, on an unlabeled corpus.

Third, we use the NNER model to assign weak labels to the corpus.

Finally, we retrain our NNER model iteratively over the augmented training set, including the seed, the reference-set examples, and the weakly-labeled examples, which results in refined labels.

We show empirically that this augmented bootstrapping process significantly improves NER performance, and discuss the factors impacting the efficacy of the approach.

Augmented bootstrapping approach combining information from a reference set with iterative refinements of soft labels to improve Name Entity Recognition from biomedical literature.

Biomedical Named Entity Recognition via Reference-Set Augmented Bootstrapping


Quantum machine learning methods have the potential to facilitate learning using extremely large datasets.

While the availability of data for training machine learning models is steadily increasing, oftentimes it is much easier to collect feature vectors that to obtain the corresponding labels.

One of the approaches for addressing this issue is to use semi-supervised learning, which leverages not only the labeled samples, but also unlabeled feature vectors.

Here, we present a quantum machine learning algorithm for training Semi-Supervised Kernel Support Vector Machines.

The algorithm uses recent advances in quantum sample-based Hamiltonian simulation to extend the existing Quantum LS-SVM algorithm to handle the semi-supervised term in the loss, while maintaining the same quantum speedup as the Quantum LS-SVM.

We extend quantum SVMs to semi-supervised setting, to deal with the likely problem of many missing class labels in huge datasets.

Quantum Semi-Supervised Kernel Learning


Deep neural networks have become the state-of-the-art models in numerous machine learning tasks.

However, general guidance to network architecture design is still missing.

In our work, we bridge deep neural network design with numerical differential equations.

We show that many effective networks, such as ResNet, PolyNet, FractalNet and RevNet, can be interpreted as different numerical discretizations of differential equations.

This finding brings us a brand new perspective on the design of effective deep architectures.

We can take advantage of the rich knowledge in numerical analysis to guide us in designing new and potentially more effective deep networks.

As an example, we propose a linear multi-step architecture (LM-architecture) which is inspired by the linear multi-step method solving ordinary differential equations.

The LM-architecture is an effective structure that can be used on any ResNet-like networks.

In particular, we demonstrate that LM-ResNet and LM-ResNeXt (i.e. the networks obtained by applying the LM-architecture on ResNet and ResNeXt respectively) can achieve noticeably higher accuracy than ResNet and ResNeXt on both CIFAR and ImageNet with comparable numbers of trainable parameters.

In particular, on both CIFAR and ImageNet, LM-ResNet/LM-ResNeXt can significantly compress (>50%) the original networks while maintaining a similar performance.

This can be explained mathematically using the concept of modified equation from numerical analysis.

Last but not least, we also establish a connection between stochastic control and noise injection in the training process which helps to improve generalization of the networks.

Furthermore, by relating stochastic training strategy with stochastic dynamic system, we can easily apply stochastic training to the networks with the LM-architecture.

As an example, we introduced stochastic depth to LM-ResNet and achieve significant improvement over the original LM-ResNet on CIFAR10.

This paper bridges deep network architectures with numerical (stochastic) differential equations. This new perspective enables new designs of more effective deep neural networks.

Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations

architecture numerical
neural network
bridge deep
differential equation


Transforming a graphical user interface screenshot created by a designer into computer code is a typical task conducted by a developer in order to build customized software, websites, and mobile applications.

In this paper, we show that deep learning methods can be leveraged to train a model end-to-end to automatically generate code from a single input image with over 77% of accuracy for three different platforms (i.e. iOS, Android and web-based technologies).

CNN and LSTM to generate markup-like code describing graphical user interface images.

pix2code: Generating Code from a Graphical User Interface Screenshot


Computer vision tasks such as image classification, image retrieval and few-shot learning are currently dominated by Euclidean and spherical embeddings, so that the final decisions about class belongings or the degree of similarity are made using linear hyperplanes, Euclidean distances, or spherical geodesic distances (cosine similarity).

In this work, we demonstrate that in many practical scenarios hyperbolic embeddings provide a better alternative.

We show that hyperbolic embeddings are useful for high-level computer vision tasks, especially for few-shot classification.

Hyperbolic Image Embeddings


High-dimensional time series are common in many domains.

Since human cognition is not optimized to work well in high-dimensional spaces, these areas could benefit from interpretable low-dimensional representations.

However, most representation learning algorithms for time series data are difficult to interpret.

This is due to non-intuitive mappings from data features to salient properties of the representation and non-smoothness over time.

To address this problem, we propose a new representation learning framework building on ideas from interpretable discrete dimensionality reduction and deep generative modeling.

This framework allows us to learn discrete representations of time series, which give rise to smooth and interpretable embeddings with superior clustering performance.

We introduce a new way to overcome the non-differentiability in discrete representation learning and present a gradient-based version of the traditional self-organizing map algorithm that is more performant than the original.

Furthermore, to allow for a probabilistic interpretation of our method, we integrate a Markov model in the representation space.

This model uncovers the temporal transition structure, improves clustering performance even further and provides additional explanatory insights as well as a natural representation of uncertainty.

We evaluate our model in terms of clustering performance and interpretability on static (Fashion-)MNIST data, a time series of linearly interpolated (Fashion-)MNIST images, a chaotic Lorenz attractor system with two macro states, as well as on a challenging real world medical time series application on the eICU data set.

Our learned representations compare favorably with competitor methods and facilitate downstream tasks on the real world data.

We present a method to learn interpretable representations on time series using ideas from variational autoencoders, self-organizing maps and probabilistic models.

SOM-VAE: Interpretable Discrete Representation Learning on Time Series

time series
representation
interpretable


We propose Significance-Offset Convolutional Neural Network, a deep convolutional network architecture for regression of multivariate asynchronous time series.

The model is inspired by standard autoregressive (AR) models and gating mechanisms used in recurrent neural networks.

It involves an AR-like weighting system, where the final predictor is obtained as a weighted sum of adjusted regressors, while the weights are data-dependent functions learnt through a convolutional network.

The architecture was designed for applications on asynchronous time series and is evaluated on such datasets: a hedge fund proprietary dataset of over 2 million quotes for a credit derivative index, an artificially generated noisy autoregressive  series  and  household  electricity  consumption  dataset.

The  pro-posed architecture achieves promising results as compared to convolutional and recurrent neural networks.

The code for the numerical experiments and the architecture implementation will be shared online to make the research reproducible.

Convolutional architecture for learning data-dependent weights for autoregressive forecasting of time series.

Autoregressive Convolutional Neural Networks for Asynchronous Time Series

time series
autoregressive
convolutional


MixUp is a data augmentation scheme in which pairs of training samples and their corresponding labels are mixed using linear coefficients.

Without label mixing, MixUp becomes a more conventional scheme: input samples are moved but their original labels are retained.

Because samples are preferentially moved in the direction of other classes \iffalse -- which are typically clustered in input space -- \fi we refer to this method as directional adversarial training, or DAT.

We show that under two mild conditions, MixUp asymptotically convergences to a subset of DAT.

We define untied MixUp (UMixUp), a superset of MixUp wherein training labels are mixed with different linear coefficients to those of their corresponding samples.

We show that under the same mild conditions, untied MixUp converges to the entire class of DAT schemes.

Motivated by the understanding that UMixUp is both a generalization of MixUp and a form of adversarial training, we experiment with different datasets and loss functions to show that UMixUp provides improved performance over MixUp.

In short, we present a novel interpretation of MixUp as belonging to a class highly analogous to adversarial training, and on this basis we introduce a simple generalization which outperforms MixUp.

We present a novel interpretation of MixUp as belonging to a class highly analogous to adversarial training, and on this basis we introduce a simple generalization which outperforms MixUp

MixUp as Directional Adversarial Training


Plan recognition aims to look for target plans to best explain the observed actions based on plan libraries and/or domain models.

Despite the success of previous approaches on plan recognition, they mostly rely on correct action observations.

Recent advances in visual activity recognition have the potential of enabling applications such as automated video surveillance.

Effective approaches for such problems would require the ability to recognize the plans of agents from video information.

Traditional plan recognition algorithms rely on access to detailed planning domain models.

One recent promising direction involves learning approximate (or shallow) domain models directly from the observed activity sequences.

Such plan recognition approaches expect observed action sequences as inputs.

However, visual inference results are often noisy and uncertain, typically represented as a distribution over possible actions.

In this work, we develop a visual plan recognition framework that recognizes plans with an approximate domain model learned from uncertain visual data.

Handling Uncertainty in Visual Perception for Plan Recognition

Recognizing Plans by Learning Embeddings from Observed Action Distributions


We consider the task of answering complex multi-hop questions using a corpus as a virtual knowledge base (KB).

In particular, we describe a neural module, DrKIT, that traverses textual data like a virtual KB, softly following paths of relations between mentions of entities in the corpus.

At each step the operation uses a combination of sparse-matrix TFIDF indices and maximum inner product search (MIPS) on a special index of contextual representations.

This module is differentiable, so the full system can be trained completely end-to-end using gradient based methods, starting from natural language inputs.

We also describe a pretraining scheme for the index mention encoder by generating hard negative examples using existing knowledge bases.

We show that DrKIT improves accuracy by 9 points on 3-hop questions in the MetaQA dataset, cutting the gap between text-based and KB-based state-of-the-art by 70%.

DrKIT is also very efficient, processing upto 10x more queries per second than existing state-of-the-art QA systems.

Differentiable multi-hop access to a textual knowledge base of indexed contextual representations

Differentiable Reasoning over a Virtual Knowledge Base


In spite of their great success, traditional factorization algorithms typically do not support features (e.g., Matrix Factorization), or their complexity scales quadratically with the number of features (e.g, Factorization Machine).

On the other hand, neural methods allow large feature sets, but are often designed for a specific application.

We propose novel deep factorization methods that allow efficient and flexible feature representation.

For example, we enable describing items with natural language with complexity linear to the vocabulary size—this enables prediction for unseen items and avoids the cold start problem.

We show that our architecture can generalize some previously published single-purpose neural architectures.

Our experiments suggest improved training times and accuracy compared to shallow methods.

Scalable general-purpose factorization algorithm-- also helps to circumvent cold start problem.

Structured Deep Factorization Machine: Towards General-Purpose Architectures


Augmented Reality (AR) can assist with physical tasks such as object assembly through the use of “situated instructions”.

These instructions can be in the form of videos, pictures, text or guiding animations, where the most helpful media among these is highly dependent on both the user and the nature of the task.

Our work supports the authoring of AR tutorials for assembly tasks with little overhead beyond simply performing the task itself.

The presented system, AuthAR reduces the time and effort required to build interactive AR tutorials by automatically generating key components of the AR tutorial while the author is assembling the physical pieces.

Further, the system guides authors through the process of adding videos, pictures, text and animations to the tutorial.

This concurrent assembly and tutorial generation approach allows for authoring of portable tutorials that fit the preferences of different end users.

We present a mixed media assembly tutorial authoring system that streamlines creation of videos, images, text and dynamic instructions in situ.

AuthAR: Concurrent Authoring of Tutorials for AR Assembly Guidance


Monitoring patients in ICU is a challenging and high-cost task.

Hence, predicting the condition of patients during their ICU stay can help provide better acute care and plan the hospital's resources.

There has been continuous progress in machine learning research for ICU management, and most of this work has focused on using time series signals recorded by ICU instruments.

In our work, we show that adding clinical notes as another modality improves the performance of the model for three benchmark tasks: in-hospital mortality prediction, modeling decompensation, and length of stay forecasting that play an important role in ICU management.

While the time-series data is measured at regular intervals, doctor notes are charted at irregular times, making it challenging to model them together.

We propose a method to model them jointly, achieving considerable improvement across benchmark tasks over baseline time-series model.

We demostarte that using clinical notes in conjuntion with ICU instruments data improves the perfomance on ICU management benchmark tasks

Using Clinical Notes for ICU Management


Existing works in deep Multi-Agent Reinforcement Learning (MARL) mainly focus on coordinating cooperative agents to complete certain tasks jointly.

However, in many cases of the real world, agents are self-interested such as employees in a company and clubs in a league.

Therefore, the leader, i.e., the manager of the company or the league, needs to provide bonuses to followers for efficient coordination, which we call expensive coordination.

The main difficulties of expensive coordination are that

i) the leader has to consider the long-term effect and predict the followers' behaviors when assigning bonuses and

ii) the complex interactions between followers make the training process hard to converge, especially when the leader's policy changes with time.

In this work, we address this problem through an event-based deep RL approach.

(1) We model the leader's decision-making process as a semi-Markov Decision Process and propose a novel multi-agent event-based policy gradient to learn the leader's long-term policy.

(2) We exploit the leader-follower consistency scheme to design a follower-aware module and a follower-specific attention module to predict the followers' behaviors and make accurate response to their behaviors.

(3) We propose an action abstraction-based policy gradient algorithm to reduce the followers' decision space and thus accelerate the training process of followers.

Experiments in resource collections, navigation, and the predator-prey game reveal that our approach outperforms the state-of-the-art methods dramatically.

We propose an event-based policy gradient  to train the leader and an action abstraction policy gradient to train the followers in leader-follower Markov game.

Learning Expensive Coordination: An Event-Based Deep RL Approach


Recent work has studied the emergence of language among deep reinforcement learning agents that must collaborate to solve a task.

Of particular interest are the factors that cause language to be compositional---i.e., express meaning by combining words which themselves have meaning.

Evolutionary linguists have found that in addition to structural priors like those already studied in deep learning, the dynamics of transmitting language from generation to generation contribute significantly to the emergence of  compositionality.

In this paper, we introduce these cultural evolutionary dynamics into language emergence by periodically replacing agents in a population to create a knowledge gap, implicitly inducing cultural transmission of language.

We show that this implicit cultural transmission encourages the resulting languages to exhibit better compositional generalization.

We use cultural transmission to encourage compositionality in languages that emerge from interactions between neural agents.

Emergence of Compositional Language with Deep Generational Transmission


Based on our observation that there exists a dramatic drop for the singular values of the fully connected layers or a single feature map of the convolutional layer, and that the dimension of the concatenated feature vector almost equals the summation of the dimension on each feature map, we propose a singular value decomposition (SVD) based approach to estimate the dimension of the deep manifolds for a typical convolutional neural network VGG19.

We choose three categories from the ImageNet, namely Persian Cat, Container Ship and Volcano, and determine the local dimension of the deep manifolds of the deep layers through the tangent space of a target image.

Through several augmentation methods, we found that the Gaussian noise method is closer to the intrinsic dimension, as by adding random noise to an image we are moving in an arbitrary dimension, and when the rank of the feature matrix of the augmented images does not increase we are very close

We also estimate the dimension of the deep manifold based on the tangent space for each of the maxpooling layers.

Our results show that the dimensions of different categories are close to each other and decline quickly along the convolutional layers and fully connected layers.

Furthermore, we show that the dimensions decline quickly inside the Conv5 layer.

Our work provides new insights for the intrinsic structure of deep neural networks and helps unveiling the inner organization of the black box of deep neural networks.

We propose a SVD based method to explore the local dimension of activation manifold in deep neural networks.

THE LOCAL DIMENSION OF DEEP MANIFOLD


Large pre-trained Transformers such as BERT have been tremendously effective for many NLP tasks.

However, inference in these large-capacity models is prohibitively slow and expensive

. Transformers are essentially a stack of self-attention layers which encode each input position using the entire input sequence as its context

. However, we find that it may not be necessary to apply this expensive sequence-wide self-attention over at all layers

. Based on this observation, we propose a decomposition to a pre-trained Transformer that allows the lower layers to process segments of the input independently enabling parallelism and caching

. We show that the information loss due to this decomposition can be recovered in the upper layers with auxiliary supervision during fine-tuning

.  We evaluate de-composition with pre-trained BERT models on five different paired-input tasks in question answering, sentence similarity, and natural language inference

.  Results show that decomposition enables faster inference (up to 4x), significant memory reduction (up to 70%) while retaining most (up to 99%) of the original performance

. We will release the code at<anonymized url>.

Inference in large Transformers is expensive due to the self-attention in multiple layers. We show a simple decomposition technique can yield a faster, low memory-footprint model that is just as accurate of the original models.

Faster and Just As Accurate: A Simple Decomposition for Transformer Models

accurate
fast
simple decomposition
transformer
model


Exploration while learning representations is one of the main challenges Deep

Reinforcement Learning (DRL) faces today.

As the learned representation is dependant in the observed data, the exploration strategy has a crucial role.

The popular DQN algorithm has improved significantly the capabilities of Reinforcement

Learning (RL) algorithms to learn state representations from raw data, yet, it uses

a naive exploration strategy which is statistically inefficient.

Least Squares Value Iteration (RLSVI) algorithm (Osband et al., 2016), on the

other hand, explores and generalizes efficiently via linearly parameterized value

However, it is based on hand-designed state representation that requires

prior engineering work for every environment.

Rather than using hand-design state representation, we use a state representation that is being learned directly from the data by a

As the representation is being optimized during the learning process,

a key component for the suggested method is a likelihood matching mechanism,

which adapts to the changing representations.

the various properties of our algorithm on a toy problem and show that our method

outperforms DQN in five Atari benchmarks, reaching competitive results with the

A Deep Learning adaptation of Randomized Least Squares Value Iteration

Deep Randomized Least Squares Value Iteration

randomize least square value iteration
deep


The complexity of large-scale neural networks can lead to poor understanding of their internal  details.

We show that this opaqueness provides an opportunity for adversaries to embed unintended functionalities into the network in the form of Trojan horse attacks.

Our novel framework hides the existence of a malicious network within a benign transport network.

Our attack is flexible, easy to execute, and difficult to detect.

We prove theoretically that the malicious network's detection is computationally infeasible and demonstrate empirically that the transport network does not compromise its disguise.

Our attack exposes an important, previously unknown loophole that unveils a new direction in machine learning security.

Parameters of a trained neural network can be permuted to produce a completely separate model for a different task, enabling the embedding of Trojan horse networks inside another network.

TrojanNet: Exposing the Danger of Trojan Horse Attack on Neural Networks


In this paper, we introduce Random Path Generative Adversarial Network (RPGAN) --- an alternative scheme of GANs that can serve as a tool for generative model analysis.

While the latent space of a typical GAN consists of input vectors, randomly sampled from the standard Gaussian distribution, the latent space of RPGAN consists of random paths in a generator network.

As we show, this design allows to associate different layers of the generator with different regions of the latent space, providing their natural interpretability.

With experiments on standard benchmarks, we demonstrate that RPGAN reveals several interesting insights about roles that different layers play in the image generation process.

Aside from interpretability, the RPGAN model also provides competitive generation quality and allows efficient incremental learning on new data.

We introduce an alternative GAN design based on random routes in generator, which can serve as a tool for generative models interpretability.

RPGAN: random paths as a latent space for GAN interpretability


Deep artificial neural networks can achieve an extremely small difference between training and test accuracies on identically distributed training and test sets, which is a standard measure of generalization.

However, the training and test sets may not be sufficiently representative of the empirical sample set, which consists of real-world input samples.

When samples are drawn from an underrepresented or unrepresented subset during inference, the gap between the training and inference accuracies can be significant.

To address this problem, we first reformulate a classification algorithm as a procedure for searching for a source code that maps input features to classes.

We then derive a necessary and sufficient condition for generalization using a universal cognitive similarity metric, namely information distance, based on Kolmogorov complexity.

Using this condition, we formulate an optimization problem to learn a more general classification function.

To achieve this end, we extend the input features by concatenating encodings of them, and then train the classifier on the extended features.

As an illustration of this idea, we focus on image classification, where we use channel codes on the input features as a systematic way to improve the degree to which the training and test sets are representative of the empirical sample set.

To showcase our theoretical findings, considering that corrupted or perturbed input features belong to the empirical sample set, but typically not to the training and test sets, we demonstrate through extensive systematic experiments that, as a result of learning a more general classification function, a model trained on encoded input features is significantly more robust to common corruptions, e.g., Gaussian and shot noise, as well as adversarial perturbations, e.g., those found via projected gradient descent, than the model trained on uncoded input features.

We present a theoretical and experimental framework for defining, understanding, and achieving generalization, and as a result robustness, in deep learning by drawing on algorithmic information theory and coding theory.

A Kolmogorov Complexity Approach to Generalization in Deep Learning


Many approaches to causal discovery are limited by their inability to discriminate between Markov equivalent graphs given only observational data.

We formulate causal discovery as a marginal likelihood based Bayesian model selection problem.

We adopt a parameterization based on the notion of the independence of causal mechanisms which renders Markov equivalent graphs distinguishable.

We complement this with an empirical Bayesian approach to setting priors so that the actual underlying causal graph is assigned a higher marginal likelihood than its alternatives.

Adopting a Bayesian approach also allows for straightforward modeling of unobserved confounding variables, for which we provide a variational algorithm to approximate the marginal likelihood, since this desirable feat renders the computation of the marginal likelihood intractable.

We believe that the Bayesian approach to causal discovery both allows the rich methodology of Bayesian inference to be used in various difficult aspects of this problem and provides a unifying framework to causal discovery research.

We demonstrate promising results in experiments conducted on real data, supporting our modeling approach and our inference methodology.

We cast causal structure discovery as a Bayesian model selection in a way that allows us to discriminate between Markov equivalent graphs to identify the unique causal graph.

Bayesian Model Selection for Identifying Markov Equivalent Causal Graphs

bayesian model selection
identify
markov equivalent
causal graph


  The goal of compressed sensing is to learn a structured signal $x$
  from a limited number of noisy linear measurements $y

\approx Ax$.  In
  traditional compressed sensing, ``structure'' is represented by
  sparsity in some known basis.

Inspired by the success of deep
  learning in modeling images, recent work starting with~\cite{BDJP17}
  has instead considered structure to come from a generative model
  $G: \R^k \to \R^n$.  We present two results establishing the
  difficulty of this latter task, showing that existing bounds are
  tight.

First, we provide a lower bound matching the~\cite{BDJP17} upper
  bound for compressed sensing from $L$-Lipschitz generative models
  $G$.  In particular, there exists such a function that requires
  roughly $\Omega(k \log L)$ linear measurements for sparse recovery
  to be possible.

This holds even for the more relaxed goal of
  \emph{nonuniform} recovery.

Second, we show that generative models generalize sparsity as a
  representation of structure.

In particular, we construct a
  ReLU-based neural network $G: \R^{2k} \to \R^n$ with $O(1)$ layers
  and $O(kn)$ activations per layer, such that the range of $G$
  contains all $k$-sparse vectors.

Lower bound for compressed sensing w/ generative models that matches known upper bounds

Lower Bounds for Compressed Sensing with Generative Models


We hypothesize that end-to-end neural image captioning systems work seemingly well because they exploit and learn ‘distributional similarity’ in a multimodal feature space, by mapping a test image to similar training images in this space and generating a caption from the same space.

To validate our hypothesis, we focus on the ‘image’ side of image captioning, and vary the input image representation but keep the RNN text generation model of a CNN-RNN constant.

We propose a sparse bag-of-objects vector as an interpretable representation to investigate our distributional similarity hypothesis.

(i) are capable of separating structure from noisy input representations;

(ii) experience virtually no significant performance loss when a high dimensional representation is compressed to a lower dimensional space;

(iii) cluster images with similar visual and linguistic information together;

(iv) are heavily reliant on test sets with a similar distribution as the training set;

(v) repeatedly generate the same captions by matching images and ‘retrieving’ a caption in the joint visual-textual space.

Our experiments all point to one fact: that our distributional similarity hypothesis holds.

We conclude that, regardless of the image representation, image captioning systems seem to match images and generate captions in a learned joint image-text semantic subspace.

This paper presents an empirical analysis on the role of different types of image representations and probes the properties of these representations for the task of image captioning.

What is image captioning made of?


We present a sequence-to-action parsing approach for the natural language to SQL task that incrementally fills the slots of a SQL query with feasible actions from a pre-defined inventory.

To account for the fact that typically there are multiple correct SQL queries with the same or very similar semantics, we draw inspiration from syntactic parsing techniques and propose to train our sequence-to-action models with non-deterministic oracles.

We evaluate our models on the WikiSQL dataset and achieve an execution accuracy of 83.7% on the test set, a 2.1% absolute improvement over the models trained with traditional static oracles assuming a single correct target SQL query.

When further combined with the execution-guided decoding strategy, our model sets a new state-of-the-art performance at an execution accuracy of 87.1%.

We design incremental sequence-to-action parsers for text-to-SQL task and achieve SOTA results. We further improve by using non-deterministic oracles to allow multiple correct action sequences.

IncSQL: Training Incremental Text-to-SQL Parsers with Non-Deterministic Oracles

non-deterministic oracle
parser
text-to-sql
incremental


We propose Efficient Neural Architecture Search (ENAS), a faster and less expensive approach to automated model design than previous methods.

In ENAS, a controller learns to discover neural network architectures by searching for an optimal path within a larger model.

The controller is trained with policy gradient to select a path that maximizes the expected reward on the validation set.

Meanwhile the model corresponding to the selected path is trained to minimize the cross entropy loss.

On the Penn Treebank dataset, ENAS can discover a novel architecture thats achieves a test perplexity of 57.8, which is state-of-the-art among automatic model design methods on Penn Treebank.

On the CIFAR-10 dataset, ENAS can design novel architectures that achieve a test error of 2.89%, close to the 2.65% achieved by standard NAS (Zoph et al., 2017).

Most importantly, our experiments show that ENAS is more than 10x faster and 100x less resource-demanding than NAS.

An approach that speeds up neural architecture search by 10x, whilst using 100x less computing resource.

Faster Discovery of Neural Architectures by Searching for Paths in a Large Model


Nowadays, deep neural networks (DNNs) have become the main instrument for machine learning tasks within a wide range of domains, including vision, NLP, and speech.

Meanwhile, in an important case of heterogenous tabular data, the advantage of DNNs over shallow counterparts remains questionable.

In particular, there is no sufficient evidence that deep learning machinery allows constructing methods that outperform gradient boosting decision trees (GBDT), which are often the top choice for tabular problems.

In this paper, we introduce Neural Oblivious Decision Ensembles (NODE), a new deep learning architecture, designed to work with any tabular data.

In a nutshell, the proposed NODE architecture generalizes ensembles of oblivious decision trees, but benefits from both end-to-end gradient-based optimization and the power of multi-layer hierarchical representation learning.

With an extensive experimental comparison to the leading GBDT packages on a large number of tabular datasets, we demonstrate the advantage of the proposed NODE architecture, which outperforms the competitors on most of the tasks.

We open-source the PyTorch implementation of NODE and believe that it will become a universal framework for machine learning on tabular data.

We propose a new DNN architecture for deep learning on tabular data

Neural Oblivious Decision Ensembles for Deep Learning on Tabular Data


Person re-identification (re-ID) aims at identifying the same persons' images across different cameras.

However, domain diversities between different datasets pose an evident challenge for adapting the re-ID model trained on one dataset to another one.

State-of-the-art unsupervised domain adaptation methods for person re-ID transferred the learned knowledge from the source domain by optimizing with pseudo labels created by clustering algorithms on the target domain.

Although they achieved state-of-the-art performances, the inevitable label noise caused by the clustering procedure was ignored.

Such noisy pseudo labels substantially hinders the model's capability on further improving feature representations on the target domain.

In order to mitigate the effects of noisy pseudo labels, we propose to softly refine the pseudo labels in the target domain by proposing an unsupervised framework, Mutual Mean-Teaching (MMT), to learn better features from the target domain via off-line refined hard pseudo labels and on-line refined soft pseudo labels in an alternative training manner.

In addition, the common practice is to adopt both the classification loss and the triplet loss jointly for achieving optimal performances in person re-ID models.

However, conventional triplet loss cannot work with softly refined labels.

To solve this problem, a novel soft softmax-triplet loss is proposed to support learning with soft pseudo triplet labels for achieving the optimal domain adaptation performance.

The proposed MMT framework achieves considerable improvements of 14.4%, 18.2%, 13.1% and 16.4% mAP on Market-to-Duke, Duke-to-Market, Market-to-MSMT and Duke-to-MSMT unsupervised domain adaptation tasks.

A framework that conducts online refinement of pseudo labels with a novel soft softmax-triplet loss for unsupervised domain adaptation on person re-identification.

Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification

unsupervised domain adaptation person re-identification


We present the first end-to-end verifier of audio classifiers.

Compared to existing methods, our approach enables analysis of both, the entire audio processing stage as well as recurrent neural network architectures (e.g., LSTM).

The audio processing is verified using novel convex relaxations tailored to feature extraction operations used in audio (e.g., Fast Fourier Transform) while recurrent architectures are certified via a novel binary relaxation for the recurrent unit update.

We show the verifier scales to large networks while computing significantly tighter bounds than existing methods for common audio classification benchmarks: on the challenging Google Speech Commands dataset we certify 95% more inputs than the interval approximation (only prior scalable method), for a perturbation of -90dB.

We present the first approach to certify robustness of neural networks against noise-based perturbations in the audio domain.

Certifying Neural Network Audio Classifiers