Graph neural networks have shown promising results on representing and analyzing diverse graph-structured data such as social, citation, and protein interaction networks.Existing approaches commonly suffer from the oversmoothing issue, regardless of whether policies are edge-based or node-based for neighborhood aggregation.Most methods also focus on transductive scenarios for fixed graphs, leading to poor generalization performance for unseen graphs.To address these issues, we propose a new graph neural network model that considers both edge-based neighborhood relationships and node-based entity features, i.e. Graph Entities with Step Mixture via random walk (GESM).GESM employs a mixture of various steps through random walk to alleviate the oversmoothing problem and attention to use node information explicitly.These two mechanisms allow for a weighted neighborhood aggregation which considers the properties of entities and relations.With intensive experiments, we show that the proposed GESM achieves state-of-the-art or comparable performances on four benchmark graph datasets comprising transductive and inductive learning tasks.Furthermore, we empirically demonstrate the significance of considering global information.The source code will be publicly available in the near future.

Einfaches und effektives neuronales Graphennetz mit einer Mischung aus Zufallsschritten und Aufmerksamkeit


Basis pursuit is a compressed sensing optimization in which the l1-norm is minimized subject to model error constraints.Here we use a deep neural network prior instead of l1-regularization.Using known noise statistics, we jointly learn the prior and reconstruct images without access to ground-truth data.During training, we use alternating minimization across an unrolled iterative network and jointly solve for the neural network weights and training set image reconstructions.At inference, we fix the weights and pass the measurements through the network.We compare reconstruction performance between unsupervised and supervised (i.e. with ground-truth) methods.We hypothesize this technique could be used to learn reconstruction when ground-truth data are unavailable, such as in high-resolution dynamic MRI.

Wir stellen eine unbeaufsichtigte Deep Learning-Rekonstruktion für inverse Bildgebungsprobleme vor, die neuronale Netze mit modellbasierten Einschränkungen kombiniert.


Deep learning approaches usually require a large amount of labeled data to generalize.However, humans can learn a new concept only by a few samples.One of the high cogntition human capablities is to learn several concepts at the same time.In this paper, we address the task of classifying multiple objects by seeing only a few samples from each category.To the best of authors' knowledge, there is no dataset specially designed for few-shot multiclass classification.We design a task of mutli-object few class classification and an environment for easy creating controllable datasets for this task.We demonstrate that the proposed dataset is sound using a method which is an extension of prototypical networks.

Wir stellen eine diagnostische Aufgabe vor, die eine Variation des "few-shot learning" ist und stellen einen Datensatz dafür vor.


We present a new approach to defining a sequence loss function to train a summarizer by using a secondary encoder-decoder as a loss function, alleviating a shortcoming of word level training for sequence outputs.The technique is based on the intuition that if a summary is a good one, it should contain the most essential information from the original article, and therefore should itself be a good input sequence, in lieu of the original, from which a summary can be generated.We present experimental results where we apply this additional loss function to a general abstractive summarizer on a news summarization dataset.The result is an improvement in the ROUGE metric and an especially large improvement in human evaluations, suggesting enhanced performance that is competitive with specialized state-of-the-art models.

Wir stellen die Verwendung eines sekundären Kodierers/Dekodierers als Verlustfunktion vor, um das Training eines Zusammenfassers zu unterstützen.


Existing unsupervised video-to-video translation methods fail to produce translated videos which are frame-wise realistic, semantic information preserving and video-level consistent.In this work, we propose a novel unsupervised video-to-video translation model.Our model decomposes the style and the content, uses specialized encoder-decoder structure and propagates the inter-frame information through bidirectional recurrent neural network (RNN) units.The style-content decomposition mechanism enables us to achieve long-term style-consistent video translation results as well as provides us with a good interface for modality flexible translation.In addition, by changing the input frames and style codes incorporated in our translation, we propose a video interpolation loss, which captures temporal information within the sequence to train our building blocks in a self-supervised manner.Our model can produce photo-realistic, spatio-temporal consistent translated videos in a multimodal way.Subjective and objective experimental results validate the superiority of our model over the existing methods.

Ein zeitlich konsistentes und modalitätsflexibles unüberwachtes Video-zu-Video-Übersetzungssystem, das auf selbstüberwachte Weise trainiert wird.


Providing transparency of AI planning systems is crucial for their success in practical applications.In order to create a transparent system, a user must be able to query it for explanations about its outputs.We argue that a key underlying principle for this is the use of causality within a planning model, and that argumentation frameworks provide an intuitive representation of such causality.In this paper, we discuss how argumentation can aid in extracting causalities in plans and models, and how they can create explanations from them.

Argumentationsrahmen werden verwendet, um die Kausalität von Plänen/Modellen darzustellen, die für Erklärungen herangezogen werden sollen.


Point clouds, as a form of Lagrangian representation, allow for powerful and flexible applications in a large number of computational disciplines.We propose a novel deep-learning method to learn stable and temporally coherent feature spaces for points clouds that change over time.We identify a set of inherent problems with these approaches: without knowledge of the time dimension, the inferred solutions can exhibit strong flickering, and easy solutions to suppress this flickering can result in undesirable local minima that manifest themselves as halo structures.We propose a novel temporal loss function that takes into account higher time derivatives of the point positions, and encourages mingling, i.e., to prevent the aforementioned halos.We combine these techniques in a super-resolution method with a truncation approach to flexibly adapt the size of the generated positions.We show that our method works for large, deforming point sets from different sources to demonstrate the flexibility of our approach.

Wir schlagen einen generativen neuronalen Netzwerkansatz für zeitlich kohärente Punktansammlungen vor.


We study the problem of generating adversarial examples in a black-box setting in which only loss-oracle access to a model is available.We introduce a framework that conceptually unifies much of the existing work on black-box attacks, and demonstrate that the current state-of-the-art methods are optimal in a natural sense.Despite this optimality, we show how to improve black-box attacks by bringing a new element into the problem: gradient priors.We give a bandit optimization-based algorithm that allows us to seamlessly integrate any such priors, and we explicitly identify and incorporate two examples.The resulting methods use two to four times fewer queries and fail two to five times less than the current state-of-the-art.The code for reproducing our work is available at https://git.io/fAjOJ.

Wir stellen eine vereinheitlichende Sichtweise auf Black-Box-Angriffe als Gradientenschätzungsproblem vor und präsentieren dann einen Rahmen (basierend auf Bandits-Optimierung), um Prioritäten in die Gradientenschätzung zu integrieren, was zu einer signifikanten Leistungssteigerung führt.


Collecting high-quality, large scale datasets typically requires significant resources.The aim of the present work is to improve the label efficiency of large neural networks operating on audio data through multitask learning with self-supervised tasks on unlabeled data.To this end, we trained an end-to-end audio feature extractor based on WaveNet that feeds into simple, yet versatile task-specific neural networks.We describe three self-supervised learning tasks that can operate on any large, unlabeled audio corpus.We demonstrate that, in a scenario with limited labeled training data, one can significantly improve the performance of a supervised classification task by simultaneously training it with these additional self-supervised tasks.We show that one can improve performance on a diverse sound events classification task by nearly 6\% when jointly trained with up to three distinct self-supervised tasks.This improvement scales with the number of additional auxiliary tasks as well as the amount of unsupervised data.We also show that incorporating data augmentation into our multitask setting leads to even further gains in performance.

Verbesserung der Kennzeichnungseffizienz durch Multitasking-Lernen bei auditiven Daten


Information need of humans is essentially multimodal in nature, enabling maximum exploitation of situated context.We introduce a dataset for sequential procedural (how-to) text generation from images in cooking domain.The dataset consists of 16,441 cooking recipes with 160,479 photos associated with different steps.We setup a baseline motivated by the best performing model in terms of human evaluation for the Visual Story Telling (ViST) task.In addition, we introduce two models to incorporate high level structure learnt by a Finite State Machine (FSM) in neural sequential generation process by: (1) Scaffolding Structure in Decoder (SSiD) (2) Scaffolding Structure in Loss (SSiL).These models show an improvement in empirical as well as human evaluation.Our best performing model (SSiL) achieves a METEOR score of 0.31, which is an improvement of 0.6 over the baseline model.We also conducted human evaluation of the generated grounded recipes, which reveal that 61% found that our proposed (SSiL) model is better than the baseline model in terms of overall recipes, and 72.5% preferred our model in terms of coherence and structure.We also discuss analysis of the output highlighting key important NLP issues for prospective directions.

In diesem Beitrag werden zwei Techniken vorgestellt, die eine Struktur auf hoher Ebene in die Erzeugung von prozeduralem Text aus einer Bildsequenz einbeziehen.


Implicit models, which allow for the generation of samples but not for point-wise evaluation of probabilities, are omnipresent in real-world problems tackled by machine learning and a hot topic of current research.Some examples include data simulators that are widely used in engineering and scientific research, generative adversarial networks (GANs) for image synthesis, and hot-off-the-press approximate inference techniques relying on implicit distributions.The majority of existing approaches to learning implicit models rely on approximating the intractable distribution or optimisation objective for gradient-based optimisation, which is liable to produce inaccurate updates and thus poor models.This paper alleviates the need for such approximations by proposing the \emph{Stein gradient estimator}, which directly estimates the score function of the implicitly defined distribution.The efficacy of the proposed estimator is empirically demonstrated by examples that include meta-learning for approximate inference and entropy regularised GANs that provide improved sample diversity.

Wir haben einen neuartigen Gradientenschätzer nach der Stein-Methode eingeführt und mit anderen Methoden zum Erlernen impliziter Modelle für ungefähre Schlussfolgerungen und Bilderzeugung verglichen.


Noise injection is a fundamental tool for data augmentation, and yet there is no widely accepted procedure to incorporate it with learning frameworks.This study analyzes the effects of adding or applying different noise models of varying magnitudes to Convolutional Neural Network (CNN) architectures.Noise models that are distributed with different density functions are given common magnitude levels via Structural Similarity (SSIM) metric in order to create an appropriate ground for comparison.The basic results are conforming with the most of the common notions in machine learning, and also introduces some novel heuristics and recommendations on noise injection.The new approaches will provide better understanding on optimal learning procedures for image classification.

Ideale Methode, um während des CNN-Trainings Rauschen in die Eingangsdaten zu injizieren


State-of-the-art Unsupervised Domain Adaptation (UDA) methods learn transferable features by minimizing the feature distribution discrepancy between the source and target domains.Different from these methods which do not model the feature distributions explicitly, in this paper, we explore explicit feature distribution modeling for UDA.In particular, we propose Distribution Matching Prototypical Network (DMPN) to model the deep features from each domain as Gaussian mixture distributions.With explicit feature distribution modeling, we can easily measure the discrepancy between the two domains.In DMPN, we propose two new domain discrepancy losses with probabilistic interpretations.The first one minimizes the distances between the corresponding Gaussian component means of the source and target data.The second one minimizes the pseudo negative log likelihood of generating the target features from source feature distribution.To learn both discriminative and domain invariant features, DMPN is trained by minimizing the classification loss on the labeled source data and the domain discrepancy losses together.Extensive experiments are conducted over two UDA tasks.Our approach yields a large margin in the Digits Image transfer task over state-of-the-art approaches.More remarkably, DMPN obtains a mean accuracy of 81.4% on VisDA 2017 dataset.The hyper-parameter sensitivity analysis shows that our approach is robust w.r.t hyper-parameter changes.

Wir schlagen vor, tiefe Merkmalsverteilungen von Quell- und Zieldaten explizit als Gaußsche Mischverteilungen für die unüberwachte Domänenanpassung (UDA) zu modellieren und erzielen bei mehreren UDA-Aufgaben bessere Ergebnisse als State-of-the-Art-Methoden.


Efficiently learning to solve tasks in complex environments is a key challenge for reinforcement learning (RL) agents. We propose to decompose a complex environment using a task-agnostic world graphs, an abstraction that accelerates learning by enabling agents to focus exploration on a subspace of the environment.The nodes of a world graph are important waypoint states and edges represent feasible traversals between them.  Our framework has two learning phases: 1) identifying world graph nodes and edges by training a binary recurrent variational auto-encoder (VAE) on trajectory data and2) a hierarchical RL framework that leverages structural and connectivity knowledge from the learned world graph to bias exploration towards task-relevant waypoints and regions.We show that our approach significantly accelerates RL on a suite of challenging 2D grid world tasks: compared to baselines, world graph integration doubles achieved rewards on simpler tasks, e.g. MultiGoal, and manages to solve more challenging tasks, e.g. Door-Key, where baselines fail.

Wir erlernen eine aufgabenunabhängige Weltgraphenabstraktion der Umgebung und zeigen, wie deren Verwendung für die strukturierte Exploration die nachgelagerte aufgabenspezifische RL erheblich beschleunigen kann.


We introduce the notion of property signatures, a representation for programs andprogram specifications meant for consumption by machine learning algorithms.Given a function with input type τ_in and output type τ_out, a property is a functionof type: (τ_in, τ_out) → Bool that (informally) describes some simple propertyof the function under consideration.For instance, if τ_in and τ_out are both listsof the same type, one property might ask ‘is the input list the same length as theoutput list?’.If we have a list of such properties, we can evaluate them all for ourfunction to get a list of outputs that we will call the property signature.Crucially,we can ‘guess’ the property signature for a function given only a set of input/outputpairs meant to specify that function.We discuss several potential applications ofproperty signatures and show experimentally that they can be used to improveover a baseline synthesizer so that it emits twice as many programs in less thanone-tenth of the time.

Wir stellen ein Computerprogramm durch eine Reihe von einfacheren Programmen dar und verwenden diese Darstellung zur Verbesserung von Programmsynthesetechniken.


Social dilemmas are situations where individuals face a temptation to increase their payoffs at a cost to total welfare.Building artificially intelligent agents that achieve good outcomes in these situations is important because many real world interactions include a tension between selfish interests and the welfare of others.We show how to modify modern reinforcement learning methods to construct agents that act in ways that are simple to understand, nice (begin by cooperating), provokable (try to avoid being exploited), and forgiving (try to return to mutual cooperation).We show both theoretically and experimentally that such agents can maintain cooperation in Markov social dilemmas.Our construction does not require training methods beyond a modification of self-play, thus if an environment is such that good strategies can be constructed in the zero-sum case (eg. Atari) then we can construct agents that solve social dilemmas in this environment.

Wie können wir künstliche Agenten entwickeln, die soziale Dilemmasituationen lösen (Situationen, in denen Individuen versucht sind, ihren Nutzen auf Kosten des Gesamtwohls zu erhöhen)?


The reparameterization trick has become one of the most useful tools in the field of variational inference.However, the reparameterization trick is based on the standardization transformation which restricts the scope of application of this method to distributions that have tractable inverse cumulative distribution functions or are expressible as deterministic transformations of such distributions.In this paper, we generalized the reparameterization trick by allowing a general transformation.Unlike other similar works, we develop the generalized transformation-based gradient model formally and rigorously.We discover that the proposed model is a special case of control variate indicating that the proposed model can combine the advantages of CV and generalized reparameterization.Based on the proposed gradient model, we propose a new polynomial-based gradient estimator which has better theoretical performance than the reparameterization trick under certain condition and can be applied to a larger class of variational distributions.In studies of synthetic and real data, we show that our proposed gradient estimator has a significantly lower gradient variance than other state-of-the-art methods thus enabling a faster inference procedure.

Wir schlagen ein neuartiges verallgemeinertes transformationsbasiertes Gradientenmodell vor und entwickeln einen polynomialen Gradientenschätzer, der auf diesem Modell basiert.


The fault diagnosis in a modern communication system is traditionally supposed to be difficult, or even impractical for a purely data-driven machine learning approach, for it is a humanmade system of intensive knowledge.A few labeled raw packet streams extracted from fault archive can hardly be sufficient to deduce the intricate logic of underlying protocols.In this paper, we supplement these limited samples with two inexhaustible data sources: the unlabeled records probed from a system in service, and the labeled data simulated in an emulation environment.To transfer their inherent knowledge to the target domain, we construct a directed information flow graph, whose nodes are neural network components consisting of two generators, three discriminators and one classifier, and whose every forward path represents a pair of adversarial optimization goals, in accord with the semi-supervised and transfer learning demands.The multi-headed network can be trained in an alternative approach, at each iteration of which we select one target to update the weights along the path upstream, and refresh the residual layer-wisely to all outputs downstream.The actual results show that it can achieve comparable accuracy on classifying Transmission Control Protocol (TCP) streams without deliberate expert features.The solution has relieved operation engineers from massive works of understanding and maintaining rules, and provided a quick solution independent of specific protocols.

Semi-überwachtes und Transfer-Lernen zur Klassifizierung von Paketflüssen über ein System kooperativer oder adversarischer neuronaler Blöcke


Our work addresses two important issues with recurrent neural networks: (1) they are over-parameterized, and (2) the recurrent weight matrix is ill-conditioned.The former increases the sample complexity of learning and the training time.The latter causes the vanishing and exploding gradient problem.We present a flexible recurrent neural network model called Kronecker Recurrent Units (KRU).KRU achieves parameter efficiency in RNNs through a Kronecker factored recurrent matrix.It overcomes the ill-conditioning of the recurrent matrix by enforcing soft unitary constraints on the factors.Thanks to the small dimensionality of the factors, maintaining these constraints is computationally efficient.Our experimental results on seven standard data-sets reveal that KRU can reduce the number of parameters by three orders of magnitude in the recurrent weight matrix compared to the existing recurrent models, without trading the statistical performance.These results in particular show that while there are advantages in having a high dimensional recurrent space, the capacity of the recurrent part of the model can be dramatically reduced.

Unsere Arbeit stellt eine Kronecker-Faktorisierung von rekurrenten Gewichtsmatrizen für parameter-effiziente und gut konditionierte Recurrent Neural Networks vor.


This paper studies the undesired phenomena of over-sensitivity of representations learned by deep networks to semantically-irrelevant changes in data.We identify a cause for this shortcoming in the classical Variational Auto-encoder (VAE) objective, the evidence lower bound (ELBO).We show that the ELBO fails to control the behaviour of the encoder out of the support of the empirical data distribution and this behaviour of the VAE can lead to extreme errors in the learned representation.This is a key hurdle in the effective use of representations for data-efficient learning and transfer.To address this problem, we propose to augment the data with specifications that enforce insensitivity of the representation with respect to families of transformations.To incorporate these specifications, we propose a regularization method that is based on a selection mechanism that creates a fictive data point by explicitly perturbing an observed true data point.For certain choices of parameters, our formulation naturally leads to the minimization of the entropy regularized Wasserstein distance between representations.We illustrate our approach on standard datasets and experimentally show that significant improvements in the downstream adversarial accuracy can be achieved by learning robust representations completely in an unsupervised manner, without a reference to a particular downstream task and without a costly supervised adversarial training procedure.

Wir schlagen eine Methode zur Berechnung von adversarial robusten Repräsentationen in einer völlig unbeaufsichtigten Weise vor.


We propose a novel score-based approach to learning a directed acyclic graph (DAG) from observational data.We adapt a recently proposed continuous constrained optimization formulation to allow for nonlinear relationships between variables using neural networks.This extension allows to model complex interactions while being more global in its search compared to other greedy approaches.In addition to comparing our method to existing continuous optimization methods, we provide missing empirical comparisons to nonlinear greedy search methods.On both synthetic and real-world data sets, this new method outperforms current continuous methods on most tasks while being competitive with existing greedy search methods on important metrics for causal inference.

Wir schlagen einen neuen, auf Punkten basierenden Ansatz für strukturelles/kausales Lernen vor, der neuronale Netze und eine neue kontinuierliche, eingeschränkte Formulierung für dieses Problem nutzt


We study the problem of designing provably optimal adversarial noise algorithms that induce misclassification in settings where a learner aggregates decisions from multiple classifiers.Given the demonstrated vulnerability of state-of-the-art models to adversarial examples, recent efforts within the field of robust machine learning have focused on the use of ensemble classifiers as a way of boosting the robustness of individual models.In this paper, we design provably optimal attacks against a set of classifiers.We demonstrate how this problem can be framed as finding strategies at equilibrium in a two player, zero sum game between a learner and an adversary and consequently illustrate the need for randomization in adversarial attacks.The main technical challenge we consider is the design of best response oracles that can be implemented in a Multiplicative Weight Updates framework to find equilibrium strategies in the zero-sum game.We develop a series of scalable noise generation algorithms for deep neural networks, and show that it outperforms state-of-the-art attacks on various image classification tasks.Although there are generally no guarantees for deep learning, we show this is a well-principled approach in that it is provably optimal for linear classifiers.The main insight is a geometric characterization of the decision space that reduces the problem of designing best response oracles to minimizing a quadratic function over a set of convex polytopes.

Das Papier analysiert das Problem des Entwurfs von Angriffen gegen mehrere Klassifizierer und stellt Algorithmen vor, die für lineare Klassifizierer optimal sind und die modernste Ergebnisse für Deep Learning liefern.


Multiagent systems where the agents interact among themselves and with an stochastic environment can be formalized as stochastic games.We study a subclass of these games, named Markov potential games (MPGs), that appear often in economic and engineering applications when the agents share some common resource.We consider MPGs with continuous state-action variables, coupled constraints and nonconvex rewards.Previous analysis followed a variational approach that is only valid for very simple cases (convex rewards, invertible dynamics, and no coupled constraints); or considered deterministic dynamics and provided open-loop (OL) analysis, studying strategies that consist in predefined action sequences, which are not optimal for stochastic environments.We present a closed-loop (CL) analysis for MPGs and consider parametric policies that depend on the current state and where agents adapt to stochastic transitions.We provide easily verifiable, sufficient and necessary conditions for a stochastic game to be an MPG, even for complex parametric functions (e.g., deep neural networks); and show that a closed-loop Nash equilibrium (NE) can be found (or at least approximated) by solving a related optimal control problem (OCP).This is useful since solving an OCP---which is a single-objective problem---is usually much simpler than solving the original set of coupled OCPs that form the game---which is a multiobjective control problem.This is a considerable improvement over the previously standard approach for the CL analysis of MPGs, which gives no approximate solution if no NE belongs to the chosen parametric family, and which is practical only for simple parametric forms.We illustrate the theoretical contributions with an example by applying our approach to a noncooperative communications engineering game.We then solve the game with a deep reinforcement learning algorithm that learns policies that closely approximates an exact variational NE of the game.

Wir präsentieren eine allgemeine Closed-Loop-Analyse für Markov-Potential-Spiele und zeigen, dass Deep Reinforcement Learning zum Erlernen eines approximativen Closed-Loop-Nash-Gleichgewichts verwendet werden kann.


We make the following striking observation: fully convolutional VAE models trained on 32x32 ImageNet can generalize well, not just to 64x64 but also to far larger photographs, with no changes to the model.We use this property, applying fully convolutional models to lossless compression, demonstrating a method to scale the VAE-based 'Bits-Back with ANS' algorithm for lossless compression to large color photographs, and achieving state of the art for compression of full size ImageNet images.We release Craystack, an open source library for convenient prototyping of lossless compression using probabilistic models, along with full implementations of all of our compression results.

Wir erhöhen die verlustfreie Komprimierung mit latenten Variablen und übertreffen damit bestehende Ansätze für ImageNet-Bilder in voller Größe.


State of the art computer vision models have been shown to be vulnerable to small adversarial perturbations of the input.In other words, most images in the data distribution are both correctly classified by the model and are very close to a visually similar misclassified image.Despite substantial research interest, the cause of the phenomenon is still poorly understood and remains unsolved.We hypothesize that this counter intuitive behavior is a naturally occurring result of the high dimensional geometry of the data manifold.As a first step towards exploring this hypothesis, we study a simple synthetic dataset of classifying between two concentric high dimensional spheres.For this dataset we show a fundamental tradeoff between the amount of test error and the average distance to nearest error.In particular, we prove that any model which misclassifies a small constant fraction of a sphere will be vulnerable to adversarial perturbations of size $O(1/\sqrt{d})$.Surprisingly, when we train several different architectures on this dataset, all of their error sets naturally approach this theoretical bound.As a result of the theory, the vulnerability of neural networks to small adversarial perturbations is a logical consequence of the amount of test error observed.We hope that our theoretical analysis of this very simple case will point the way forward to explore how the geometry of complex real-world data sets leads to adversarial examples.

Wir stellen die Hypothese auf, dass die Anfälligkeit von Bildmodellen für kleine Störungen eine natürliche Folge der hochdimensionalen Geometrie der Datenvielfalt ist. Wir untersuchen und beweisen diese Hypothese theoretisch für einen einfachen synthetischen Datensatz.


It has been established that diverse behaviors spanning the controllable subspace of a Markov decision process can be trained by rewarding a policy for being distinguishable from other policies.However, one limitation of this formulation is the difficulty to generalize beyond the finite set of behaviors being explicitly learned, as may be needed in subsequent tasks.Successor features provide an appealing solution to this generalization problem, but require defining the reward function as linear in some grounded feature space.In this paper, we show that these two techniques can be combined, and that each method solves the other's primary limitation.To do so we introduce Variational Intrinsic Successor FeatuRes (VISR), a novel algorithm which learns controllable features that can be leveraged to provide enhanced generalization and fast task inference through the successor features framework.We empirically validate VISR on the full Atari suite, in a novel setup wherein the rewards are only exposed briefly after a long unsupervised phase.Achieving human-level performance on 12 games and beating all baselines, we believe VISR represents a step towards agents that rapidly learn from limited feedback.

Wir stellen Variational Intrinsic Successor FeatuRes (VISR) vor, einen neuartigen Algorithmus, der kontrollierbare Merkmale erlernt, die genutzt werden können, um schnelle Aufgabeninferenz durch den Rahmen der Nachfolgermerkmale zu ermöglichen.


Predicting structured outputs such as semantic segmentation relies on expensive per-pixel annotations to learn strong supervised models like convolutional neural networks.However, these models trained on one data domain may not generalize well to other domains unequipped with annotations for model finetuning.To avoid the labor-intensive process of annotation, we develop a domain adaptation method to adapt the source data to the unlabeled target domain.To this end, we propose to learn discriminative feature representations of patches based on label histograms in the source domain, through the construction of a disentangled space.With such representations as guidance, we then use an adversarial learning scheme to push the feature representations in target patches to the closer distributions in source ones.In addition, we show that our framework can integrate a global alignment process with the proposed patch-level alignment and achieve state-of-the-art performance on semantic segmentation.Extensive ablation studies and experiments are conducted on numerous benchmark datasets with various settings, such as synthetic-to-real and cross-city scenarios.

Eine Methode zur Anpassung von strukturierten Ausgaben durch das Erlernen von diskriminierenden Merkmalsdarstellungen auf Patch-Ebene


Many machine learning algorithms are vulnerable to almost imperceptible perturbations of their inputs.So far it was unclear how much risk adversarial perturbations carry for the safety of real-world machine learning applications because most methods used to generate such perturbations rely either on detailed model information (gradient-based attacks) or on confidence scores such as class probabilities (score-based attacks), neither of which are available in most real-world scenarios.In many such cases one currently needs to retreat to transfer-based attacks which rely on cumbersome substitute models, need access to the training data and can be defended against.Here we emphasise the importance of attacks which solely rely on the final model decision.Such decision-based attacks are (1) applicable to real-world black-box models such as autonomous cars, (2) need less knowledge and are easier to apply than transfer-based attacks and (3) are more robust to simple defences than gradient- or score-based attacks.Previous attacks in this category were limited to simple models or simple datasets.Here we introduce the Boundary Attack, a decision-based attack that starts from a large adversarial perturbation and then seeks to reduce the perturbation while staying adversarial.The attack is conceptually simple, requires close to no hyperparameter tuning, does not rely on substitute models and is competitive with the best gradient-based attacks in standard computer vision tasks like ImageNet.We apply the attack on two black-box algorithms from Clarifai.com.The Boundary Attack in particular and the class of decision-based attacks in general open new avenues to study the robustness of machine learning models and raise new questions regarding the safety of deployed machine learning systems.An implementation of the attack is available as part of Foolbox (https://github.com/bethgelab/foolbox).

Ein neuartiger feinflicher Angriff, der reale Black-Box-Modelle des maschinellen Lernens ohne Übertragung direkt angreifen kann.


We demonstrate that it is possible to train large recurrent language models with user-level differential privacy guarantees with only a negligible cost in predictive accuracy.  Our work builds on recent advances in the training of deep networks on user-partitioned data and privacy accounting for stochastic gradient descent.In particular, we add user-level privacy protection to the federated averaging algorithm, which makes large step updates from user-level data.Our work demonstrates that given a dataset with a sufficiently large number of users (a requirement easily met by even small internet-scale datasets), achieving differential privacy comes at the cost of increased computation, rather than in decreased utility as in most prior work.We find that our private LSTM language models are quantitatively and qualitatively similar to un-noised models when trained on a large dataset.

Differenzierter Datenschutz auf Benutzerebene für Recurrent Neural Network-Sprachmodelle ist mit einem ausreichend großen Datensatz möglich.


Convolutional neural networks (CNNs) are commonly trained using a fixed spatial image size predetermined for a given model.Although trained on images of a specific size, it is well established that CNNs can be used to evaluate a wide range of image sizes at test time, by adjusting the size of intermediate feature maps. In this work, we describe and evaluate a novel mixed-size training regime that mixes several image sizes at training time.We demonstrate that models trained using our method are more resilient to image size changes and generalize well even on small images.This allows faster inference by using smaller images at test time.For instance, we receive a 76.43% top-1 accuracy using ResNet50 with an image size of 160, which matches the accuracy of the baseline model with 2x fewer computations.Furthermore, for a given image size used at test time, we show this method can be exploited either to accelerate training or the final test accuracy.For example, we are able to reach a 79.27% accuracy with a model evaluated at a 288 spatial size for a relative improvement of 14% over the baseline.

Das Training von Convnets mit gemischten Bildgrößen kann die Ergebnisse bei der Auswertung über mehrere Größen hinweg verbessern


We propose a simple technique for encouraging generative RNNs to plan ahead.We train a ``backward'' recurrent network to generate a given sequence in reverse order, and we encourage states of the forward model to predict cotemporal states of the backward model.The backward network is used only during training, and plays no role during sampling or inference.We hypothesize that our approach eases modeling of long-term dependencies by implicitly forcing the forward states to hold information about the longer-term future (as contained in the backward states).We show empirically that our approach achieves 9% relative improvement for a speech recognition task, and achieves significant improvement on a COCO caption generation task.

In diesem Beitrag wird eine Methode zur Ausbildung generativer Recurrent Networks vorgestellt, die eine Vorausplanung ermöglicht. Wir lassen ein zweites RNN in umgekehrter Richtung laufen und machen eine weiche Einschränkung zwischen zeitgleichen Vorwärts- und Rückwärtszuständen.


Deep generative models seek to recover the process with which the observed data was generated.They may be used to synthesize new samples or to subsequently extract representations.Successful approaches in the domain of images are driven by several core inductive biases.However, a bias to account for the compositional way in which humans structure a visual scene in terms of objects has frequently been overlooked.In this work we propose to structure the generator of a GAN to consider objects and their relations explicitly, and generate images by means of composition.This provides a way to efficiently learn a more accurate generative model of real-world images, and serves as an initial step towards learning corresponding object representations.We evaluate our approach on several multi-object image datasets, and find that the generator learns to identify and disentangle information corresponding to different objects at a representational level.A human study reveals that the resulting generative model is better at generating images that are more faithful to the reference distribution.

Wir schlagen vor, den Generator eines GAN so zu strukturieren, dass er Objekte und ihre Beziehungen explizit berücksichtigt und Bilder durch Komposition erzeugt


Current literature in machine learning holds that unaligned, self-interested agents do not learn to use an emergent communication channel.We introduce a new sender-receiver game to study emergent communication for this spectrum of partially-competitive scenarios and put special care into evaluation.We find that communication can indeed emerge in partially-competitive scenarios, and we discover three things that are tied to improving it.First, that selfish communication is proportional to cooperation, and it naturally occurs for situations that are more cooperative than competitive.Second, that stability and performance are improved by using LOLA (Foerster et al, 2018), especially in more competitive scenarios.And third, that discrete protocols lend themselves better to learning cooperative communication than continuous ones.

Wir schaffen es, die Kommunikation mit egoistischen Agenten hervorzubringen, im Gegensatz zur aktuellen Ansicht in ML


Deep neural networks (DNNs) typically have enough capacity to fit random data by brute force even when conventional data-dependent regularizations focusing on the geometry of the features are imposed.We find out that the reason for this is the inconsistency between the enforced geometry and the standard softmax cross entropy loss.To resolve this, we propose a new framework for data-dependent DNN regularization, the Geometrically-Regularized-Self-Validating neural Networks (GRSVNet).During training, the geometry enforced on one batch of features is simultaneously validated on a separate batch using a validation loss consistent with the geometry.We study  a particular case of GRSVNet, the Orthogonal-Low-rank Embedding (OLE)-GRSVNet, which is capable of producing highly discriminative features residing in orthogonal low-rank subspaces.Numerical experiments show that OLE-GRSVNet outperforms DNNs with conventional regularization when trained on real data.More importantly, unlike conventional DNNs, OLE-GRSVNet refuses to memorize random data or random labels, suggesting it only learns intrinsic patterns by reducing the memorizing capacity of the baseline DNN.

Wir schlagen einen neuen Rahmen für datenabhängige DNN-Regularisierung vor, der verhindern kann, dass DNNs sich zufälligen Daten oder zufälligen Bezeichnungen zu sehr anpassen.


End-to-end automatic speech recognition (ASR) commonly transcribes audio signals into sequences of characters while its performance is evaluated by measuring the word-error rate (WER).This suggests that predicting sequences of words directly may be helpful instead.However, training with word-level supervision can be more difficult due to the sparsity of examples per label class.In this paper we analyze an end-to-end ASR model that combines a word-and-character representation in a multi-task learning (MTL) framework.We show that it improves on the WER and study how the word-level model can benefit from character-level supervision by analyzing the learned inductive preference bias of each model component empirically.We find that by adding character-level supervision, the MTL model interpolates between recognizing more frequent words (preferred by the word-level model) and shorter words (preferred by the character-level model).

Multitasking-Lernen verbessert die Spracherkennung auf Wort- und Zeichenebene durch Interpolation der Präferenzvorlieben seiner Komponenten: Frequenz- und Wortlängenpräferenz.


Discretizing floating-point vectors is a fundamental step of modern indexing methods.State-of-the-art techniques learn parameters of the quantizers on training data for optimal performance, thus adapting quantizers to the data.In this work, we propose to reverse this paradigm and adapt the data to the quantizer: we train a neural net whose last layers form a fixed parameter-free quantizer, such as pre-defined points of a sphere.As a proxy objective, we design and train a neural network that favors uniformity in the spherical latent space, while preserving the neighborhood structure after the mapping.  For this purpose, we propose a new regularizer derived from the Kozachenko-Leonenko differential entropy estimator and combine it with a locality-aware triplet loss. Experiments show that our end-to-end approach outperforms most learned quantization methods, and is competitive with the state of the art on widely adopted benchmarks.Further more, we show that training without the quantization step results in almost no difference in accuracy, but yields a generic catalyser that can be applied with any subsequent quantization technique.

Wir lernen ein neuronales Netz, das die Eingabeverteilung vereinheitlicht, was zu einer konkurrenzfähigen Indexierungsleistung im hochdimensionalen Raum führt


Temporal difference (TD) learning is a popular algorithm for policy evaluation in reinforcement learning, but the vanilla TD can substantially suffer from the inherent optimization variance.A variance reduced TD (VRTD) algorithm was proposed by Korda and La (2015), which applies the variance reduction technique directly to the online TD learning with Markovian samples.In this work, we first point out the technical errors in the analysis of VRTD in Korda and La (2015), and then provide a mathematically solid analysis of the non-asymptotic convergence of VRTD and its variance reduction performance.We show that VRTD is guaranteed to converge to a neighborhood of the fixed-point solution of TD at a linear convergence rate.Furthermore, the variance error (for both i.i.d. and Markovian sampling) and the bias error (for Markovian sampling) of VRTD are significantly reduced by the batch size of variance reduction in comparison to those of vanilla TD.

Dieses Papier bietet eine rigorose Studie des varianzreduzierten TD-Lernens und charakterisiert seinen Vorteil gegenüber dem einfachen TD-Lernen


We tackle unsupervised domain adaptation by accounting for the fact that different domains may need to be processed differently to arrive to a common feature representation effective for recognition.To this end, we introduce a deep learning framework where each domain undergoes a different sequence of operations, allowing some, possibly more complex, domains to go through more computations than others.This contrasts with  state-of-the-art domain adaptation techniques that force all domains to be processed with the same series of operations, even when using multi-stream architectures whose parameters are not shared.As evidenced by our experiments, the greater flexibility of our method translates to higher accuracy.Furthermore, it allows us to handle any number of domains simultaneously.

Ein Multiflow-Netzwerk ist eine dynamische Architektur für die Domänenanpassung, die potenziell unterschiedliche Berechnungsgraphen pro Domäne erlernt, um sie auf eine gemeinsame Darstellung abzubilden, in der Inferenzen auf domänenunabhängige Weise durchgeführt werden können.


The practical usage of reinforcement learning agents is often bottlenecked by the duration of training time.To accelerate training, practitioners often turn to distributed reinforcement learning architectures to parallelize and accelerate the training process.However, modern methods for scalable reinforcement learning (RL) often tradeoff between the throughput of samples that an RL agent can learn from (sample throughput) and the quality of learning from each sample (sample efficiency).In these scalable RL architectures, as one increases sample throughput (i.e. increasing parallelization in IMPALA (Espeholt et al., 2018)), sample efficiency drops significantly.To address this, we propose a new distributed reinforcement learning algorithm, IMPACT.IMPACT extends PPO with three changes: a target network for stabilizing the surrogate objective, a circular buffer, and truncated importance sampling.In discrete action-space environments, we show that IMPACT attains higher reward and, simultaneously, achieves up to 30% decrease in training wall-time than that of IMPALA.For continuous control environments, IMPACT trains faster than existing scalable agents while preserving the sample efficiency of synchronous PPO.

IMPACT hilft RL-Agenten, schneller zu trainieren, indem es die Trainingszeit an der Wall-Clock verkürzt und gleichzeitig die Effizienz der Beispiele erhöht.


In this paper, we show that a simple coloring scheme can improve, both theoretically and empirically, the expressive power of Message Passing Neural Networks (MPNNs).More specifically, we introduce a graph neural network called Colored Local Iterative Procedure (CLIP) that uses colors to disambiguate identical node attributes, and show that this representation is a universal approximator of continuous functions on graphs with node attributes.Our method relies on separability, a key topological characteristic that allows to extend well-chosen neural networks into universal representations.Finally, we show experimentally that CLIP is capable of capturing structural characteristics that traditional MPNNs fail to distinguish, while being state-of-the-art on benchmark graph classification datasets.

In diesem Papier wird ein Färbungsschema für die Knoten-Disambiguierung in graphischen neuronalen Netzen vorgestellt, das auf Trennbarkeit basiert und sich als universelle MPNN-Erweiterung erweist.


In this paper we present a method for algorithmic melody generation using a generative adversarial network without recurrent components.Music generation has been successfully done using recurrent neural networks, where the model learns sequence information that can help create authentic sounding melodies.  Here, we use DCGAN architecture with dilated convolutions and towers to capture sequential information as spatial image information, and learn long-range dependencies in fixed-length melody forms such as Irish traditional reel.

Wenn wir Melodien als Bilder mit semantischen Einheiten darstellen, können wir sie mit einem DCGAN ohne rekurrente Komponenten erzeugen.


Neural machine translation (NMT) systems have reached state of the art performance in translating text and widely deployed.  Yet little is understood about how these systems function or break.  Here we show that NMT systems are susceptible to producing highly pathological translations that are completely untethered from the source material, which we term hallucinations.  Such pathological translations are problematic because they are are deeply disturbing of user trust and easy to find.  We describe a method t generate hallucinations and show that many common variations of the NMT architecture are susceptible to them.We study a variety of approaches to reduce the frequency of hallucinations, including data augmentation, dynamical systems and regularization techniques and show that data augmentation significantly reduces hallucination frequency.Finally, we analyze networks that produce hallucinations and show signatures of hallucinations in the attention matrix and in the stability measures of the decoder.

Wir analysieren und stellen das Phänomen der "Halluzinationen" in der NMT vor, auch falsche Übersetzungen ohne Bezug zum Quelltext, und schlagen Methoden vor, um deren Häufigkeit zu reduzieren.


For AI systems to garner widespread public acceptance, we must develop methods capable of explaining the decisions of black-box models such as neural networks.In this work, we identify two issues of current explanatory methods.First, we show that two prevalent perspectives on explanations—feature-additivity and feature-selection—lead to fundamentally different instance-wise explanations.In the literature, explainers from different perspectives are currently being directly compared, despite their distinct explanation goals.The second issue is that current post-hoc explainers have only been thoroughly validated on simple models, such as linear regression, and, when applied to real-world neural networks, explainers are commonly evaluated under the assumption that the learned models behave reasonably.However, neural networks often rely on unreasonable correlations, even when producing correct decisions.We introduce a verification framework for explanatory methods under the feature-selection perspective.Our framework is based on a non-trivial neural network architecture trained on a real-world task, and for which we are able to provide guarantees on its inner workings.We validate the efficacy of our evaluation by showing the failure modes of current explainers.We aim for this framework to provide a publicly available,1 off-the-shelf evaluation when the feature-selection perspective on explanations is needed.

Ein Bewertungsrahmen auf der Grundlage eines realen neuronalen Netzes für post-hoc-Erklärungsmethoden


Planning in high-dimensional space remains a challenging problem, even with recent advances in algorithms and computational power.We are inspired by efference copy and sensory reafference theory from neuroscience.  Our aim is to allow agents to form mental models of their environments for planning.  The cerebellum is emulated with a two-stream, fully connected, predictor network.The network receives as inputs the efference as well as the features of the current state.Building on insights gained from knowledge distillation methods, we choose as our features the outputs of a pre-trained network,  yielding a compressed representation of the current state.  The representation is chosen such that it allows for fast search using classical graph search algorithms.We display the effectiveness of our approach on a viewpoint-matching task using a modified best-first search algorithm.

Wir stellen eine von den Neurowissenschaften inspirierte Methode vor, die auf neuronalen Netzen für die Suche im latenten Raum basiert


Experimental evidence indicates that simple models outperform complex deep networks on many unsupervised similarity tasks.Introducing the concept of an optimal representation space, we provide a simple theoretical resolution to this apparent paradox.In addition, we present a straightforward procedure that, without any retraining or architectural modifications, allows deep recurrent models to perform equally well (and sometimes better) when compared to shallow models.To validate our analysis, we conduct a set of consistent empirical evaluations and introduce several new sentence embedding models in the process.Even though this work is presented within the context of natural language processing, the insights are readily applicable to other domains that rely on distributed representations for transfer tasks.

Durch die Einführung des Begriffs eines optimalen Repräsentationsraums liefern wir ein theoretisches Argument und eine experimentelle Validierung, dass ein unbeaufsichtigtes Modell für Sätze sowohl bei überwachten Ähnlichkeitsaufgaben als auch bei unbeaufsichtigten Transferaufgaben gut abschneiden kann.


Cloze test is widely adopted in language exams to evaluate students' language proficiency.In this paper, we propose the first large-scale human-designed cloze test dataset CLOTH in which the questions were used in middle-school and high-school language exams.With the missing blanks carefully created by teachers and candidate choices purposely designed to be confusing, CLOTH requires a deeper language understanding and a wider attention span than previous automatically generated cloze datasets.We show humans outperform dedicated designed baseline models by a significant margin, even when the model is trained on sufficiently large external data.We investigate the source of the performance gap, trace model deficiencies to some distinct properties of CLOTH, and identify the limited ability of comprehending a long-term context to be the key bottleneck.In addition, we find that human-designed data leads to a larger gap between the model's performance and human performance when compared to automatically generated data.

Ein Lückentext-Datensatz, der von Lehrern zur Bewertung von Sprachkenntnissen entwickelt wurde


Recent work suggests goal-driven training of neural networks can be used to model neural activity in the brain.While response properties of neurons in artificial neural networks bear similarities to those in the brain, the network architectures are often constrained to be different.Here we ask if a neural network can recover both neural representations and, if the architecture is unconstrained and optimized, also the anatomical properties of neural circuits.We demonstrate this in a system where the connectivity and the functional organization have been characterized, namely, the head direction circuit of the rodent and fruit fly.We trained recurrent neural networks (RNNs) to estimate head direction through integration of angular velocity.We found that the two distinct classes of neurons observed in the head direction system, the Ring neurons and the Shifter neurons, emerged naturally in artificial neural networks as a result of training.Furthermore, connectivity analysis and in-silico neurophysiology revealed structural and mechanistic similarities between artificial networks and the head direction system.Overall, our results show that optimization of RNNs in a goal-driven task can recapitulate the structure and function of biological circuits, suggesting that artificial neural networks can be used to study the brain at the level of both neural activity and anatomical organization.

Künstliche neuronale Netze, die mit Gradientenabstieg trainiert werden, sind in der Lage, sowohl realistische neuronale Aktivität als auch die anatomische Organisation eines biologischen Kreises zu rekapitulieren.


Convolutional Neural Networks (CNNs) are computationally intensive, which limits their application on mobile devices.Their energy is dominated by the number of multiplies needed to perform the convolutions.Winograd’s minimal filtering algorithm (Lavin, 2015) and network pruning (Han et al., 2015) can reduce the operation count, but these two methods cannot be straightforwardly combined — applying the Winograd transform fills in the sparsity in both the weights and the activations.We propose two modifications to Winograd-based CNNs to enable these methods to exploit sparsity.First, we move the ReLU operation into the Winograd domain to increase the sparsity of the transformed activations.Second, we prune the weights in the Winograd domain to exploit static weight sparsity.For models on CIFAR-10, CIFAR-100 and ImageNet datasets, our method reduces the number of multiplications by 10.4x, 6.8x and 10.8x respectively with loss of accuracy less than 0.1%, outperforming previous baselines by 2.0x-3.0x.We also show that moving ReLU to the Winograd domain allows more aggressive pruning.

Prune und ReLU im Winograd-Bereich für effiziente Convolutional Neural Networks


In this paper we present a novel optimization algorithm called Advanced Neuroevolution.The aim for this algorithm is to train deep neural networks, and eventually act as an alternative to Stochastic Gradient Descent (SGD) and its variants as needed.We evaluated our algorithm on the MNIST dataset, as well as on several global optimization problems such as the Ackley function.We find the algorithm performing relatively well for both cases, overtaking other global optimization algorithms such as Particle Swarm Optimization (PSO) and Evolution Strategies (ES).

Ein neuer Algorithmus zum Trainieren tiefer neuronaler Netze. Getestet an Optimierungsfunktionen und MNIST.


Stochastic neural net weights are used in a variety of contexts, including regularization, Bayesian neural nets, exploration in reinforcement learning, and evolution strategies.Unfortunately, due to the large number of weights, all the examples in a mini-batch typically share the same weight perturbation, thereby limiting the variance reduction effect of large mini-batches.We introduce flipout, an efficient method for decorrelating the gradients within a mini-batch by implicitly sampling pseudo-independent weight perturbations for each example.Empirically, flipout achieves the ideal linear variance reduction for fully connected networks, convolutional networks, and RNNs.We find significant speedups in training neural networks with multiplicative Gaussian perturbations.We show that flipout is effective at regularizing LSTMs, and outperforms previous methods.Flipout also enables us to vectorize evolution strategies: in our experiments, a single GPU with flipout can handle the same throughput as at least 40 CPU cores using existing methods, equivalent to a factor-of-4 cost reduction on Amazon Web Services.

Wir stellen Flipout vor, eine effiziente Methode zur Dekorrelation der Gradienten, die durch stochastische neuronale Netzgewichte innerhalb eines Mini-Batches berechnet werden, indem implizit pseudo-unabhängige Gewichtsstörungen für jedes Beispiel gesammelt werden.


Deep generative models such as Variational AutoEncoder (VAE) and Generative Adversarial Network (GAN) play an increasingly important role in machine learning and computer vision.However, there are two fundamental issues hindering their real-world applications: the difficulty of conducting variational inference in VAE and the functional absence of encoding real-world samples in GAN.In this paper, we propose a novel algorithm named Latently Invertible Autoencoder (LIA) to address the above two issues in one framework.An invertible network and its inverse mapping are symmetrically embedded in the latent space of VAE.Thus the partial encoder first transforms the input into feature vectors and then the distribution of these feature vectors is reshaped to fit a prior by the invertible network.The decoder proceeds in the reverse order of the encoder's composite mappings.A two-stage stochasticity-free training scheme is designed to train LIA via adversarial learning, in the sense that the decoder of LIA is first trained as a standard GAN with the invertible network and then the partial encoder is learned from an autoencoder by detaching the invertible network from LIA.  Experiments conducted on the FFHQ face dataset and three LSUN datasets validate the effectiveness of LIA for inference and generation.

Ein neues Modell eines Latently Invertible Autoencoders wird vorgeschlagen, um das Problem der variationalen Inferenz in VAE mit dem invertible Netzwerk und zweistufige adversarial Ausbildung zu lösen.


Neural programs are highly accurate and structured policies that perform algorithmic tasks by controlling the behavior of a computation mechanism.Despite the potential to increase the interpretability and the compositionality of the behavior of artificial agents, it remains difficult to learn from demonstrations neural networks that represent computer programs.The main challenges that set algorithmic domains apart from other imitation learning domains are the need for high accuracy, the involvement of specific structures of data, and the extremely limited observability.To address these challenges, we propose to model programs as Parametrized Hierarchical Procedures (PHPs).A PHP is a sequence of conditional operations, using a program counter along with the observation to select between taking an elementary action, invoking another PHP as a sub-procedure, and returning to the caller.We develop an algorithm for training PHPs from a set of supervisor demonstrations, only some of which are annotated with the internal call structure, and apply it to efficient level-wise training of multi-level PHPs.We show in two benchmarks, NanoCraft and long-hand addition, that PHPs can learn neural programs more accurately from smaller amounts of both annotated and unannotated demonstrations.

Wir stellen das PHP-Modell für die hierarchische Darstellung neuronaler Programme und einen Algorithmus zum Lernen von PHPs aus einer Mischung von starker und schwacher Überwachung vor.


Deep Reinforcement Learning has managed to achieve state-of-the-art results in learning control policies directly from raw pixels.However, despite its remarkable success, it fails to generalize, a fundamental component required in a stable Artificial Intelligence system.Using the Atari game Breakout, we demonstrate the difficulty of a trained agent in adjusting to simple modifications in the raw image, ones that a human could adapt to trivially.In transfer learning, the goal is to use the knowledge gained from the source task to make the training of the target task faster and better.We show that using various forms of fine-tuning, a common method for transfer learning, is not effective for adapting to such small visual changes.In fact, it is often easier to re-train the agent from scratch than to fine-tune a trained agent.We suggest that in some cases transfer learning can be improved by adding a dedicated component whose goal is to learn to visually map between the known domain and the new one.Concretely, we use Unaligned Generative Adversarial Networks (GANs) to create a mapping function to translate images in the target task to corresponding images in the source task.These mapping functions allow us to transform between various variations of the Breakout game, as well as between different levels of a Nintendo game, Road Fighter.We show that learning this mapping is substantially more efficient than re-training.A visualization of a trained agent playing Breakout and Road Fighter, with and without the GAN transfer, can be seen in \url{https://streamable.com/msgtm} and \url{https://streamable.com/5e2ka}.

Wir schlagen eine Methode zum Wissenstransfer zwischen verwandten RL-Aufgaben unter Verwendung visueller Mappings vor und demonstrieren ihre Effektivität an visuellen Varianten des Atari Breakout-Spiels und verschiedenen Levels von Road Fighter, einem Nintendo-Autofahrspiel.


A commonplace belief in the machine learning community is that using adaptive gradient methods hurts generalization.We re-examine this belief both theoretically and experimentally, in light of insights and trends from recent years.We revisit some previous oft-cited experiments and theoretical accounts in more depth, and provide a new set of experiments in larger-scale, state-of-the-art settings.We conclude that with proper tuning, the improved training performance of adaptive optimizers does not in general carry an overfitting penalty, especially in contemporary deep learning.Finally, we synthesize a ``user's guide'' to adaptive optimizers, including some proposed modifications to AdaGrad to mitigate some of its empirical shortcomings.

Richtig angewandte adaptive Gradientenmethoden sind nicht mit einem Generalisierungsnachteil verbunden.


The ability to generalize quickly from few observations is crucial for intelligent systems.In this paper we introduce APL, an algorithm that approximates probability distributions by remembering the most surprising observations it has encountered.These past observations are recalled from an external memory module and processed by a decoder network that can combine information from different memory slots to generalize beyond direct recall.We show this algorithm can perform as well as state of the art baselines on few-shot classification benchmarks with a smaller memory footprint.  In addition, its memory compression allows it to scale to thousands of unknown labels.  Finally, we introduce a meta-learning reasoning task which is more challenging than direct classification.In this setting, APL is able to generalize with fewer than one example per class via deductive reasoning.

Wir stellen ein Modell vor, das ausgehend von wenigen Beobachtungen schnell verallgemeinert, indem es überraschende Informationen speichert und zu jedem Zeitpunkt die wichtigsten Daten berücksichtigt.


Recent advances in recurrent neural nets (RNNs) have shown much promise in many applications in natural language processing.For most of these tasks, such as sentiment analysis of customer reviews, a recurrent neural net model parses the entire review before forming a decision.We argue that reading the entire input is not always necessary in practice, since a lot of reviews are often easy to classify, i.e., a decision can be formed after reading some crucial sentences or words in the provided text.In this paper, we present an approach of fast reading for text classification.Inspired by several well-known human reading techniques, our approach implements an intelligent recurrent agent which evaluates the importance of the current snippet in order to decide whether to make a prediction, or to skip some texts, or to re-read part of the sentence.Our agent uses an RNN module to encode information from the past and the current tokens, and applies a policy module to form decisions.With an end-to-end training algorithm based on policy gradient, we train and test our agent on several text classification datasets and achieve both higher efficiency and better accuracy compared to previous approaches.

Wir entwickeln einen durchgängig trainierbaren Ansatz für das Überfliegen, Wiederlesen und frühzeitige Beenden von Klassifikationsaufgaben.


Reinforcement learning agents need to explore their unknown environments to solve the tasks given to them.The Bayes optimal solution to exploration is intractable for complex environments, and while several exploration methods have been proposed as approximations, it remains unclear what underlying objective is being optimized by existing exploration methods, or how they can be altered to incorporate prior knowledge about the task.Moreover, it is unclear how to acquire a single exploration strategy that will be useful for solving multiple downstream tasks.We address these shortcomings by learning a single exploration policy that can quickly solve a suite of downstream tasks in a multi-task setting, amortizing the cost of learning to explore.We recast exploration as a problem of State Marginal Matching (SMM), where we aim to learn a policy for which the state marginal distribution matches a given target state distribution, which can incorporate prior knowledge about the task.We optimize the objective by reducing it to a two-player, zero-sum game between a state density model and a parametric policy.Our theoretical analysis of this approach suggests that prior exploration methods do not learn a policy that does distribution matching, but acquire a replay buffer that performs distribution matching, an observation that potentially explains these prior methods' success in single-task settings.On both simulated and real-world tasks, we demonstrate that our algorithm explores faster and adapts more quickly than prior methods.

Wir betrachten Exploration in RL als ein Problem der Anpassung einer Randverteilung über Zustände.


The effectiveness of Convolutional Neural Networks stems in large part from their ability to exploit the translation invariance that is inherent in many learning problems.Recently, it was shown that CNNs can exploit other invariances, such as rotation invariance, by using group convolutions instead of planar convolutions.However, for reasons of performance and ease of implementation, it has been necessary to limit the group convolution to transformations that can be applied to the filters without interpolation.Thus, for images with square pixels, only integer translations, rotations by multiples of 90 degrees, and reflections are admissible.Whereas the square tiling provides a 4-fold rotational symmetry, a hexagonal tiling of the plane has a 6-fold rotational symmetry.In this paper we show how one can efficiently implement planar convolution and group convolution over hexagonal lattices, by re-using existing highly optimized convolution routines.We find that, due to the reduced anisotropy of hexagonal filters, planar HexaConv provides better accuracy than planar convolution with square filters, given a fixed parameter budget.Furthermore, we find that the increased degree of symmetry of the hexagonal grid increases the effectiveness of group convolutions, by allowing for more parameter sharing.We show that our method significantly outperforms conventional CNNs on the AID aerial scene classification dataset, even outperforming ImageNet pre-trained models.

Wir stellen G-HexaConv vor, ein gruppenäquivariantes Convolutional Neural Network auf hexagonalen Gittern.


Deep Convolutional Neural Networks (CNNs) have been repeatedly shown to perform well on image classification tasks, successfully recognizing a broad array of objects when given sufficient training data.Methods for object localization, however, are still in need of substantial improvement.Common approaches to this problem involve the use of a sliding window, sometimes at multiple scales, providing input to a deep CNN trained to classify the contents of the window.In general, these approaches are time consuming, requiring many classification calculations.In this paper, we offer a fundamentally different approach to the localization of recognized objects in images.Our method is predicated on the idea that a deep CNN capable of recognizing an object must implicitly contain knowledge about object location in its connection weights.We provide a simple method to interpret classifier weights in the context of individual classified images.This method involves the calculation of the derivative of network generated activation patterns, such as the activation of output class label units, with regard to each in- put pixel, performing a sensitivity analysis that identifies the pixels that, in a local sense, have the greatest influence on internal representations and object recognition.These derivatives can be efficiently computed using a single backward pass through the deep CNN classifier, producing a sensitivity map of the image.We demonstrate that a simple linear mapping can be learned from sensitivity maps to bounding box coordinates, localizing the recognized object.Our experimental results, using real-world data sets for which ground truth localization information is known, reveal competitive accuracy from our fast technique.

Vorschlag eines neuartigen Ansatzes zur Objektlokalisierung (Erkennung) auf der Grundlage der Interpretation des tiefen CNN unter Verwendung der internen Darstellung und der Gedanken des Netzwerks


We present trellis networks, a new architecture for sequence modeling.On the one hand, a trellis network is a temporal convolutional network with special structure, characterized by weight tying across depth and direct injection of the input into deep layers.On the other hand, we show that truncated recurrent networks are equivalent to trellis networks with special sparsity structure in their weight matrices.Thus trellis networks with general weight matrices generalize truncated recurrent networks.We leverage these connections to design high-performing trellis networks that absorb structural and algorithmic elements from both recurrent and convolutional models.Experiments demonstrate that trellis networks outperform the current state of the art methods on a variety of challenging benchmarks, including word-level language modeling and character-level language modeling tasks, and stress tests designed to evaluate long-term memory retention.The code is available at https://github.com/locuslab/trellisnet .

Trellis-Netzwerke sind eine neue Architektur zur Sequenzmodellierung, die rekurrente und konvolutionale Modelle miteinander verbindet und einen neuen Stand der Technik für die Sprachmodellierung auf Wort- und Zeichenebene darstellt.


We propose an end-to-end framework for training domain specific models (DSMs) to obtain both high accuracy and computational efficiency for object detection tasks.DSMs are trained with distillation and focus on achieving high accuracy at a limited domain (e.g. fixed view of an intersection).We argue that DSMs can capture essential features well even with a small model size, enabling higher accuracy and efficiency than traditional techniques.  In addition, we improve the training efficiency by reducing the dataset size by culling easy to classify images from the training set.For the limited domain, we observed that compact DSMs significantly surpass the accuracy of COCO trained models of the same size.By training on a compact dataset, we show that with an accuracy drop of only 3.6%, the training time can be reduced by 93%.

Eine hohe Objekterkennungsgenauigkeit kann durch das Training von domänenspezifischen kompakten Modellen erreicht werden, und das Training kann sehr kurz sein.


We compare the model-free reinforcement learning with the model-based approaches through the lens of the expressive power of neural networks for policies, $Q$-functions, and dynamics.  We show, theoretically and empirically, that even for one-dimensional continuous state space, there are many MDPs whose optimal $Q$-functions and policies are much more complex than the dynamics.We hypothesize many real-world MDPs also have a similar property.For these MDPs, model-based planning is a favorable algorithm, because the resulting policies can approximate the optimal policy significantly better than a neural network parameterization can, and model-free or model-based policy optimization rely on policy parameterization.Motivated by the theory, we apply a simple multi-step model-based bootstrapping planner (BOOTS) to bootstrap a weak $Q$-function into a stronger policy.Empirical results show that applying BOOTS on top of model-based or model-free policy optimization algorithms at the test time improves the performance on MuJoCo benchmark tasks.

Wir vergleichen tiefe modellbasierte und modellfreie RL-Algorithmen, indem wir die Approximierbarkeit von $Q$-Funktionen, Richtlinien und Dynamik durch neuronale Netze untersuchen.


Common-sense physical reasoning is an essential ingredient for any intelligent agent operating in the real-world.For example, it can be used to simulate the environment, or to infer the state of parts of the world that are currently unobserved.In order to match real-world conditions this causal knowledge must be learned without access to supervised data.To address this problem we present a novel method that learns to discover objects and model their physical interactions from raw visual images in a purely unsupervised fashion.It incorporates prior knowledge about the compositional nature of human perception to factor interactions between object-pairs and learn efficiently.On videos of bouncing balls we show the superior modelling capabilities of our method compared to other unsupervised neural approaches that do not incorporate such prior knowledge.We demonstrate its ability to handle occlusion and show that it can extrapolate learned knowledge to scenes with different numbers of objects.

Wir stellen einen neuartigen Ansatz für physikalische Schlussfolgerungen mit gesundem Menschenverstand vor, der lernt, Objekte zu erkennen und ihre physikalischen Interaktionen aus visuellen Rohbildern zu modellieren, und zwar auf rein unüberwachte Weise


The idea that neural networks may exhibit a bias towards simplicity has a long history.Simplicity bias provides a way to quantify this intuition.  It predicts, for a broad class of input-output maps which can describe many systems in science and engineering, that simple outputs are exponentially more likely to occur upon uniform random sampling of inputs than complex outputs are.  This simplicity bias behaviour has been observed for systems ranging from the RNA sequence to secondary structure map, to systems of coupled differential equations, to models of plant growth.   Deep neural networks can be viewed as a mapping from the space of parameters (the weights) to the space of functions (how inputs get transformed to  outputs by the network).  We show that this parameter-function map obeys the necessary  conditions for simplicity bias, and numerically show that it is hugely biased towards functions with low descriptional complexity.  We also demonstrate a Zipf like power-law probability-rank relation.   A bias towards simplicity may help explain why neural nets generalize so well.

Bei vielen einfachen Input-Output-Zuordnungen ist eine sehr starke Tendenz zu einfachen Outputs zu beobachten. Die Parameter-Funktions Zuordnung von tiefen Netzen ist auf die gleiche Weise verzerrt.


Imitation Learning (IL) is an appealing approach to learn desirable autonomous behavior.However, directing IL to achieve arbitrary goals is difficult.In contrast, planning-based algorithms use dynamics models and reward functions to achieve goals.Yet, reward functions that evoke desirable behavior are often difficult to specify.In this paper, we propose "Imitative Models" to combine the benefits of IL and goal-directed planning.Imitative Models are probabilistic predictive models of desirable behavior able to plan interpretable expert-like trajectories to achieve specified goals.We derive families of flexible goal objectives, including constrained goal regions, unconstrained goal sets, and energy-based goals.We show that our method can use these objectives to successfully direct behavior.Our method substantially outperforms six IL approaches and a planning-based approach in a dynamic simulated autonomous driving task, and is efficiently learned from expert demonstrations without online data collection.  We also show our approach is robust to poorly-specified goals, such as goals on the wrong side of the road.

In diesem Beitrag schlagen wir Imitative Modelle vor, um die Vorteile von IL und zielgerichteter Planung zu kombinieren: probabilistische Vorhersagemodelle für wünschenswertes Verhalten, die in der Lage sind, interpretierbare expertenähnliche Trajektorien zu planen, um bestimmte Ziele zu erreichen.


Learning communication via deep reinforcement learning has recently been shown to be an effective way to solve cooperative multi-agent tasks.However, learning which communicated information is beneficial for each agent's decision-making remains a challenging task.In order to address this problem, we introduce a fully differentiable framework for communication and reasoning, enabling agents to solve cooperative tasks in partially-observable environments.The framework is designed to facilitate explicit reasoning between agents, through a novel memory-based attention network that can learn selectively from its past memories.The model communicates through a series of reasoning steps that decompose each agent's intentions into learned representations that are used first to compute the relevance of communicated information, and second to extract information from memories given newly received information.By selectively interacting with new information, the model effectively learns a communication protocol directly, in an end-to-end manner.We empirically demonstrate the strength of our model in cooperative multi-agent tasks, where inter-agent communication and reasoning over prior information substantially improves performance compared to baselines.

Neuartige Architektur eines speicherbasierten Aufmerksamkeitsmechanismus für Multi-Agenten-Kommunikation.


In this paper, we introduce Symplectic ODE-Net (SymODEN), a deep learning framework which can infer the dynamics of a physical system from observed state trajectories.To achieve better generalization with fewer training samples, SymODEN incorporates appropriate inductive bias by designing the associated computation graph in a physics-informed manner.In particular, we enforce Hamiltonian dynamics with control to learn the underlying dynamics in a transparent way which can then be leveraged to draw insight about relevant physical aspects of the system, such as mass and potential energy.In addition, we propose a parametrization which can enforce this Hamiltonian formalism even when the generalized coordinate data is embedded in a high-dimensional space or we can only access velocity data instead of generalized momentum.This framework, by offering interpretable, physically-consistent models for physical systems, opens up new possibilities for synthesizing model-based control strategies.

Diese Arbeit erzwingt eine Hamilton-Dynamik mit Steuerung, um Systemmodelle aus eingebetteten Positions- und Geschwindigkeitsdaten zu lernen, und nutzt diese physikalisch konsistente Dynamik, um eine modellbasierte Steuerung durch Energieformung zu synthetisieren.


Federated learning, where a global model is trained by iterative parameter averaging of locally-computed updates, is a promising approach for distributed training of deep networks; it provides high communication-efficiency and privacy-preservability, which allows to fit well into decentralized data environments, e.g., mobile-cloud ecosystems.However, despite the advantages, the federated learning-based methods still have a challenge in dealing with non-IID training data of local devices (i.e., learners).In this regard, we study the effects of a variety of hyperparametric conditions under the non-IID environments, to answer important concerns in practical implementations:(i) We first investigate parameter divergence of local updates to explain performance degradation from non-IID data.The origin of the parameter divergence is also found both empirically and theoretically.(ii) We then revisit the effects of optimizers, network depth/width, and regularization techniques; our observations show that the well-known advantages of the hyperparameter optimization strategies could rather yield diminishing returns with non-IID data.(iii) We finally provide the reasons of the failure cases in a categorized way, mainly based on metrics of the parameter divergence.

Wir untersuchen die internen Gründe für unsere Beobachtungen, nämlich die abnehmenden Auswirkungen der bekannten Hyperparameter-Optimierungsmethoden auf das föderierte Lernen aus dezentralen Nicht-IID-Daten.


Deep learning models are known to be vulnerable to adversarial examples.A practical adversarial attack should require as little as possible knowledge of attacked models T. Current substitute attacks need pre-trained models to generate adversarial examples and their attack success rates heavily rely on the transferability of adversarial examples.Current score-based and decision-based attacks require lots of queries for the T. In this study, we propose a novel adversarial imitation attack.First, it produces a replica of the T by a two-player game like the generative adversarial networks (GANs).The objective of the generative model G is to generate examples which lead D returning different outputs with T. The objective of the discriminative model D is to output the same labels with T under the same inputs.Then, the adversarial examples generated by D are utilized to fool the T. Compared with the current substitute attacks, imitation attack can use less training data to produce a replica of T and improve the transferability of adversarial examples.Experiments demonstrate that our imitation attack requires less training data than the black-box substitute attacks, but achieves an attack success rate close to the white-box attack on unseen data with no query.

Ein neuartiger Nachahmungsangriff zur Täuschung von Modellen des maschinellen Lernens.


Stochastic Gradient Descent (SGD) methods using randomly selected batches are widely-used to train neural network (NN) models.Performing design exploration to find the best NN for a particular task often requires extensive training with different models on a large dataset,  which is very computationally expensive.The most straightforward method to accelerate this computation is to distribute the batch of SGD over multiple processors.However, large batch training often times leads to degradation in accuracy, poor generalization, and even poor robustness to adversarial attacks.  Existing solutions for large batch training either do not work or require massive hyper-parameter tuning.To address this issue, we propose a novel large batch training method which combines recent results in adversarial training (to regularize against ``sharp minima'') and second order optimization (to use curvature information to change batch size adaptively during training).We extensively evaluate our method on Cifar-10/100, SVHN, TinyImageNet, and ImageNet datasets, using multiple NNs, including residual networks as well as compressed networks such as SqueezeNext.  Our new approach exceeds the performance of the existing solutions in terms of both accuracy and the number of SGD iterations (up to 1\% and $3\times$, respectively).We emphasize that this is achieved without any additional hyper-parameter tuning to tailor our method to any of these experiments.

Training mit großen Batches unter Verwendung von adversarialem Training und Informationen zweiter Ordnung


Inspired by neurophysiological discoveries of navigation cells in the mammalianbrain, we introduce the first deep neural network architecture for modeling EgocentricSpatial Memory (ESM).It learns to estimate the pose of the agent andprogressively construct top-down 2D global maps from egocentric views in a spatiallyextended environment.During the exploration, our proposed ESM networkmodel updates belief of the global map based on local observations using a recurrentneural network.It also augments the local mapping with a novel externalmemory to encode and store latent representations of the visited places based ontheir corresponding locations in the egocentric coordinate.This enables the agentsto perform loop closure and mapping correction.This work contributes in thefollowing aspects: first, our proposed ESM network provides an accurate mappingability which is vitally important for embodied agents to navigate to goal locations.In the experiments, we demonstrate the functionalities of the ESM network inrandom walks in complicated 3D mazes by comparing with several competitivebaselines and state-of-the-art Simultaneous Localization and Mapping (SLAM)algorithms.Secondly, we faithfully hypothesize the functionality and the workingmechanism of navigation cells in the brain.Comprehensive analysis of our modelsuggests the essential role of individual modules in our proposed architecture anddemonstrates efficiency of communications among these modules.We hope thiswork would advance research in the collaboration and communications over bothfields of computer science and computational neuroscience.

erstes tiefes neuronales Netz zur Modellierung des egozentrischen räumlichen Gedächtnisses, inspiriert durch neurophysiologische Entdeckungen von Navigationszellen im Säugetiergehirn


We show that if the usual training loss is augmented by a Lipschitz regularization term, then the networks generalize.  We prove generalization by first establishing a stronger convergence result, along with a rate of convergence.   A second result resolves a question posed in Zhang et al. (2016): how can a model distinguish between the case of clean labels, and randomized labels?  Our answer is that Lipschitz regularization using the Lipschitz constant of the clean data makes this distinction.  In this case, the model learns a different function which we hypothesize correctly fails to learn the dirty labels.

Wir beweisen die Generalisierung von DNNs durch Hinzufügen eines Lipschitz-Regularisierungsterms zum Trainingsverlust. Wir lösen eine Frage, die in Zhang et al. (2016) gestellt wurde.


For fast and energy-efficient deployment of trained deep neural networks on resource-constrained embedded hardware,  each learned weight parameter should ideally be represented and stored using a single bit.  Error-rates usually increase when this requirement is imposed.Here, we report large improvements in error rates on multiple datasets, for deep convolutional neural networks deployed with 1-bit-per-weight.Using wide residual networks as our main baseline, our approach simplifies existing methods that binarize weights by applying the sign function in training; we apply  scaling factors for each layer with constant unlearned values equal to the layer-specific standard deviations used for initialization.For CIFAR-10, CIFAR-100 and ImageNet, and models with 1-bit-per-weight requiring less than 10 MB of parameter memory, we achieve error rates of 3.9%, 18.5% and 26.0% / 8.5% (Top-1 / Top-5) respectively.We also considered MNIST, SVHN and ImageNet32, achieving 1-bit-per-weight test results of 0.27%, 1.9%, and 41.3% / 19.1%  respectively.For CIFAR, our error rates halve previously reported values, and are within about 1% of our error-rates for the same network with full-precision weights.For networks that overfit, we also show significant improvements in error rate by not learning batch normalization scale and offset parameters.This applies to both full precision and 1-bit-per-weight networks.Using a warm-restart learning-rate schedule, we found that training for 1-bit-per-weight is just as fast as full-precision networks, with better accuracy than standard schedules, and achieved about 98%-99% of peak performance in just 62 training epochs for CIFAR-10/100.For full training code and trained models in MATLAB, Keras and PyTorch see https://github.com/McDonnell-Lab/1-bit-per-weight/ .

Wir trainieren breite Residualnetzwerke, die sofort eingesetzt werden können, indem wir nur ein einziges Bit für jedes Convolutional Gewicht verwenden, und zwar mit deutlich besserer Genauigkeit als bisherige Methoden.


This paper presents a system for immersive visualization of Non-Euclidean spaces using real-time ray tracing.It exploits the capabilities of the new generation of GPU’s based on the NVIDIA’s Turing architecture in order to develop new methods for intuitive exploration of landscapes featuring non-trivial geometry and topology in virtual reality.

Immersive Visualisierung der klassischen nicht-euklidischen Räume mit Echtzeit-Raytracing.


We propose the set autoencoder, a model for unsupervised representation learning for sets of elements.It is closely related to sequence-to-sequence models, which learn fixed-sized latent representations for sequences, and have been applied to a number of challenging supervised sequence tasks such as machine translation, as well as unsupervised representation learning for sequences.In contrast to sequences, sets are permutation invariant.The proposed set autoencoder considers this fact, both with respect to the input as well as the output of the model.On the input side, we adapt a recently-introduced recurrent neural architecture using a content-based attention mechanism.On the output side, we use a stable marriage algorithm to align predictions to labels in the learning phase.We train the model on synthetic data sets of point clouds and show that the learned representations change smoothly with translations in the inputs, preserve distances in the inputs, and that the set size is represented directly.We apply the model to supervised tasks on the point clouds using the fixed-size latent representation.For a number of difficult classification problems, the results are better than those of a model that does not consider the permutation invariance.Especially for small training sets, the set-aware model benefits from unsupervised pretraining.

Wir schlagen den Set-Autoencoder vor, ein Modell zum unbeaufsichtigten Lernen von Repräsentationen für Mengen von Elementen.


Deep reinforcement learning algorithms can learn complex behavioral skills, but real-world application of these methods requires a considerable amount of experience to be collected by the agent.In practical settings, such as robotics, this involves repeatedly attempting a task, resetting the environment between each attempt.However, not all tasks are easily or automatically reversible.In practice, this learning process requires considerable human intervention.In this work, we propose an autonomous method for safe and efficient reinforcement learning that simultaneously learns a forward and backward policy, with the backward policy resetting the environment for a subsequent attempt.By learning a value function for the backward policy, we can automatically determine when the forward policy is about to enter a non-reversible state, providing for uncertainty-aware safety aborts.Our experiments illustrate that proper use of the backward policy can greatly reduce the number of manual resets required to learn a task and can reduce the number of unsafe actions that lead to non-reversible states.

Wir schlagen eine autonome Methode für sicheres und effizientes Reinforcement Learning vor, die gleichzeitig eine Vorwärts- und eine Rückwärtsstrategie erlernt, wobei die Rückwärtsstrategie die Umgebung für einen nachfolgenden Versuch zurücksetzt.


It has been argued that current machine learning models do not have commonsense, and therefore must be hard-coded with prior knowledge (Marcus, 2018).Here we show surprising evidence that language models can already learn to capture certain common sense knowledge.Our key observation is that a language model can compute the probability of any statement, and this probability can be used to evaluate the truthfulness of that statement.  On the Winograd Schema Challenge (Levesque et al., 2011), language models are 11% higher in accuracy than previous state-of-the-art supervised methods.Language models can also be fine-tuned for the task of Mining Commonsense Knowledge on ConceptNet to achieve an F1 score of 0.912 and 0.824, outperforming previous best results (Jastrzebskiet al., 2018).  Further analysis demonstrates that language models can discover unique features of Winograd Schema contexts that decide the correct answers without explicit supervision.

Wir präsentieren Beweise dafür, dass LMs den gesunden Menschenverstand erfassen, mit Spitzenergebnissen sowohl bei der Winograd Schema Challenge als auch beim Commonsense Knowledge Mining.