In this paper, we propose two methods, namely Trace-norm regression (TNR) and Stable Trace-norm Analysis (StaTNA), to improve performances of recommender systems with side information.Our trace-norm regression approach extracts low-rank latent factors underlying the side information that drives user preference under different context.Furthermore, our novel recommender framework StaTNA not only captures latent low-rank common drivers for user preferences, but also considers idiosyncratic taste for individual users.We compare performances of TNR and StaTNA on the MovieLens datasets against state-of-the-art models, and demonstrate that StaTNA and TNR in general outperforms these methods.

Methoden für Empfehlungssysteme mit Nebeninformationen auf der Grundlage der Spurennorm-Regularisierung.


This work presents an exploration and imitation-learning-based agent capable of state-of-the-art performance in playing text-based computer games.Text-based computer games describe their world to the player through natural language and expect the player to interact with the game using text.These games are of interest as they can be seen as a testbed for language understanding, problem-solving, and language generation by artificial agents.Moreover, they provide a learning environment in which these skills can be acquired through interactions with an environment rather than using fixed corpora. One aspect that makes these games particularly challenging for learning agents is the combinatorially large action space.Existing methods for solving text-based games are limited to games that are either very simple or have an action space restricted to a predetermined set of admissible actions.In this work, we propose to use the exploration approach of Go-Explore (Ecoffet et al., 2019) for solving text-based games.More specifically, in an initial exploration phase, we first extract trajectories with high rewards, after which we train a policy to solve the game by imitating these trajectories.Our experiments show that this approach outperforms existing solutions in solving text-based games, and it is more sample efficient in terms of the number of interactions with the environment.Moreover, we show that the learned policy can generalize better than existing solutions to unseen games without using any restriction on the action space.

In dieser Arbeit wird ein auf Exploration und Imitation Learning basierender Agent vorgestellt, der beim Spielen textbasierter Computerspiele Spitzenleistungen erbringt.


The recent “Lottery Ticket Hypothesis” paper by Frankle & Carbin showed that a simple approach to creating sparse networks (keep the large weights) results in models that are trainable from scratch, but only when starting from the same initial weights.The performance of these networks often exceeds the performance of the non-sparse base model, but for reasons that were not well understood.In this paper we study the three critical components of the Lottery Ticket (LT) algorithm, showing that each may be varied significantly without impacting the overall results.Ablating these factors leads to new insights for why LT networks perform as well as they do.We show why setting weights to zero is important, how signs are all you need to make the re-initialized network train, and why masking behaves like training.Finally, we discover the existence of Supermasks, or masks that can be applied to an untrained, randomly initialized network to produce a model with performance far better than chance (86% on MNIST, 41% on CIFAR-10).

Beim Pruning von neuronalen Netzen ist es wichtig, die beschnittenen Gewichte auf Null zu setzen, das Zeichen der Initialisierung ist entscheidend, und die Maskierung kann als Training angesehen werden.


Fine-tuning with pre-trained models has achieved exceptional results for many language tasks.In this study, we focused on one such self-attention network model, namely BERT, which has performed well in terms of stacking layers across diverse language-understanding benchmarks.However, in many downstream tasks, information between layers is ignored by BERT for fine-tuning.In addition, although self-attention networks are well-known for their ability to capture global dependencies, room for improvement remains in terms of emphasizing the importance of local contexts.In light of these advantages and disadvantages, this paper proposes SesameBERT, a generalized fine-tuning method that (1) enables the extraction of global information among all layers through Squeeze and Excitation and (2) enriches local information by capturing neighboring contexts via Gaussian blurring.Furthermore, we demonstrated the effectiveness of our approach in the HANS dataset, which is used to determine whether models have adopted shallow heuristics instead of learning underlying generalizations.The experiments revealed that SesameBERT outperformed BERT with respect to GLUE benchmark and the HANS evaluation set.

Wir haben SesameBERT vorgeschlagen, eine verallgemeinerte Feinabstimmungsmethode, die die Extraktion globaler Informationen zwischen allen Schichten durch Squeeze und Excitation ermöglicht und lokale Informationen durch die Erfassung benachbarter Kontexte mittels Gaußscher Unschärfe anreichert.


A growing number of learning methods are actually differentiable games whose players optimise multiple, interdependent objectives in parallel – from GANs and intrinsic curiosity to multi-agent RL.Opponent shaping is a powerful approach to improve learning dynamics in these games, accounting for player influence on others’ updates.Learning with Opponent-Learning Awareness (LOLA) is a recent algorithm that exploits this response and leads to cooperation in settings like the Iterated Prisoner’s Dilemma.Although experimentally successful, we show that LOLA agents can exhibit ‘arrogant’ behaviour directly at odds with convergence.In fact, remarkably few algorithms have theoretical guarantees applying across all (n-player, non-convex) games.In this paper we present Stable Opponent Shaping (SOS), a new method that interpolates between LOLA and a stable variant named LookAhead.We prove that LookAhead converges locally to equilibria and avoids strict saddles in all differentiable games.SOS inherits these essential guarantees, while also shaping the learning of opponents and consistently either matching or outperforming LOLA experimentally.

Opponent Shaping ist ein leistungsfähiger Ansatz für das Lernen von Multi-Agenten, kann aber Konvergenz verhindern; unser SOS-Algorithmus behebt dieses Problem mit starken Garantien in allen differenzierbaren Spielen.


The rate at which medical questions are asked online significantly exceeds the capacity of qualified people to answer them, leaving many questions unanswered or inadequately answered.Many of these questions are not unique, and reliable identification of similar questions would enable more efficient and effective question answering schema.While many research efforts have focused on the problem of general question similarity, these approaches do not generalize well to the medical domain, where medical expertise is often required to determine semantic similarity.In this paper, we show how a semi-supervised approach of pre-training a neural network on medical question-answer pairs is a particularly useful intermediate task for the ultimate goal of determining medical question similarity.While other pre-training tasks yield an accuracy below 78.7% on this task, our model achieves an accuracy of 82.6% with the same number of training examples, an accuracy of 80.0% with a much smaller training set, and an accuracy of 84.5% when the full corpus of medical question-answer data is used.

Wir zeigen, dass das Frage-Antwort-Matching eine besonders gute Pre-Training-Aufgabe für Fragenähnlichkeit ist und veröffentlichen einen Datensatz für medizinische Fragenähnlichkeit


We study the benefit of sharing representations among tasks to enable the effective use of deep neural networks in Multi-Task Reinforcement Learning.We leverage the assumption that learning from different tasks, sharing common properties, is helpful to generalize the knowledge of them resulting in a more effective feature extraction compared to learning a single task.Intuitively, the resulting set of features offers performance benefits when used by Reinforcement Learning algorithms.We prove this by providing theoretical guarantees that highlight the conditions for which is convenient to share representations among tasks, extending the well-known finite-time bounds of Approximate Value-Iteration to the multi-task setting.In addition, we complement our analysis by proposing multi-task extensions of three Reinforcement Learning algorithms that we empirically evaluate on widely used Reinforcement Learning benchmarks showing significant improvements over the single-task counterparts in terms of sample efficiency and performance.

Eine Studie über den Nutzen der gemeinsamen Repräsentation beim Multi-Task Reinforcement Learning.


We present a 3D capsule architecture for processing of point clouds that is equivariant with respect to the SO(3) rotation group, translation and permutation of the unordered input sets.The network operates on a sparse set of local reference frames, computed from an input point cloud and establishes end-to-end equivariance through a novel 3D quaternion group capsule layer, including an equivariant dynamic routing procedure.The capsule layer enables us to disentangle geometry from pose, paving the way for more informative descriptions and a structured latent space.In the process, we theoretically connect the process of dynamic routing between capsules to the well-known Weiszfeld algorithm, a scheme for solving iterative re-weighted least squares (IRLS) problems with provable convergence properties, enabling robust pose estimation between capsule layers.Due to the sparse equivariant quaternion capsules, our architecture allows joint object classification and orientation estimation, which we validate empirically on common benchmark datasets.

Tiefgehende Architekturen für 3D-Punktwolken, die äquivariant zu SO(3)-Drehungen sowie zu Translationen und Permutationen sind.


Vector semantics, especially sentence vectors, have recently been used successfully in many areas of natural language processing.However, relatively little work has explored the internal structure and properties of spaces of sentence vectors.In this paper, we will explore the properties of sentence vectors by studying a particular real-world application: Automatic Summarization.In particular, we show that cosine similarity between sentence vectors and document vectors is strongly correlated with sentence importance and that vector semantics can identify and correct gaps between the sentences chosen so far and the document.In addition, we identify specific dimensions which are linked to effective summaries.To our knowledge, this is the first time specific dimensions of sentence embeddings have been connected to sentence properties.We also compare the features of different methods of sentence embeddings.Many of these insights have applications in uses of sentence embeddings far beyond summarization.

Ein Vergleich und eine detaillierte Analyse verschiedener Modelle zur Satzeinbettung anhand der realen Aufgabe der automatischen Zusammenfassung.


We present Value Propagation (VProp), a parameter-efficient differentiable planning module built on Value Iteration which can successfully be trained in a reinforcement learning fashion to solve unseen tasks, has the capability to generalize to larger map sizes, and can learn to navigate in dynamic environments.We evaluate on configurations of MazeBase grid-worlds, with randomly generated environments of several different sizes.Furthermore, we show that the module enables to learn to plan when the environment also includes stochastic elements, providing a cost-efficient learning system to build low-level size-invariant planners for a variety of interactive navigation problems.

Wir schlagen Value Propagation vor, einen neuartigen Ende-zu-Ende Planer, der lernen kann, 2D-Navigationsaufgaben durch Reinforcement Learning zu lösen, und der sich auf größere und dynamische Umgebungen verallgemeinern lässt.


Recommendation is a prevalent application of machine learning that affects many users; therefore, it is crucial for recommender models to be accurate and interpretable.In this work, we propose a method to both interpret and augment the predictions of black-box recommender systems.In particular, we propose to extract feature interaction interpretations from a source recommender model and explicitly encode these interactions in a target recommender model, where both source and target models are black-boxes.By not assuming the structure of the recommender system, our approach can be used in general settings.  In our experiments, we focus on a prominent use of machine learning recommendation: ad-click prediction.We found that our interaction interpretations are both informative and predictive, i.e., significantly outperforming existing recommender models.What's more, the same approach to interpreting interactions can  provide new insights into domains even beyond recommendation.

Vorschlag für eine Methode zur Extraktion und Nutzung von Interpretationen der Wechselwirkungen zwischen Merkmalen


Rectified linear units, or ReLUs, have become a preferred activation function for artificial neural networks.In this paper we consider the problem of learning a generative model in the presence of nonlinearity (modeled by the ReLU functions).Given a set of signal vectors $\mathbf{y}^i \in \mathbb{R}^d, i =1, 2, \dots , n$, we  aim to learn the network parameters, i.e., the $d\times k$ matrix $A$, under the model $\mathbf{y}^i = \mathrm{ReLU}(A\mathbf{c}^i +\mathbf{b})$, where $\mathbf{b}\in \mathbb{R}^d$ is a random bias vector, and {$\mathbf{c}^i \in \mathbb{R}^k$ are arbitrary unknown latent vectors}.We show that it is possible to recover the column space of $A$ within an error of $O(d)$ (in Frobenius norm) under certain conditions on the  distribution of $\mathbf{b}$.

Wir zeigen, dass es möglich ist, die Parameter eines 1-schichtigen generativen ReLU-Modells aus der Betrachtung der von ihm erzeugten Beispiele zu ermitteln.


Methods that calculate dense vector representations for features in unstructured data—such as words in a document—have proven to be very successful for knowledge representation.We study how to estimate dense representations when multiple feature types exist within a dataset for supervised learning where explicit labels are available, as well as for unsupervised learning where there are no labels.Feat2Vec calculates embeddings for data with multiple feature types enforcing that all different feature types exist in a common space.In the supervised case, we show that our method has advantages over recently proposed methods; such as enabling higher prediction accuracy, and providing a way to avoid the cold-startproblem.In the unsupervised case, our experiments suggest that Feat2Vec significantly outperforms existing algorithms that do not leverage the structure of the data.We believe that we are the first to propose a method for learning unsuper vised embeddings that leverage the structure of multiple feature types.

Lernen von dichten Vektordarstellungen beliebiger Arten von Merkmalen in beschrifteten und unbeschrifteten Datensätzen.


We investigate the internal representations that a recurrent neural network (RNN) uses while learning to recognize a regular formal language.Specifically, we train a RNN on positive and negative examples from a regular language, and ask if there is a simple decoding function that maps states of this RNN to states of the minimal deterministic finite automaton (MDFA) for the language.Our experiments show that such a decoding function indeed exists, and that it maps states of the RNN not to MDFA states, but to states of an {\em abstraction} obtained by clustering small sets of MDFA states into ``''superstates''.A qualitative analysis reveals that the abstraction often has a simple interpretation.Overall, the results suggest a strong structural relationship between internal representations used by RNNs and finite automata, and explain the well-known ability of RNNs to recognize formal grammatical structure.

Endliche Automaten können linear aus spracherkennenden RNNs dekodiert werden, indem Abstraktionsfunktionen mit geringer Grobheit und hochgenaue Dekodierer verwendet werden.


Designing accurate and efficient convolutional neural architectures for vast amount of hardware is challenging because hardware designs are complex and diverse.This paper addresses the hardware diversity challenge in Neural Architecture Search (NAS).Unlike previous approaches that apply search algorithms on a small, human-designed search space without considering hardware diversity, we propose HURRICANE that explores the automatic hardware-aware search over a much larger search space and a multistep search scheme in coordinate ascent framework, to generate tailored models for different types of hardware.Extensive experiments on  ImageNet show that our algorithm consistently achieves a much lower inference latency with a similar or better accuracy than state-of-the-art NAS methods on three types of hardware.Remarkably, HURRICANE achieves a 76.63% top-1 accuracy on ImageNet with a inference latency of only 16.5 ms for DSP, which is a 3.4% higher accuracy and a 6.35x inference speedup than FBNet-iPhoneX.For VPU, HURRICANE achieves a 0.53% higher top-1 accuracy than Proxyless-mobile with a 1.49x speedup.Even for well-studied mobile CPU, HURRICANE achieves a 1.63% higher top-1  accuracy than FBNet-iPhoneX with a comparable inference latency.HURRICANE also reduces the training time by 54.7% on average compared to SinglePath-Oneshot.

Wir schlagen HURRICANE vor, um die Herausforderung der Hardware-Vielfalt bei der Suche in einer neuronalen One-Shot-Architektur anzugehen.


In this paper, we present Neural Phrase-based Machine Translation (NPMT).Our method explicitly models the phrase structures in output sequences using Sleep-WAke Networks (SWAN), a recently proposed segmentation-based sequence modeling method.To mitigate the monotonic alignment requirement of SWAN, we introduce a new layer to perform (soft) local reordering of input sequences.Different from existing neural machine translation (NMT) approaches, NPMT does not use attention-based decoding mechanisms.  Instead, it directly outputs phrases in a sequential order and can decode in linear time.Our experiments show that NPMT achieves superior performances on IWSLT 2014 German-English/English-German and IWSLT 2015 English-Vietnamese machine translation tasks compared with strong NMT baselines.We also observe that our method produces meaningful phrases in output languages.

Neuronale phrasenbasierte maschinelle Übersetzung mit linearer Dekodierzeit.


Generative Adversarial Networks (GANs) have shown impressive results in modeling distributions over complicated manifolds such as those of natural images.However, GANs often suffer from mode collapse, which means they are prone to characterize only a single or a few modes of the data distribution.In order to address this problem, we propose a novel framework called LDMGAN.We ﬁrst introduce Latent Distribution Matching (LDM) constraint which regularizes the generator by aligning distribution of generated samples with that of real samples in latent space.To make use of such latent space, we propose a regularized AutoEncoder (AE) that maps the data distribution to prior distribution in encoded space.Extensive experiments on synthetic data and real world datasets show that our proposed framework signiﬁcantly improves GAN’s stability and diversity.

Wir schlagen ein AE-basiertes GAN vor, das den Mode-Kollaps in GANs mildert.


Achieving faster execution with shorter compilation time can foster further diversity and innovation in neural networks.However, the current paradigm of executing neural networks either relies on hand-optimized libraries, traditional compilation heuristics, or very recently genetic algorithms and other stochastic methods.These methods suffer from frequent costly hardware measurements rendering them not only too time consuming but also suboptimal.As such, we devise a solution that can learn to quickly adapt to a previously unseen design space for code optimization, both accelerating the search and improving the output performance.This solution dubbed CHAMELEON leverages reinforcement learning whose solution takes fewer steps to converge, and develops an adaptive sampling algorithm that not only focuses on the costly samples (real hardware measurements) on representative points but also uses a domain knowledge inspired logic to improve the samples itself.Experimentation with real hardware shows that CHAMELEON provides 4.45×speed up in optimization time over AutoTVM, while also improving inference time of the modern deep networks by 5.6%.

Reinforcement Learning und adaptives Sampling für optimierte Kompilierung von tiefen neuronalen Netzen.


In this paper, we propose a differentiable adversarial grammar model for future prediction.The objective is to model a formal grammar in terms of differentiable functions and latent representations, so that their learning is possible through standard backpropagation.Learning a formal grammar represented with latent terminals, non-terminals, and productions rules allows capturing sequential structures with multiple possibilities from data.The adversarial grammar is designed so that it can learn stochastic production rules from the data distribution.Being able to select multiple production rules leads to different predicted outcomes, thus efficiently modeling many plausible futures.  We confirm the benefit of the adversarial grammar on two diverse tasks: future 3D human pose prediction and future activity prediction.For all settings, the proposed adversarial grammar outperforms the state-of-the-art approaches, being able to predict much more accurately and further in the future, than prior work.

Wir entwerfen eine Grammatik, die in einer kontradiktorischen Umgebung gelernt wird, und wenden sie auf die Zukunftsvorhersage in Videos an.


We consider a simple and overarching representation for permutation-invariant functions of sequences (or set functions).Our approach, which we call Janossy pooling, expresses a permutation-invariant function as the average of a permutation-sensitive function applied to all reorderings of the input sequence.This allows us to leverage the rich and mature literature on permutation-sensitive functions to construct novel and flexible permutation-invariant functions.If carried out naively, Janossy pooling can be computationally prohibitive.To allow computational tractability, we consider three kinds of approximations: canonical orderings of sequences, functions with k-order interactions, and stochastic optimization algorithms with random permutations.Our framework unifies a variety of existing work in the literature, and suggests possible modeling and algorithmic extensions.We explore a few in our experiments, which demonstrate improved performance over current state-of-the-art methods.

Wir schlagen Janossy-Pooling vor, eine Methode zum Erlernen tiefer permutationsinvarianter Funktionen, die entwickelt wurde, um Beziehungen innerhalb der Eingabesequenz und nachvollziehbare Inferenzstrategien wie ein stochastisches Optimierungsverfahren, das wir piSGD nennen, zu nutzen.


While tasks could come with varying the number of instances and classes in realistic settings, the existing meta-learning approaches for few-shot classification assume that number of instances per task and class is fixed.Due to such restriction, they learn to equally utilize the meta-knowledge across all the tasks, even when the number of instances per task and class largely varies.Moreover, they do not consider distributional difference in unseen tasks, on which the meta-knowledge may have less usefulness depending on the task relatedness.To overcome these limitations, we propose a novel meta-learning model that adaptively balances the effect of the meta-learning and task-specific learning within each task.Through the learning of the balancing variables, we can decide whether to obtain a solution by relying on the meta-knowledge or task-specific learning.We formulate this objective into a Bayesian inference framework and tackle it using variational inference.We validate our Bayesian Task-Adaptive Meta-Learning (Bayesian TAML) on two realistic task- and class-imbalanced datasets, on which it significantly outperforms existing meta-learning approaches.Further ablation study confirms the effectiveness of each balancing component and the Bayesian learning framework.

Ein neuartiges Meta-Lernmodell, das die Auswirkungen des Meta-Lernens und des aufgabenspezifischen Lernens sowie des klassenspezifischen Lernens innerhalb jeder Aufgabe adaptiv ausgleicht.


Many tasks in artificial intelligence require the collaboration of multiple agents.We exam deep reinforcement learning for multi-agent domains.Recent research efforts often take the form of two seemingly conflicting perspectives, the decentralized perspective, where each agent is supposed to have its own controller; and the centralized perspective, where one assumes there is a larger model controlling all agents.In this regard, we revisit the idea of the master-slave architecture by incorporating both perspectives within one framework.Such a hierarchical structure naturally leverages advantages from one another.The idea of combining both perspective is intuitive and can be well motivated from many real world systems, however, out of a variety of possible realizations, we highlights three key ingredients, i.e. composed action representation, learnable communication and independent reasoning.With network designs to facilitate these explicitly, our proposal consistently outperforms latest competing methods both in synthetics experiments and when applied to challenging StarCraft  micromanagement tasks.

Wir greifen die Idee der Master-Slave-Architektur im Multi-Agenten Reinforcement Learning wieder auf und übertreffen den Stand der Technik.


We study the implicit bias of gradient descent methods in solving a binary classification problem over a linearly separable dataset.The classifier is described by a nonlinear ReLU model and the objective function adopts the exponential loss function.We first characterize the landscape of the loss function and show that there can exist spurious asymptotic local minima besides asymptotic global minima.We then show that gradient descent (GD) can converge to either a global or a local max-margin direction, or may diverge from the desired max-margin direction in a general context.For stochastic gradient descent (SGD), we show that it converges in expectation to either the global or the local max-margin direction if SGD converges.We further explore the implicit bias of these algorithms in learning a multi-neuron network under certain stationary conditions, and show that the learned classifier maximizes the margins of each sample pattern partition under the ReLU activation.

Wir untersuchen die implizite Verzerrung von Gradientenmethoden bei der Lösung eines binären Klassifikationsproblems mit nichtlinearen ReLU-Modellen.


Model pruning has become a useful technique that improves the computational efficiency of deep learning, making it possible to deploy solutions in resource-limited scenarios.A widely-used practice in relevant work assumes that a smaller-norm parameter or feature plays a less informative role at the inference time.In this paper, we propose a channel pruning technique for accelerating the computations of deep convolutional neural networks (CNNs) that does not critically rely on this assumption.Instead, it focuses on direct simplification of the channel-to-channel computation graph of a CNN without the need of performing a computationally difficult and not-always-useful task of making high-dimensional tensors of CNN structured sparse.Our approach takes two stages: first to adopt an end-to-end stochastic training method that eventually forces the outputs of some channels to be constant, and then to prune those constant channels from the original neural network by adjusting the biases of their impacting layers such that the resulting compact model can be quickly fine-tuned.Our approach is mathematically appealing from an optimization perspective and easy to reproduce.We experimented our approach through several image learning benchmarks and demonstrate its interest- ing aspects and competitive performance.

Eine Pruning-Methode für CNN-Modelle, die ISTA und einen Reskalierungstrick verwendet, um die Spärlichkeit der Skalierungsparameter bei der Stapelnormalisierung zu erzwingen.


Stochastic gradient descent (SGD) has been the dominant optimization method for training deep neural networks due to its many desirable properties.One of the more remarkable and least understood quality of SGD is that it generalizes relatively wellon unseen data even when the neural network has millions of parameters.We hypothesize that in certain cases it is desirable to relax its intrinsic generalization properties and introduce an extension of SGD called deep gradient boosting (DGB).The key idea of DGB is that back-propagated gradients inferred using the chain rule can be viewed as pseudo-residual targets of a gradient boosting problem.Thus at each layer of a neural network the weight update is calculated by solving the corresponding boosting problem using a linear base learner.The resulting weight update formula can also be viewed as a normalization procedure of the data that arrives at each layer during the forward pass.When implemented as a separate input normalization layer (INN) the new architecture shows improved performance on image recognition tasks when compared to the same architecture without normalization layers.As opposed to batch normalization (BN), INN has no learnable parameters however it matches its performance on CIFAR10 and ImageNet classification tasks.

Was können wir über das Training neuronaler Netze lernen, wenn wir jede Schicht als ein Gradientenverstärkungsproblem behandeln?


We propose to extend existing deep reinforcement learning  (Deep RL) algorithms by allowing them to additionally choose sequences of actions as a part of their policy.  This modification forces the network to anticipate the reward of action sequences, which, as we show, improves the exploration leading to better convergence.Our proposal is simple, flexible, and can be easily incorporated into any Deep RL framework.We show the power of our scheme by consistently outperforming the state-of-the-art GA3C algorithm on several popular Atari Games.

Antizipation verbessert die Konvergenz von Deep Reinforcement Learning.


To optimize a neural network one often thinks of optimizing its parameters, but it is ultimately a matter of optimizing the function that maps inputs to outputs.Since a change in the parameters might serve as a poor proxy for the change in the function, it is of some concern that primacy is given to parameters but that the correspondence has not been tested.Here, we show that it is simple and computationally feasible to calculate distances between functions in a $L^2$ Hilbert space.We examine how typical networks behave in this space, and compare how parameter $\ell^2$ distances compare to function $L^2$ distances between various points of an optimization trajectory.We find that the two distances are nontrivially related.In particular, the $L^2/\ell^2$ ratio decreases throughout optimization, reaching a steady value around when test error plateaus.We then investigate how the $L^2$ distance could be applied directly to optimization.We first propose that in multitask learning, one can avoid catastrophic forgetting by directly limiting how much the input/output function changes between tasks.Secondly, we propose a new learning rule that constrains the distance a network can travel through $L^2$-space in any one update.This allows new examples to be learned in a way that minimally interferes with what has previously been learned.These applications demonstrate how one can measure and regularize function distances directly, without relying on parameters or local approximations like loss curvature.

Wir stellen fest, dass die Bewegung im Funktionsraum nicht proportional zur Bewegung im Parameterraum während der Optimierung ist. Wir schlagen einen neuen Optimierer im Stil eines natürlichen Gradienten vor, um dieses Problem zu lösen.


The information bottleneck (IB) problem tackles the issue of obtaining relevant compressed representations T of some random variable X for the task of predicting Y. It is defined as a constrained optimization problem which maximizes the information the representation has about the task, I(T;Y), while ensuring that a minimum level of compression r is achieved (i.e., I(X;T) <= r).For practical reasons the problem is usually solved by maximizing the IB Lagrangian for many values of the Lagrange multiplier, therefore drawing the IB curve (i.e., the curve of maximal I(T;Y) for a given I(X;Y)) and selecting the representation of desired predictability and compression.It is known when Y is a deterministic function of X, the IB curve cannot be explored and other Lagrangians have been proposed to tackle this problem (e.g., the squared IB Lagrangian).In this paper we(i) present a general family of Lagrangians which allow for the exploration of the IB curve in all scenarios;(ii) prove that if these Lagrangians are used, there is a one-to-one mapping between the Lagrange multiplier and the desired compression rate r for known IB curve shapes, hence, freeing from the burden of solving the optimization problem for many values of the Lagrange multiplier.

Wir führen eine allgemeine Familie von Lagranges ein, die es ermöglichen, die IB-Kurve in allen Szenarien zu erforschen. Wenn diese verwendet werden und die IB-Kurve bekannt ist, kann man direkt für ein Leistungs-/Komprimierungsniveau optimieren.


We propose the Neural Logic Machine (NLM), a neural-symbolic architecture for both inductive learning and logic reasoning.NLMs exploit the power of both neural networks---as function approximators, and logic programming---as a symbolic processor for objects with properties, relations, logic connectives, and quantifiers.  After being trained on small-scale tasks (such as sorting short arrays), NLMs can recover lifted rules, and generalize to large-scale tasks (such as sorting longer arrays).In our experiments, NLMs achieve perfect generalization in a number of tasks, from relational reasoning tasks on the family tree and general graphs, to decision making tasks including sorting arrays, finding shortest paths, and playing the blocks world.Most of these tasks are hard to accomplish for neural networks or inductive logic programming alone.

Wir schlagen die Neural Logic Machine (NLM) vor, eine neuronal-symbolische Architektur für induktives Lernen und logisches Denken.


Sequence-to-sequence (seq2seq) neural models have been actively investigated for abstractive summarization.Nevertheless, existing neural abstractive systems frequently generate factually incorrect summaries and are vulnerable to adversarial information, suggesting a crucial lack of semantic understanding.In this paper, we propose a novel semantic-aware neural abstractive summarization model that learns to generate high quality summaries through semantic interpretation over salient content.A novel evaluation scheme with adversarial samples is introduced to measure how well a model identifies off-topic information, where our model yields significantly better performance than the popular pointer-generator summarizer.Human evaluation also confirms that our system summaries are uniformly more informative and faithful as well as less redundant than the seq2seq model.

Wir schlagen ein semantikbewusstes neuronales abstraktes Zusammenfassungsmodell und ein neuartiges automatisches Zusammenfassungsbewertungsschema vor, das misst, wie gut ein Modell themenfremde Informationen aus gegnerischen Stichproben identifiziert.


The recent work of Super Characters method using two-dimensional word embedding achieved state-of-the-art results in text classification tasks, showcasing the promise of this new approach.This paper borrows the idea of Super Characters method and two-dimensional embedding, and proposes a method of generating conversational response for open domain dialogues.The experimental results on a public dataset shows that the proposed SuperChat method generates high quality responses.An interactive demo is ready to show at the workshop.And code will be available at github soon.

Drucken Sie den Eingabesatz und den aktuellen Antwortsatz auf ein Bild und verwenden Sie ein fine-tuned ImageNet CNN-Modell, um die nächste Antwort vorherzusagen.


Convolutional neural networks (CNNs) have been generally acknowledged as one of the driving forces for the advancement of computer vision.Despite their promising performances on many tasks, CNNs still face major obstacles on the road to achieving ideal machine intelligence.One is that CNNs are complex and hard to interpret.Another is that standard CNNs require large amounts of annotated data, which is sometimes very hard to obtain, and it is desirable to be able to learn them from few examples.In this work, we address these limitations of CNNs by developing novel, simple, and interpretable models for few-shot learn- ing. Our models are based on the idea of encoding objects in terms of visual concepts, which are interpretable visual cues represented by the feature vectors within CNNs.We first adapt the learning of visual concepts to the few-shot setting, and then uncover two key properties of feature encoding using visual concepts, which we call category sensitivity and spatial pattern.Motivated by these properties, we present two intuitive models for the problem of few-shot learning.Experiments show that our models achieve competitive performances, while being much more flexible and interpretable than alternative state-of-the-art few-shot learning methods.We conclude that using visual concepts helps expose the natural capability of CNNs for few-shot learning.

Wir befähigen gewöhnliche CNNs zum Few-Shot Learning, indem wir visuelle Konzepte nutzen, die interpretierbare visuelle Hinweise sind, die innerhalb von CNNs gelernt werden.


Recently various neural networks have been proposed for irregularly structured data such as graphs and manifolds.To our knowledge, all existing graph networks have discrete depth.Inspired by neural ordinary differential equation (NODE) for data in the Euclidean domain, we extend the idea of continuous-depth models to graph data, and propose graph ordinary differential equation (GODE).The derivative of hidden node states are parameterized with a graph neural network, and the output states are the solution to this ordinary differential equation.We demonstrate two end-to-end methods for efficient training of GODE: (1) indirect back-propagation with the adjoint method; (2) direct back-propagation through the ODE solver, which accurately computes the gradient.We demonstrate that direct backprop outperforms the adjoint method in experiments.We then introduce a family of bijective blocks, which enables $\mathcal{O}(1)$ memory consumption.We demonstrate that GODE can be easily adapted to different existing graph neural networks and improve accuracy.We validate the performance of GODE in both semi-supervised node classification tasks and graph classification tasks.Our GODE model achieves a continuous model in time, memory efficiency, accurate gradient estimation, and generalizability with different graph networks.

Anwendung des Modells der gewöhnlichen Differentialgleichung auf graphstrukturierte Daten.


Unsupervised image-to-image translation is a recently proposed task of translating an image to a different style or domain given only unpaired image examples at training time.In this paper, we formulate a new task of unsupervised video-to-video translation, which poses its own unique challenges.Translating video implies learning not only the appearance of objects and scenes but also realistic motion and transitions between consecutive frames.We investigate the performance of per-frame video-to-video translation using existing image-to-image translation networks, and propose a spatio-temporal 3D translator as an alternative solution to this problem.We evaluate our 3D method on multiple synthetic datasets, such as moving colorized digits, as well as the realistic segmentation-to-video GTA dataset and a new CT-to-MRI volumetric images translation dataset.Our results show that frame-wise translation produces realistic results on a single frame level but underperforms significantly on the scale of the whole video compared to our three-dimensional translation approach, which is better able to learn the complex structure of video and motion and continuity of object appearance.

Vorgeschlagene neue Aufgabe, Datensätze und Baselines; 3D Conv CycleGAN bewahrt Objekteigenschaften über Frames hinweg; die Stapelstruktur in Methoden auf Frame-Ebene ist wichtig.


Capsule networks are constrained by the parameter-expensive nature of their layers, and the general lack of provable equivariance guarantees.We present a variation of capsule networks that aims to remedy this.We identify that learning all pair-wise part-whole relationships between capsules of successive layers is inefficient.Further, we also realise that the choice of prediction networks and the routing mechanism are both key to equivariance.Based on these, we propose an alternative framework for capsule networks that learns to projectively encode the manifold of pose-variations, termed the space-of-variation (SOV), for every capsule-type of each layer.This is done using a trainable, equivariant function defined over a grid of group-transformations.Thus, the prediction-phase of routing involves projection into the SOV of a deeper capsule using the corresponding function.As a specific instantiation of this idea, and also in order to reap the benefits of increased parameter-sharing, we use type-homogeneous group-equivariant convolutions of shallower capsules in this phase.We also introduce an equivariant routing mechanism based on degree-centrality.We show that this particular instance of our general model is equivariant, and hence preserves the compositional representation of an input under transformations.We conduct several experiments on standard object-classification datasets that showcase the increased transformation-robustness, as well as general performance, of our model to several capsule baselines.

Ein neues skalierbares, gruppenäquivariantes Modell für Kapselnetzwerke, das die Kompositionalität bei Transformationen beibehält und empirisch stabiler ist als ältere Kapselnetzwerkmodelle.


Recently deep neural networks have shown their capacity to memorize training data, even with noisy labels, which hurts generalization performance.To mitigate this issue, we propose a simple but effective method that is robust to noisy labels, even with severe noise.  Our objective involves a variance regularization term that implicitly penalizes the Jacobian norm of the neural network on the whole training set (including the noisy-labeled data), which encourages generalization and prevents overfitting to the corrupted labels.Experiments on noisy benchmarks demonstrate that our approach achieves state-of-the-art performance with a high tolerance to severe noise.

In dem Werk wird eine einfache, aber effektive Basis für das Lernen mit verrauschten Labels vorgeschlagen.


Recent research suggests that neural machine translation achieves parity with professional human translation on the WMT Chinese--English news translation task.We empirically test this claim with alternative evaluation protocols, contrasting the evaluation of single sentences and entire documents.In a pairwise ranking experiment, human raters assessing adequacy and fluency show a stronger preference for human over machine translation when evaluating documents as compared to isolated sentences.Our findings emphasise the need to shift towards document-level evaluation as machine translation improves to the degree that errors which are hard or impossible to spot at the sentence-level become decisive in discriminating quality of different translation outputs.

Die Bewerter bevorzugen die Angemessenheit der menschlichen gegenüber der maschinellen Übersetzung, wenn sie ganze Dokumente bewerten, aber nicht, wenn sie einzelne Sätze bewerten.


Imitation learning aims to inversely learn a policy from expert demonstrations, which has been extensively studied in the literature for both single-agent setting with Markov decision process (MDP) model, and multi-agent setting with Markov game (MG) model.However, existing approaches for general multi-agent Markov games are not applicable to multi-agent extensive Markov games, where agents make asynchronous decisions following a certain order, rather than simultaneous decisions.We propose a novel framework for asynchronous multi-agent generative adversarial imitation learning (AMAGAIL) under general extensive Markov game settings, and the learned expert policies are proven to guarantee subgame perfect equilibrium (SPE), a more general and stronger equilibrium than Nash equilibrium (NE).The experiment results demonstrate that compared to state-of-the-art baselines, our AMAGAIL model can better infer the policy of each expert agent using their demonstration data collected from asynchronous decision-making scenarios (i.e., extensive Markov games).

Diese Arbeit erweitert das generative adversarial Imitation Learning für Multi-Agenten auf Markov-Spiele in extensiver Form.


Self-training is one of the earliest and simplest semi-supervised methods.The key idea is to augment the original labeled dataset with unlabeled data paired with the model’s prediction.Self-training has mostly been well-studied to classification problems.However, in complex sequence generation tasks such as machine translation, it is still not clear how self-training woks due to the compositionality of the target space.In this work, we first show that it is not only possible but recommended to apply self-training in sequence generation.Through careful examination of the performance gains, we find that the noise added on the hidden states (e.g. dropout) is critical to the success of self-training, as this acts like a regularizer which forces the model to yield similar predictions for similar inputs from unlabeled data.To further encourage this mechanism, we propose to inject noise to the input space, resulting in a “noisy” version of self-training.Empirical study on standard benchmarks across machine translation and text summarization tasks under different resource settings shows that noisy self-training is able to effectively utilize unlabeled data and improve the baseline performance by large margin.

Wir überdenken das Selbsttraining als halbüberwachte Lernmethode für das Problem der neuronalen Sequenzerzeugung und zeigen, dass das Selbsttraining mit eingespeistem Rauschen recht erfolgreich sein kann.


We present an end-to-end trained memory system that quickly adapts to new data and generates samples like them.Inspired by Kanerva's sparse distributed memory, it has a robust  distributed reading and writing mechanism.The memory is analytically tractable, which enables optimal on-line compression via a Bayesian update-rule.We formulate it as a hierarchical conditional generative model, where memory provides a rich data-dependent prior distribution.Consequently, the top-down memory and bottom-up perception are combined to produce the code representing an observation.Empirically, we demonstrate that the adaptive memory significantly improves generative models trained on both the Omniglot and CIFAR datasets.Compared with the Differentiable Neural Computer (DNC) and its variants, our memory model has greater capacity and is significantly easier to train.

Ein generatives Speichermodell, das langsam lernende neuronale Netze und ein sich schnell anpassendes lineares Gauß-Modell als Speicher kombiniert.


Pruning large neural networks while maintaining their performance is often desirable due to the reduced space and time complexity.In existing methods, pruning is done within an iterative optimization procedure with either heuristically designed pruning schedules or additional hyperparameters, undermining their utility.In this work, we present a new approach that prunes a given network once at initialization prior to training.To achieve this, we introduce a saliency criterion based on connection sensitivity that identifies structurally important connections in the network for the given task.This eliminates the need for both pretraining and the complex pruning schedule while making it robust to architecture variations.After pruning, the sparse network is trained in the standard way.Our method obtains extremely sparse networks with virtually the same accuracy as the reference network on the MNIST, CIFAR-10, and Tiny-ImageNet classification tasks and is broadly applicable to various architectures including convolutional, residual and recurrent networks.Unlike existing methods, our approach enables us to demonstrate that the retained connections are indeed relevant to the given task.

Wir stellen einen neuen Ansatz, SNIP, vor, der einfach, vielseitig und interpretierbar ist. Er entfernt irrelevante Verbindungen für eine bestimmte Aufgabe auf einen Schlag vor dem Training und ist ohne Änderungen auf eine Vielzahl von neuronalen Netzwerkmodellen anwendbar.


Transfer learning uses trained weights from a source model as the initial weightsfor the training of a target dataset.  A well chosen source with a large numberof labeled data leads to significant improvement in accuracy.  We demonstrate atechnique that automatically labels large unlabeled datasets so that they can trainsource models for transfer learning.We experimentally evaluate this method, usinga baseline dataset of human-annotated ImageNet1K labels, against five variationsof this technique.  We show that the performance of these automatically trainedmodels come within 17% of baseline on average.

Eine Technik zur automatischen Kennzeichnung großer unbeschrifteter Datensätze, damit diese Quellmodelle für das Transferlernen trainieren können, und ihre experimentelle Bewertung.


Recent studies in attention modules have enabled higher performance in computer vision tasks by capturing global contexts and accordingly attending important features.In this paper, we propose a simple and highly parametrically efficient module named Tree-structured Attention Module (TAM) which recursively encourages neighboring channels to collaborate in order to produce a spatial attention map as an output.Unlike other attention modules which try to capture long-range dependencies at each channel, our module focuses on imposing non-linearities be- tween channels by utilizing point-wise group convolution.This module not only strengthens representational power of a model but also acts as a gate which controls signal flow.Our module allows a model to achieve higher performance in a highly parameter-efficient manner.We empirically validate the effectiveness of our module with extensive experiments on CIFAR-10/100 and SVHN datasets.With our proposed attention module employed, ResNet50 and ResNet101 models gain 2.3% and 1.2% accuracy improvement with less than 1.5% parameter over- head.Our PyTorch implementation code is publicly available.

In unserem Beitrag wird ein Aufmerksamkeitsmodul vorgeschlagen, das die Beziehungen zwischen den Kanälen erfasst und große Leistungsgewinne bietet.


A significant challenge for the practical application of reinforcement learning toreal world problems is the need to specify an oracle reward function that correctly defines a task.Inverse reinforcement learning (IRL) seeks to avoid this challenge by instead inferring a reward function from expert behavior.  While appealing, it can be impractically expensive to collect datasets of demonstrations that cover the variation common in the real world (e.g. opening any type of door).Thus in practice, IRL must commonly be performed with only a limited set of demonstrations where it can be exceedingly difficult to unambiguously recover a reward function.In this work, we exploit the insight that demonstrations from other tasks can be used to constrain the set of possible reward functions by learning a "prior" that is specifically optimized for the ability to infer expressive reward functions from limited numbers of demonstrations.  We demonstrate that our method can efficiently recover rewards from images for novel tasks and provide intuition as to how our approach is analogous to learning a prior.

Die Anwendbarkeit des inversen Reinforcement Learnings wird oft durch die Kosten für das Sammeln von Expertendemonstrationen behindert. In diesem Beitrag wird versucht, die Anwendbarkeit zu erweitern, indem vorherige Aufgabeninformationen durch Meta-Lernen einbezogen werden.


Recent work has focused on combining kernel methods and deep learning.With this in mind, we introduce Deepström networks -- a new architecture of neural networks which we use to replace top dense layers of standard convolutional architectures with an approximation of a kernel function by relying on the Nyström approximation. Our approach is easy highly flexible.It is compatible with any kernel function and it allows exploiting multiple kernels. We show that Deepström networks reach state-of-the-art performance on standard datasets like SVHN and CIFAR100.One benefit of the method lies in its limited number of learnable parameters which make it particularly suited for small training set sizes, e.g. from 5 to 20 samples per class.Finally we illustrate two ways of using multiple kernels, including a multiple Deepström  setting, that exploits a kernel on each feature map output by the convolutional part of the model.

Eine neue neuronale Architektur, bei der die obersten dichten Schichten von Standard Convolutionalarchitekturen durch eine Annäherung an eine Kernel-Funktion ersetzt werden, indem man sich auf die Nyström-Näherung stützt.


The main goal of this short paper is to inform the neural art community at large on the ethical ramifications of using models trained on the imagenet dataset, or using seed images from classes 445 -n02892767- [’bikini, two-piece’] and 459- n02837789- [’brassiere, bra, bandeau’] of the same.We discovered that many of the images belong to these classes were verifiably pornographic, shot in a non-consensual setting, voyeuristic and also entailed underage nudity.Akin to the \textit{ivory carving-illegal poaching} and \textit{diamond jewelry art-blood diamond} nexuses, we posit there is a similar moral conundrum at play here and would like to instigate a conversation amongst the neural artists in the community.

Der ImageNet-Datensatz enthält nicht-einvernehmliche und pornografische Bilder.


Inductive and unsupervised graph learning is a critical technique for predictive or information retrieval tasks where label information is difficult to obtain.It is also challenging to make graph learning inductive and unsupervised at the same time, as learning processes guided by reconstruction error based loss functions inevitably demand graph similarity evaluation that is usually computationally intractable.In this paper, we propose a general framework SEED (Sampling, Encoding, and Embedding Distributions) for inductive and unsupervised representation learning on graph structured objects.Instead of directly dealing with the computational challenges raised by graph similarity evaluation, given an input graph, the SEED framework samples a number of subgraphs whose reconstruction errors could be efficiently evaluated, encodes the subgraph samples into a collection of subgraph vectors, and employs the embedding of the subgraph vector distribution as the output vector representation for the input graph.By theoretical analysis, we demonstrate the close connection between SEED and graph isomorphism.Using public benchmark datasets, our empirical study suggests the proposed SEED framework is able to achieve up to 10% improvement, compared with competitive baseline methods.

In diesem Papier wird ein neuartiger Rahmen für das Lernen von Graphenähnlichkeit in einem induktiven und unbeaufsichtigten Szenario vorgeschlagen.


Neural population responses to sensory stimuli can exhibit both nonlinear stimulus- dependence and richly structured shared variability.Here, we show how adversarial training can be used to optimize neural encoding models to capture both the deterministic and stochastic components of neural population data.To account for the discrete nature of neural spike trains, we use the REBAR method to estimate unbiased gradients for adversarial optimization of neural encoding models.We illustrate our approach on population recordings from primary visual cortex.We show that adding latent noise-sources to a convolutional neural network yields a model which captures both the stimulus-dependence and noise correlations of the population activity.

Wir zeigen, wie neuronale Kodierungsmodelle trainiert werden können, um sowohl die Signal- als auch die Höchststand Variabilität von neuronalen Populationsdaten mit Hilfe von GANs zu erfassen.


A weakly supervised learning based clustering framework is proposed in this paper.As the core of this framework, we introduce a novel multiple instance learning task based on a bag level label called unique class count (ucc), which is the number of unique classes among all instances inside the bag.In this task, no annotations on individual instances inside the bag are needed during training of the models.We mathematically prove that with a perfect ucc classifier, perfect clustering of individual instances inside the bags is possible even when no annotations on individual instances are given during training.We have constructed a neural network based ucc classifier and experimentally shown that the clustering performance of our framework with our weakly supervised ucc classifier is comparable to that of fully supervised learning models where labels for all instances are known.Furthermore, we have tested the applicability of our framework to a real world task of semantic segmentation of breast cancer metastases in histological lymph node sections and shown that the performance of our weakly supervised framework is comparable to the performance of a fully supervised Unet model.

Ein auf schwach überwachtem Lernen basierender Clustering-Rahmen erbringt vergleichbare Leistungen wie vollständig überwachte Lernmodelle, indem er die eindeutige Klassenzahl ausnutzt.


Model-free reinforcement learning (RL) methods are succeeding in a growing number of tasks, aided by recent advances in deep learning.  However, they tend to suffer from high sample complexity, which hinders their use in real-world domains.  Alternatively, model-based reinforcement learning promises to reduce sample complexity, but tends to require careful tuning and to date have succeeded mainly in restrictive domains where simple models are sufficient for learning.In this paper, we analyze the behavior of vanilla model-based reinforcement learning methods when deep neural networks are used to learn both the model and the policy, and show that the learned policy tends to exploit regions where insufficient data is available for the model to be learned, causing instability in training.To overcome this issue, we propose to use an ensemble of models to maintain the model uncertainty and regularize the learning process.We further show that the use of likelihood ratio derivatives yields much more stable learning than backpropagation through time.Altogether, our approach Model-Ensemble Trust-Region Policy Optimization (ME-TRPO) significantly reduces the sample complexity compared to model-free deep RL methods on challenging continuous control benchmark tasks.

Tiefes modellbasiertes RL, das gut funktioniert.


The high computational and parameter complexity of neural networks makes their training very slow and difficult to deploy on energy and storage-constrained comput- ing systems.Many network complexity reduction techniques have been proposed including fixed-point implementation.However, a systematic approach for design- ing full fixed-point training and inference of deep neural networks remains elusive.We describe a precision assignment methodology for neural network training in which all network parameters, i.e., activations and weights in the feedforward path, gradients and weight accumulators in the feedback path, are assigned close to minimal precision.The precision assignment is derived analytically and enables tracking the convergence behavior of the full precision training, known to converge a priori.Thus, our work leads to a systematic methodology of determining suit- able precision for fixed-point training.The near optimality (minimality) of the resulting precision assignment is validated empirically for four networks on the CIFAR-10, CIFAR-100, and SVHN datasets.The complexity reduction arising from our approach is compared with other fixed-point neural network designs.

Wir analysieren und bestimmen die Präzisionsanforderungen für das Training neuronaler Netze, wenn alle Tensoren, einschließlich der back-propagated Signale und Gewichtsakkumulatoren, im Festkommaformat quantisiert sind.


Machine learning (ML) research has investigated prototypes: examples that are representative of the behavior to be learned.We systematically evaluate five methods for identifying prototypes, both ones previously introduced as well as new ones we propose, finding all of them to provide meaningful but different interpretations.Through a human study, we confirm that all five metrics are well matched to human intuition.Examining cases where the metrics disagree offers an informative perspective on the properties of data and algorithms used in learning, with implications for data-corpus construction, efficiency, adversarial robustness, interpretability, and other ML aspects.In particular, we confirm that the "train on hard" curriculum approach can improve accuracy on many datasets and tasks, but that it is strictly worse when there are many mislabeled or ambiguous examples.

Wir können beim maschinellen Lernen prototypische Beispiele und Ausreißer identifizieren, die sich quantitativ stark voneinander unterscheiden, und sie nutzen, um viele Aspekte neuronaler Netze zu verbessern.


In this work, we propose the Sparse Deep Scattering Croisé Network (SDCSN) a novel architecture based on the Deep Scattering Network (DSN).The DSN is achieved by cascading  wavelet transform convolutions with a complex modulus and a time-invariant operator.We extend this work by first,crossing multiple wavelet family transforms to increase the feature diversity while avoiding any learning.Thus providing a more informative latent representation and benefit from the development of highly specialized wavelet filters over the last decades.Beside, by combining all the different wavelet representations, we reduce the amount of prior information needed regarding the signals at hand.Secondly, we develop an optimal thresholding strategy for over-complete filter banks that regularizes the network and controls instabilities such as inherent non-stationary noise in the signal.Our systematic and principled solution sparsifies the latent representation of the network by acting as a local mask distinguishing between activity and noise.Thus, we propose to enhance the DSN by increasing the variance of the scattering coefficients representation as well as improve its robustness with respect to non-stationary noise.We show that our new approach is more robust and outperforms the DSN on a bird detection task.

Wir schlagen vor, das Deep Scattering Network zu verbessern, um die Kontrolle und Stabilität jeder beliebigen maschinellen Lernpipeline zu erhöhen, indem wir ein kontinuierliches Wavelet-Schwellwertverfahren vorschlagen.


We propose a neural clustering model that jointly learns both latent features and how they cluster.Unlike similar methods our model does not require a predefined number of clusters.Using a supervised approach, we agglomerate latent features towards randomly sampled targets within the same space whilst progressively removing the targets until we are left with only targets which represent cluster centroids.To show the behavior of our model across different modalities we apply our model on both text and image data and very competitive results on MNIST.Finally, we also provide results against baseline models for fashion-MNIST, the 20 newsgroups dataset, and a Twitter dataset we ourselves create.

Neuronales Clustering, ohne dass eine Anzahl von Clustern benötigt wird.


Recent work on explanation generation for decision-making problems has viewed the explanation process as one of model reconciliation where an AI agent brings the human mental model (of its capabilities, beliefs, and goals) to the same page with regards to a task at hand.This formulation succinctly captures many possible types of explanations, as well as explicitly addresses the various properties -- e.g. the social aspects, contrastiveness, and selectiveness -- of explanations studied in social sciences among human-human interactions.However, it turns out that the same process can be hijacked into producing "alternative explanations" -- i.e. explanations that are not true but still satisfy all the properties of a proper explanation.In previous work, we have looked at how such explanations may be perceived by the human in the loop and alluded to one possible way of generating them.In this paper, we go into more details of this curious feature of the model reconciliation process and discuss similar implications to the overall notion of explainable decision-making.

Die Modellabstimmung ist ein bewährter Rahmen für Planerklärungen, kann aber leicht für Lügen missbraucht werden.


We consider new variants of optimization algorithms.Our algorithms are based on the observation that mini-batch of stochastic gradients in consecutive iterations do not change drastically and consequently may be predictable.Inspired by the similar setting in online learning literature called Optimistic Online learning, we propose two new optimistic algorithms for AMSGrad and Adam, respectively, by  exploiting the predictability of gradients.  The new algorithms combine the idea of momentum method, adaptive gradient method, and algorithms in Optimistic Online learning, which leads to speed up in training deep neural nets in practice.

Wir betrachten neue Varianten von Optimierungsalgorithmen für das Training tiefer Netze.


Using Recurrent Neural Networks (RNNs) in sequence modeling tasks is promising in delivering high-quality results but challenging to meet stringent latency requirements because of the memory-bound execution pattern of RNNs.We propose a big-little dual-module inference to dynamically skip unnecessary memory access and computation to speedup RNN inference.Leveraging the error-resilient feature of nonlinear activation functions used in RNNs, we propose to use a lightweight little module that approximates the original RNN layer, which is referred to as the big module, to compute activations of the insensitive region that are more error-resilient.The expensive memory access and computation of the big module can be reduced as the results are only used in the sensitive region.Our method can reduce the overall memory access by 40% on average and achieve 1.54x to 1.75x speedup on CPU-based server platform with negligible impact on model quality.

Wir beschleunigen die RNN-Inferenz, indem wir redundante Speicherzugriffe durch eine Mischung aus genauen und approximativen Modulen dynamisch reduzieren.


Fine-grained Entity Recognition (FgER) is the task of detecting and classifying entity mentions to a large set of types spanning diverse domains such as biomedical, finance and sports.   We  observe  that  when  the  type  set  spans  several  domains,  detection  of  entity mention becomes a limitation for supervised learning models.  The primary reason being lack  of  dataset  where  entity  boundaries  are  properly  annotated  while  covering  a  large spectrum of entity types.  Our work directly addresses this issue.  We propose Heuristics Allied with Distant Supervision (HAnDS) framework to automatically construct a quality dataset suitable for the FgER task.  HAnDS framework exploits the high interlink among Wikipedia  and  Freebase  in  a  pipelined  manner,  reducing  annotation  errors  introduced by naively using distant supervision approach.  Using HAnDS framework,  we create two datasets, one suitable for building FgER systems recognizing up to 118 entity types based on the FIGER type hierarchy and another for up to 1115 entity types based on the TypeNet hierarchy.  Our extensive empirical experimentation warrants the quality of the generated datasets.  Along with this, we also provide a manually annotated dataset for benchmarking FgER systems.

Wir initiieren einen Vorstoß zum Aufbau von ER-Systemen, die Tausende von Typen erkennen können, indem wir eine Methode zur automatischen Konstruktion geeigneter Datensätze auf der Grundlage der Typenhierarchie bereitstellen.


Implementing correct method invocation is an important task for software developers.However, this is challenging work, since the structure of method invocation can be complicated.In this paper, we propose InvocMap, a code completion tool allows developers to obtain an implementation of multiple method invocations from a list of method names inside code context.InvocMap is able to predict the nested method invocations which their names didn’t appear in the list of input method names given by developers.To achieve this, we analyze the Method Invocations by four levels of abstraction.We build a Machine Translation engine to learn the mapping from the first level to the third level of abstraction of multiple method invocations, which only requires developers to manually add local variables from generated expression to get the final code.We evaluate our proposed approach on six popular libraries: JDK, Android, GWT, Joda-Time, Hibernate, and Xstream.With the training corpus of 2.86 million method invocations extracted from 1000 Java Github projects and the testing corpus extracted from 120 online forums code snippets, InvocMap achieves the accuracy rate up to 84 in F1- score depending on how much information of context provided along with method names, that shows its potential for auto code completion.

In dieser Arbeit wird eine Theorie zur Klassifizierung von Methodenaufrufen nach verschiedenen Abstraktionsebenen vorgeschlagen und ein statistischer Ansatz zur Codevervollständigung vom Methodennamen bis zum Methodenaufruf durchgeführt.


Adversaries in neural networks have drawn much attention since their first debut. While most existing methods aim at deceiving image classification models into misclassification or crafting attacks for specific object instances in the object setection tasks, we focus on creating universal adversaries to fool object detectors and hide objects from the detectors. The adversaries we examine are universal in three ways: (1) They are not specific for specific object instances; (2) They are image-independent; (3) They can further transfer to different unknown models. To achieve this, we propose two novel techniques to improve the transferability of the adversaries: \textit{piling-up} and \textit{monochromatization}. Both techniques prove to simplify the patterns of generated adversaries, and ultimately result in higher transferability.

Wir konzentrieren uns darauf, universelle Gegner zu schaffen, um Objektdetektoren zu täuschen und Objekte vor den Detektoren zu verbergen.


This work presents the Poincaré Wasserstein Autoencoder, a reformulation ofthe recently proposed Wasserstein autoencoder framework on a non-Euclideanmanifold, the Poincaré ball model of the hyperbolic space H n .By assuming thelatent space to be hyperbolic, we can use its intrinsic hierarchy to impose structureon the learned latent space representations.We show that for datasets with latenthierarchies, we can recover the structure in a low-dimensional latent space.Wealso demonstrate the model in the visual domain to analyze some of its propertiesand show competitive results on a graph link prediction task.

Wasserstein-Autoencoder mit hyperbolischem latentem Raum.


In this paper, we introduce a method to compress intermediate feature maps of deep neural networks (DNNs) to decrease memory storage and bandwidth requirements during inference.Unlike previous works, the proposed method is based on converting fixed-point activations into vectors over the smallest GF(2) finite field followed by nonlinear dimensionality reduction (NDR) layers embedded into a DNN.Such an end-to-end learned representation finds more compact feature maps by exploiting quantization redundancies within the fixed-point activations along the channel or spatial dimensions.We apply the proposed network architecture to the tasks of ImageNet classification and PASCAL VOC object detection.Compared to prior approaches, the conducted experiments show a factor of 2 decrease in memory requirements with minor degradation in accuracy while adding only bitwise computations.

Methode zur Komprimierung von Merkmalskarten, die quantisierte Aktivierungen in binäre Vektoren umwandelt, gefolgt von nichtlinearen Dimensionalitätsreduktionsschichten, die in ein DNN eingebettet sind.


Adversarial training is one of the strongest defenses against adversarial attacks, but it requires adversarial examples to be generated for every mini-batch during optimization.  The expense of producing these examples during training often precludes adversarial training from use on complex image datasets. In this study, we explore the mechanisms by which adversarial training improves classifier robustness, and show that these mechanisms can be effectively mimicked using simple regularization methods, including label smoothing and logit squeezing.  Remarkably, using these simple regularization methods in combination with Gaussian noise injection, we are able to achieve strong adversarial robustness -- often exceeding that of adversarial training -- using no adversarial examples.

Erzielung einer starken Robustheit gegenüber Gegnern, die mit gegnerischem Training vergleichbar ist, ohne Training an gegnerischen Beispielen.


A major goal of unsupervised learning is to discover data representations that are useful for subsequent tasks, without access to supervised labels during training.Typically, this involves minimizing a surrogate objective, such as the negative log likelihood of a generative model, with the hope that representations useful for subsequent tasks will arise as a side effect.In this work, we propose instead to directly target later desired tasks by meta-learning an unsupervised learning rule which leads to representations useful for those tasks.  Specifically, we target semi-supervised classification performance, and we meta-learn an algorithm -- an unsupervised weight update rule -- that produces representations useful for this task.Additionally, we constrain our unsupervised update rule to a be a biologically-motivated, neuron-local function, which enables it to generalize to different neural network architectures, datasets, and data modalities.We show that the meta-learned update rule produces useful features and sometimes outperforms existing unsupervised learning techniques.We further show that the meta-learned unsupervised update rule generalizes to train networks with different widths, depths, and nonlinearities.It also generalizes to train on data with randomly permuted input dimensions and even generalizes from image datasets to a text task.

Wir lernen einen unüberwachten Lernalgorithmus, der nützliche Repräsentationen aus einer Reihe von überwachten Aufgaben erzeugt. Zur Testzeit wenden wir diesen Algorithmus auf neue Aufgaben ohne Überwachung an und zeigen eine Leistung, die mit einer VAE vergleichbar ist.


There is significant recent evidence in supervised learning that, in the over-parametrized setting, wider networks achieve better test error.In other words, the bias-variance tradeoff is not directly observable when increasing network width arbitrarily.We investigate whether a corresponding phenomenon is present in reinforcement learning.We experiment on four OpenAI Gym environments, increasing the width of the value and policy networks beyond their prescribed values.Our empirical results lend support to this hypothesis.However, tuning the hyperparameters of each network width separately remains as important future work in environments/algorithms where the optimal hyperparameters vary noticably across widths, confounding the results when the same hyperparameters are used for all widths.

Eine Überparametrisierung in der Breite scheint beim Deep Reinforcement Learning ebenso hilfreich zu sein wie beim überwachten Lernen.


Learning disentangled representations of  data is one of the central themes in unsupervised learning in general and generative modelling in particular.  In this work,  we tackle a slightly more intricate scenario where the observations are generated from a conditional distribution of some known control variate and some latent noise variate.  To this end, we present a hierarchical model and a training method (CZ-GEM) that leverages some of the recent developments in likelihood-based and likelihood-free generative models.  We show that by formulation, CZ-GEM introduces the right inductive biases that ensure the disentanglement of the control from the noise variables, while also keeping the components of the control variate disentangled.This is achieved without compromising on the quality of the generated samples.Our approach is simple, general, and can be applied both in supervised and unsupervised settings.

Hierarchisches generatives Modell (Hybrid aus VAE und GAN), das eine entwirrte Darstellung der Daten lernt, ohne die generative Qualität zu beeinträchtigen.


Deep learning has yielded state-of-the-art performance on many natural language processing tasks including named entity recognition (NER).However, this typically requires large amounts of labeled data.In this work, we demonstrate that the amount of labeled training data can be drastically reduced when deep learning is combined with active learning.While active learning is sample-efficient, it can be computationally expensive since it requires iterative retraining.To speed this up, we introduce a lightweight architecture for NER, viz., the CNN-CNN-LSTM model consisting of convolutional character and word encoders and a  long short term memory (LSTM) tag decoder.The model achieves nearly state-of-the-art performance on standard datasets for the task while being computationally much more efficient than best performing models.We carry out incremental active learning, during the training process, and are able to nearly match state-of-the-art performance with just 25\% of the original training data.

Wir stellen eine leichtgewichtige Architektur für die Erkennung von Named Entities vor und führen ein inkrementelles aktives Lernen durch, das in der Lage ist, mit nur 25 % der ursprünglichen Trainingsdaten eine dem Stand der Technik entsprechende Leistung zu erzielen.


Network quantization is a model compression and acceleration technique that has become essential to neural network deployment.Most quantization methods per- form fine-tuning on a pretrained network, but this sometimes results in a large loss in accuracy compared to the original network.We introduce a new technique to train quantization-friendly networks, which can be directly converted to an accurate quantized network without the need for additional fine-tuning.Our technique allows quantizing the weights and activations of all network layers down to 4 bits, achieving high efficiency and facilitating deployment in practical settings.Com- pared to other fully quantized networks operating at 4 bits, we show substantial improvements in accuracy, for example 66.68% top-1 accuracy on ImageNet using ResNet-18, compared to the previous state-of-the-art accuracy of 61.52% Louizos et al. (2019) and a full precision reference accuracy of 69.76%.We performed a thorough set of experiments to test the efficacy of our method and also conducted ablation studies on different aspects of the method and techniques to improve training stability and accuracy.Our codebase and trained models are available on GitHub.

Wir trainieren genaue, vollständig quantisierte Netzwerke unter Verwendung einer Verlustfunktion, die die Genauigkeit des Modells mit voller Präzision maximiert und die Differenz zwischen dem Netzwerk mit voller Präzision und dem quantisierten Netzwerk minimiert.


While much of the work in the design of convolutional networks over the last five years has revolved around the empirical investigation of the importance of depth, filter sizes, and number of feature channels, recent studies have shown that  branching, i.e., splitting the computation along parallel but distinct threads and then aggregating their outputs, represents a new promising dimension for significant improvements in performance.To combat the complexity of design choices in multi-branch architectures, prior work has adopted simple strategies, such as a fixed branching factor, the same input being fed to all parallel branches, and an additive combination of the outputs produced by all branches at aggregation points. In this work we remove these predefined choices and propose an algorithm to learn the connections between branches in the network.Instead of being chosen a priori by the human designer, the multi-branch connectivity is learned simultaneously with the weights of the network by optimizing a single loss function defined with respect to the end task.We demonstrate our approach on the problem of multi-class image classification using four different datasets where it yields consistently higher accuracy compared to the state-of-the-art ``ResNeXt'' multi-branch network given the same learning capacity.

In diesem Beitrag stellen wir einen Algorithmus vor, mit dem die Konnektivität von tiefen Mehrfachverzweigungsnetzen erlernt werden kann. Der Ansatz wird anhand von Bildkategorisierungen evaluiert, bei denen er durchgängig Genauigkeitsgewinne gegenüber modernen Modellen mit fester Konnektivität erzielt.


Although deep convolutional networks have achieved improved performance in many natural language tasks, they have been treated as black boxes because they are difficult to interpret.Especially, little is known about how they represent language in their intermediate layers.In an attempt to understand the representations of deep convolutional networks trained on language tasks, we show that individual units are selectively responsive to specific morphemes, words, and phrases, rather than responding to arbitrary and uninterpretable patterns.In order to quantitatively analyze such intriguing phenomenon, we propose a concept alignment method based on how units respond to replicated text.We conduct analyses with different architectures on multiple datasets for classification and translation tasks and provide new insights into how deep models understand natural language.

Wir zeigen, dass einzelne Einheiten in CNN-Repräsentationen, die in NLP-Aufgaben gelernt wurden, selektiv auf natürlichsprachliche Konzepte reagieren.


Applying reinforcement learning (RL) to real-world problems will require reasoning about action-reward correlation over long time horizons.Hierarchical reinforcement learning (HRL) methods handle this by dividing the task into hierarchies, often with hand-tuned network structure or pre-defined subgoals.We propose a novel HRL framework TAIC, which learns the temporal abstraction from past experience or expert demonstrations without task-specific knowledge.We formulate the temporal abstraction problem as learning latent representations of action sequences and present a novel approach of regularizing the latent space by adding information-theoretic constraints.Specifically, we maximize the mutual information between the latent variables and the state changes.A visualization of the latent space demonstrates that our algorithm learns an effective abstraction of the long action sequences.The learned abstraction allows us to learn new tasks on higher level more efficiently.We convey a significant speedup in convergence over benchmark learning problems.These results demonstrate that learning temporal abstractions is an effective technique in increasing the convergence rate and sample efficiency of RL algorithms.

Wir schlagen ein neuartiges HRL-Framework vor, in dem wir das zeitliche Abstraktionsproblem als Lernen einer latenten Repräsentation einer Handlungssequenz formulieren.


Recurrent neural networks (RNNs) have achieved state-of-the-art performance on many diverse tasks, from machine translation to surgical activity recognition, yet training RNNs to capture long-term dependencies remains difficult.To date, the vast majority of successful RNN architectures alleviate this problem using nearly-additive connections between states, as introduced by long short-term memory (LSTM).We take an orthogonal approach and introduce MIST RNNs, a NARX RNN architecture that allows direct connections from the very distant past.We show that MIST RNNs1) exhibit superior vanishing-gradient properties in comparison to LSTM and previously-proposed NARX RNNs;2) are far more efficient than previously-proposed NARX RNN architectures, requiring even fewer computations than LSTM; and3) improve performance substantially over LSTM and Clockwork RNNs on tasks requiring very long-term dependencies.

Wir stellen MIST-RNNs vor, die a) im Vergleich zu LSTMs überragende Vanishing-Gradient-Eigenschaften aufweisen; b) die Leistung von LSTMs und Clockwork-RNNs bei Aufgaben, die sehr langfristige Abhängigkeiten erfordern, erheblich verbessern; und c) viel effizienter sind als die zuvor vorgeschlagenen NARX-RNNs, mit sogar weniger Parametern und Operationen als LSTMs.


A well-trained model should classify objects with unanimous score for every category.This requires the high-level semantic features should be alike among samples, despite a wide span in resolution, texture, deformation, etc.Previous works focus on re-designing the loss function or proposing new regularization constraints on the loss.In this paper, we address this problem via a new perspective.For each category, it is assumed that there are two sets in the feature space: one with more reliable information and the other with less reliable source.We argue that the reliable set could guide the feature learning of the less reliable set during training - in spirit of student mimicking teacher’s behavior and thus pushing towards a more compact class centroid in the high-dimensional space.Such a scheme also benefits the reliable set since samples become more closer within the same category - implying that it is easilier for the classifier to identify.We refer to this mutual learning process as feature intertwiner and embed the spirit into object detection.It is well-known that objects of low resolution are more difficult to detect due to the loss of detailed information during network forward pass.We thus regard objects of high resolution as the reliable set and objects of low resolution as the less reliable set.Specifically, an intertwiner is achieved by minimizing the distribution divergence between two sets.We design a historical buffer to represent all previous samples in the reliable set and utilize them to guide the feature learning of the less reliable set.The design of obtaining an effective feature representation for the reliable set is further investigated, where we introduce the optimal transport (OT) algorithm into the framework.Samples in the less reliable set are better aligned with the reliable set with aid of OT metric.Incorporated with such a plug-and-play intertwiner, we achieve an evident improvement over previous state-of-the-arts on the COCO object detection benchmark.

Ein Modul zur Verflechtung von Merkmalen, um Merkmale aus einem genauen Satz zu nutzen, um das Lernen eines anderen, weniger zuverlässigen Satzes zu unterstützen.


Approaches to continual learning aim to successfully learn a set of related tasks that arrive in an online manner.Recently, several frameworks have been developed which enable deep learning to be deployed in this learning scenario.A key modelling decision is to what extent the architecture should be shared across tasks.On the one hand, separately modelling each task avoids catastrophic forgetting but it does not support transfer learning and leads to large models.On the other hand, rigidly specifying a shared component and a task-specific part enables task transfer and limits the model size, but it is vulnerable to catastrophic forgetting and restricts the form of task-transfer that can occur.Ideally, the network should adaptively identify which parts of the network to share in a data driven way.Here we introduce such an approach called Continual Learning with Adaptive Weights (CLAW), which is based on probabilistic modelling and variational inference.Experiments show that CLAW achieves state-of-the-art performance on six benchmarks in terms of overall continual learning performance, as measured by classification accuracy, and in terms of addressing catastrophic forgetting.

Ein kontinuierlich lernender Rahmen, der lernt, seine Architektur auf der Grundlage eines vorgeschlagenen Variationsinferenzalgorithmus automatisch anzupassen.


High-dimensional data often lie in or close to low-dimensional subspaces.Sparse subspace clustering methods with sparsity induced by L0-norm, such as L0-Sparse Subspace Clustering (L0-SSC), are demonstrated to be more effective than its L1 counterpart such as Sparse Subspace Clustering (SSC).However, these L0-norm based subspace clustering methods are restricted to clean data that lie exactly in subspaces.Real data often suffer from noise and they may lie close to subspaces.We propose noisy L0-SSC to handle noisy data so as to improve the robustness.We show that the optimal solution to the optimization problem of noisy L0-SSC achieves subspace detection property (SDP), a key element with which data from different subspaces are separated, under deterministic and randomized models.Our results provide theoretical guarantee on the correctness of noisy L0-SSC in terms of SDP on noisy data.We further propose Noisy-DR-L0-SSC which provably recovers the subspaces on dimensionality reduced data.Noisy-DR-L0-SSC first projects the data onto a lower dimensional space by linear transformation, then performs noisy L0-SSC on the dimensionality reduced data so as to improve the efficiency.The experimental results demonstrate the effectiveness of noisy L0-SSC and Noisy-DR-L0-SSC.

Wir schlagen Noisy-DR-L0-SSC (Noisy Dimension Reduction L0-Sparse Subspace Clustering) vor, um verrauschte Daten entsprechend ihrer zugrunde liegenden Unterraumstruktur effizient zu partitionieren.


Mode connectivity provides novel geometric insights on analyzing loss landscapes and enables building high-accuracy pathways between well-trained neural networks.In this work, we propose to employ mode connectivity in loss landscapes to study the adversarial robustness of deep neural networks, and provide novel methods for improving this robustness.  Our experiments cover various types of adversarial attacks applied to different network architectures and datasets.When network models are tampered with backdoor or error-injection attacks, our results demonstrate that the path connection learned using limited amount of bonafide data can effectively mitigate adversarial effects while maintaining the original accuracy on clean data.Therefore, mode connectivity provides users with the power to repair backdoored or error-injected models.  We also use mode connectivity to investigate the loss landscapes of regular and robust models against evasion attacks.Experiments show that there exists a barrier in adversarial robustness loss on the path connecting regular and adversarially-trained models.  A high correlation is observed between the adversarial robustness loss and the largest eigenvalue of the input Hessian matrix, for which theoretical justifications are provided.  Our results suggest that mode connectivity offers a holistic tool and practical means for evaluating and improving adversarial robustness.

Ein neuartiger Ansatz, der die Konnektivität von Modi in Verlustlandschaften nutzt, um nachteilige Auswirkungen abzuschwächen, manipulierte Modelle zu reparieren und die Widerstandsfähigkeit gegenüber nachteiligen Einflüssen zu bewerten.


Generative adversarial networks (GANs) learn to map samples from a noise distribution to a chosen data distribution.Recent work has demonstrated that GANs are consequently sensitive to, and limited by, the shape of the noise distribution.For example, a single generator struggles to map continuous noise (e.g. a uniform distribution) to discontinuous output (e.g. separate Gaussians) or complex output (e.g. intersecting parabolas).We address this problem by learning to generate from multiple models such that the generator's output is actually the combination of several distinct networks.We contribute a novel formulation of multi-generator models where we learn a prior over the generators conditioned on the noise, parameterized by a neural network.Thus, this network not only learns the optimal rate to sample from each generator but also optimally shapes the noise received by each generator.The resulting Noise Prior GAN (NPGAN) achieves expressivity and flexibility that surpasses both single generator models and previous multi-generator models.

Ein GAN-Rahmenwerk mit mehreren Generatoren und einem zusätzlichen Netz, das einen Prior über das Eingangsrauschen lernt.


Recent advances in Generative Adversarial Networks (GANs) – in architectural design,  training strategies,  and empirical tricks – have led nearly photorealistic samples on large-scale datasets such as ImageNet.  In fact, for one model in particular, BigGAN, metrics such as Inception Score or Frechet Inception Distance nearly match those of the dataset, suggesting that these models are close to match-ing the distribution of the training set.   Given the quality of these models,  it is worth understanding to what extent these samples can be used for data augmentation, a task expressed as a long-term goal of the GAN research project.  To that end, we train ResNet-50 classifiers using either purely BigGAN images or mixtures of ImageNet and BigGAN images, and test on the ImageNet validation set.Our  preliminary  results  suggest  both a measured view of  state-of-the-art  GAN quality and highlight limitations of current metrics.Using only BigGAN images, we find that Top-1 and Top-5 error increased by 120% and 384%, respectively, and furthermore, adding more BigGAN data to the ImageNet training set at best only marginally improves classifier performance.Finally, we find that neither Inception Score, nor FID, nor combinations thereof are predictive of classification accuracy.   These results suggest that as GANs are beginning to be deployed in downstream tasks, we should create metrics that better measure downstream task performance.  We propose classification performance as one such metric that, in addition to assessing per-class sample quality, is more suited to such downstream tasks.

BigGANs erfassen nicht die ImageNet-Datenverteilungen und sind nur mäßig erfolgreich bei der Datenerweiterung.


Modern federated networks, such as those comprised of wearable devices, mobile phones, or autonomous vehicles, generate massive amounts of data each day.This wealth of data can help to learn models that can improve the user experience on each device.However, the scale and heterogeneity of federated data presents new challenges in research areas such as federated learning, meta-learning, and multi-task learning.As the machine learning community begins to tackle these challenges, we are at a critical time to ensure that developments made in these areas are grounded with realistic benchmarks.To this end, we propose Leaf, a modular benchmarking framework for learning in federated settings.Leaf includes a suite of open-source federated datasets, a rigorous evaluation framework, and a set of reference implementations, all geared towards capturing the obstacles and intricacies of practical federated environments.

Wir präsentieren Leaf, ein modulares Benchmarking-Framework für das Lernen in föderierten Daten, mit Anwendungen für Lernparadigmen wie föderiertes Lernen, Meta-Lernen und Multi-Task-Lernen.


Understanding object motion is one of the core problems in computer vision.It requires segmenting and tracking objects over time.Significant progress has been made in instance segmentation, but such models cannot track objects, and more crucially, they are unable to reason in both 3D space and time.We propose a new spatio-temporal embedding loss on videos that generates temporally consistent video instance segmentation.Our model includes a temporal network that learns to model temporal context and motion, which is essential to produce smooth embeddings over time.Further, our model also estimates monocular depth, with a self-supervised loss, as the relative distance to an object effectively constrains where it can be next, ensuring a time-consistent embedding.Finally, we show that our model can accurately track and segment instances, even with occlusions and missed detections, advancing the state-of-the-art on the KITTI Multi-Object and Tracking Dataset.

Wir führen einen neuen räumlich-zeitlichen Einbettungsverlust für Videos ein, der eine zeitlich konsistente Segmentierung von Videoinstanzen erzeugt, selbst bei Okklusion und verpassten Erkennungen, unter Verwendung von Aussehen, Geometrie und zeitlichem Kontext.