We study the role of intrinsic motivation as an exploration bias for reinforcement learning in sparse-reward synergistic tasks, which are tasks where multiple agents must work together to achieve a goal they could not individually.Our key idea is that a good guiding principle for intrinsic motivation in synergistic tasks is to take actions which affect the world in ways that would not be achieved if the agents were acting on their own.Thus, we propose to incentivize agents to take (joint) actions whose effects cannot be predicted via a composition of the predicted effect for each individual agent.We study two instantiations of this idea, one based on the true states encountered, and another based on a dynamics model trained concurrently with the policy.While the former is simpler, the latter has the benefit of being analytically differentiable with respect to the action taken.We validate our approach in robotic bimanual manipulation tasks with sparse rewards; we find that our approach yields more efficient learning than both1) training with only the sparse reward and2) using the typical surprise-based formulation of intrinsic motivation, which does not bias toward synergistic behavior.Videos are available on the project webpage: https://sites.google.com/view/iclr2020-synergistic.

我们提出了一种内在动机的表述，它适合作为多代理稀疏奖励协同任务中的探索偏向，通过鼓励代理以单独行动时无法实现的方式影响世界。

本文重点讨论了在需要多代理实现的任务中，利用内在动机来改善强化学习代理的探索过程。


A general graph-structured neural network architecture operates on graphs through two core components: (1) complex enough message functions; (2) a fixed information aggregation process.In this paper, we present the Policy Message Passing algorithm, which takes a probabilistic perspective and reformulates the whole information aggregation as stochastic sequential processes.The algorithm works on a much larger search space, utilizes reasoning history to perform inference, and is robust to noisy edges.We apply our algorithm to multiple complex graph reasoning and prediction tasks and show that our algorithm consistently outperforms state-of-the-art graph-structured models by a significant margin.

由神经网络驱动的图结构模型的概率推理算法

本文介绍了策略性消息传递，这是一个具有推理机制的图神经网络，它以循环的方式将消息分配给边，表明在视觉推理任务上具有竞争性的性能。


Deep multitask networks, in which one neural network produces multiple predictive outputs, are more scalable and often better regularized than their single-task counterparts.Such advantages can potentially lead to gains in both speed and performance, but multitask networks are also difficult to train without finding the right balance between tasks.We present a novel gradient normalization (GradNorm) technique which automatically balances the multitask loss function by directly tuning the gradients to equalize task training rates.We show that for various network architectures, for both regression and classification tasks, and on both synthetic and real datasets, GradNorm improves accuracy and reduces overfitting over single networks, static baselines, and other adaptive multitask loss balancing techniques.GradNorm also matches or surpasses the performance of exhaustive grid search methods, despite only involving a single asymmetry hyperparameter $\alpha$.Thus, what was once a tedious search process which incurred exponentially more compute for each task added can now be accomplished within a few training runs, irrespective of the number of tasks.Ultimately, we hope to demonstrate that gradient manipulation affords us great control over the training dynamics of multitask networks and may be one of the keys to unlocking the potential of multitask learning.

我们展示了如何通过调整一个自适应的多任务损失函数来提高多任务网络的性能，该函数是通过直接平衡网络梯度学习的。

这项工作提出了一个动态权重更新方案，通过利用不同任务的损失率，在训练时间内更新不同任务损失的权重。


Image segmentation aims at grouping pixels that belong to the same object or region.At the heart of image segmentation lies the problem of determining whether a pixel is inside or outside a region, which we denote as the "insideness" problem.Many Deep Neural Networks (DNNs) variants excel in segmentation benchmarks, but regarding insideness, they have not been well visualized or understood: What representations do DNNs use to address the long-range relationships of insideness?How do architectural choices affect the learning of these representations?In this paper, we take the reductionist approach by analyzing DNNs solving the insideness problem in isolation, i.e. determining the inside of closed (Jordan) curves.We demonstrate analytically that state-of-the-art feed-forward and recurrent architectures can implement solutions of the insideness problem for any given curve.Yet, only recurrent networks could  learn these general solutions when the training enforced a specific "routine" capable of breaking down the long-range relationships.Our results highlights the need for new training strategies that decompose the learning into appropriate stages, and that lead to the general class of solutions necessary for DNNs to understand insideness.

用于图像分割的DNN可以实现内部性问题的解决方案，但只有一些循环网络可以在特定类型的监督下学习它们。

本文引入insideness来研究深度学习时代的语义分割，结果可以帮助模型更好地泛化。


We address the challenging problem of deep representation learning--the efficient adaption of a pre-trained deep network to different tasks.Specifically, we propose to explore gradient-based features.These features are gradients of the model parameters with respect to a task-specific loss given an input sample.Our key innovation is the design of a linear model that incorporates both gradient features and the activation of the network.We show that our model provides a local linear approximation to a underlying deep model, and discuss important theoretical insight.Moreover, we present an efficient algorithm for the training and inference of our model without computing the actual gradients.Our method is evaluated across a number of representation learning tasks on several datasets and using different network architectures.We demonstrate strong results in all settings.And our results are well-aligned with our theoretical insight.

考虑到一个预先训练好的模型，我们探索了模型参数相对于特定任务损失的每个样本梯度，并构建了一个结合模型参数梯度和模型激活的线性模型。

本文提出将卷积网络特定层的梯度作为线性化模型中的特征，用于转移学习和快速适应。


Recovering 3D geometry shape, albedo and lighting from a single image has wide applications in many areas, which is also a typical ill-posed problem.In order to eliminate the ambiguity, face prior knowledge like linear 3D morphable models (3DMM) learned from limited scan data are often adopted to the reconstruction process.However, methods based on linear parametric models cannot generalize well for facial images in the wild with various ages, ethnicity, expressions, poses, and lightings.Recent methods aim to learn a nonlinear parametric model using convolutional neural networks (CNN) to regress the face shape and texture directly.However, the models were only trained on a dataset that is generated from a linear 3DMM.Moreover, the identity and expression representations are entangled in these models, which hurdles many facial editing applications.In this paper, we train our model with adversarial loss in a semi-supervised manner on hybrid batches of unlabeled and labeled face images to exploit the value of large amounts of unlabeled face images from unconstrained photo collections.A novel center loss is introduced to make sure that different facial images from the same person have the same identity shape and albedo.Besides, our proposed model disentangles identity, expression, pose, and lighting representations, which improves the overall reconstruction performance and facilitates facial editing applications, e.g., expression transfer.Comprehensive experiments demonstrate that our model produces high-quality reconstruction compared to state-of-the-art methods and is robust to various expression, pose, and lighting conditions.

我们以半监督的方式在未标记和标记的混合批次的人脸图像上训练我们的人脸重建模型，以利用来自无约束的照片集的大量未标记的人脸图像的价值。

本文提出了一个半监督和对抗性的训练过程，从具有损失函数的人脸图像中精确地获得非线性的分离表征，在人脸重建中达到最先进的性能。


Human conversations naturally evolve around related entities and connected concepts, while may also shift from topic to topic.This paper presents ConceptFlow, which leverages commonsense knowledge graphs to explicitly model such conversation flows for better conversation response generation.ConceptFlow grounds the conversation inputs to the latent concept space and represents the potential conversation flow as a concept flow along the commonsense relations.The concept is guided by a graph attention mechanism that models the possibility of the conversation evolving towards different concepts.The conversation response is then decoded using the encodings of both utterance texts and concept flows, integrating the learned conversation structure in the concept space.Our experiments on Reddit conversations demonstrate the advantage of ConceptFlow over previous commonsense aware dialog models and fine-tuned GPT-2 models, while using much fewer parameters but with explicit modeling of conversation structures.

本文提出了ConceptFlow，它在常识性知识图中明确地对对话流进行建模，以更好地生成对话。

本文提出了一个系统，用于在开放领域的对话环境中，利用扩散到基础概念的邻居中，生成对所发布的语词的单匝回应。


Biological neural networks face homeostatic and resource constraints that restrict the allowed configurations of connection weights.If a constraint is tight it defines a very small solution space, and the size of these constraint spaces determines their potential overlap with the solutions for computational tasks.We study the geometry of the solution spaces for constraints on neurons' total synaptic weight and on individual synaptic weights, characterizing the connection degrees (numbers of partners) that maximize the size of these solution spaces.We then hypothesize that the size of constraints' solution spaces could serve as a cost function governing neural circuit development.We develop analytical approximations and bounds for the model evidence of the maximum entropy degree distributions under these cost functions.We test these on a published electron microscopic connectome of an associative learning center in the fly brain, finding evidence for a developmental progression in circuit structure.

我们研究了这样一个假设：对突触权重进行约束的解空间的熵（约束的 "灵活性"）可以作为神经回路发展的成本函数。


In  this  preliminary  work,  we  study  the  generalization  properties  of  infinite  ensembles  of infinitely-wide neural networks.  Amazingly, this model family admits tractable calculations for many information-theoretic quantities.  We report analytical and empirical investigations in the search for signals that correlate with generalization.

从信息论的角度来看，无限宽的神经网络的无限集合是一个有趣的模型系列。


Learning multilingual representations of text has proven a successful method for many cross-lingual transfer learning tasks.There are two main paradigms for learning such representations: (1) alignment, which maps different independently trained monolingual representations into a shared space, and (2) joint training, which directly learns unified multilingual representations using monolingual and cross-lingual objectives jointly.In this paper, we first conduct direct comparisons of representations learned using both of these methods across diverse cross-lingual tasks.Our empirical results reveal a set of pros and cons for both methods, and show that the relative performance of alignment versus joint training is task-dependent.Stemming from this analysis, we propose a simple and novel framework that combines these two previously mutually-exclusive approaches.Extensive experiments on various tasks demonstrate that our proposed framework alleviates limitations of both approaches, and outperforms existing methods on the MUSE bilingual lexicon induction (BLI) benchmark.We further show that our proposed framework can generalize to contextualized representations and achieves state-of-the-art results on the CoNLL cross-lingual NER benchmark.

我们对跨语言一致与联合训练方法进行了比较研究，并在一个新的框架中统一了这两个以前排他性的范式。

本文比较了双语词库归纳的方法，并说明哪种方法在词库、归纳、NER和MT任务上表现更好。


Large number of weights in deep neural networks make the models difficult to be deployed in low memory environments such as, mobile phones, IOT edge devices as well as "inferencing as a service" environments on the cloud. Prior work has considered reduction in the size of the models, through compression techniques like weight pruning, filter pruning, etc. or through low-rank decomposition of the convolution layers.In this paper, we demonstrate the use of multiple techniques to achieve not only higher model compression but also reduce the compute resources required during inferencing.We do filter pruning followed by low-rank decomposition using Tucker decomposition for model compression.We show that our approach achieves upto 57\% higher model compression when compared to either Tucker Decomposition or Filter pruning alone  at similar accuracy for GoogleNet.Also, it reduces the Flops by upto 48\% thereby making the inferencing faster.

结合正交的模型压缩技术，在推理过程中获得模型大小和所需的翻转次数的显著减少。 本文提出将Tucker分解与滤波器修剪相结合。


We review the limitations of BLEU and ROUGE -- the most popular metrics used to assess reference summaries against hypothesis summaries, and introduce JAUNE:  a set of criteria for what a good metric should behave like and propose concrete ways to use recent Transformers-based Language Models to assess reference summaries against hypothesis summaries.

介绍JAUNE：一种用多维的、基于模型的评价器来取代BLEU和ROUGE分数的方法，用于评估总结。

本文提出了一个新的JAUNE指标，用于评估机器翻译和文本总结系统，表明他们的模型比BLEU更好地对应于基本事实相似度标签。


This paper presents a new Graph Neural Network (GNN) type using feature-wise linear modulation (FiLM).Many standard GNN variants propagate information along the edges of a graph by computing ``messages'' based only on the representation of the source of each edge.In GNN-FiLM, the representation of the target node of an edge is additionally used to compute a transformation that can be applied to all incoming messages, allowing feature-wise modulation of the passed information.Results of experiments comparing different GNN architectures on three tasks from the literature are presented, based on re-implementations of baseline methods.Hyperparameters for all methods were found using extensive search, yielding somewhat surprising results: differences between baseline models are smaller than reported in the literature.Nonetheless, GNN-FiLM outperforms baseline methods on a regression task on molecular graphs and performs competitively on other tasks.

新的GNN形式主义+广泛的实验；显示GGNN/GCN/GAT之间的差异比想象的要小

本文提出了一种新的图神经网络架构，该架构使用特征线性调制（Feature-wise Linear Modulation），根据目标节点的表示，对源节点到目标节点的信息传递进行调节。


To deal simultaneously with both, the attributed network embedding and clustering, we propose a new model.It exploits both content and structure information, capitalising on their simultaneous use.The proposed model relies on the approximation of the relaxed continuous embedding solution by the true discrete clustering one.Thereby, we show that incorporating an embedding representation provides simpler and more interpretable solutions.Experiment results demonstrate that the proposed algorithm performs better, in terms of clustering and embedding, than the state-of-art algorithms, including deep learning methods devoted to similar tasks for attributed network datasets with different proprieties.

本文提出了一个新颖的矩阵分解框架，用于同时进行属性网络数据的嵌入和聚类。 本文提出了一种共同进行属性网络嵌入和聚类的算法。


We propose a learned image-guided rendering technique that combines the benefits of image-based rendering and GAN-based image synthesis.The goal of our method is to generate photo-realistic re-renderings of reconstructed objects for virtual and augmented reality applications (e.g., virtual showrooms, virtual tours and sightseeing, the digital inspection of historical artifacts).A core component of our work is the handling of view-dependent effects.Specifically, we directly train an object-specific deep neural network to synthesize the view-dependent appearance of an object.As input data we are using an RGB video of the object.This video is used to reconstruct a proxy geometry of the object via multi-view stereo.Based on this 3D proxy, the appearance of a captured view can be warped into a new target view as in classical image-based rendering.This warping assumes diffuse surfaces, in case of view-dependent effects, such as specular highlights, it leads to artifacts.To this end, we propose EffectsNet, a deep neural network that predicts view-dependent effects.Based on these estimations, we are able to convert observed images to diffuse images.These diffuse images can be projected into other views.In the target view, our pipeline reinserts the new view-dependent effects.To composite multiple reprojected images to a final output, we learn a composition network that outputs photo-realistic results.Using this image-guided approach, the network does not have to allocate capacity on ``remembering'' object appearance, instead it learns how to combine the appearance of captured images.We demonstrate the effectiveness of our approach both qualitatively and quantitatively on synthetic as well as on real data.

我们提出了一种学习型图像引导的渲染技术，该技术结合了基于图像的渲染和基于GAN的图像合成的优点，同时考虑了与视图相关的效果。

本报告提出了一种处理神经渲染中视线依赖效应的方法，它提高了现有神经渲染方法的稳健性。


We evaluate the distribution learning capabilities of generative adversarial networks by testing them on synthetic datasets.The datasets include common distributions of points in $R^n$ space and images containing polygons of various shapes and sizes.We find that by and large GANs fail to faithfully recreate point datasets which contain discontinous support or sharp bends with noise.Additionally, on image datasets, we find that GANs do not seem to learn to count the number of objects of the same kind in an image.We also highlight the apparent tension between generalization and learning in GANs.

在合成数据集上对GAN进行评估


This paper proposes a new approach for step size adaptation in gradient methods.The proposed method called step size optimization (SSO) formulates the step size adaptation as an optimization problem which minimizes the loss function with respect to the step size for the given model parameters and gradients.Then, the step size is optimized based on alternating direction method of multipliers (ADMM).SSO does not require the second-order information or any probabilistic models for adapting the step size, so it is efficient and easy to implement.Furthermore, we also introduce stochastic SSO for stochastic learning environments.In the experiments, we integrated SSO to vanilla SGD and Adam, and they outperformed state-of-the-art adaptive gradient methods including RMSProp, Adam, L4-Adam, and AdaBound on extensive benchmark datasets.

我们为梯度方法提出了一种高效的、有效的步长适应方法。

一阶梯度方法中新的步长适应性，用损失函数的一阶扩展和正则化建立了一个新的优化问题，其中步长被视为变量。


Despite the fact that generative models are extremely successful in practice, the theory underlying this phenomenon is only starting to catch up with practice.In this work we address the question of the universality of generative models: is it true that neural networks can approximate any data manifold arbitrarily well?We provide a positive answer to this question and show that under mild assumptions on the activation function one can always find a feedforward neural network that maps the latent space onto a set located within the specified Hausdorff distance from the desired data manifold.We also prove similar theorems for the case of multiclass generative models and cycle generative models, trained to map samples from one manifold to another and vice versa.

我们拍到，ReLU和sigmoid网络可以以任意的精度生成一类广泛的流形。

本文对流形何时能被写成由神经网络近似的地图的图像提供了某些基本保证，并将流形几何的定理和标准的普遍近似结果缝合在一起。

本文从理论上表明，基于神经网络的生成模型可以近似于数据流形，并证明在温和的假设下，神经网络可以在很小的Hausdorff距离内将潜在空间映射到接近于给定数据流形的集合。


Model-based reinforcement learning (RL) is considered to be a promising approach to reduce the sample complexity that hinders model-free RL.However, the theoretical understanding of such methods has been rather limited.This paper introduces a novel algorithmic framework for designing and analyzing model-based RL algorithms with theoretical guarantees.We design a meta-algorithm with a theoretical guarantee of monotone improvement to a local maximum of the expected reward.The meta-algorithm iteratively builds a lower bound of the expected reward based on the estimated dynamical model and sample trajectories, and then maximizes the lower bound jointly over the policy and the model.The framework extends the optimism-in-face-of-uncertainty principle to non-linear dynamical models in a way that requires no explicit uncertainty quantification.Instantiating our framework with simplification gives a  variant of model-based RL algorithms Stochastic Lower Bounds Optimization (SLBO).Experiments demonstrate that SLBO achieves the state-of-the-art performance when only 1M or fewer samples are permitted on a range of continuous control benchmark tasks.

我们设计了具有理论保证的基于模型的强化学习算法，并在允许一百万或更少样本的情况下，在Mujuco基准任务上取得最先进的结果。

本文提出了一个框架，设计基于OFU的RL算法，在MuJoCo任务上实现了SOTA性能。


We study the use of knowledge distillation to compress the U-net architecture.We show that, while standard distillation is not sufficient to reliably train a compressed U-net, introducing other regularization methods, such as batch normalization and class re-weighting, in knowledge distillation significantly improves the training process.This allows us to compress a U-net by over 1000x, i.e., to 0.1% of its original number of parameters, at a negligible decrease in performance.

我们提出了额外的技术，利用知识蒸馏将U-net压缩1000倍以上。

作者介绍了一种改进的蒸馏策略，将U-net结构压缩了1000多倍，同时保留了接近原始U-net的精度。


Learning neural networks with gradient descent over a long sequence of tasks is problematic as their fine-tuning to new tasks overwrites the network weights that are important for previous tasks.This leads to a poor performance on old tasks â€“ a phenomenon framed as catastrophic forgetting.  While early approaches use task rehearsal and growing networks that both limit the scalability of the task sequence orthogonal approaches build on regularization.  Based on the Fisher information matrix (FIM) changes to parameters that are relevant to old tasks are penalized, which forces the task to be mapped into the available remaining capacity of the network.This requires to calculate the Hessian around a mode, which makes learning tractable.In this paper, we introduce Hessian-free curvature estimates as an alternative method to actually calculating the Hessian.  In contrast to previous work, we exploit the fact that most regions in the loss surface are flat and hence only calculate a Hessian-vector-product around the surface that is relevant for the current task.Our experiments show that on a variety of well-known task sequences we either significantly outperform or are en par with previous work.

本文提供了一种通过无Hessian的曲率估计来解决灾难性遗忘的方法

本文提出了一种近似的拉普拉斯方法在持续学习环境下的神经网络训练中具有较低的空间复杂性。


There has been recent interest in improving performance of simple models for multiple reasons such as interpretability, robust learning from small data, deployment in memory constrained settings as well as environmental considerations.In this paper, we propose a novel method SRatio that can utilize information from high performing complex models (viz. deep neural networks, boosted trees, random forests) to reweight a training dataset for a potentially low performing simple model such as a decision tree or a shallow network enhancing its performance.Our method also leverages the per sample hardness estimate of the simple model which is not the case with the prior works which primarily consider the complex model's confidences/predictions and is thus conceptually novel.Moreover, we generalize and formalize the concept of attaching probes to intermediate layers of a neural network, which was one of the main ideas in previous work \citep{profweight}, to other commonly used classifiers and incorporate this into our method.The benefit of these contributions is witnessed in the experiments where on 6 UCI datasets and CIFAR-10 we outperform competitors in a majority (16 out of 27) of the cases and tie for best performance in the remaining cases.In fact, in a couple of cases, we even approach the complex model's performance.We also conduct further experiments to validate assertions and intuitively understand why our method works.Theoretically, we motivate our approach by showing that the weighted loss minimized by simple models using our weighting upper bounds the loss of the complex model.

给出一个（准确的）复杂模型，改善简单模型性能的方法。 本文提出了一种改进低容量模型预测的方法，显示出比现有方法更多的好处。


We propose a principled method for kernel learning, which relies on a Fourier-analytic characterization of translation-invariant or rotation-invariant kernels.Our method produces a sequence of feature maps, iteratively refining the SVM margin.We provide rigorous guarantees for optimality and generalization, interpreting our algorithm as online equilibrium-finding dynamics in a certain two-player min-max game.Evaluations on synthetic and real-world datasets demonstrate scalability and consistent improvements over related random features-based methods.

利用傅里叶分析和遗憾最小化的工具，从训练数据中学习边际最大化的平移不变量或球形对称核的简单实用算法。

本文提出在傅里叶表示中学习一个自定义的平移或旋转不变的内核，以使SVM的边际最大化。

作者提出了一种有趣的算法，用于一起学习l1-SVM和傅里叶表示的核

作者考虑为机器学习应用直接学习移位/平移不变核的傅里叶表示，将核与数据的一致性作为目标函数来优化。


We elaborate on using importance sampling for causal reasoning, in particular for counterfactual inference.We show how this can be implemented natively in probabilistic programming.By considering the structure of the counterfactual query, one can significantly optimise the inference process.We also consider design choices to enable further optimisations.We introduce MultiVerse, a probabilistic programming prototype engine for approximate causal reasoning.We provide experimental results and compare with Pyro, an existing probabilistic programming framework with some of causal reasoning tools.

本质上支持因果、反事实推理的概率编程


We consider the problem of representing collective behavior of large populations and predicting the evolution of a population distribution over a discrete state space.A discrete time mean field game (MFG) is motivated as an interpretable model founded on game theory for understanding the aggregate effect of individual actions and predicting the temporal evolution of population distributions.We achieve a synthesis of MFG and Markov decision processes (MDP) by showing that a special MFG is reducible to an MDP.This enables us to broaden the scope of mean field game theory and infer MFG models of large real-world systems via deep inverse reinforcement learning.Our method learns both the reward function and forward dynamics of an MFG from real data, and we report the first empirical test of a mean field game model of a real-world social media population.

通过均值场博弈和马尔科夫决策过程的综合，推断大群体行为的均值场博弈（MFG）模型。

作者通过使用反强化学习来学习模型中代理的奖励函数来处理集体行为模型中的推理。


We study the problem of training sequential generative models for capturing coordinated multi-agent trajectory behavior, such as  offensive basketball gameplay.  When modeling such settings, it is often beneficial to design hierarchical models that can capture long-term coordination using intermediate variables.  Furthermore, these intermediate variables should capture interesting high-level behavioral semantics in an interpretable and manipulable way.We present a hierarchical framework that can effectively learn such sequential generative models.  Our approach is inspired by recent work on leveraging programmatically produced weak labels, which we extend to the spatiotemporal regime.In addition to synthetic settings, we show how to instantiate our framework to effectively model complex interactions between basketball players and generate realistic multi-agent trajectories of basketball gameplay over long time periods.We validate our approach using both quantitative and qualitative evaluations, including a user study comparison conducted with professional sports analysts.

我们将深度生成模型与程序化的弱监督相融合，以生成质量明显高于以往基线的协调的多代理轨迹。

提出了多代理顺序生成模型。

本文提出了训练生成模型，使用启发式函数产生多代理轨迹，这些函数标记了在训练数据中本来是潜在的变量。


Many automated machine learning methods, such as those for hyperparameter and neural architecture optimization, are computationally expensive because they involve training many different model configurations.In this work, we present a new method that saves computational budget by terminating poor configurations early on in the training.In contrast to existing methods, we consider this task as a ranking and transfer learning problem.We qualitatively show that by optimizing a pairwise ranking loss and leveraging learning curves from other data sets, our model is able to effectively rank learning curves without having to observe many or very long learning curves.We further demonstrate that our method can be used to accelerate a neural architecture search by a factor of up to 100 without a significant performance degradation of the discovered architecture.In further experiments we analyze the quality of ranking, the influence of different model components as well as the predictive behavior of the model.

学会对学习曲线进行排序，以便及早停止没有前途的训练工作。新颖之处：使用成对的排序损失来直接模拟跨数据集的改进和迁移学习的概率，以减少所需训练数据。

本文提出了一种对神经网络的学习曲线进行排序的方法，可以对不同数据集的学习曲线进行建模，在图像分类任务上实现更高的速度提升。


Continual learning is the problem of learning new tasks or knowledge while protecting old knowledge and ideally generalizing from old experience to learn new tasks faster.Neural networks trained by stochastic gradient descent often degrade on old tasks when trained successively on new tasks with different data distributions.This phenomenon, referred to as catastrophic forgetting, is considered a major hurdle to learning with non-stationary data or sequences of new tasks, and prevents networks from continually accumulating knowledge and skills.We examine this issue in the context of reinforcement learning, in a setting where an agent is exposed to tasks in a sequence.Unlike most other work, we do not provide an explicit indication to the model of task boundaries, which is the most general circumstance for a learning agent exposed to continuous experience.While various methods to counteract catastrophic forgetting have recently been proposed, we explore a straightforward, general, and seemingly overlooked solution - that of using experience replay buffers for all past events - with a mixture of on- and off-policy learning, leveraging behavioral cloning.We show that this strategy can still learn new tasks quickly yet can substantially reduce catastrophic forgetting in both Atari and DMLab domains, even matching the performance of methods that require task identities.When buffer storage is constrained, we confirm that a simple mechanism for randomly discarding data allows a limited size buffer to perform almost as well as an unbounded one.

我们表明，在持续学习的环境中，通过将非策略性RL应用于新的和重放的经验的混合物，以及行为上的克隆损失，可以避免灾难性的遗忘。

提出了一种特殊的经验重放与行为克隆的变体，作为一种持续学习的方法。


We present a method which learns to integrate temporal information, from a learned dynamics model, with ambiguous visual information, from a learned vision model, in the context of interacting agents.Our method is based on a graph-structured variational recurrent neural network, which is trained end-to-end to infer the current state of the (partially observed) world, as well as to forecast future states.We show that our method outperforms various baselines on two sports datasets, one based on real basketball trajectories, and one generated by a soccer game engine.

我们提出了一种方法，它可以学习在互动代理的背景下整合时间信息和模糊的视觉信息。

作者提出了图形VRNN，它通过为每个代理部署一个VRNN来模拟多个代理的互动。

本文提出了一个基于图形神经网络的架构，该架构经过训练可以直接从像素中定位和模拟环境中的代理的相互作用，并显示了模型在跟踪任务和预测代理位置方面的优势。


In this paper we study the problem of learning the weights of a deep convolutional neural network.We consider a network where convolutions are carried out over non-overlapping patches with a single kernel in each layer.We develop an algorithm for simultaneously learning all the kernels from the training data.Our approach dubbed Deep Tensor Decomposition (DeepTD) is based on a rank-1 tensor decomposition.We theoretically investigate DeepTD under a realizable model for the training data where the inputs are chosen i.i.d. from a Gaussian distribution and the labels are generated according to planted convolutional kernels.We show that DeepTD is data-efficient and provably works as soon as the sample size exceeds the total number of convolutional weights in the network.Our numerical experiments demonstrate the effectiveness of DeepTD and verify our theoretical findings.

我们考虑一个简化的深度卷积神经网络模型。我们表明，这个网络的所有层都可以通过适当应用张量分解来近似学习。

为使用一阶张量分解学习深度卷积神经网络提供理论保证。

本文提出了一种用于深度卷积网络的限制性案例的学习方法，其中各层仅限于非重叠情况，每层只有一个输出通道。

分析了学习一类非常特殊的CNN的问题：每层由一个过滤器组成，应用于输入的非重叠块。


Neural network pruning techniques can reduce the parameter counts of trained networks by over 90%, decreasing storage requirements and improving computational performance of inference without compromising accuracy.However, contemporary experience is that the sparse architectures produced by pruning are difficult to train from the start, which would similarly improve training performance.We find that a standard pruning technique naturally uncovers subnetworks whose initializations made them capable of training effectively.Based on these results, we articulate the "lottery ticket hypothesis:" dense, randomly-initialized, feed-forward networks contain subnetworks ("winning tickets") that - when trained in isolation - reach test accuracy comparable to the original network in a similar number of iterations.The winning tickets we find have won the initialization lottery: their connections have initial weights that make training particularly effective.We present an algorithm to identify winning tickets and a series of experiments that support the lottery ticket hypothesis and the importance of these fortuitous initializations.We consistently find winning tickets that are less than 10-20% of the size of several fully-connected and convolutional feed-forward architectures for MNIST and CIFAR10.Above this size, the winning tickets that we find learn faster than the original network and reach higher test accuracy.

训练后可以修剪权重的前馈神经网络，在训练前也可以修剪同样的权重

显示存在可以从头开始训练并具有良好泛化性能的稀疏子网络，并提出了一个未经修剪的、随机初始化的NN包含的子网络，可以从头开始训练并具有类似的泛化精度。

本文研究了这样一个假设：随机初始化的神经网络包含的子网络收敛速度同样快或更快，并能达到相同或更好的分类精度。


We investigate the difficulties of training sparse neural networks and make new observations about optimization dynamics and the energy landscape within the sparse regime.Recent work of \citep{Gale2019, Liu2018} has shown that sparse ResNet-50 architectures trained on ImageNet-2012 dataset converge to solutions that are significantly worse than those found by pruning.We show that, despite the failure of optimizers, there is a linear path with a monotonically decreasing objective from the initialization to the ``good'' solution.Additionally, our attempts to find a decreasing objective path from ``bad'' solutions to the ``good'' ones in the sparse subspace fail.However, if we allow the path to traverse the dense subspace, then we consistently find a path between two solutions.These findings suggest traversing extra dimensions may be needed to escape stationary points found in the sparse subspace.

在本文中，我们通过在能量景观中做插值实验来强调训练稀疏神经网络的难度


Neural network training depends on the structure of the underlying loss landscape, i.e. local minima, saddle points, flat plateaus, and loss barriers.In relation to the structure of the landscape, we study the permutation symmetry of neurons in each layer of a deep neural network, which gives rise not only to multiple equivalent global minima of the loss function but also to critical points in between partner minima.In a network of $d-1$ hidden layers with $n_k$ neurons in layers $k = 1, \ldots, d$, we construct continuous paths between equivalent global minima that lead through a `permutation point' where the input and output weight vectors of two neurons in the same hidden layer $k$ collide and interchange.We show that such permutation points are critical points which lie inside high-dimensional subspaces of equal loss, contributing to the global flatness of the landscape.We also find that a permutation point for the exchange of neurons $i$ and $j$ transits into a flat high-dimensional plateau that enables all $n_k!$permutations of neurons in a given layer $k$ at the same loss value. Moreover, we introduce higher-order permutation points by exploiting the hierarchical structure in the loss landscapes of neural networks, and find that the number of $K$-th order permutation points is much larger than the (already huge) number of equivalent global minima -- at least by a polynomial factor of order $K$. In twotasks, we demonstrate numerically with our path finding method that continuous paths between partner minima exist: first, in a toy network with a single hidden layer on a function approximation task and, second, in a multilayer network on the MNIST task. Our geometricapproach yields a lower bound on the number of critical points generated by weight-space symmetries and provides a simple intuitive link between previous theoretical results and numerical observations.

神经网络景观中的权重空间对称性产生了无数的鞍座和平坦的高维子空间。

论文提出了一种低损失的方法，从权重空间对称性的角度研究神经网络中与参数有关的损失函数。


The training of stochastic neural network models with binary ($\pm1$) weights and activations via continuous surrogate networks is investigated.We derive, using mean field theory, a set of scalar equations describing how input signals propagate through surrogate networks.The equations reveal that depending on the choice of surrogate model, the networks may or may not exhibit an order to chaos transition, and the presence of depth scales that limit the maximum trainable depth.Specifically, in solving the equations for edge of chaos conditions, we show that surrogates derived using the Gaussian local reparameterisation trick have no critical initialisation, whereas a deterministic surrogates based on analytic Gaussian integration do.The theory is applied to a range of binary neuron and weight design choices, such as different neuron noise models, allowing the categorisation of algorithms in terms of their behaviour at initialisation.Moreover, we predict theoretically and confirm numerically, that common weight initialization schemes used in standard continuous networks, when applied to the mean values of the stochastic binary weights, yield poor training performance.This study shows that, contrary to common intuition, the means of the stochastic binary weights should be initialised close to close to $\pm 1$ for deeper networks to be trainable.

信号传播理论应用于二进制网的连续替代；反直觉的初始化；重新参数化的技巧没有帮助

作者研究了使用连续代用物时二元神经网络的训练动态，研究了网络在初始化时应具有哪些属性以进行最佳训练，并对初始化时的随机权重提出了具体建议。

对随机二元网络、连续代用品及其训练动态的深入探索，以及对如何初始化权重以获得最佳性能的深刻见解。


Semantic dependency parsing, which aims to find rich bi-lexical relationships, allows words to have multiple dependency heads, resulting in graph-structured representations.We propose an approach to semi-supervised learning of semantic dependency parsers based on the CRF autoencoder framework.Our encoder is a discriminative neural semantic dependency parser that predicts the latent parse graph of the input sentence.Our decoder is a generative neural model that reconstructs the input sentence conditioned on the latent parse graph.Our model is arc-factored and therefore parsing and learning are both tractable.Experiments show our model achieves significant and consistent improvement over the supervised baseline.

我们提出了一种基于CRF自动编码器框架的半监督学习语义依赖分析器的方法。

本文重点讨论了半监督语义依赖分析，使用CRF-自动编码器以半监督的方式训练模型，表明在低资源标记数据任务上的有效性。


For sequence models with large word-level vocabularies, a majority of network parameters lie in the input and output layers.In this work, we describe a new method, DeFINE, for learning deep word-level representations efficiently.Our architecture uses a hierarchical structure with novel skip-connections which allows for the use of low dimensional input and output layers, reducing total parameters and training time while delivering similar or better performance versus existing methods.DeFINE can be incorporated easily in new or existing sequence models.Compared to state-of-the-art methods including adaptive input representations, this technique results in a 6% to 20% drop in perplexity.On WikiText-103, DeFINE reduces total parameters of Transformer-XL by half with minimal impact on performance.On the Penn Treebank, DeFINE improves AWD-LSTM by 4 points with a 17% reduction in parameters,  achieving comparable performance to state-of-the-art methods with fewer parameters.For machine translation, DeFINE improves a Transformer model by 2% while simultaneously reducing total parameters by 26%

DeFINE使用一个深度的、分层的、带有新的跳过连接的稀疏网络来有效地学习更好的词嵌入。

本文介绍了一种新的方法，通过使用低维输入和输出层的跳过连接的分层结构，有效地学习深层的词级表征。


In this paper, we present a reproduction of the paper of Bertinetto et al. [2019] "Meta-learning with differentiable closed-form solvers" as part of the ICLR 2019 Reproducibility Challenge.In successfully reproducing the most crucial part of the paper, we reach a performance that is comparable with or superior to the original paper on two benchmarks for several settings.We evaluate new baseline results, using a new dataset presented in the paper.Yet, we also provide multiple remarks and recommendations about reproducibility and comparability.  After we brought our reproducibility work to the authorsâ€™ attention, they have updated the original paper on which this work is based and released code as well.Our contributions mainly consist in reproducing the most important results of their original paper, in giving insight in the reproducibility and in providing a first open-source implementation.

我们成功地重现了一个用于少样本分类的元学习方法，并给出了与基线的比较意见，该方法通过闭式求解器的解法进行反向传播。


Network pruning has emerged as a powerful technique for reducing the size of deep neural networks.Pruning uncovers high-performance subnetworks by taking a trained dense network and gradually removing unimportant connections.Recently, alternative techniques have emerged for training sparse networks directly without having to train a large dense model beforehand, thereby achieving small memory footprints during both training and inference.These techniques are based on dynamic reallocation of non-zero parameters during training.Thus, they are in effect executing a training-time search for the optimal subnetwork.We investigate a most recent one of these techniques and conduct additional experiments to elucidate its behavior in training sparse deep convolutional networks.Dynamic parameter reallocation converges early during training to a highly trainable subnetwork.We show that neither the structure, nor the initialization of the discovered high-performance subnetwork is sufficient to explain its good performance.Rather, it is the dynamics of parameter reallocation that are responsible for successful learning.Dynamic parameter reallocation thus improves the trainability of deep convolutional networks, playing a similar role as overparameterization, without incurring the memory and computational cost of the latter.

动态参数重新分配能够成功地直接训练紧凑的稀疏网络，即使在我们预先知道最佳稀疏网络的情况下，它也发挥着不可或缺的作用。


n this paper we present a thrust in three directions of visual development us- ing supervised and semi-supervised techniques.The first is an implementation of semi-supervised object detection and recognition using the principles of Soft At- tention and Generative Adversarial Networks (GANs).The second and the third are supervised networks that learn basic concepts of spatial locality and quantity respectively using Convolutional Neural Networks (CNNs).The three thrusts to- gether are based on the approach of Experiential Robot Learning, introduced in previous publication.While the results are unripe for implementation, we believe they constitute a stepping stone towards autonomous development of robotic vi- sual modules.

3 thrusts serving as stepping stones for robot experiential learning of vision module

调查了现有图像分类器和物体检测器的性能。


Characterization of the representations learned in intermediate layers of deep networks can provide valuable insight into the nature of a task and can guide the development of well-tailored learning strategies.Here we study convolutional neural network-based acoustic models in the context of automatic speech recognition.Adapting a method proposed by Yosinski et al. [2014], we measure the transferability of each layer between German and English to assess the their language-specifity.We observe three distinct regions of transferability: (1) the first two layers are entirely transferable between languages, (2) layers 2â€“8 are also highly transferable but we find evidence of some language specificity, (3) the subsequent fully connected layers are more language specific but can be successfully finetuned to the target language.To further probe the effect of weight freezing, we performed follow-up experiments using freeze-training [Raghu et al., 2017].Our results are consistent with the observation that CCNs converge 'bottom up' during training and demonstrate the benefit of freeze training, especially for transfer learning.

除了基于CNN的声学模型的前两层之外，所有这些模型都表现出一定程度的语言特异性，但冻结训练使其能够在语言之间成功转移。

本文衡量了基于CNN的声学模型中各层特征在不同语言间的可转移性，结论是用 "冻结训练 "技术训练的AM比其他转移的模型表现更好。


Policy gradients methods often achieve better performance when the change in policy is limited to a small Kullback-Leibler divergence.We derive policy gradients where the change in policy is limited to a small Wasserstein distance (or trust region).This is done in the discrete and continuous multi-armed bandit settings with entropy regularisation.We show that in the small steps limit with respect to the Wasserstein distance $W_2$, policy dynamics are governed by the heat equation, following the Jordan-Kinderlehrer-Otto result.This means that policies undergo diffusion and advection, concentrating near actions with high reward.This helps elucidate the nature of convergence in the probability matching setup, and provides justification for empirical practices such as Gaussian policy priors and additive gradient noise.

将Wasserstein-信任区的熵策略梯度与热方程联系起来。

本文探讨了强化学习和二次优化运输理论之间的联系

作者研究了在多臂强盗背景下，由Wasserstein距离的信任区域限制的策略变化的政策梯度，显示在小步数限制下，策略动态受热方程（Fokker-Planck方程）支配。


The softmax function is widely used to train deep neural networks for multi-class classification.Despite its outstanding performance in classification tasks, the features derived from the supervision of softmax are usually sub-optimal in some scenarios where Euclidean distances apply in feature spaces.To address this issue, we propose a new loss, dubbed the isotropic loss, in the sense that the overall distribution of data points is regularized to approach the isotropic normal one.Combined with the vanilla softmax, we formalize a novel criterion called the isotropic softmax, or isomax for short, for supervised learning of deep neural networks.By virtue of the isomax, the intra-class features are penalized by the isotropic loss while inter-class distances are well kept by the original softmax loss.Moreover, the isomax loss does not require any additional modifications to the network, mini-batches or the training process.Extensive experiments on classification and clustering are performed to demonstrate the superiority and robustness of the isomax loss.

通过对数据点的全局分布进行各向同性的归一化处理，有效提高了softmax学习对象特征向量的辨别能力。


A fundamental question in reinforcement learning is whether model-free algorithms are sample efficient.Recently,  Jin et al. (2018) proposed a Q-learning algorithm with UCB exploration policy, and proved it has nearly optimal regret bound for finite-horizon episodic MDP.In this paper, we adapt Q-learning with UCB-exploration bonus to infinite-horizon MDP with discounted rewards \emph{without} accessing a generative model.We show that the \textit{sample complexity of exploration} of our algorithm is bounded by $\tilde{O}({\frac{SA}{\epsilon^2(1-\gamma)^7}})$.This improves the previously best known result of $\tilde{O}({\frac{SA}{\epsilon^4(1-\gamma)^8}})$ in this setting achieved by delayed Q-learning (Strehlet al., 2006),, and matches the lower bound in terms of $\epsilon$ as well as $S$ and $A$ up to logarithmic factors.

我们将带有UCB探索奖励的Q-learning调整为无限范围的MDP，无需访问生成模型即可获得折扣奖励，并改进了之前已知的最佳结果。

本文考虑了一种带有UCB探索策略的Q-learning算法，用于无限期的MDP。


Backpropagation is driving today's artificial neural networks (ANNs).However, despite extensive research, it remains unclear if the brain implements this algorithm.Among neuroscientists, reinforcement learning (RL) algorithms are often seen as a realistic alternative: neurons can randomly introduce change, and use unspecific feedback signals to observe their effect on the cost and thus approximate their gradient.However, the convergence rate of such learning scales poorly with the number of involved neurons.Here we propose a hybrid learning approach.Each neuron uses an RL-type strategy to learn how to approximate the gradients that backpropagation would provide.We provide proof that our approach converges to the true gradient for certain classes of networks.In both feedforward and convolutional networks, we empirically show that our approach learns to approximate the gradient, and can match the performance of gradient-based learning.Learning feedback weights provides a biologically plausible mechanism of achieving good performance, without the need for precise, pre-specified learning rules.

扰动可用于训练反馈权重，以便在全连接和卷积神经网络中学习。

本文提出了一种方法，通过使用基于噪声的估计器来估计后向通道的权重，解决 "权重传递 "问题


This paper proposes and demonstrates a surprising pattern in the training of neural networks: there is a one to one relation between the values of any pair of losses (such as cross entropy, mean squared error, 0/1 error etc.) evaluated for a model arising at (any point of) a training run.This pattern is universal in the sense that this one to one relationship is identical across architectures (such as VGG, Resnet, Densenet etc.), algorithms (SGD and SGD with momentum) and training loss functions (cross entropy and mean squared error).

我们在训练神经网络时，在不同的代用损失（CE、MSE、0-1损失）的行为中发现了一些普遍的模式（即在不同的架构中保持不变），并提出了支持的实证证据。