Visual Active Tracking (VAT) aims at following a target object by autonomously controlling the motion system of a tracker given visual observations. Previous work has shown that the tracker can be trained in a simulator via reinforcement learning and deployed in real-world scenarios. However, during training, such a method requires manually specifying the moving path of the target object to be tracked, which cannot ensure the tracker’s generalization on the unseen object moving patterns. To learn a robust tracker for VAT, in this paper, we propose a novel adversarial RL method which adopts an Asymmetric Dueling mechanism, referred to as AD-VAT. In AD-VAT, both the tracker and the target are approximated by end-to-end neural networks, and are trained via RL in a dueling/competitive manner: i.e., the tracker intends to lockup the target, while the target tries to escape from the tracker. They are asymmetric in that the target is aware of the tracker, but not vice versa. Specifically, besides its own observation, the target is fed with the tracker’s observation and action, and learns to predict the tracker’s reward as an auxiliary task. We show that such an asymmetric dueling mechanism produces a stronger target, which in turn induces a more robust tracker. To stabilize the training, we also propose a novel partial zero-sum reward for the tracker/target. The experimental results, in both 2D and 3D environments, demonstrate that the proposed method leads to a faster convergence in training and yields more robust tracking behaviors in different testing scenarios. For supplementary videos, see: https://www.youtube.com/playlist?list=PL9rZj4Mea7wOZkdajK1TsprRg8iUf51BS   The code is available at https://github.com/zfw1226/active_tracking_rl

我们提出了AD-VAT，其中跟踪器和目标对象被看作是两个可学习的代理，它们是对手，在训练中可以相互增强。

这项工作的目的是用一种训练机制来解决视觉主动跟踪问题，在这种机制中，跟踪者和目标作为相互的对手。

本文提出了一个简单的多代理深度RL任务，一个移动的跟踪器试图跟踪一个移动的目标。

提出了一个新的奖励函数--"部分零和"，它只在跟踪者和目标的竞争接近时鼓励他们，在他们太远时惩罚他们。


Identifying the hypernym relations that hold between words is a fundamental task in NLP. Word embedding methods have recently shown some capability to encode hypernymy. However, such methods tend not to explicitly encode the hypernym hierarchy that exists between words. In this paper, we propose a method to learn a hierarchical word embedding in a speciﬁc order to capture the hypernymy. To learn the word embeddings, the proposed method considers not only the hypernym relations that exists between words on a taxonomy, but also their contextual information in a large text corpus. The experimental results on a supervised hypernymy detection and a newly-proposed hierarchical path completion tasks show the ability of the proposed method to encode the hierarchy. Moreover, the proposed method outperforms previously proposed methods for learning word and hypernym-speciﬁc word embeddings on multiple benchmarks.

我们提出了一种利用语料库和分类法联合学习层次词嵌入（HWE）的方法，用于识别词之间的超义关系。

本文提出了一种利用共同发生统计学以及结合语义网络的层次信息来联合学习词嵌入的方法。

本文提出了一种从原始文本和监督分类法数据中联合学习超词的方法。

本文建议在GloVE目标中增加一个 "分布性包容 "差异的测量，以表示超词关系。


While self-organizing principles have motivated much of early learning models, such principles have rarely been included in deep learning architectures. Indeed, from a supervised learning perspective it seems that topographic constraints are rather decremental to optimal performance. Here we study a network model that incorporates self-organizing maps into a supervised network and show how gradient learning results in a form of a self-organizing learning rule. Moreover, we show that such a model is robust in the sense of its application to a variety of  areas, which is believed to be a hallmark of biological learning systems.

分层神经网络中自组织和监督学习的整合

本文讨论了在一个有三层的神经网络中的学习，其中中间层是按拓扑组织的，并研究了生物背景下无监督学习和分层监督学习之间的相互作用。

科霍宁自组织地图（SOM）的监督变体，但线性输出层被一个具有交叉熵的softmax层取代了平方误差。

提出了一个使用具有自组织激活函数的隐性神经元的模型，其输出反馈给具有softmax输出函数的分类器。


Quantization of a neural network has an inherent problem called accumulated quantization error, which is the key obstacle towards ultra-low precision, e.g., 2- or 3-bit precision. To resolve this problem, we propose precision highway, which forms an end-to-end high-precision information flow while performing the ultra-low-precision computation. First, we describe how the precision highway reduce the accumulated quantization error in both convolutional and recurrent neural networks. We also provide the quantitative analysis of the benefit of precision highway and evaluate the overhead on the state-of-the-art hardware accelerator. In the experiments, our proposed method outperforms the best existing quantization methods while offering 3-bit weight/activation quantization with no accuracy loss and 2-bit quantization with a 2.45 % top-1 accuracy loss in ResNet-50. We also report that the proposed method significantly outperforms the existing method in the 2-bit quantization of an LSTM for language modeling.

精密公路；4位以下量化的高精度信息流的一般概念

研究神经网络量化问题，采用端到端的精度高速公路，减少累积量化误差，实现深度神经网络的超低精度。

本文研究改善量化神经网络性能的方法

本文提出在ResNet和LSTM这两种网络结构中保持高激活/梯度流。


The vast majority of natural sensory data is temporally redundant. For instance, video frames or audio samples which are sampled at nearby points in time tend to have similar values.   Typically, deep learning algorithms take no advantage of this redundancy to reduce computations.   This can be an obscene waste of energy.   We present a variant on backpropagation for neural networks in which computation scales with the rate of change of the data - not the rate at which we process the data.   We do this by implementing a form of Predictive Coding wherein neurons communicate a combination of their state, and their temporal change in state, and quantize this signal using Sigma-Delta modulation.   Intriguingly, this simple communication rule give rise to units that resemble biologically-inspired leaky integrate-and-fire neurons, and to a spike-timing-dependent weight-update similar to Spike-Timing Dependent Plasticity (STDP), a synaptic learning rule observed in the brain.   We demonstrate that on MNIST, on a temporal variant of MNIST, and on Youtube-BB, a dataset with videos in the wild, our algorithm performs about as well as a standard deep network trained with backpropagation, despite only communicating discrete values between layers.

一种在时间冗余数据上有效训练神经网络的算法。

本文介绍了一种用于深度神经网络中基于尖峰学习的神经编码方案

本文提出了一种基于尖峰的学习方法，旨在减少学习和测试过程中对时间冗余数据进行分类时所需要的计算。

本文应用Sigma-Delta编码方案的预测编码版本来减少深度学习网络的计算负荷，以一种以前没有见过的方式将这三个部分结合起来。


Information bottleneck (IB) is a method for extracting information from one random variable X that is relevant for predicting another random variable Y. To do so, IB identifies an intermediate "bottleneck" variable T that has low mutual information I(X;T) and high mutual information I(Y;T). The "IB curve" characterizes the set of bottleneck variables that achieve maximal I(Y;T) for a given I(X;T), and is typically explored by maximizing the "IB Lagrangian", I(Y;T) - βI(X;T). In some cases, Y is a deterministic function of X, including many classification problems in supervised learning where the output class Y is a deterministic function of the input X. We demonstrate three caveats when using IB in any situation where Y is a deterministic function of X: (1) the IB curve cannot be recovered by maximizing the IB Lagrangian for different values of β; (2) there are "uninteresting" trivial solutions at all points of the IB curve; and (3) for multi-layer classifiers that achieve low prediction error, different layers cannot exhibit a strict trade-off between compression and prediction, contrary to a recent proposal. We also show that when Y is a small perturbation away from being a deterministic function of X, these three caveats arise in an approximate way. To address problem (1), we propose a functional that, unlike the IB Lagrangian, can recover the IB curve in all cases. We demonstrate the three caveats on the MNIST dataset.

只要输出是输入的确定性函数，信息瓶颈就会以令人惊讶的方式表现出来。

论证了大多数真实的分类问题在类别标签和输入X之间显示出这样一种确定性的关系，并探讨了由这种病理导致的几个问题。

探讨将信息瓶颈概念应用于确定性的监督学习模型时出现的问题

作者阐明了信息瓶颈法在确定性规则的监督学习中的几个反直觉行为。


We prove, under two sufficient conditions, that idealised models can have no adversarial examples. We discuss which idealised models satisfy our conditions, and show that idealised Bayesian neural networks (BNNs) satisfy these. We continue by studying near-idealised BNNs using HMC inference, demonstrating the theoretical ideas in practice. We experiment with HMC on synthetic data derived from MNIST for which we know the ground-truth image density, showing that near-perfect epistemic uncertainty correlates to density under image manifold, and that adversarial images lie off the manifold in our setting. This suggests why MC dropout, which can be seen as performing approximate inference, has been observed to be an effective defence against adversarial examples in practice; We highlight failure-cases of non-idealised BNNs relying on dropout, suggesting a new attack for dropout models and a new defence as well. Lastly, we demonstrate the defence on a cats-vs-dogs image classification task with a VGG13 variant.

我们证明了理想化的贝叶斯神经网络可以没有对抗性的例子，并且用现实世界的BNN给出了实证证据。

该论文研究了贝叶斯分类器的对抗性稳健性，并陈述了两个条件，他们表明这两个条件足以证明 "理想化模型 "在 "理想化数据集 "上不存在对抗性实例

论文提出了一类没有任何对抗性例子的判别性贝叶斯分类器。


Deep neural networks are susceptible to adversarial attacks. In computer vision, well-crafted perturbations to images can cause neural networks to make mistakes such as confusing a cat with a computer. Previous adversarial attacks have been designed to degrade performance of models or cause machine learning models to produce specific outputs chosen ahead of time by the attacker. We introduce attacks that instead reprogram the target model to perform a task chosen by the attacker without the attacker needing to specify or compute the desired output for each test-time input. This attack finds a single adversarial perturbation, that can be added to all test-time inputs to a machine learning model in order to cause the model to perform a task chosen by the adversary—even if the model was not trained to do this task. These perturbations can thus be considered a program for the new task. We demonstrate adversarial reprogramming on six ImageNet classification models, repurposing these models to perform a counting task, as well as classification tasks: classification of MNIST and CIFAR-10 examples presented as inputs to the ImageNet model.

我们介绍了对抗性攻击的第一个实例，即重新编程目标模型以执行攻击者选择的任务--而攻击者不需要为每个测试时间的输入指定或计算期望的输出。

作者提出了一种新的对抗性攻击方案，即神经网被重新利用来完成与最初训练时不同的任务。

本文提出了对训练有素的固定神经网络进行 "对抗性重新编程"，并表明对抗性重新编程对未训练的网络效果较差。

本文将NN监督学习中的 "对抗性攻击 "的概念扩展到对训练过的网络的解决方案进行全面的重新利用。


As shown in recent research, deep neural networks can perfectly fit randomly labeled data, but with very poor accuracy on held out data. This phenomenon indicates that loss functions such as cross-entropy are not a reliable indicator of generalization. This leads to the crucial question of how generalization gap should be predicted from the training data and network parameters. In this paper, we propose such a measure, and conduct extensive empirical studies on how well it can predict the generalization gap. Our measure is based on the concept of margin distribution, which are the distances of training points to the decision boundary. We find that it is necessary to use margin distributions at multiple layers of a deep network. On the CIFAR-10 and the CIFAR-100 datasets, our proposed measure correlates very strongly with the generalization gap. In addition, we find the following other factors to be of importance: normalizing margin values for scale independence, using characterizations of margin distribution rather than just the margin (closest distance to decision boundary), and working in log space instead of linear space (effectively using a product of margins rather than a sum).  Our measure can be easily applied to feedforward deep networks with any architecture and may point towards new training loss functions that could enable better generalization.

我们开发了一种新的方案来预测深度网络的泛化差距，并具有很高的准确性。

作者建议使用几何边际和层间边际分布来预测泛化差距。

经验表明，所提出的边际统计量和泛化差距之间存在有趣的联系，这可以用来为理解深度神经网的泛化提供一些规范性的见解。


We propose a new algorithm to learn a one-hidden-layer convolutional neural network where both the convolutional weights and the outputs weights are parameters to be learned. Our algorithm works for a general class of (potentially overlapping) patches, including commonly used structures for computer vision tasks. Our algorithm draws ideas from (1) isotonic regression for learning neural networks and (2) landscape analysis of non-convex matrix factorization problems. We believe these findings may inspire further development in designing provable algorithms for learning neural networks and other complex models. While our focus is theoretical, we also present experiments that illustrate our theoretical findings.

我们提出了一种可证明的恢复具有重叠图像块的卷积网络参数（卷积和输出权重）的算法。

本文研究了单隐层卷积神经网的理论学习，得出了一个学习算法和使用该算法的可证明保证。

本文给出了一种学习两层神经网络的新算法，该算法涉及单个卷积滤波器和不同位置的权重向量。


Since their invention, generative adversarial networks (GANs) have become a popular approach for learning to model a distribution of real (unlabeled) data. Convergence problems during training are overcome by Wasserstein GANs which minimize the distance between the model and the empirical distribution in terms of a different metric, but thereby introduce a Lipschitz constraint into the optimization problem. A simple way to enforce the Lipschitz constraint on the class of functions, which can be modeled by the neural network, is weight clipping. Augmenting the loss by a regularization term that penalizes the deviation of the gradient norm of the critic (as a function of the network's input) from one, was proposed as an alternative that improves training. We present theoretical arguments why using a weaker regularization term enforcing the Lipschitz constraint is preferable. These arguments are supported by experimental results on several data sets.

一个新的正则化术语可以改善你对Waserstein Gan的训练

本文提出了一种基于放宽1的Lipschitz常数约束的Wasserstein GAN的正则化方案。

文章讨论了基于L_1 Wasserstein度量的GAN拟合中的正则化/惩罚问题。


We introduce a new method for training GANs by applying the Wasserstein-2 metric proximal on the generators.   The approach is based on the gradient operator induced by optimal transport, which connects the geometry of sample space and parameter space in implicit deep generative models. From this theory, we obtain an easy-to-implement regularizer for the parameter updates. Our experiments demonstrate that this method improves the speed and stability in training GANs in terms of wall-clock time and Fr\'echet Inception Distance (FID) learning curves.

提出了一个新的GAN过程，考虑到前一次迭代中产生的点，并更新生成器为进行l次。

考虑到GAN学习中的自然梯度学习，其中采用了由Wasserstein-2距离引起的黎曼结构。

本文打算利用Wasserstein-2距离引起的自然梯度来训练GAN中的生成器，作者提出Wasserstein近似算子作为正则化。


Influence diagrams provide a modeling and inference framework for sequential decision problems, representing the probabilistic knowledge by a Bayesian network and the preferences of an agent by utility functions over the random variables and decision variables.  MDPs and POMDPS, widely used for planning under uncertainty can also be represented by influence diagrams.  The time and space complexity of computing the maximum expected utility (MEU) and its maximizing policy is exponential in the induced width of the underlying graphical model, which is often prohibitively large due to the growth of the information set under the sequence of decisions.  In this paper, we develop a weighted mini-bucket approach for bounding the MEU.  These bounds can be used as a stand-alone approximation that can be improved as a function of a controlling i-bound parameter . They can also be used as heuristic  functions to guide search, especially for planning   such as MDPs and POMDPs.  We evaluate the scheme empirically against state-of-the-art, thus illustrating its potential.

本文介绍了一种基于排除法的启发式决策函数用于顺序决策，适用于指导解决影响图的AND/OR搜索算法。

将minibuckets的推理启发式推广到影响图中。


Probabilistic Neural Networks deal with various sources of stochasticity: input noise, dropout, stochastic neurons, parameter uncertainties modeled as random variables, etc.  In this paper we revisit a feed-forward propagation approach that allows one to estimate for each neuron its mean and variance w.r.t. all mentioned sources of stochasticity. In contrast, standard NNs propagate only point estimates, discarding the uncertainty.  Methods propagating also the variance have been proposed by several authors in different context. The view presented here attempts to clarify the assumptions and derivation behind such methods, relate them to classical NNs and broaden their scope of applicability.  The main technical contributions are new approximations for the distributions of argmax and max-related transforms, which allow for fully analytic uncertainty propagation in networks with softmax and max-pooling layers as well as leaky ReLU activations.  We evaluate the accuracy of the approximation and suggest a simple calibration. Applying the method to networks with dropout allows for faster training and gives improved test likelihoods without the need of sampling.

噪声输入/丢弃/不确定参数下NN输出的平均值和方差的近似。argmax、softmax和max层的分析性近似。

作者专注于不确定性传播DNN的问题

本文重新审视了神经元中均值和方差的前馈传播，解决了通过最大集合层和softmax传播不确定性的问题。


Generative Adversarial Networks are one of the leading tools in generative modeling, image editing and content creation.   However, they are hard to train as they require a delicate balancing act between two deep networks fighting a never ending duel. Some of the most promising adversarial models today minimize a Wasserstein objective. It is smoother and more stable to optimize. In this paper, we show that the Wasserstein distance is just one out of a large family of objective functions that yield these properties. By making the discriminator of a GAN robust to adversarial attacks we can turn any GAN objective into a smooth and stable loss. We experimentally show that any GAN objective, including Wasserstein GANs, benefit from adversarial robustness both quantitatively and qualitatively. The training additionally becomes more robust to suboptimal choices of hyperparameters, model architectures, or objective functions.

一个不容易被对抗性例子愚弄的判别器使GAN训练更加稳健，并导致一个更平滑的目标。

本文提出了一种新的方法来稳定GAN的训练过程，通过对判别器进行正则化，使其对对抗性例子具有鲁棒性。

本文提出了一种系统化的带有稳健性正则化条款的GAN训练方式，使GAN的训练更加平滑。

提出的观点是，使判别器对对抗性扰动具有稳健性，可以使GAN目标变得平滑，从而在视觉上和FID方面取得更好的结果。


We propose a method to learn stochastic activation functions for use in probabilistic neural networks.  First, we develop a framework to embed stochastic activation functions based on Gaussian processes in probabilistic neural networks.  Second, we analytically derive expressions for the propagation of means and covariances in such a network, thus allowing for an efficient implementation and training without the need for sampling.  Third, we show how to apply variational Bayesian inference to regularize and efficiently train this model.  The resulting model can deal with uncertain inputs and implicitly provides an estimate of the confidence of its predictions.  Like a conventional neural network it can scale to datasets of arbitrary size and be extended with convolutional and recurrent connections, if desired.

我们将每个神经元的激活函数建模为高斯过程，并通过变分推理法与权重一起学习。 提出在神经网中每个激活函数的函数形式上放置高斯过程先验，以学习激活函数的形式。


Recent results from linear algebra stating that any matrix can be decomposed into products of diagonal and circulant matrices has lead to the design of compact deep neural network architectures that perform well in practice. In this paper, we bridge the gap between these good empirical results   and the theoretical approximation capabilities of Deep diagonal-circulant ReLU networks. More precisely, we first demonstrate  that a Deep diagonal-circulant ReLU networks of  bounded width and small depth can approximate a deep ReLU network in which the dense matrices are  of low rank. Based on this result, we provide new bounds on the expressive power and universal approximativeness of this type of networks. We support our experimental results with thorough experiments on a large, real world video classification problem.

我们对Deep circulant-diagonal ReLU网络的特性进行了理论研究，并证明它们是有界宽度的通用近似器。

本文提出使用环形矩阵和对角矩阵来加快计算速度，减少神经网络的内存需求。

本文证明了有界宽度的对角线-循环ReLU网络（DC-ReLU）是通用的近似器。


Camera drones, a rapidly emerging technology, offer people the ability to remotely inspect an environment with a high degree of mobility and agility. However, manual remote piloting of a drone is prone to errors. In contrast, autopilot systems can require a significant degree of environmental knowledge and are not necessarily designed to support flexible visual inspections. Inspired by camera manipulation techniques in interactive graphics, we designed StarHopper, a novel touch screen interface for efficient object-centric camera drone navigation, in which a user directly specifies the navigation of a drone camera relative to a specified object of interest. The system relies on minimal environmental information and combines both manual and automated control mechanisms to give users the freedom to remotely explore an environment with efficiency and accuracy. A lab study shows that StarHopper offers an efficiency gain of 35.4% over manual piloting, complimented by an overall user preference towards our object-centric navigation system.

StarHopper是一个新颖的触摸屏界面，用于高效和灵活的以物体为中心的摄像无人机导航

作者概述了他们开发的新的无人机控制界面StarHopper，将自动和手动驾驶结合到一个新的混合导航界面中，并通过使用一个额外的顶置摄像机摆脱了目标物体已经在无人机FOV中的假设。

本文介绍了StarHopper，一个在远程检查背景下用于半自动无人机导航的系统。

介绍了StarHopper，一个使用计算机视觉技术与触摸输入的应用程序，支持以物体为中心的无人机驾驶。


Recurrent neural networks (RNN), convolutional neural networks (CNN) and self-attention networks (SAN) are commonly used to produce context-aware representations. RNN can capture long-range dependency but is hard to parallelize and not time-efficient. CNN focuses on local dependency but does not perform well on some tasks. SAN can model both such dependencies via highly parallelizable computation, but memory requirement grows rapidly in line with sequence length. In this paper, we propose a model, called "bi-directional block self-attention network (Bi-BloSAN)", for RNN/CNN-free sequence encoding. It requires as little memory as RNN but with all the merits of SAN. Bi-BloSAN splits the entire sequence into blocks, and applies an intra-block SAN to each block for modeling local context, then applies an inter-block SAN to the outputs for all blocks to capture long-range dependency. Thus, each SAN only needs to process a short sequence, and only a small amount of memory is required. Additionally, we use feature-level attention to handle the variation of contexts around the same word, and use forward/backward masks to encode temporal order information. On nine benchmark datasets for different NLP tasks, Bi-BloSAN achieves or improves upon state-of-the-art accuracy, and shows better efficiency-memory trade-off than existing RNN/CNN/SAN.

用于RNN/CNN-free序列编码的自我注意力网络，内存消耗小，计算高度可并行，在几个NLP任务上具有最先进的性能

建议在两个层面上应用自我注意力，以限制基于注意力的模型的记忆要求，对速度的影响可以忽略不计。

本文介绍了双向块状自我注意力模型，作为一种通用的编码器，用于NLP中的各种序列建模任务。


End-to-end neural models have made significant progress in question answering, however recent studies show that these models implicitly assume that the answer and evidence appear close together in a single document. In this work, we propose the Coarse-grain Fine-grain Coattention Network (CFC), a new question answering model that combines information from evidence across multiple documents. The CFC consists of a coarse-grain module that interprets documents with respect to the query then finds a relevant answer, and a fine-grain module which scores each candidate answer by comparing its occurrences across all of the documents with the query. We design these modules using hierarchies of coattention and self-attention, which learn to emphasize different parts of the input. On the Qangaroo WikiHop multi-evidence question answering task, the CFC obtains a new state-of-the-art result of 70.6% on the blind test set, outperforming the previous best by 3% accuracy despite not using pretrained contextual encoders.

使用粗粒度细粒度分层注意力的多证据问答的新的最先进的模型。 提出了一种基于两个独立模块（粗粒度和细粒度模块）的多跳问答方法。 本文提出了一个有趣的粗粒度细粒度的共同注意力网络架构，以解决多证据问题的回答。 专注于多选问答，并提出了一个从粗到细的评分框架。


Generative adversarial networks (GANs) are an expressive class of neural generative models with tremendous success in modeling high-dimensional continuous measures. In this paper, we present a scalable method for unbalanced optimal transport (OT) based on the generative-adversarial framework. We formulate unbalanced OT as a problem of simultaneously learning a transport map and a scaling factor that push a source measure to a target measure in a cost-optimal manner. We provide theoretical justification for this formulation, showing that it is closely related to an existing static formulation by Liero et al. (2018). We then propose an algorithm for solving this problem based on stochastic alternating gradient updates, similar in practice to GANs, and perform numerical experiments demonstrating how this methodology can be applied to population modeling.

我们提出了使用生成式对抗网络进行不平衡最优运输的新方法。

作者使用随机最小最大算法和局部缩放法考虑了两个总质量不同的措施之间的不平衡最优运输问题

作者提出了一种估计采样措施之间的不平衡最优传输的方法，该方法在维度和采样数量上有很好的扩展性。

本文通过同时学习运输图T和缩放因子xi，引入了非平衡最优运输的静态公式。


Extracting saliency maps, which indicate parts of the image important to classification, requires many tricks to achieve satisfactory performance when using classifier-dependent methods. Instead, we propose classifier-agnostic saliency map extraction, which finds all parts of the image that any classifier could use, not just one given in advance. We observe that the proposed approach extracts higher quality saliency maps and outperforms existing weakly-supervised localization techniques, setting the new state of the art result on the ImageNet dataset.

我们提出了一种新的显著性地图提取方法，它能提取出更高质量的地图。

提出了一种与分类器无关的显著图提取方法。

本文介绍了一种新的显著图提取器，似乎可以改善最先进的结果。

作者认为，当提取的显著图直接依赖于一个模型时，那么它可能对不同的分类器没有用处，并提出了一个近似的解决方案。


Unsupervised image-to-image translation has gained considerable attention due to the recent impressive progress based on generative adversarial networks (GANs). However, previous methods often fail in challenging cases, in particular, when an image has multiple target instances and a translation task involves significant changes in shape, e.g., translating pants to skirts in fashion images. To tackle the issues, we propose a novel method, coined instance-aware GAN (InstaGAN), that incorporates the instance information (e.g., object segmentation masks) and improves multi-instance transfiguration. The proposed method translates both an image and the corresponding set of instance attributes while maintaining the permutation invariance property of the instances. To this end, we introduce a context preserving loss that encourages the network to learn the identity function outside of target instances. We also propose a sequential mini-batch inference/training technique that handles multiple instances with a limited GPU memory and enhances the network to generalize better for multiple instances. Our comparative evaluation demonstrates the effectiveness of the proposed method on different image datasets, in particular, in the aforementioned challenging cases. Code and results are available in https://github.com/sangwoomo/instagan

我们提出了一种新颖的方法，将实例属性集纳入图像到图像的转换。

本文提出了一种方法--InstaGAN--它建立在CycleGAN的基础上，以每个实例分割掩码的形式考虑到了实例信息，其结果与CycleGAN和其他基线相比更有优势。

提出针对未配对的图像与图像之间的转换问题，增加实例感知的分割掩码。


Deep neural networks (DNNs) generalize remarkably well without explicit regularization even in the strongly over-parametrized regime  where classical learning theory would instead predict that they would severely overfit.   While many proposals for some kind of implicit regularization have been made to rationalise this success, there is no consensus for the fundamental reason why DNNs do not strongly overfit.   In this paper, we provide a new explanation. By applying a very general probability-complexity bound recently derived from  algorithmic information theory (AIT), we argue that the parameter-function map of many DNNs should be exponentially biased towards simple functions. We then provide clear evidence for this strong simplicity bias in a model DNN for Boolean functions, as well as in much larger fully connected and convolutional networks trained on CIFAR10 and MNIST.  As the target functions in many real problems are expected to be highly structured, this intrinsic simplicity bias helps explain why deep networks generalize well on real world problems.  This picture also facilitates a novel PAC-Bayes approach where the prior is taken over the DNN input-output function space, rather than  the more conventional prior over parameter space.   If we assume that the training algorithm samples parameters close to uniformly within the zero-error region then the PAC-Bayes theorem can be used to guarantee good expected generalization for target functions producing high-likelihood training sets.   By exploiting recently discovered connections between DNNs and Gaussian processes to estimate the marginal likelihood,  we produce relatively tight generalization PAC-Bayes error bounds which correlate well with the true error on realistic datasets such as MNIST and CIFAR10 and for architectures including convolutional and fully connected networks.

深度网络的参数-函数图是有巨大偏差的；这可以解释为什么它们会泛化。我们使用PAC-Bayes和高斯过程来获得非空边界。

本文在PAC-Bayesian学习理论和经验支持的直觉的帮助下，研究了深度神经网络的泛化能力。

本文提出了对大型过度参数化神经网络泛化行为的解释，声称神经网络中的参数-函数图偏向于 "简单 "的函数，如果目标概念也是 "简单 "的，则泛化行为会很好。


We establish a theoretical link between evolutionary algorithms and variational parameter optimization of probabilistic generative models with binary hidden variables.  While the novel approach is independent of the actual generative model, here we use two such models to investigate its applicability and scalability: a noisy-OR Bayes Net (as a standard example of binary data) and Binary Sparse Coding (as a model for continuous data).   Learning of probabilistic generative models is first formulated as approximate maximum likelihood optimization using variational expectation maximization (EM).  We choose truncated posteriors as variational distributions in which discrete latent states serve as variational parameters. In the variational E-step,  the latent states are then   optimized according to a tractable free-energy objective . Given a data point, we can show that evolutionary algorithms can be used for the variational optimization loop by (A)~considering the bit-vectors of the latent states as genomes of individuals, and by (B)~defining the fitness of the  individuals as the (log) joint probabilities given by the used generative model.   As a proof of concept, we apply the novel evolutionary EM approach to the optimization of the parameters of noisy-OR Bayes nets and binary sparse coding on artificial and real data (natural image patches). Using point mutations and single-point cross-over for the evolutionary algorithm, we find that scalable variational EM algorithms are obtained which efficiently improve the data likelihood. In general we believe that, with the link established here, standard as well as recent results in the field of evolutionary optimization can be leveraged to address the difficult problem of parameter optimization in generative models.

我们提出了进化EM，作为一种新的算法，用于对具有二元潜变量的生成模型进行无监督训练，该算法将变异EM与进化优化紧密联系起来。

本文提出了一种结合进化计算和变异EM的方法，适用于通过基于粒子的近似表示的二元潜变量模型。

本文尝试将期望最大化训练算法与进化算法紧密结合。


While deep neural networks have achieved groundbreaking prediction results in many tasks, there is a class of data where existing architectures are not optimal -- sequences of probability distributions. Performing forward prediction on sequences of distributions has many important applications. However, there are two main challenges in designing a network model for this task. First, neural networks are unable to encode distributions compactly as each node encodes just a real value. A recent work of Distribution Regression Network (DRN) solved this problem with a novel network that encodes an entire distribution in a single node, resulting in improved accuracies while using much fewer parameters than neural networks. However, despite its compact distribution representation, DRN does not address the second challenge, which is the need to model time dependencies in a sequence of distributions. In this paper, we propose our Recurrent Distribution Regression Network (RDRN) which adopts a recurrent architecture for DRN. The combination of compact distribution representation and shared weights architecture across time steps makes RDRN suitable for modeling the time dependencies in a distribution sequence. Compared to neural networks and DRN, RDRN achieves the best prediction performance while keeping the network compact.

我们提出了一个高效的循环网络模型，用于对时变分布进行前向预测。

本文提出了一种创建神经网络的方法，将历史分布映射到分布上，并将该方法应用于几个分布预测任务。

提出了一种循环分布回归网络，它在以前的模型分布回归网络的基础上使用了一个循环结构。

本文通过研究循环神经网络环境下的时间变化分布，对概率分布进行回归。


We present graph attention networks (GATs), novel neural network architectures that operate on graph-structured data, leveraging masked self-attentional layers to address the shortcomings of prior methods based on graph convolutions or their approximations. By stacking layers in which nodes are able to attend over their neighborhoods' features, we enable (implicitly) specifying different weights to different nodes in a neighborhood, without requiring any kind of computationally intensive matrix operation (such as inversion) or depending on knowing the graph structure upfront. In this way, we address several key challenges of spectral-based graph neural networks simultaneously, and make our model readily applicable to inductive as well as transductive problems. Our GAT models have achieved or matched state-of-the-art results across four established transductive and inductive graph benchmarks: the Cora, Citeseer and Pubmed citation network datasets, as well as a protein-protein interaction dataset (wherein test graphs remain unseen during training).

用神经网络处理图形结构数据的新方法，利用对节点邻域的关注。在归纳引文网络任务和归纳蛋白质-蛋白质相互作用任务上取得了最先进的结果。

本文提出了一种对图的节点进行分类的新方法，该方法可用于半监督的场景和全新的图上。

本文介绍了一个在图结构数据上操作的神经网络架构，名为图注意网络。

对学习图的节点的向量表示的最新方法进行了公平和几乎全面的讨论。


While bigger and deeper neural network architectures continue to advance the state-of-the-art for many computer vision tasks, real-world adoption of these networks is impeded by hardware and speed constraints. Conventional model compression methods attempt to address this problem by modifying the architecture manually or using pre-defined heuristics. Since the space of all reduced architectures is very large, modifying the architecture of a deep neural network in this way is a difficult task. In this paper, we tackle this issue by introducing a principled method for learning reduced network architectures in a data-driven way using reinforcement learning. Our approach takes a larger 'teacher' network as input and outputs a compressed 'student' network derived from the 'teacher' network. In the first stage of our method, a recurrent policy network aggressively removes layers from the large 'teacher' model. In the second stage, another  recurrent policy network carefully reduces the size of each remaining layer. The resulting network is then evaluated to obtain a reward -- a score based on the accuracy and compression of the network. Our approach uses this reward signal with policy gradients to train the policies to find a locally optimal student network. Our experiments show that we can achieve compression rates of more than 10x for models such as ResNet-34 while maintaining similar performance to the input 'teacher' network. We also present a valuable transfer learning result which shows that policies which are pre-trained on smaller 'teacher' networks can be used to rapidly speed up training on larger 'teacher' networks.

基于强化学习的新型方法，用知识蒸馏法压缩深度神经网络

本文提出在知识蒸馏过程中使用强化学习而不是预先定义的启发式方法来确定压缩模型的结构

介绍了一种网络对网络压缩的原则性方法，它使用策略梯度来优化两个策略，将一个强大的教师压缩成一个强大但较小的学生模型。


Recent advances in conditional image generation tasks, such as image-to-image translation and image inpainting, are largely accounted to the success of conditional GAN models, which are often optimized by the joint use of the GAN loss with the reconstruction loss. However, we reveal that this training recipe shared by almost all existing methods causes one critical side effect: lack of diversity in output samples. In order to accomplish both training stability and multimodal output generation, we propose novel training schemes with a new set of losses named moment reconstruction losses that simply replace the reconstruction loss. We show that our approach is applicable to any conditional generation tasks by performing thorough experiments on image-to-image translation, super-resolution and image inpainting using Cityscapes and CelebA dataset. Quantitative evaluations also confirm that our methods achieve a great diversity in outputs while retaining or even improving the visual fidelity of generated samples.

我们证明了条件型GAN的模式崩溃主要是由于重建损失和GAN损失之间的不匹配，并介绍了一套新的损失函数作为重建损失的替代品。

本文提出了对传统的条件性GAN目标的修改，以促进图像的多样化、多模态生成。

本文提出了一种替代L1/L2误差的方法，这些误差在训练条件GAN时被用来增加对抗性损失。


Generative models are important tools to capture and investigate the properties of complex empirical data. Recent developments such as Generative Adversarial Networks (GANs) and Variational Auto-Encoders (VAEs) use two very similar, but \textit{reverse}, deep convolutional architectures, one to generate and one to extract information from data. Does learning the parameters of both architectures obey the same rules? We exploit the causality principle of independence of mechanisms to quantify how the weights of successive layers adapt to each other. Using the recently introduced Spectral Independence Criterion, we quantify the dependencies between the kernels of successive convolutional layers and show that those are more independent for the generative process than for information extraction, in line with results from the field of causal inference. In addition, our experiments on generation of human faces suggest that more independence between successive layers of generators results in improved performance of these architectures.

我们使用因果推理来描述生成模型的结构特点 本文研究了VAE的编码器和解码器以及GAN的发生器和判别器中卷积滤波器的性质。 这项工作利用了因果关系原理来量化连续层的权重如何相互适应。


Many deep reinforcement learning approaches use graphical state representations,  this means visually distinct games that share the same underlying structure cannot  effectively share knowledge. This paper outlines a new approach for learning  underlying game state embeddings irrespective of the visual rendering of the game  state. We utilise approaches from multi-task learning and domain adaption in  order to place visually distinct game states on a shared embedding manifold. We  present our results in the context of deep reinforcement learning agents.

一种学习视觉上不同的游戏之间共享嵌入空间的方法。

一种学习视觉上不同的游戏底层结构的新方法，结合了处理输入图像的卷积层、用于深度强化学习的异步优势演员批判和强迫嵌入表示独立于游戏的视觉表示的对抗性方法。

介绍了一种通过适应深度强化学习来学习视觉上不同游戏的策略的方法。

本文讨论了一个代理架构，该架构使用一个共享的表示法来训练具有不同精灵级别视觉统计的多个任务


We study discrete time dynamical systems governed by the state equation $h_{t+1}=ϕ(Ah_t+Bu_t)$. Here A,B are weight matrices, ϕ is an activation function, and $u_t$ is the input data. This relation is the backbone of recurrent neural networks (e.g. LSTMs) which have broad applications in sequential learning tasks. We utilize stochastic gradient descent to learn the weight matrices from a finite input/state trajectory $(u_t,h_t)_{t=0}^N$. We prove that SGD estimate linearly converges to the ground truth weights while using near-optimal sample size. Our results apply to increasing activations whose derivatives are bounded away from zero. The analysis is based on i) an SGD convergence result with nonlinear activations and ii) careful statistical characterization of the state vector. Numerical experiments verify the fast convergence of SGD on ReLU and leaky ReLU in consistence with our theory.

我们研究了一个循环神经网络的状态方程。我们表明，在适当的假设下，SGD可以有效地从少数输入/输出观测中学习未知的动态。

本文研究了具有非线性状态方程的离散时间动力系统，证明了在固定长度的轨迹上运行SGD可以得到对数收敛。

这项工作考虑了学习一个非线性动态系统的问题，其中输出等于状态。

本文研究了SGD学习线性系统的动态和非线性激活的能力。


Although deep neural networks show their extraordinary power in various tasks, they are not feasible for deploying such large models on embedded systems due to high computational cost and storage space limitation. The recent work knowledge distillation (KD) aims at transferring model knowledge from a well-trained teacher model to a small and fast student model which can significantly help extending the usage of large deep neural networks on portable platform. In this paper, we show that, by properly defining the neuron manifold of deep neuron network (DNN), we can significantly improve the performance of student DNN networks through approximating neuron manifold of powerful teacher network. To make this, we propose several novel methods for learning neuron manifold from DNN model. Empowered with neuron manifold knowledge, our experiments show the great improvement across a variety of DNN architectures and training data. Compared with other KD methods, our Neuron Manifold Transfer (NMT) has best transfer ability of the learned features.

一种用于迁移学习的新的知识蒸馏方法 该工作介绍了一种使用所提出的神经元流形概念的知识蒸馏方法。 提出了一种知识蒸馏的方法，其中神经流形被作为转移的知识。


We present a simple and general method to train a single neural network executable at different widths (number of channels in a layer), permitting instant and adaptive accuracy-efficiency trade-offs at runtime. Instead of training individual networks with different width configurations, we train a shared network with switchable batch normalization. At runtime, the network can adjust its width on the fly according to on-device benchmarks and resource constraints, rather than downloading and offloading different models. Our trained networks, named slimmable neural networks, achieve similar (and in many cases better) ImageNet classification accuracy than individually trained models of MobileNet v1, MobileNet v2, ShuffleNet and ResNet-50 at different widths respectively. We also demonstrate better performance of slimmable models compared with individual ones across a wide range of applications including COCO bounding-box object detection, instance segmentation and person keypoint detection without tuning hyper-parameters. Lastly we visualize and discuss the learned features of slimmable networks. Code and models are available at: https://github.com/JiahuiYu/slimmable_networks

我们提出了一种简单而通用的方法来训练一个可在不同宽度（一个层中的通道数）下执行的单一神经网络，允许在运行时进行即时和自适应的准确性-效率权衡。

本文提出了将不同大小的模型组合成一个共享网的想法，大大提高了检测的性能。

本文在不同的宽度下训练了一个单一的可执行网络。


Measuring visual (dis)similarity between two or more instances within a data distribution is a fundamental task in many applications, specially in image retrieval. Theoretically, non-metric distances are able to generate a more complex and accurate similarity model than metric distances, provided that the non-linear data distribution is precisely captured by the similarity model. In this work, we analyze a simple approach for deep learning networks to be used as an approximation of non-metric similarity functions and we study how these models generalize across different image retrieval datasets.

学习一对图像之间的非度量视觉相似性估计的相似性网络

作者提出了视觉相似性的学习相似性措施，并以此在非常著名的牛津和巴黎的图像检索数据集中获得了改进。

本文认为，使用非公制距离而不是公制距离更合适。


Training a model to perform a task typically requires a large amount of data from the domains in which the task will be applied.  However, it is often the case that data are abundant in some domains but scarce in others. Domain adaptation deals with the challenge of adapting a model trained from a data-rich source domain to perform well in a data-poor target domain. In general, this requires learning plausible mappings between domains. CycleGAN is a powerful framework that efficiently learns to map inputs from one domain to another using adversarial training and a cycle-consistency constraint. However, the conventional approach of enforcing cycle-consistency via reconstruction may be overly restrictive in cases where one or more domains have limited training data. In this paper, we propose an augmented cyclic adversarial learning model that enforces the cycle-consistency constraint via an external task specific model, which encourages the preservation of task-relevant content as opposed to exact reconstruction. We explore digit classification in a low-resource setting in supervised, semi and unsupervised situation, as well as high resource unsupervised. In low-resource supervised setting, the results show that our approach improves absolute performance by 14% and 4% when adapting SVHN to MNIST and vice versa, respectively, which outperforms unsupervised domain adaptation methods that require high-resource unlabeled target domain.   Moreover, using only few unsupervised target data, our approach can still outperforms many high-resource unsupervised models. Our model also outperforms on USPS to MNIST and synthetic digit to SVHN for high resource unsupervised adaptation. In speech domains, we similarly adopt a speech recognition model from each domain as the task specific model. Our approach improves absolute performance of speech recognition by 2% for female speakers in the TIMIT dataset, where the majority of training samples are from male voices.

一种新的循环对抗学习与辅助任务模型相结合，提高了低资源监督和无监督情况下的领域适应性能

提出了循环一致的逆向适应方法的扩展，以解决有监督的目标数据有限时的领域适应。

本文介绍了一种基于Cyclic GAN思想的领域适应方法，并提出了两种不同的算法。


Nodes residing in different parts of a graph can have similar structural roles within their local network topology. The identification of such roles provides key insight into the organization of networks and can also be used to inform machine learning on graphs. However, learning structural representations of nodes is a challenging unsupervised-learning task, which typically involves manually specifying and tailoring topological features for each node. Here we develop GraphWave, a method that represents each node’s local network neighborhood via a low-dimensional embedding by leveraging spectral graph wavelet diffusion patterns. We prove that nodes with similar local network neighborhoods will have similar GraphWave embeddings even though these nodes may reside in very different parts of the network. Our method scales linearly with the number of edges and does not require any hand-tailoring of topological features. We evaluate performance on both synthetic and real-world datasets, obtaining improvements of up to 71% over state-of-the-art baselines.

我们开发了一种基于谱图小波扩散的学习网络结构特征的方法。 利用节点局部近邻的谱图小波扩散模式将节点嵌入低维空间中 本文基于图拉普拉斯算子的小波分析，得出了一种比较图中节点的方法。


Driving simulators play an important role in vehicle research. However, existing virtual reality simulators do not give users a true sense of presence. UniNet is our driving simulator, designed to allow users to interact with and visualize simulated traffic in mixed reality. It is powered by SUMO and Unity. UniNet's modular architecture allows us to investigate interdisciplinary research topics such as vehicular ad-hoc networks, human-computer interaction, and traffic management. We accomplish this by giving users the ability to observe and interact with simulated traffic in a high fidelity driving simulator. We present a user study that subjectively measures user's sense of presence in UniNet. Our findings suggest that our novel mixed reality system does increase this sensation.

一个使用立体摄像机和穿透式VR的混合现实驾驶模拟器在24名参与者的用户研究中进行了评估。

提出了一个复杂的驾驶模拟系统。

本文介绍了一个混合现实驾驶模拟器的设置，以增强存在感。

提出了一个混合现实驾驶模拟器，其中包括交通生成，并声称由于MR系统而增强了 "存在感"。


We consider the problem of improving kernel approximation via feature maps. These maps arise as Monte Carlo approximation to integral representations of kernel functions and scale up kernel methods for larger datasets. We propose to use more efficient numerical integration technique to obtain better estimates of the integrals compared to the state-of-the-art methods. Our approach allows to use information about the integrand to enhance approximation and facilitates fast computations. We derive the convergence behavior and conduct an extensive empirical study that supports our hypothesis.

本文提出通过使用正交规则（如随机球状辐射规则）来改进随机特征的核近似。

作者提出了一种新型的随机特征图方法来近似解决大规模的核问题。

本文表明，由于Genz & Monahan（1998）的技术可用于在随机傅里叶特征的框架下实现低内核近似误差，这是一种应用正交规则改善内核近似的新方法。


Human world knowledge is both structured and flexible. When people see an object, they represent it not as a pixel array but as a meaningful arrangement of semantic parts. Moreover, when people refer to an object, they provide descriptions that are not merely true but also relevant in the current context. Here, we combine these two observations in order to learn fine-grained correspondences between language and contextually relevant geometric properties of 3D objects. To do this, we employed an interactive communication task with human participants to construct a large dataset containing natural utterances referring to 3D objects from ShapeNet in a wide variety of contexts. Using this dataset, we developed neural listener and speaker models with strong capacity for generalization. By performing targeted lesions of visual and linguistic input, we discovered that the neural listener depends heavily on part-related words and associates these words correctly with the corresponding geometric properties of objects, suggesting that it has learned task-relevant structure linking the two input modalities. We further show that a neural speaker that is `listener-aware' --- that plans its utterances according to how an imagined listener would interpret its words in context --- produces more discriminative referring expressions than an `listener-unaware' speaker, as measured by human performance in identifying the correct object.

如何建立神经说话人/听众，从指代语言中学习三维物体的细微特征。

作者提供了一项关于学习指代三维物体的研究，收集了一个指代表达的数据集，并通过试验一些建筑选择来训练几个模型


Object-based factorizations provide a useful level of abstraction for interacting with the world. Building explicit object representations, however, often requires supervisory signals that are difficult to obtain in practice. We present a paradigm for learning object-centric representations for physical scene understanding without direct supervision of object properties. Our model, Object-Oriented Prediction and Planning (O2P2), jointly learns a perception function to map from image observations to object representations, a pairwise physics interaction function to predict the time evolution of a collection of objects, and a rendering function to map objects back to pixels. For evaluation, we consider not only the accuracy of the physical predictions of the model, but also its utility for downstream tasks that require an actionable representation of intuitive physics. After training our model on an image prediction task, we can use its learned representations to build block towers more complicated than those observed during training.

我们提出了一个学习以物体为中心的表征的框架，该框架适用于需要理解物理的任务中的计划。

本文提出了一个预测物体在引力作用下相互作用的图像的平台。

本文提出了一种方法，可以从给定的图像中学习复制 "块状塔"。

提出了一种方法，可以学习推理不同物体的物理互动，而不需要监督物体的属性。


We study the error landscape of deep linear and nonlinear neural networks with the squared error loss. Minimizing the loss of a deep linear neural network is a nonconvex problem, and despite recent progress, our understanding of this loss surface is still incomplete. For deep linear networks, we present necessary and sufficient conditions for a critical point of the risk function to be a global minimum. Surprisingly, our conditions provide an efficiently checkable test for global optimality, while such tests are typically intractable in nonconvex optimization. We further extend these results to deep nonlinear neural networks and prove similar sufficient conditions for global optimality, albeit in a more limited function space setting.

我们为深度线性神经网络的全局最优性提供了有效的可检查的必要条件和充分条件，并对非线性设置进行了一些初步扩展。

本文给出了深度线性神经网络损失函数的全局最优性的条件

本文给出了关于深度神经网络目标函数中存在局部最小值的理论结果。

研究深度线性网络的一些理论特性。


Recurrent auto-encoder model can summarise sequential data through an encoder structure into a fixed-length vector and then reconstruct into its original sequential form through the decoder structure. The summarised information can be used to represent time series features. In this paper, we propose relaxing the dimensionality of the decoder output so that it performs partial reconstruction. The fixed-length vector can therefore represent features only in the selected dimensions. In addition, we propose using rolling fixed window approach to generate samples. The change of time series features over time can be summarised as a smooth trajectory path. The fixed-length vectors are further analysed through additional visualisation and unsupervised clustering techniques.    This proposed method can be applied in large-scale industrial processes for sensors signal analysis purpose where clusters of the vector representations can be used to reflect the operating states of selected aspects of the industrial system.

使用循环自动编码器模型来提取多维时间序列特征

本文介绍了循环自动编码器在多维时间序列分析中的应用。

本文描述了一个序列到序列的自动编码器模型，该模型用于学习序列表征，表明对于它们的应用，当网络只被训练为重建数据测量的一个子集时，可以获得更好的性能。

提出了一种受循环自动编码器模型启发的策略，这样就可以根据背景向量对多维时间序列数据进行聚类。


We view molecule optimization as a graph-to-graph translation problem. The goal is to learn to map from one molecular graph to another with better properties based on an available corpus of paired molecules. Since molecules can be optimized in different ways, there are multiple viable translations for each input graph. A key challenge is therefore to model diverse translation outputs. Our primary contributions include a junction tree encoder-decoder for learning diverse graph translations along with a novel adversarial training method for aligning distributions of molecules. Diverse output distributions in our model are explicitly realized by low-dimensional latent vectors that modulate the translation process. We evaluate our model on multiple molecule optimization tasks and show that our model outperforms previous state-of-the-art baselines by a significant margin.

我们引入了一个图到图的编码器-解码器框架，用于学习多样化的图转换。

在匹配分子对分析的启发下，提出了分子优化的图-图转换模型。

通过增加潜伏变量来捕捉多模态和潜伏空间中的对抗性正则化，将JT-VAE扩展到图与图的转换场景中

提出了一个相当复杂的系统，涉及许多不同的选择和组成部分，以便从一个给定的语料库开始获得具有改进性质的化学复合物。


Partial differential equations (PDEs) are widely used across the physical and computational sciences. Decades of research and engineering went into designing fast iterative solution methods. Existing solvers are general purpose, but may be sub-optimal for specific classes of problems. In contrast to existing hand-crafted solutions, we propose an approach to learn a fast iterative solver tailored to a specific domain. We achieve this goal by learning to modify the updates of an existing solver using a deep neural network. Crucially, our approach is proven to preserve strong correctness and convergence guarantees. After training on a single geometry, our model generalizes to a wide variety of geometries and boundary conditions, and achieves 2-3 times speedup compared to state-of-the-art solvers.

我们学习了一种具有收敛保证的PDEs的快速神经解算器。

开发了一种加速解决PDEs的有限差分方法，并提出了离散化后固定点迭代的修订框架。

作者提出了一种加快PDE求解器速度的线性方法。


Variational Bayesian neural networks (BNN) perform variational inference over weights, but it is difficult to specify meaningful priors and approximating posteriors in a high-dimensional weight space. We introduce functional variational Bayesian neural networks (fBNNs), which maximize an Evidence Lower BOund (ELBO) defined directly on stochastic processes, i.e. distributions over functions. We prove that the KL divergence between stochastic processes is equal to the supremum of marginal KL divergences over all finite sets of inputs. Based on this, we introduce a practical training objective which approximates the functional ELBO using finite measurement sets and the spectral Stein gradient estimator. With fBNNs, we can specify priors which entail rich structure, including Gaussian processes and implicit stochastic processes. Empirically, we find that fBNNs extrapolate well using various structured priors, provide reliable uncertainty estimates, and can scale to large datasets.

我们对贝叶斯神经网络定义的随机过程进行功能变分推理。

拟合函数形式的变分贝叶斯神经网络近似，并考虑通过样本隐含地与随机过程先验匹配。

提出了一种用于训练BNN的新型ELBO目标，允许在模型中编码更有意义的先验，而不是文献中信息量较小的权重先验特征。

提出了一种新的贝叶斯神经网络模型的变分推理算法，在这种算法中，先验是以函数方式而不是通过权重的先验来指定的。


Words are not created equal. In fact, they form an aristocratic graph with a latent hierarchical structure that the next generation of unsupervised learned word embeddings should reveal. In this paper, justified by the notion of delta-hyperbolicity or tree-likeliness of a space, we propose to embed words in a Cartesian product of hyperbolic spaces which we theoretically connect to the Gaussian word embeddings and their Fisher geometry. This connection allows us to introduce a novel principled hypernymy score for word embeddings. Moreover, we adapt the well-known Glove algorithm to learn unsupervised word embeddings in this type of Riemannian manifolds. We further explain how to solve the analogy task using the Riemannian parallel transport that generalizes vector arithmetics to this new type of geometry. Empirically, based on extensive experiments, we prove that our embeddings, trained unsupervised, are the first to simultaneously outperform strong and popular baselines on the tasks of similarity, analogy and hypernymy detection. In particular, for word hypernymy, we obtain new state-of-the-art on fully unsupervised WBLESS classification accuracy.

我们将词嵌入双曲空间，并与高斯词嵌入建立了联系。

本文将Glove词嵌入改编为由Poincare半平面模型给出的双曲空间

本文提出了一种实现基于GLOVE的双曲词嵌入模型的方法，它通过黎曼优化方法进行优化。


Answering questions about a text frequently requires aggregating information from multiple places in that text. End-to-end neural network models, the dominant approach in the current literature, can theoretically learn how to distill and manipulate representations of the text without explicit supervision about how to do so. We investigate a canonical architecture for this task, the memory network, and analyze how effective it really is in the context of three multi-hop reasoning settings. In a simple synthetic setting, the path-finding task of the bAbI dataset, the model fails to learn the correct reasoning without additional supervision of its attention mechanism. However, with this supervision, it can perform well. On a real text dataset, WikiHop, the memory network gives nearly state-of-the-art performance, but does so without using its multi-hop capabilities. A tougher anonymized version of the WikiHop dataset is qualitatively similar to bAbI: the model fails to perform well unless it has additional supervision. We hypothesize that many "multi-hop" architectures do not truly learn this reasoning as advertised, though they could learn this reasoning if appropriately supervised.

除非我们监督它们，否则记忆网络不会学习多跳推理。

声称多跳推理不容易直接学习，需要直接监督，在WikiHop上做得很好并不一定意味着模型真的在学习跳跃。

本文拟研究众所周知的记忆网络学习问题，更确切地说，是用这种模型进行注意力学习监督的难度。

本文认为，记忆网络未能学会合理的多跳推理。


Generative Adversarial Nets (GANs) and Variational Auto-Encoders (VAEs) provide impressive image generations from Gaussian white noise, but the underlying mathematics are not well understood. We compute deep convolutional network generators by inverting a fixed embedding operator. Therefore, they do not require to be optimized with a discriminator or an encoder. The embedding is Lipschitz continuous to deformations so that generators transform linear interpolations between input white noise vectors into deformations between output images. This embedding is computed with a wavelet Scattering transform. Numerical experiments demonstrate that the resulting Scattering generators have similar properties as GANs or VAEs, without learning a discriminative network or an encoder.

我们引入了生成网络，它不需要用判别器或编码器来学习；它们是通过倒置一个由小波散射变换定义的特殊嵌入算子得到的。

介绍散射变换作为生成对抗网络背景下的图像生成模型，并提出为什么它们可以被视为具有可控信息损失和可逆性的高斯化变换。

本文提出了一种图像的生成模型，不需要学习判别器（如GAN）或学习嵌入。


Recurrent neural networks (RNNs) can model natural language by sequentially ''reading'' input tokens and outputting a distributed representation of each token. Due to the sequential nature of RNNs, inference time is linearly dependent on the input length, and all inputs are read regardless of their importance. Efforts to speed up this inference, known as ''neural speed reading'', either ignore or skim over part of the input. We present Structural-Jump-LSTM: the first neural speed reading model to both skip and jump text during inference. The model consists of a standard LSTM and two agents: one capable of skipping single words when reading, and one capable of exploiting punctuation structure (sub-sentence separators (,:), sentence end symbols (.!?), or end of text markers) to jump ahead after reading a word.  A comprehensive experimental evaluation of our model against all five state-of-the-art neural reading models shows that   Structural-Jump-LSTM achieves the best overall floating point operations (FLOP) reduction (hence is faster), while keeping the same accuracy or even improving it compared to a vanilla LSTM that reads the whole text.

我们提出了一个新的神经速读模型，利用文本固有的标点结构来定义有效的跳跃和跳过行为。

本文提出了一个结构跳跃-LSTM模型，用两个代理而不是一个代理来加快机器阅读速度

提出了一个新的神经速读模型，其中新读者有能力跳过一个词或一连串的词。

本文提出了一种使用跳过和跳跃动作的快速阅读方法，表明所提出的方法与LSTM一样准确，但使用的计算量要少得多。


One of the key challenges of session-based recommender systems is to enhance users’ purchase intentions. In this paper, we formulate the sequential interactions between user sessions and a recommender agent as a Markov Decision Process (MDP). In practice, the purchase reward is delayed and sparse, and may be buried by clicks, making it an impoverished signal for policy learning. Inspired by the prediction error minimization (PEM) and embodied cognition, we propose a simple architecture to augment reward, namely Imagination Reconstruction Network (IRN). Speciﬁcally, IRN enables the agent to explore its environment and learn predictive representations via three key components. The imagination core generates predicted trajectories, i.e., imagined items that users may purchase. The trajectory manager controls the granularity of imagined trajectories using the planning strategies, which balances the long-term rewards and short-term rewards. To optimize the action policy, the imagination-augmented executor minimizes the intrinsic imagination error of simulated trajectories by self-supervised reconstruction, while maximizing the extrinsic reward using model-free algorithms. Empirically, IRN promotes quicker adaptation to user interest, and shows improved robustness to the cold-start scenario and ultimately higher purchase performance compared to several baselines. Somewhat surprisingly, IRN using only the purchase reward achieves excellent next-click prediction performance, demonstrating that the agent can "guess what you like" via internal planning.

我们提出了IRN架构，以增强基于会话的推荐的稀疏和延迟的购买奖励。 本文提出通过使用想象力重构网络的强化学习来提高推荐系统的性能。 本文通过关注用户的购买行为而不是点击量，提出了一种基于会话的推荐方法。


The question why deep learning algorithms generalize so well has attracted increasing  research interest. However, most of the well-established approaches,  such as hypothesis capacity, stability or sparseness, have not provided complete  explanations (Zhang et al., 2016; Kawaguchi et al., 2017). In this work, we focus  on the robustness approach (Xu & Mannor, 2012), i.e., if the error of a hypothesis  will not change much due to perturbations of its training examples, then it  will also generalize well. As most deep learning algorithms are stochastic (e.g.,  Stochastic Gradient Descent, Dropout, and Bayes-by-backprop), we revisit the robustness  arguments of Xu & Mannor, and introduce a new approach – ensemble  robustness – that concerns the robustness of a population of hypotheses. Through  the lens of ensemble robustness, we reveal that a stochastic learning algorithm can  generalize well as long as its sensitiveness to adversarial perturbations is bounded  in average over training examples. Moreover, an algorithm may be sensitive to  some adversarial examples (Goodfellow et al., 2015) but still generalize well. To  support our claims, we provide extensive simulations for different deep learning  algorithms and different network architectures exhibiting a strong correlation between  ensemble robustness and the ability to generalize.

通过集合稳健性从理论和经验上解释随机深度学习算法的泛化问题

本文提出了对Xu&Mannor'12的算法稳健性的改编，并提出了学习边界和一个显示经验性集合稳健性和泛化误差之间相关性的实验。

提出了对深度学习算法泛化能力的研究，使用了稳定性概念的扩展，称为集合稳健性，并给出了随机算法的泛化误差在稳定性参数方面的边界，并提供了试图将理论与实践联系起来的实证研究。

该论文从稳健性的角度研究了深度学习背景下学习算法的泛化能力


Deep autoregressive models have shown state-of-the-art performance in density estimation for natural images on large-scale datasets such as ImageNet.   However, such models require many thousands of gradient-based weight updates and unique image examples for training. Ideally, the models would rapidly learn visual concepts from only a handful of examples, similar to the manner in which humans learns across many vision tasks.   In this paper, we show how 1) neural attention and 2) meta learning techniques can be used in combination with autoregressive models to enable effective few-shot density estimation. Our proposed modifications to PixelCNN result in state-of-the art few-shot density estimation on the Omniglot dataset.   Furthermore, we visualize the learned attention policy and find that it learns intuitive algorithms for simple tasks such as image mirroring on ImageNet and handwriting on Omniglot without supervision. Finally, we extend the model to natural images and demonstrate few-shot image generation on the Stanford Online Products dataset.

本文提出，当训练数据的可用性较低时，通过使用元学习模型来使用密度估计。

本文考虑了单/少样本密度估计的问题，使用已经应用于单/少样本监督学习的金属学习技术。

本文重点介绍了自回归密度估计的少样本学习，并利用神经注意力和元学习技术改进了PixelCNN。


Neural networks exhibit good generalization behavior in the  over-parameterized regime, where the number of network parameters  exceeds the number of observations. Nonetheless,  current generalization bounds for neural networks fail to explain this  phenomenon. In an attempt to bridge this gap, we study the problem of  learning a two-layer over-parameterized neural network, when the data is generated by a linearly separable function. In the case where the network has Leaky  ReLU activations, we provide both optimization and generalization guarantees for over-parameterized networks.  Specifically, we prove convergence rates of SGD to a global  minimum and provide generalization guarantees for this global minimum  that are independent of the network size.   Therefore, our result clearly shows that the use of SGD for optimization both finds a global minimum, and avoids overfitting despite the high capacity of the model. This is the first theoretical demonstration that SGD can avoid overfitting, when learning over-specified neural network classifiers.

我们表明，SGD可以学习具有Leaky ReLU激活的两层超参数神经网络，并且可以证明在线性可分离数据上的泛化。

本文研究了超参数模型能够通过使用具有固定输出层的1个隐藏层网络来学习具有良好泛化的解决方案。

本文表明，在线性可分离数据上，过参数化网络上的SGD仍能靠出一个可证明是泛化的分类器。


A central challenge in reinforcement learning is discovering effective policies for tasks where rewards are sparsely distributed. We postulate that in the absence of useful reward signals, an effective exploration strategy should seek out {\it decision states}. These states lie at critical junctions in the state space from where the agent can transition to new, potentially unexplored regions. We propose to learn about decision states from prior experience. By training a goal-conditioned model with an information bottleneck, we can identify decision states by examining where the model accesses the goal state through the bottleneck. We find that this simple mechanism effectively identifies decision states, even in partially observed settings. In effect, the model learns the sensory cues that correlate with potential subgoals. In new environments, this model can then identify novel subgoals for further exploration, guiding the agent through a sequence of potential decision  states and through new regions of the state space.

用目标策略信息瓶颈训练代理，促进转移并产生强大的探索奖励

提出了在多目标RL环境下，用负条件互信息对标准RL损失进行正则化的策略搜索。

本文提出了决策状态的概念，并提出了KL发散正则化来学习任务的结构，以利用这些信息来鼓励策略访问决策状态。

本文提出了一种用相互信息项来规范目标条件策略的方法。


Many applications in machine learning require optimizing a function whose true gradient is unknown, but where surrogate gradient information (directions that may be correlated with, but not necessarily identical to, the true gradient) is available instead. This arises when an approximate gradient is easier to compute than the full gradient (e.g. in meta-learning or unrolled optimization), or when a true gradient is intractable and is replaced with a surrogate (e.g. in certain reinforcement learning applications or training networks with discrete variables). We propose Guided Evolutionary Strategies, a method for optimally using surrogate gradient directions along with random search. We define a search distribution for evolutionary strategies that is elongated along a subspace spanned by the surrogate gradients. This allows us to estimate a descent direction which can then be passed to a first-order optimizer. We analytically and numerically characterize the tradeoffs that result from tuning how strongly the search distribution is stretched along the guiding subspace, and use this to derive a setting of the hyperparameters that works well across problems. Finally, we apply our method to example problems including truncated unrolled optimization and training neural networks with discrete variables, demonstrating improvement over both standard evolutionary strategies and first-order methods (that directly follow the surrogate gradient). We provide a demo of Guided ES at: redacted URL

我们提出了一种优化方法，适用于只有有偏差的梯度的情况--我们为这种情况定义了一个新的梯度估计器，推导出这个估计器的偏差和方差，并将其应用于实例问题。

作者提出了一种将随机搜索与代用梯度信息相结合的方法，并对方差-偏置的权衡进行了讨论，还对超参数优化进行了讨论。

本文提出了一种通过建立前k个代用梯度的子空间来改进随机搜索的方法。

本文试图通过引入一个具有I+UU^t形式的协方差矩阵的非等熵分布和外部信息（如代用梯度）来加速OpenAI类型的演化，以确定U


Point clouds are an important type of geometric data and have widespread use in computer graphics and vision. However, learning representations for point clouds is particularly challenging due to their nature as being an unordered collection of points irregularly distributed in 3D space. Graph convolution, a generalization of the convolution operation for data defined over graphs, has been recently shown to be very successful at extracting localized features from point clouds in supervised or semi-supervised tasks such as classification or segmentation. This paper studies the unsupervised problem of a generative model exploiting graph convolution. We focus on the generator of a GAN and define methods for graph convolution when the graph is not known in advance as it is the very output of the generator. The proposed architecture learns to generate localized features that approximate graph embeddings of the output geometry. We also study the problem of defining an upsampling layer in the graph-convolutional generator, such that it learns to exploit a self-similarity prior on the data distribution to sample more effectively.

一个使用图卷积操作的GAN，从隐藏的特征中动态地计算出图来 本文提出了一个专门用于生成点云的GAN版本，该工作的核心贡献是上采样操作。 本文提出了用于不规则三维点云的图形卷积GAN，它同时学习领域和特征。


Memorization in over-parameterized neural networks can severely hurt generalization in the presence of mislabeled examples. However, mislabeled examples are to hard avoid in extremely large datasets. We address this problem using the implicit regularization effect of stochastic gradient descent with large learning rates, which we find to be able to separate clean and mislabeled examples with remarkable success using loss statistics. We leverage this to identify and on-the-fly discard mislabeled examples using a threshold on their losses. This leads to On-the-fly Data Denoising (ODD), a simple yet effective algorithm that is robust to mislabeled examples, while introducing almost zero computational overhead. Empirical results demonstrate the effectiveness of ODD on several datasets containing artificial and real-world mislabeled examples.

我们介绍了一种快速且易于实施的算法，该算法对数据集的噪声具有稳健性。 本文旨在通过丢弃训练过程中损失较大的例子来去除潜在的标签噪音。


Binarized Neural Networks (BNNs) have recently attracted significant interest due to their computational efficiency. Concurrently, it has been shown that neural networks may be overly sensitive to ``attacks" -- tiny adversarial changes in the input -- which may be detrimental to their use in safety-critical domains. Designing attack algorithms that effectively fool trained models is a key step towards learning robust neural networks.  The discrete, non-differentiable nature of BNNs, which distinguishes them from their full-precision counterparts, poses a challenge to gradient-based attacks. In this work, we study the problem of attacking a BNN through the lens of combinatorial and integer optimization. We propose a Mixed Integer Linear Programming (MILP) formulation of the problem. While exact and flexible, the MILP quickly becomes intractable as the network and perturbation space grow. To address this issue, we propose IProp, a decomposition-based algorithm that solves a sequence of much smaller MILP problems. Experimentally, we evaluate both proposed methods against the standard gradient-based attack (PGD) on MNIST and Fashion-MNIST, and show that IProp performs favorably compared to PGD, while scaling beyond the limits of the MILP.

由于二值化神经网络的非差异性，基于梯度的攻击并不有效；我们的IPROP算法利用整数优化解决了这个问题。

提出了一种新的目标传播式算法，以产生对二值化神经网络的强对抗性攻击。

本文提出了一种基于二元神经网络的MILP的新攻击算法。

本文提出了一种寻找二元神经网络对抗性攻击的算法，该算法从顶层到输入层逐层迭代寻找所需的表征，比解决完整的混合整数线性规划（MILP）求解器更有效率。


Highly regularized LSTMs achieve impressive results on several benchmark datasets in language modeling. We propose a new regularization method based on decoding the last token in the context using the predicted distribution of the next token. This biases the model towards retaining more contextual information, in turn improving its ability to predict the next token. With negligible overhead in the number of parameters and training time, our Past Decode Regularization (PDR) method achieves a word level perplexity of 55.6 on the Penn Treebank and 63.5 on the WikiText-2 datasets using a single softmax. We also show gains by using PDR in combination with a mixture-of-softmaxes, achieving a word level perplexity of 53.8 and 60.5 on these datasets. In addition, our method achieves 1.169 bits-per-character on the Penn Treebank Character dataset for character level language modeling. These results constitute a new state-of-the-art in their respective settings.

使用预测的下一个token分布对上下文中的最后一个token进行解码，可以作为正则器，改善语言建模。

作者介绍了过去解码的想法，目的是在Penn Treebank上进行正则化以改善困惑性。

建议在训练LSTM LM时使用一个额外的损失项，并表明通过增加这个损失项，他们可以在一些LM的基准上实现SOTA的困惑性。

提出了一种新的正则化技术，可以在Merity等人（2017）的AWD-LSTM中使用的正则化技术的基础上添加，而且开销很小。


The assumption that data samples are independently identically distributed is the backbone of many learning algorithms. Nevertheless, datasets often exhibit rich structures in practice, and we argue that there exist some unknown orders within the data instances. Aiming to find such orders, we introduce a novel Generative Markov Network (GMN) which we use to extract the order of data instances automatically. Specifically, we assume that the instances are sampled from a Markov chain. Our goal is to learn the transitional operator of the chain as well as the generation order by maximizing the generation probability under all possible data permutations. One of our key ideas is to use neural networks as a soft lookup table for approximating the possibly huge, but discrete transition matrix. This strategy allows us to amortize the space complexity with a single model and make the transitional operator generalizable to unseen instances. To ensure the learned Markov chain is ergodic, we propose a greedy batch-wise permutation scheme that allows fast training.   Empirically, we evaluate the learned Markov chain by showing that GMNs are able to discover orders among data instances and also perform comparably well to state-of-the-art methods on the one-shot recognition benchmark task.

提出以生成模型的观点观察数据集中的隐性顺序。

作者处理了数据集中的隐性排序问题和恢复它的挑战，并提议学习一个无距离计量的模型，该模型假定马尔可夫链为数据的生成机制。

本文提出了生成马尔可夫网络--一种基于深度学习的方法，用于对序列进行建模并发现数据集中的顺序。

提出了通过学习马尔可夫链来学习无序数据样本的顺序。


We present a Neural Program Search, an algorithm to generate programs from natural language description and a small number of input / output examples. The algorithm combines methods from Deep Learning and Program Synthesis fields by designing rich domain-specific language (DSL) and defining efficient search algorithm guided by a Seq2Tree model on it. To evaluate the quality of the approach we also present a semi-synthetic dataset of descriptions with test examples and corresponding programs. We show that our algorithm significantly outperforms sequence-to-sequence model with attention baseline.

通过Seq2Tree模型上的Tree-Beam搜索，从自然语言描述和输入/输出实例中进行程序合成

提出了一个seq2Tree模型，将自然语言中的问题陈述翻译成DSL中的相应功能程序，这比seq2seq基线方法有了改进。

本文解决了在给定问题描述和少量输入输出实例时进行程序综合的问题。

本文介绍了一种程序合成的技术，涉及到一个限制性的问题语法，该语法使用注意力编码器-解码器网络进行束状搜索。


Generative adversarial training can be generally understood as minimizing certain moment matching loss defined by a set of discriminator functions, typically  neural networks. The discriminator set should be large enough to be able to uniquely identify the true distribution (discriminative), and also be small enough to go beyond memorizing samples (generalizable). In this paper, we show that a discriminator set is guaranteed to be discriminative whenever its linear span is dense in the set of bounded continuous functions. This is a very mild condition satisfied even by neural networks with a single neuron. Further, we develop generalization bounds between the learned distribution and true distribution under different evaluation metrics. When evaluated with neural distance, our bounds show that generalization is guaranteed as long as the discriminator set is small enough, regardless of the size of the generator or hypothesis set. When evaluated with KL divergence, our bound provides an explanation on the counter-intuitive behaviors of testing likelihood in GAN training. Our analysis sheds lights on understanding the practical performance of GANs.

本文研究了当判别器集是一个像神经网络一样的限制性函数类时，GAN的判别和泛化特性。

通过保证诱导的IPM是度量而非伪度量，平衡GAN中生成器和判别器类的能力

本文对GAN中对手/判别器集的大小的作用进行了数学分析


Normalization layers are a staple in state-of-the-art deep neural network architectures. They are widely believed to stabilize training, enable higher learning rate, accelerate convergence and improve generalization, though the reason for their effectiveness is still an active research topic. In this work, we challenge the commonly-held beliefs by showing that none of the perceived benefits is unique to normalization. Specifically, we propose fixed-update initialization (Fixup), an initialization motivated by solving the exploding and vanishing gradient problem at the beginning of training via properly rescaling a standard initialization. We find training residual networks with Fixup to be as stable as training with normalization -- even for networks with 10,000 layers. Furthermore, with proper regularization, Fixup enables residual networks without normalization to achieve state-of-the-art performance in image classification and machine translation.

训练深层残差网络所需要的只是一个好的初始化；规范化层是不必要的。

提出了一种用于深度残差网络的初始化和规范化的方法。这是基于对此类网络中前向和后向爆炸的观察。该方法的性能与其他具有更明确规范化的网络获得的最佳结果相当。

作者提出了一种新的初始化残差网络的方法，其动机是为了避免梯度爆炸/消失的需要。

提出了一种新的初始化方法，用于训练非常深的RedNets，而不使用批量规范。


Designing a metric manually for unsupervised sequence generation tasks, such as text generation, is essentially difficult. In a such situation, learning a metric of a sequence from data is one possible solution. The previous study, SeqGAN, proposed the framework for unsupervised sequence generation, in which a metric is learned from data, and a generator is optimized with regard to the learned metric with policy gradient, inspired by generative adversarial nets (GANs) and reinforcement learning. In this paper, we make two proposals to learn better metric than SeqGAN's: partial reward function and expert-based reward function training. The partial reward function is a reward function for a partial sequence of a certain length. SeqGAN employs a reward function for completed sequence only. By combining long-scale and short-scale partial reward functions, we expect a learned metric to be able to evaluate a partial correctness as well as a coherence of a sequence, as a whole. In expert-based reward function training, a reward function is trained to discriminate between an expert (or true) sequence and a fake sequence that is produced by editing an expert sequence. Expert-based reward function training is not a kind of GAN frameworks. This makes the optimization of the generator easier. We examine the effect of the partial reward function and expert-based reward function training on synthetic data and real text data, and show improvements over SeqGAN and the model trained with MLE. Specifically, whereas SeqGAN gains 0.42 improvement of NLL over MLE on synthetic data, our best model gains 3.02 improvement, and whereas SeqGAN gains 0.029 improvement of BLEU over MLE, our best model gains 0.250 improvement.

本文旨在为无监督学习（如文本生成）学习一个更好的衡量标准，并显示出对SeqGAN的显著改进。

描述了一种通过学习状态-动作值来生成时间序列的方法，其中状态是到目前为止生成的序列，而动作是对下一个值的选择。

本文考虑了通过学习更好的指标来改进序列生成的问题，特别是曝光偏差问题


One of the most successful techniques in generative models has been decomposing a complicated generation task into a series of simpler generation tasks.   For example, generating an image at a low resolution and then learning to refine that into a high resolution image often improves results substantially.   Here we explore a novel strategy for decomposing generation for complicated objects in which we first generate latent variables which describe a subset of the observed variables, and then map from these latent variables to the observed space.   We show that this allows us to achieve decoupled training of complicated generative models and present both theoretical and experimental results supporting the benefit of such an approach.

将学习生成模型的任务分解为学习数据子集的分解潜因素，然后学习这些潜因素的联合。

分层潜在变量生成模型的局部解缠因素，可以看作是逆向学习推理的分层变体

本文研究了分层潜变量模型在生成图像和图像序列方面的潜力，并提议训练几个ALI模型叠加在一起，形成数据的分层表示。

本文旨在直接学习层次优化计划中训练GAN的层次结构，而不是由人类来设计


Visual grounding of language is an active research field aiming at enriching text-based representations with visual information. In this paper, we propose a new way to leverage visual knowledge for sentence representations. Our approach transfers the structure of a visual representation space to the textual space by using two complementary sources of information: (1) the cluster information: the implicit knowledge that two sentences associated with the same visual content describe the same underlying reality and (2) the perceptual information contained within the structure of the visual space. We use a joint approach to encourage beneficial interactions during training between textual, perceptual, and cluster information. We demonstrate the quality of the learned representations on semantic relatedness, classification, and cross-modal retrieval tasks.

我们提出一个联合模型，将视觉知识纳入句子表征中 本文提出了一种使用视频与字幕配对的方法来改进句子嵌入的方法 这份材料提出了一个基于相关视频数据的句子学习句子表征模型，该模型是有依据的。 提出了一种通过多模态联合框架改善基于文本的句子嵌入的方法。