Many real world learning tasks involve complex or hard-to-specify objectives, and using an easier-to-specify proxy can lead to poor performance or misaligned behavior. One solution is to have humans provide a training signal by demonstrating or judging performance, but this approach fails if the task is too complicated for a human to directly evaluate. We propose Iterated Amplification, an alternative training strategy which progressively builds up a training signal for difficult problems by combining solutions to easier subproblems. Iterated Amplification is closely related to Expert Iteration (Anthony et al., 2017; Silver et al., 2017), except that it uses no external reward function. We present results in algorithmic environments, showing that Iterated Amplification can efficiently learn complex behaviors.

複雑なタスクについて、人間のサポートを受けながら、対象のタスクをブレークダウンして学習する手法の提案。人の役割はタスクをばらすことと実際に解くことで、タスクの分解方法/回答を学習した教示役(Amplifier)を使って教師ありで学習を行う。実験自体はまだ簡単なタスクでしか行っていない。


Imitation Learning (IL) is an appealing approach to learn desirable autonomous behavior. However, directing IL to achieve arbitrary goals is difficult. In contrast, planning-based algorithms use dynamics models and reward functions to achieve goals. Yet, reward functions that evoke desirable behavior are often difficult to specify. In this paper, we propose Imitative Models to combine the benefits of IL and goal-directed planning. Imitative Models are probabilistic predictive models of desirable behavior able to plan interpretable expert-like trajectories to achieve specified goals. We derive families of flexible goal objectives, including constrained goal regions, unconstrained goal sets, and energy-based goals. We show that our method can use these objectives to successfully direct behavior. Our method substantially outperforms six IL approaches and a planning-based approach in a dynamic simulated autonomous driving task, and is efficiently learned from expert demonstrations without online data collection. We also show our approach is robust to poorly specified goals, such as goals on the wrong side of the road.

模倣学習とモデルベースの学習を組み合わせるという、ありそうでなかった手法の提案。「ある目的地が与えられた場合にエキスパートが取る軌跡」(事後確率)を、エキスパートの行動から学習した状態遷移モデル(事前確率)＋状態/軌跡がゴールへとつながる確度(尤度)の2つから学習する。


We present trellis networks, a new architecture for sequence modeling. On the one hand, a trellis network is a temporal convolutional network with special structure, characterized by weight tying across depth and direct injection of the input into deep layers. On the other hand, we show that truncated recurrent networks are equivalent to trellis networks with special sparsity structure in their weight matrices. Thus trellis networks with general weight matrices generalize truncated recurrent networks. We leverage these connections to design high-performing trellis networks that absorb structural and algorithmic elements from both recurrent and convolutional models. Experiments demonstrate that trellis networks outperform the current state of the art methods on a variety of challenging benchmarks, including word-level language modeling and character-level language modeling tasks, and stress tests designed to evaluate long-term memory retention. The code is available at this https URL .

CNNとRNN、双方の性質を兼ね備えたTrellis Networkの提案。WaveNetに近い構成だが、各層のActivationを行う際に都度元の入力を加えると共に、畳み込みの重みを各層で共有している。これにより、展開されたRNNと同等の処理が行われるようになっている。


Abstractive summarization has been studied using neural sequence transduction methods with datasets of large, paired document-summary examples. However, such datasets are rare and the models trained from them do not generalize to other domains. Recently, some progress has been made in learning sequence-to-sequence mappings with only unpaired examples. In our work, we consider the setting where there are only documents (product or business reviews) with no summaries provided, and propose an end-to-end, neural model architecture to perform unsupervised abstractive summarization. Our proposed model consists of an auto-encoder where the mean of the representations of the input reviews decodes to a reasonable summary-review while not relying on any review-specific features. We consider variants of the proposed architecture and perform an ablation study to show the importance of specific components. We show through automated metrics and human evaluation that the generated summaries are highly abstractive, fluent, relevant, and representative of the average sentiment of the input reviews. Finally, we collect a reference evaluation dataset and show that our model outperforms a strong extractive baseline.

教師なしで要約を学習する手法の提案。逆翻訳に近い構成で、元文書をモデルに入力し要約を得て、今度は要約をモデルに入れて元文書を復元する(モデルはEncoder-Decoder)。元文書と要約は潜在表現が近しいはず(=Encode結果のベクトル距離)＋元文書の再構成lossの2つで学習を行う。


In multi-task learning, multiple tasks are solved jointly, sharing inductive bias between them. Multi-task learning is inherently a multi-objective problem because different tasks may conflict, necessitating a trade-off. A common compromise is to optimize a proxy objective that minimizes a weighted linear combination of per-task losses. However, this workaround is only valid when the tasks do not compete, which is rarely the case. In this paper, we explicitly cast multi-task learning as multi-objective optimization, with the overall objective of finding a Pareto optimal solution. To this end, we use algorithms developed in the gradient-based multi-objective optimization literature. These algorithms are not directly applicable to large-scale learning problems since they scale poorly with the dimensionality of the gradients and the number of tasks. We therefore propose an upper bound for the multi-objective loss and show that it can be optimized efficiently. We further prove that optimizing this upper bound yields a Pareto optimal solution under realistic assumptions. We apply our method to a variety of multi-task deep learning problems including digit classification, scene understanding (joint semantic segmentation, instance segmentation, and depth estimation), and multi-label classification. Our method produces higher-performing models than recent multi-task learning formulations or per-task training.

マルチタスク学習を、複数目的関数のパレート最適を求める形で解く手法。タスクAを偏重したらタスクBにダメージが行くという、いわば予算制約がある形の制約を課してこれをFrank–Wolfe法で解く。タスク共通部分の勾配については、本来タスクごとに求める必要があるが上界で近似し計算を効率化している


CNNを転移学習する際、最初と最後の層「以外」をfreezeするという通常とは変わった手法の提案。最終の分類層をランダムな重みで固定してもなかなかの精度が出るという。Kaggleで開催されたベンガル語手書き数字を認識するコンペティションで6位をとっている。なお上位はより大きいモデル＋アンサンブル


We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers. Unlike recent language representation models, BERT is designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers. As a result, the pre-trained BERT model can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks, such as question answering and language inference, without substantial task-specific architecture modifications. BERT is conceptually simple and empirically powerful. It obtains new state-of-the-art results on eleven natural language processing tasks, including pushing the GLUE score to 80.5% (7.7% point absolute improvement), MultiNLI accuracy to 86.7% (4.6% absolute improvement), SQuAD v1.1 question answering Test F1 to 93.2 (1.5 point absolute improvement) and SQuAD v2.0 Test F1 to 83.1 (5.1 point absolute improvement).

Bi-directionalのTransformerを事前学習し、QAや文関係推論などのタスクに転移した研究。ELMo(#655)の双方向性と、OpenAIのTransformer転移(#790)をミックスした形になっている。言語モデル学習は双方向でないのでcropした単語を予測する形で学習、文関係学習のため次の文or notを学習させる等の工夫を行っている。


We frame Question Answering (QA) as a Reinforcement Learning task, an approach that we call Active Question Answering. We propose an agent that sits between the user and a black box QA system and learns to reformulate questions to elicit the best possible answers. The agent probes the system with, potentially many, natural language reformulations of an initial question and aggregates the returned evidence to yield the best answer. The reformulation system is trained end-to-end to maximize answer quality using policy gradient. We evaluate on SearchQA, a dataset of complex questions extracted from Jeopardy!. The agent outperforms a state-of-the-art base model, playing the role of the environment, and other benchmarks. We also analyze the language that the agent has learned while interacting with the question answering system. We find that successful question reformulations look quite different from natural language paraphrases. The agent is able to discover non-trivial reformulation strategies that resemble classic information retrieval techniques such as term re-weighting (tf-idf) and stemming.

QAシステムを構築する際に、目的のAを得るためQを変える手法(いわば、質問の仕方を変える)。目的のAが得られたら報酬、という強化学習の枠組みで学習している。言いかえはSeq2Seqがベースだが強化学習のみでは厳しいため、事前に翻訳のコーパス、さらに単言語の言い換えコーパスで学習している


Meta-learning, or learning to learn, is the science of systematically observing how different machine learning approaches perform on a wide range of learning tasks, and then learning from this experience, or meta-data, to learn new tasks much faster than otherwise possible. Not only does this dramatically speed up and improve the design of machine learning pipelines or neural architectures, it also allows us to replace hand-engineered algorithms with novel approaches learned in a data-driven way. In this chapter, we provide an overview of the state of the art in this fascinating and continuously evolving field.

メタラーニングについてのサーベイ資料。学習する「メタ」の種類として、ハイパーパラメーターなどの設定、タスクの性質(類似性)、モデルの学習結果(重み)に内在する共通特徴の3点を挙げ、このカテゴリで既存研究をまとめている。


In many reinforcement learning tasks, the goal is to learn a policy to manipulate an agent, whose design is fixed, to maximize some notion of cumulative reward. The design of the agent's physical structure is rarely optimized for the task at hand. In this work, we explore the possibility of learning a version of the agent's design that is better suited for its task, jointly with the policy. We propose an alteration to the popular OpenAI Gym framework, where we parameterize parts of an environment, and allow an agent to jointly learn to modify these environment parameters along with its policy. We demonstrate that an agent can learn a better structure of its body that is not only better suited for the task, but also facilitates policy learning. Joint learning of policy and structure may even uncover design principles that are useful for assisted-design applications. Videos of results at this https URL

強化学習において、頭脳である戦略だけでなく体(エージェントの形態: 足の角度や長さなど)も学習させるという試み。体の調整具合に応じて、報酬も変動させる。戦略と体の調整は重みを共有し、学習は素のPolicy Gradientがベースでパラメーターをサンプリングする方式を組み合わせている。


We consider negotiation settings in which two agents use natural language to bargain on goods. Agents need to decide on both high-level strategy (e.g., proposing $50) and the execution of that strategy (e.g., generating "The bike is brand new. Selling for just $50."). Recent work on negotiation trains neural models, but their end-to-end nature makes it hard to control their strategy, and reinforcement learning tends to lead to degenerate solutions. In this paper, we propose a modular approach based on coarse di- alogue acts (e.g., propose(price=50)) that decouples strategy and generation. We show that we can flexibly set the strategy using supervised learning, reinforcement learning, or domain-specific knowledge without degeneracy, while our retrieval-based generation can maintain context-awareness and produce diverse utterances. We test our approach on the recently proposed DEALORNODEAL game, and we also collect a richer dataset based on real items on Craigslist. Human evaluation shows that our systems achieve higher task success rate and more human-like negotiation behavior than previous approaches.

価格交渉を行う対話データセットと、それを解くための基礎的なモデルの紹介。1402のアイテムについて(アイテムには画像がついているものもある)、価格(値下げ)交渉を行っている6682対話が収録されている。モデルとしては、seq2seqと対話行為タイプを推定するモジュール構成タイプの2つが提供されている


We introduce Generative Neural Machine Translation (GNMT), a latent variable architecture which is designed to model the semantics of the source and target sentences. We modify an encoder-decoder translation model by adding a latent variable as a language agnostic representation which is encouraged to learn the meaning of the sentence. GNMT achieves competitive BLEU scores on pure translation tasks, and is superior when there are missing words in the source sentence. We augment the model to facilitate multilingual translation and semi-supervised learning without adding parameters. This framework significantly reduces overfitting when there is limited paired data available, and is effective for translating between pairs of languages not seen during training.

生成モデルで翻訳を行う研究。ソースとターゲットで潜在表現を共有することで、同じ意味の文は同じ意味の潜在表現になるようにしている。潜在表現の生成はガウス分布を使っており、潜在表現からソースの生成はLSTM、ターゲットの生成はソースのEncode結果+潜在表現を利用する。


We present a simple and accurate span-based model for semantic role labeling (SRL). Our model directly takes into account all possible argument spans and scores them for each label. At decoding time, we greedily select higher scoring labeled spans. One advantage of our model is to allow us to design and use span-level features, that are difficult to use in token-based BIO tagging approaches. Experimental results demonstrate that our ensemble model achieves the state-of-the-art results, 87.4 F1 and 87.0 F1 on the CoNLL-2005 and 2012 datasets, respectively.

系列ラベリングをスパンベースで行う研究。これにより単語(token)単位でなく「ここからここまで」という形でラベルを振ることができる。Bidirectionalが基本で、A=>Bのスパンの場合スパン特徴はをA+BとA-Bをコンカチしたもので、各候補スパンについてスコアを算出した後greedy searchで最適化する。


Recurrent neural networks (RNNs), such as long short-term memory networks (LSTMs), serve as a fundamental building block for many sequence learning tasks, including machine translation, language modeling, and question answering. In this paper, we consider the specific problem of word-level language modeling and investigate strategies for regularizing and optimizing LSTM-based models. We propose the weight-dropped LSTM which uses DropConnect on hidden-to-hidden weights as a form of recurrent regularization. Further, we introduce NT-ASGD, a variant of the averaged stochastic gradient method, wherein the averaging trigger is determined using a non-monotonic condition as opposed to being tuned by the user. Using these and other regularization strategies, we achieve state-of-the-art word level perplexities on two data sets: 57.3 on Penn Treebank and 65.8 on WikiText-2. In exploring the effectiveness of a neural cache in conjunction with our proposed model, we achieve an even lower state-of-the-art perplexity of 52.8 on Penn Treebank and 52.0 on WikiText-2.

言語モデルについての正則化手法をまとめた論文。DropConnectの適用、Averaged SGDの使用、embeddingへのDropout適用、sequence lengthの動的変更といった工夫が紹介されており、各手法のインパクトについても分析されている。


Generative adversarial networks (GANs) are one of the most popular methods for generating images today. While impressive results have been validated by visual inspection, a number of quantitative criteria have emerged only recently. We argue here that the existing ones are insufficient and need to be in adequation with the task at hand. In this paper we introduce two measures based on image classification---GAN-train and GAN-test, which approximate the recall (diversity) and precision (quality of the image) of GANs respectively. We evaluate a number of recent GAN approaches based on these two measures and demonstrate a clear difference in performance. Furthermore, we observe that the increasing difficulty of the dataset, from CIFAR10 over CIFAR100 to ImageNet, shows an inverse correlation with the quality of the GANs, as clearly evident from our measures.

GANを評価する指標として、GANで生成された画像を使って学習しオリジナルの画像で評価するGAN-train、オリジナル画像で学習しGAN画像で評価するGAN-testの2つを提案。前者で画像品質/クラスラベルの多様性、後者は高すぎる場合元画像を記憶しているだけと推察できる。


Adversarial learning methods have been proposed for a wide range of applications, but the training of adversarial models can be notoriously unstable. Effectively balancing the performance of the generator and discriminator is critical, since a discriminator that achieves very high accuracy will produce relatively uninformative gradients. In this work, we propose a simple and general technique to constrain information flow in the discriminator by means of an information bottleneck. By enforcing a constraint on the mutual information between the observations and the discriminator's internal representation, we can effectively modulate the discriminator's accuracy and maintain useful and informative gradients. We demonstrate that our proposed variational discriminator bottleneck (VDB) leads to significant improvements across three distinct application areas for adversarial learning algorithms. Our primary evaluation studies the applicability of the VDB to imitation learning of dynamic continuous control skills, such as running. We show that our method can learn such skills directly from \emph{raw} video demonstrations, substantially outperforming prior adversarial imitation learning methods. The VDB can also be combined with adversarial inverse reinforcement learning to learn parsimonious reward functions that can be transferred and re-optimized in new settings. Finally, we demonstrate that VDB can train GANs more effectively for image generation, improving upon a number of prior stabilization methods.

GANの枠組みで模倣学習/逆強化学習を行うGAILの安定性を向上させるための研究。VAEのように入力を基に分布のパラメーターを推定し潜在表現をサンプリングするが、この際入力と潜在表現の相関が一定値以下になるよう制約をかける(入力=潜在表現にならないようにする)。


Building on the recent successes of distributed training of RL agents, in this paper we investigate the training of RNN-based RL agents from distributed prioritized experience replay. We study the effects of parameter lag resulting in representational drift and recurrent state staleness and empirically derive an improved training strategy. Using a single network architecture and fixed set of hyper-parameters, the resulting agent, Recurrent Replay Distributed DQN, quadruples the previous state of the art on Atari-57, and matches the state of the art on DMLab-30. It is the first agent to exceed human-level performance in 52 of the 57 Atari games.

Rainbowの亜種であるAPE-X(#439 )に、RNNを組み込むことでパフォーマンスを劇的に改善した研究。任意の状態から学習を行う場合RNNの初期値をどうするかという問題があるが、RNNの状態もとっておく、RNNの状態を作るための初期ステップ(burn-in)を取っておくという対策を行っている


The field of few-shot learning has recently seen substantial advancements. Most of these advancements came from casting few-shot learning as a meta-learning problem.Model Agnostic Meta Learning or MAML is currently one of the best approaches for few-shot learning via meta-learning. MAML is simple, elegant and very powerful, however, it has a variety of issues, such as being very sensitive to neural network architectures, often leading to instability during training, requiring arduous hyperparameter searches to stabilize training and achieve high generalization and being very computationally expensive at both training and inference times. In this paper, we propose various modifications to MAML that not only stabilize the system, but also substantially improve the generalization performance, convergence speed and computational overhead of MAML, which we call MAML++.

各タスクに共通する良い初期値をみつけるMAMLの改善提案。勾配からさらに学習する形になるため二階微分が必要で計算コストが高い点と、安定性が低い点が課題だった。計算については序盤では一階の微分による近似で済ます、安定性では複数ステップの重み付平均を取るなどの対策をとっている。


The Impression section of a radiology report summarizes crucial radiology findings in natural language and plays a central role in communicating these findings to physicians. However, the process of generating impressions by summarizing findings is time-consuming for radiologists and prone to errors. We propose to automate the generation of radiology impressions with neural sequence-to-sequence learning. We further propose a customized neural model for this task which learns to encode the study background information and use this information to guide the decoding process. On a large dataset of radiology reports collected from actual hospital studies, our model outperforms existing non-neural and neural baselines under the ROUGE metrics. In a blind experiment, a board-certified radiologist indicated that 67% of sampled system summaries are at least as good as the corresponding human-written summaries, suggesting significant clinical validity. To our knowledge our work represents the first attempt in this direction.

放射線診断について、患者の症状と検査結果から所見を生成する研究。具体的には、Background(xxが痛い、など)とFindings(胸にyyがあり、など)から、所見(zzの症状が疑われる、など)を生成する。基本はFindingからの要約(Pointer-Generator)だが、Backgroundを別途EncodeしDecode時参照している。


In recent years, there have been amazing advances in deep learning methods for machine reading. In machine reading, the machine reader has to extract the answer from the given ground truth paragraph. Recently, the state-of-the-art machine reading models achieve human level performance in SQuAD which is a reading comprehension-style question answering (QA) task. The success of machine reading has inspired researchers to combine information retrieval with machine reading to tackle open-domain QA. However, these systems perform poorly compared to reading comprehension-style QA because it is difficult to retrieve the pieces of paragraphs that contain the answer to the question. In this study, we propose two neural network rankers that assign scores to different passages based on their likelihood of containing the answer to a given question. Additionally, we analyze the relative importance of semantic similarity and word level relevance matching in open-domain QA.

一般的なQAでは、回答を含む文書が既にあることは前提とできない。そこで質問に関連する文書の検索、回答が含まれる可能性に応じて再ランキング、最後に回答抽出、という3段階のプロセスを提案している。2番目のステップ(文ベクトルによる再ランキング)で精度の向上を確認。


We formalize a new modular variant of current question answering tasks by enforcing complete independence of the document encoder from the question encoder. This formulation addresses a key challenge in machine comprehension by requiring a standalone representation of the document discourse. It additionally leads to a significant scalability advantage since the encoding of the answer candidate phrases in the document can be pre-computed and indexed offline for efficient retrieval. We experiment with baseline models for the new task, which achieve a reasonable accuracy but significantly underperform unconstrained QA models. We invite the QA research community to engage in Phrase-Indexed Question Answering (PIQA, pika) for closing the gap. The leaderboard is at: this http URL

既存のQAモデルは「質問」「回答が含まれる文書」間にAttentionを張って回答を見つけるのが一般的だが、通常候補となる文書は膨大で、質問ごとに文書ベクトルを再計算するのは現実的でない。そこで、質問と文書は個別にベクトル化されることを前提としたタスクを提案している


Unsupervised representation learning algorithms such as word2vec and ELMo improve the accuracy of many supervised NLP models, mainly because they can take advantage of large amounts of unlabeled text. However, the supervised models only learn from task-specific labeled data during the main training phase. We therefore propose Cross-View Training (CVT), a semi-supervised learning algorithm that improves the representations of a Bi-LSTM sentence encoder using a mix of labeled and unlabeled data. On labeled examples, standard supervised learning is used. On unlabeled examples, CVT teaches auxiliary prediction modules that see restricted views of the input (e.g., only part of a sentence) to match the predictions of the full model seeing the whole input. Since the auxiliary modules and the full model share intermediate representations, this in turn improves the full model. Moreover, we show that CVT is particularly effective when combined with multi-task learning. We evaluate CVT on five sequence tagging tasks, machine translation, and dependency parsing, achieving state-of-the-art results.

半教師あり学習で、教師なしのデータからも有効な学習を行う方法の提案。ラベルありについては普通に学習するが、ラベルなしの場合はデータの一部を欠損させた状態で欠損のない場合と同じ確率分布が予測できるかを学習する。これにより教師なしラベルからもEncode精度を上げるための学習を行う。


While it has not yet been proven, empirical evidence suggests that model generalization is related to local properties of the optima which can be described via the Hessian. We connect model generalization with the local property of a solution under the PAC-Bayes paradigm. In particular, we prove that model generalization ability is related to the Hessian, the higher-order "smoothness" terms characterized by the Lipschitz constant of the Hessian, and the scales of the parameters. Guided by the proof, we propose a metric to score the generalization capability of the model, as well as an algorithm that optimizes the perturbed model accordingly.

DNNの汎化性能を向上させるための、適切なノイズについての研究。解はflatな方が汎化性能が高いことが知られているが(lossの高いsharpな箇所よりflatなlocal minimumの方がシンプルな決定境界を持つことも論文中で示されている)、flatに誘導するためのノイズとしてPAC-Bayesで上限を設定している。


Batch normalization (BN) is a technique to normalize activations in intermediate layers of deep neural networks. Its tendency to improve accuracy and speed up training have established BN as a favorite technique in deep learning. Yet, despite its enormous success, there remains little consensus on the exact reason and mechanism behind these improvements. In this paper we take a step towards a better understanding of BN, following an empirical approach. We conduct several experiments, and show that BN primarily enables training with larger learning rates, which is the cause for faster convergence and better generalization. For networks without BN we demonstrate how large gradient updates can result in diverging loss and activations growing uncontrollably with network depth, which limits possible learning rates. BN avoids this problem by constantly correcting activations to be zero-mean and of unit standard deviation, which enables larger gradient steps, yields faster convergence and may help bypass sharp local minima. We further show various ways in which gradients and activations of deep unnormalized networks are ill-behaved. We contrast our results against recent findings in random matrix theory, shedding new light on classical initialization schemes and their consequences.

Batch Normalizationの効果について検証した研究。発表当初提案されていた共変量シフトの解消はあまり効果がなく、それ以外の効果で精度の改善が行えているとしている。具体的には、BNなしでは層が深くなるにつれてチャンネルの分散が大きくなり、勾配が入力に非依存になってくる。BNはこの問題を解決し、結果として大きな学習率の設定が可能になり、正則化/精度が向上する。


Methods for unsupervised hypernym detection may broadly be categorized according to two paradigms: pattern-based and distributional methods. In this paper, we study the performance of both approaches on several hypernymy tasks and find that simple pattern-based methods consistently outperform distributional methods on common benchmark datasets. Our results show that pattern-based models provide important contextual constraints which are not yet captured in distributional methods.

文章から上位概念を抽出する(「笛」は「楽器」など)方法について、パターンマッチと統計的手法(分布ベースの手法)の比較を行った研究。結果としては、パターンの方が優秀で、パターンで検知されたものと似ている(=共起する)ものも評価することでさらに改善できるとのこと。


Text classification is an important and classical problem in natural language processing. There have been a number of studies that applied convolutional neural networks (convolution on regular grid, e.g., sequence) to classification. However, only a limited number of studies have explored the more flexible graph convolutional neural networks (convolution on non-grid, e.g., arbitrary graph) for the task. In this work, we propose to use graph convolutional networks for text classification. We build a single text graph for a corpus based on word co-occurrence and document word relations, then learn a Text Graph Convolutional Network (Text GCN) for the corpus. Our Text GCN is initialized with one-hot representation for word and document, it then jointly learns the embeddings for both words and documents, as supervised by the known class labels for documents. Our experimental results on multiple benchmark datasets demonstrate that a vanilla Text GCN without any external word embeddings or knowledge outperforms state-of-the-art methods for text classification. On the other hand, Text GCN also learns predictive word and document embeddings. In addition, experimental results show that the improvement of Text GCN over state-of-the-art comparison methods become more prominent as we lower the percentage of training data, suggesting the robustness of Text GCN to less training data in text classification.

文書分類にGraph Convolutionを適用した研究。ノードとして文書・単語双方をとっており、文書=>単語の重みはTF-IDF、単語=>単語の重みはコーパス全体の共起(PMI)をとっている。単語分散表現なしにCNNやLSTMのモデルを上回る効果を出せている。ただ、テストセットのノードが既知であるTransductive設定


Motivations like domain adaptation, transfer learning, and feature learning have fueled interest in inducing embeddings for rare or unseen words, n-grams, synsets, and other textual features. This paper introduces a la carte embedding, a simple and general alternative to the usual word2vec-based approaches for building such representations that is based upon recent theoretical results for GloVe-like embeddings. Our method relies mainly on a linear transformation that is efficiently learnable using pretrained word vectors and linear regression. This transform is applicable on the fly in the future when a new text feature or rare word is encountered, even if only a single usage example is available. We introduce a new dataset showing how the a la carte method requires fewer examples of words in context to learn high-quality embeddings and we obtain state-of-the-art results on a nonce task and some unsupervised document classification tasks.

未知語や低頻度語の分散表現を得る手法の提案。周辺単語(コンテキスト)の平均でそこそこ良い精度が出ることが知られているが、単純平均から単語ベクトルを復元するような重みを学習することで、より精度の高い推定が可能なことを示した。Bi-LSTMには劣るものの単純な方法ながら各タスクで良い精度


Consider learning a policy from example expert behavior, without interaction with the expert or access to reinforcement signal. One approach is to recover the expert's cost function with inverse reinforcement learning, then extract a policy from that cost function with reinforcement learning. This approach is indirect and can be slow. We propose a new general framework for directly extracting a policy from data, as if it were obtained by reinforcement learning following inverse reinforcement learning. We show that a certain instantiation of our framework draws an analogy between imitation learning and generative adversarial networks, from which we derive a model-free imitation learning algorithm that obtains significant performance gains over existing model-free methods in imitating complex behaviors in large, high-dimensional environments.

模倣学習をGANの枠組みで行う試み。Generatorは戦略を学習し、Discriminatorは生成された行動軌跡がエキスパートのものかGのものかを識別するよう訓練する。戦略はTRPOの枠組みで学習を行う。


Deep neural networks excel at function approximation, yet they are typically trained from scratch for each new function. On the other hand, Bayesian methods, such as Gaussian Processes (GPs), exploit prior knowledge to quickly infer the shape of a new function at test time. Yet GPs are computationally expensive, and it can be hard to design appropriate priors. In this paper we propose a family of neural models, Conditional Neural Processes (CNPs), that combine the benefits of both. CNPs are inspired by the flexibility of stochastic processes such as GPs, but are structured as neural networks and trained via gradient descent. CNPs make accurate predictions after observing only a handful of training data points, yet scale to complex functions and large datasets. We demonstrate the performance and versatility of the approach on a range of canonical machine learning tasks, including regression, classification and image completion.

観測データをモデル化する確率過程とニューラルネットを組み合わせた手法。確率過程の代表格であるガウス過程では観測点から分散共分散を計算しモデル化を行うが、分散共分散の更新を行うには既存のデータ点を確保しておく必要がある。そこでEncoder/Decoderの枠組みで、データ点=>ベクトル表現=>ベクトル表現を集約=>Decoderで復元、という形でNN化している。


Model-based reinforcement learning approaches carry the promise of being data efficient. However, due to challenges in learning dynamics models that sufficiently match the real-world dynamics, they struggle to achieve the same asymptotic performance as model-free methods. We propose Model-Based Meta-Policy-Optimization (MB-MPO), an approach that foregoes the strong reliance on accurate learned dynamics models. Using an ensemble of learned dynamic models, MB-MPO meta-learns a policy that can quickly adapt to any model in the ensemble with one policy gradient step. This steers the meta-policy towards internalizing consistent dynamics predictions among the ensemble while shifting the burden of behaving optimally w.r.t. the model discrepancies towards the adaptation step. Our experiments show that MB-MPO is more robust to model imperfections than previous model-based approaches. Finally, we demonstrate that our approach is able to match the asymptotic performance of model-free methods while requiring significantly less experience.

モデルベースを使用しメタラーニングを行う研究。メタラーニングでは複数のタスクにおける学習結果を統合することで転移しやすいモデルを作成するが、この研究では複数のタスクの代わりに複数のモデルベース環境/そこでの戦略を用いてメタラーニングを行っている。


In sentence compression, the task of shortening sentences while retaining the original meaning, models tend to be trained on large corpora containing pairs of verbose and compressed sentences. To remove the need for paired corpora, we emulate a summarization task and add noise to extend sentences and train a denoising auto-encoder to recover the original, constructing an end-to-end training regime without the need for any examples of compressed sentences. We conduct a human evaluation of our model on a standard text summarization dataset and show that it performs comparably to a supervised baseline based on grammatical correctness and retention of meaning. Despite being exposed to no target data, our unsupervised models learn to generate imperfect but reasonably readable sentence summaries. Although we underperform supervised models based on ROUGE scores, our models are competitive with a supervised baseline based on human evaluation for grammatical correctness and retention of meaning.

文にノイズを加えて復元する、というAuto-Encoderの枠組みで文圧縮を行う試み。文へのノイズ挿入は、他の文からサンプリングした単語を対象文に加えてシャッフルすることで行う(シャッフルは単語/n-gram単位で行う)。この他、辞書型未知語対応などいくつかの工夫を行っている


To understand a sentence like "whereas only 10% of White Americans live at or below the poverty line, 28% of African Americans do" it is important not only to identify individual facts, e.g., poverty rates of distinct demographic groups, but also the higher-order relations between them, e.g., the disparity between them. In this paper, we propose the task of Textual Analogy Parsing (TAP) to model this higher-order meaning. The output of TAP is a frame-style meaning representation which explicitly specifies what is shared (e.g., poverty rates) and what is compared (e.g., White Americans vs. African Americans, 10% vs. 28%) between its component facts. Such a meaning representation can enable new applications that rely on discourse understanding such as automated chart generation from quantitative text. We present a new dataset for TAP, baselines, and a model that successfully uses an ILP to enforce the structural constraints of the problem.

数値に関するニュース文からグラフを生成することを目指した研究。文中のフレーズをフレームにあてはめて情報を抽出するのが基本だが、比較などの情報が落ちないよう、入れ子になったフレーム(TAP frame)にあてはめることを提案。データセットも併せて提供している。


In the quest for efficient and robust reinforcement learning methods, both model-free and model-based approaches offer advantages. In this paper we propose a new way of explicitly bridging both approaches via a shared low-dimensional learned encoding of the environment, meant to capture summarizing abstractions. We show that the modularity brought by this approach leads to good generalization while being computationally efficient, with planning happening in a smaller latent state space. In addition, this approach recovers a sufficient low-dimensional representation of the environment, which opens up new strategies for interpretable AI, exploration and transfer learning.

強化学習で、モデルフリーの学習とモデルベースの学習を同時に行う手法の提案。この時環境の表現を共有することで双方に有用な潜在表現を獲得できるほか、モデルベースでモデルフリーの学習効率を上げることができる。


The reinforcement learning community has made great strides in designing algorithms capable of exceeding human performance on specific tasks. These algorithms are mostly trained one task at the time, each new task requiring to train a brand new agent instance. This means the learning algorithm is general, but each solution is not; each agent can only solve the one task it was trained on. In this work, we study the problem of learning to master not one but multiple sequential-decision tasks at once. A general issue in multi-task learning is that a balance must be found between the needs of multiple tasks competing for the limited resources of a single learning system. Many learning algorithms can get distracted by certain tasks in the set of tasks to solve. Such tasks appear more salient to the learning process, for instance because of the density or magnitude of the in-task rewards. This causes the algorithm to focus on those salient tasks at the expense of generality. We propose to automatically adapt the contribution of each task to the agent's updates, so that all tasks have a similar impact on the learning dynamics. This resulted in state of the art performance on learning to play all games in a set of 57 diverse Atari games. Excitingly, our method learned a single trained policy - with a single set of weights - that exceeds median human performance. To our knowledge, this was the first time a single agent surpassed human-level performance on this multi-task domain. The same approach also demonstrated state of the art performance on a set of 30 tasks in the 3D reinforcement learning platform DeepMind Lab.

報酬のクリッピングを見直して適応的な正規化(PopArt)を導入したという話。例えばパックマンでは幽霊を食べる、ペレットを取得する、という様々な行動があるがクリッピングするとすべて「+1」になってしまう。このため報酬(実際は価値)を正規化すること対応した。複数ゲームをまとめても効果を確認。


Dependency trees help relation extraction models capture long-range relations between words. However, existing dependency-based models either neglect crucial information (e.g., negation) by pruning the dependency trees too aggressively, or are computationally inefficient because it is difficult to parallelize over different tree structures. We propose an extension of graph convolutional networks that is tailored for relation extraction, which pools information over arbitrary dependency structures efficiently in parallel. To incorporate relevant information while maximally removing irrelevant content, we further apply a novel pruning strategy to the input trees by keeping words immediately around the shortest path between the two entities among which a relation might hold. The resulting model achieves state-of-the-art performance on the large-scale TACRED dataset, outperforming existing sequence and dependency-based neural models. We also show through detailed analysis that this model has complementary strengths to sequence models, and combining them further improves the state of the art.

Graph Convolutionを使い、文中から主語と目的語の関係を正しく推論するという研究。単純にGCNを使うだけでなく、activation前の正規化やGCNに投入する前にBi-directionalで文脈情報を得ておく、不要パスの枝刈りといった多くの工夫を行っている。


Deep learning is progressively gaining popularity as a viable alternative to i-vectors for speaker recognition. Promising results have been recently obtained with Convolutional Neural Networks (CNNs) when fed by raw speech samples directly. Rather than employing standard hand-crafted features, the latter CNNs learn low-level speech representations from waveforms, potentially allowing the network to better capture important narrow-band speaker characteristics such as pitch and formants. Proper design of the neural network is crucial to achieve this goal. This paper proposes a novel CNN architecture, called SincNet, that encourages the first convolutional layer to discover more meaningful filters. SincNet is based on parametrized sinc functions, which implement band-pass filters. In contrast to standard CNNs, that learn all elements of each filter, only low and high cutoff frequencies are directly learned from data with the proposed method. This offers a very compact and efficient way to derive a customized filter bank specifically tuned for the desired application. Our experiments, conducted on both speaker identification and speaker verification tasks, show that the proposed architecture converges faster and performs better than a standard CNN on raw waveforms.

音声を処理するCNNで、生の音声を処理する1層目を意図的にバンドパスフィルタを模すことで(フィルタする周波数領域は学習させるようにする)話者特定の精度と速度を上げた研究。具体的にはHamming windowをかけてFFT=>フィルタ適用=>逆FFTで元に戻す(窓関数は他のでもあまり大差ないらしい)。


Model-free deep reinforcement learning algorithms have been shown to be capable of learning a wide range of robotic skills, but typically require a very large number of samples to achieve good performance. Model-based algorithms, in principle, can provide for much more efficient learning, but have proven difficult to extend to expressive, high-capacity models such as deep neural networks. In this work, we demonstrate that medium-sized neural network models can in fact be combined with model predictive control (MPC) to achieve excellent sample complexity in a model-based reinforcement learning algorithm, producing stable and plausible gaits to accomplish various complex locomotion tasks. We also propose using deep neural network dynamics models to initialize a model-free learner, in order to combine the sample efficiency of model-based approaches with the high task-specific performance of model-free methods. We empirically demonstrate on MuJoCo locomotion tasks that our pure model-based approach trained on just random action data can follow arbitrary trajectories with excellent sample efficiency, and that our hybrid algorithm can accelerate model-free learning on high-speed benchmark tasks, achieving sample efficiency gains of 3-5x on swimmer, cheetah, hopper, and ant agents. Videos can be found at this https URL

モデルフリーの学習とモデルベースの学習を併用する、Dynaという手法に連なる手法。モデルとしてニューラルネットを使用し、単純に次の遷移だけでなく次の次の・・・というMulti-stepの予測をさせるようにすることで、既存の(ニューラルネットを利用した)モデルベースの研究を上回る精度を出している。またモデルベースで学習したモデルをエキスパートとすることで、モデルフリーのモデルを模倣学習するという興味深い手法も提示。


This paper presents KeypointNet, an end-to-end geometric reasoning framework to learn an optimal set of category-specific 3D keypoints, along with their detectors. Given a single image, KeypointNet extracts 3D keypoints that are optimized for a downstream task. We demonstrate this framework on 3D pose estimation by proposing a differentiable objective that seeks the optimal set of keypoints for recovering the relative pose between two views of an object. Our model discovers geometrically and semantically consistent keypoints across viewing angles and instances of an object category. Importantly, we find that our end-to-end framework using no ground-truth keypoint annotations outperforms a fully supervised baseline using the same neural network architecture on the task of pose estimation. The discovered 3D keypoints on the car, chair, and plane categories of ShapeNet are visualized at this http URL.

3D空間上のキーポイントを、ラベルからではなく視点の異なる2つの2D画像から推定する手法の提案。推定したキーポイントが他方の2D画像上でも同じ点となるか(Multi-view consistency)、それぞれ推定したキーポイントの位置関係から視点差(回転)を復元できるか、の2点を目的関数に組み込み学習する。


The web provides a rich, open-domain environment with textual, structural, and spatial properties. We propose a new task for grounding language in this environment: given a natural language command (e.g., "click on the second article"), choose the correct element on the web page (e.g., a hyperlink or text box). We collected a dataset of over 50,000 commands that capture various phenomena such as functional references (e.g. "find who made this site"), relational reasoning (e.g. "article by john"), and visual reasoning (e.g. "top-most article"). We also implemented and analyzed three baseline models that capture different phenomena present in the dataset.

自然言語でWebサイトを操作することを目指した研究。「最初の記事をクリック」や「表示言語を切り替えて」といった自然言語の指示を実現できるかというタスクを提案している(具体的には、ページのDOM/自然言語指示を基に正答エレメントを選択する形になる)。データとベースラインを提供している


Object detection, one of the most fundamental and challenging problems in computer vision, seeks to locate object instances from a large number of predefined categories in natural images. Deep learning techniques have emerged as a powerful strategy for learning feature representations directly from data and have led to remarkable breakthroughs in the field of generic object detection. Given this period of rapid evolution, the goal of this paper is to provide a comprehensive survey of the recent achievements in this field brought about by deep learning techniques. More than 300 research contributions are included in this survey, covering many aspects of generic object detection: detection frameworks, object feature representation, object proposal generation, context modeling, training strategies, and evaluation metrics. We finish the survey by identifying promising directions for future research.

物体検知の手法のまとめで、手法だけでなく物体検知というタスク自体についてもその歴史をたどり系統図にまとめている。図解も豊富で分かりやすく、まさに"Survey"の名を冠するにふさわしい出来。


A hallmark of variational autoencoders (VAEs) for text processing is their combination of powerful encoder-decoder models, such as LSTMs, with simple latent distributions, typically multivariate Gaussians. These models pose a difficult optimization problem: there is an especially bad local optimum where the variational posterior always equals the prior and the model does not use the latent variable at all, a kind of "collapse" which is encouraged by the KL divergence term of the objective. In this work, we experiment with another choice of latent distribution, namely the von Mises-Fisher (vMF) distribution, which places mass on the surface of the unit hypersphere. With this choice of prior and posterior, the KL divergence term now only depends on the variance of the vMF distribution, giving us the ability to treat it as a fixed hyperparameter. We show that doing so not only averts the KL collapse, but consistently gives better likelihoods than Gaussians across a range of modeling conditions, including recurrent language modeling and bag-of-words document modeling. An analysis of the properties of our vMF representations shows that they learn richer and more nuanced structures in their latent representations than their Gaussian counterparts.

VAEを学習するとき、潜在表現zが無視され事前分布と事後分布を一致させる方向に学習が行われてしまうことがある(KL collapse)。これを解消するために、正規分布の代わりにVon Mises–Fisher分布(球面上の分布)を導入。KL距離からパラメーター要素(μ,σ)を除去することで、学習による一致を防いでいる


We introduce a multi-task setup of identifying and classifying entities, relations, and coreference clusters in scientific articles. We create SciERC, a dataset that includes annotations for all three tasks and develop a unified framework called Scientific Information Extractor (SciIE) for with shared span representations. The multi-task setup reduces cascading errors between tasks and leverages cross-sentence relations through coreference links. Experiments show that our multi-task model outperforms previous models in scientific information extraction without using any domain-specific features. We further show that the framework supports construction of a scientific knowledge graph, which we use to analyze information in scientific literature.

AI関連研究論文のAbstractから知識を抽出する手法。各文のエンティティとエンティティ間の関係、先行文への参照を同時に解く。文内から適当な長さのスパンを抽出し双方向LSTMで共通特徴を作成、その後は被参照/関係あり、また各タスクごとの予測をそれぞれ全結合で予測


This paper addresses the problem of estimating and tracking human body keypoints in complex, multi-person video. We propose an extremely lightweight yet highly effective approach that builds upon the latest advancements in human detection and video understanding. Our method operates in two-stages: keypoint estimation in frames or short clips, followed by lightweight tracking to generate keypoint predictions linked over the entire video. For frame-level pose estimation we experiment with Mask R-CNN, as well as our own proposed 3D extension of this model, which leverages temporal information over small clips to generate more robust frame predictions. We conduct extensive ablative experiments on the newly released multi-person video pose estimation benchmark, PoseTrack, to validate various design choices of our model. Our approach achieves an accuracy of 55.2% on the validation and 51.8% on the test set using the Multi-Object Tracking Accuracy (MOTA) metric, and achieves state of the art performance on the ICCV 2017 PoseTrack keypoint tracking challenge.

多人数ビデオ映像での複雑な人体keypoint推定とトラッキングの問題で、非常に単純かつ、高効果な手法を提案。


This paper is concerned with learning to solve tasks that require a chain of interdependent steps of relational inference, like answering complex questions about the relationships between objects, or solving puzzles where the smaller elements of a solution mutually constrain each other. We introduce the recurrent relational network, a general purpose module that operates on a graph representation of objects. As a generalization of Santoro et al. [2017]'s relational network, it can augment any neural network model with the capacity to do many-step relational reasoning. We achieve state of the art results on the bAbI textual question-answering dataset with the recurrent relational network, consistently solving 20/20 tasks. As bAbI is not particularly challenging from a relational reasoning point of view, we introduce Pretty-CLEVR, a new diagnostic dataset for relational reasoning. In the Pretty-CLEVR set-up, we can vary the question to control for the number of relational reasoning steps that are required to obtain the answer. Using Pretty-CLEVR, we probe the limitations of multi-layer perceptrons, relational and recurrent relational networks. Finally, we show how recurrent relational networks can learn to solve Sudoku puzzles from supervised training data, a challenging task requiring upwards of 64 steps of relational reasoning. We achieve state-of-the-art results amongst comparable methods by solving 96.6% of the hardest Sudoku puzzles.

オブジェクト間の関係を推論するシンプルなRelation Network( #326 )を拡張し、数独を解いたという研究。Relation Networkでは一回の伝搬しか行わないため、A=>BからさらにB=>Cという連続的な伝搬が扱えなかった。そこで再帰処理を導入し、複数回の伝搬による推論を扱えるようにした。


Attention networks in multimodal learning provide an efficient way to utilize given visual information selectively. However, the computational cost to learn attention distributions for every pair of multimodal input channels is prohibitively expensive. To solve this problem, co-attention builds two separate attention distributions for each modality neglecting the interaction between multimodal inputs. In this paper, we propose bilinear attention networks (BAN) that find bilinear attention distributions to utilize given vision-language information seamlessly. BAN considers bilinear interactions among two groups of input channels, while low-rank bilinear pooling extracts the joint representations for each pair of channels. Furthermore, we propose a variant of multimodal residual networks to exploit eight-attention maps of the BAN efficiently. We quantitatively and qualitatively evaluate our model on visual question answering (VQA 2.0) and Flickr30k Entities datasets, showing that BAN significantly outperforms previous methods and achieves new state-of-the-arts on both datasets.

VQAにおいて画像と質問にCo-attentionを貼るのは有効だが、全ペアに対して計算を行うのは非常に高コストになる。そこで、行列の低ランク近似(low-rank bilinear model)を用いてAttentionを近似し、計算効率を高めたという研究。これにより重みの数も減るためCo-attentionで起こりがちな過学習も防げた


While syntactic dependency annotations concentrate on the surface or functional structure of a sentence, semantic dependency annotations aim to capture between-word relationships that are more closely related to the meaning of a sentence, using graph-structured representations. We extend the LSTM-based syntactic parser of Dozat and Manning (2017) to train on and generate these graph structures. The resulting system on its own achieves state-of-the-art performance, beating the previous, substantially more complex state-of-the-art system by 0.6% labeled F1. Adding linguistically richer input representations pushes the margin even higher, allowing us to beat it by 1.9% labeled F1.

シンプルなDependency Parserの提案。単語と品詞の情報を結合してBi-directionalに入力し、その潜在表現からエッジの有無とエッジのラベルを同時に予測させる(最終的には、単語間のエッジ情報を表すグラフ表現を出力することになる)。これで既存の複雑なモデルを超える精度


Visual Question answering is a challenging problem requiring a combination of concepts from Computer Vision and Natural Language Processing. Most existing approaches use a two streams strategy, computing image and question features that are consequently merged using a variety of techniques. Nonetheless, very few rely on higher level image representations, which can capture semantic and spatial relationships. In this paper, we propose a novel graph-based approach for Visual Question Answering. Our method combines a graph learner module, which learns a question specific graph representation of the input image, with the recent concept of graph convolutions, aiming to learn image representations that capture question specific interactions. We test our approach on the VQA v2 dataset using a simple baseline architecture enhanced by the proposed graph learner module. We obtain promising results with 66.18% accuracy and demonstrate the interpretability of the proposed method. Code can be found at this http URL.

画像を見て質問に答えるタスクに対し、Graph Convolutionを使う手法。検知したオブジェクトの画像特長と質問文の特徴を結合したものをノードとし、ノードの接続は画像特長の位置を基に行う。これにより、物体間の位置関係を質問文のコンテキストで把握することを狙っている。VQA-v2でSOTA。


We introduce LUCSS, a language-based system for interactive col- orization of scene sketches, based on their semantic understanding. LUCSS is built upon deep neural networks trained via a large-scale repository of scene sketches and cartoon-style color images with text descriptions. It con- sists of three sequential modules. First, given a scene sketch, the segmenta- tion module automatically partitions an input sketch into individual object instances. Next, the captioning module generates the text description with spatial relationships based on the instance-level segmentation results. Fi- nally, the interactive colorization module allows users to edit the caption and produce colored images based on the altered caption. Our experiments show the effectiveness of our approach and the desirability of its compo- nents to alternative choices.

自然言語でイラストの色塗りを行う研究。「オレンジの車～」「水色の空～」と入力すると、イラスト中の該当箇所の色が変わる。イラストに対するインスタンスセグメンテーション＋Image Caption＋GANを組み合わせている。


One key task of fine-grained sentiment analysis of product reviews is to extract product aspects or features that users have expressed opinions on. This paper focuses on supervised aspect extraction using deep learning. Unlike other highly sophisticated supervised deep learning models, this paper proposes a novel and yet simple CNN model employing two types of pre-trained embeddings for aspect extraction: general-purpose embeddings and domain-specific embeddings. Without using any additional supervision, this model achieves surprisingly good results, outperforming state-of-the-art sophisticated existing methods. To our knowledge, this paper is the first to report such double embeddings based CNN model for aspect extraction and achieve very good results.


Neural network-based methods for abstractive summarization produce outputs that are more fluent than other techniques, but which can be poor at content selection. This work proposes a simple technique for addressing this issue: use a data-efficient content selector to over-determine phrases in a source document that should be part of the summary. We use this selector as a bottom-up attention step to constrain the model to likely phrases. We show that this approach improves the ability to compress text, while still generating fluent summaries. This two-step process is both simpler and higher performing than other end-to-end content selection models, leading to significant improvements on ROUGE for both the CNN-DM and NYT corpus. Furthermore, the content selector can be trained with as little as 1,000 sentences, making it easy to transfer a trained summarizer to a new domain.

要約で使われるPointer-Generatorモデルは生成とコピーを併用するモデルだが、コピー幅が長くなることが多いという弱点があった。そこで、ソース文において重要な単語(要約と一致する箇所＋既出でない)を抽出して、そこ以外にAttention Maskをかけるという手法で改善を行う試み。


Machine translation systems achieve near human-level performance on some languages, yet their effectiveness strongly relies on the availability of large amounts of parallel sentences, which hinders their applicability to the majority of language pairs. This work investigates how to learn to translate when having access to only large monolingual corpora in each language. We propose two model variants, a neural and a phrase-based model. Both versions leverage a careful initialization of the parameters, the denoising effect of language models and automatic generation of parallel data by iterative back-translation. These models are significantly better than methods from the literature, while being simpler and having fewer hyper-parameters. On the widely used WMT'14 English-French and WMT'16 German-English benchmarks, our models respectively obtain 28.1 and 25.2 BLEU points without using a single parallel sentence, outperforming the state of the art by more than 11 BLEU points. On low-resource languages like English-Urdu and English-Romanian, our methods achieve even better results than semi-supervised and supervised approaches leveraging the paucity of available bitexts. Our code for NMT and PBSMT is publicly available.

Facebookが教師なしで翻訳を行う手法を公開。単語ベースの翻訳の学習(先行研究 #630 より)、単語をうまくつなげる言語モデルの学習を事前に行っておき、それらで翻訳モデルを初期化する。その後は、A->B、B->Aという風に翻訳した文を逆翻訳した際の差異を基に学習を行っていく。


Most research in reading comprehension has focused on answering questions based on individual documents or even single paragraphs. We introduce a neural model which integrates and reasons relying on information spread within documents and across multiple documents. We frame it as an inference problem on a graph. Mentions of entities are nodes of this graph while edges encode relations between different mentions (e.g., within- and cross-document co-reference). Graph convolutional networks (GCNs) are applied to these graphs and trained to perform multi-step reasoning. Our Entity-GCN method is scalable and compact, and it achieves state-of-the-art results on a multi-document question answering dataset, WikiHop (Welbl et al., 2018).

質問回答システムについて、参照文書をグラフで表現して、Graph Convolutionにより回答のエンティティを推定するという研究。複数文書の参照を想定しており、同文書内は共起・外文書へはメンションでリンクを構成し、回答候補のエンティティについてGCNでノード表現を作成し質問表現と合わせ回答する


Generative models, in particular generative adversarial networks (GANs), have received significant attention recently. A number of GAN variants have been proposed and have been utilized in many applications. Despite large strides in terms of theoretical progress, evaluating and comparing GANs remains a daunting task. While several measures have been introduced, as of yet, there is no consensus as to which measure best captures strengths and limitations of models and should be used for fair model comparison. As in other areas of computer vision and machine learning, it is critical to settle on one or few good measures to steer the progress in this field. In this paper, I review and critically discuss more than 24 quantitative and 5 qualitative measures for evaluating generative models with a particular emphasis on GAN-derived models. I also provide a set of 7 desiderata followed by an evaluation of whether a given measure or a family of measures is compatible with them.

GAN(生成モデル)の評価方法をまとめたサーベイ。指標は定量的・定性的の2つに分けられている。定量的な指標はサンプル/生成結果からそれぞれ画像特長を抽出し距離を測るのがスタンダード。定性的な指標は人手評価を行うが、過学習が高く評価される(サンプルに似ていればいいとなると、過学習していたほうが有利)のを防ぐ必要があると。


Model-based reinforcement learning (RL) has proven to be a data efficient approach for learning control tasks but is difficult to utilize in domains with complex observations such as images. In this paper, we present a method for learning representations that are suitable for iterative model-based policy improvement, even when the underlying dynamical system has complex dynamics and image observations, in that these representations are optimized for inferring simple dynamics and cost models given data from the current policy. This enables a model-based RL method based on the linear-quadratic regulator (LQR) to be used for systems with image observations. We evaluate our approach on a range of robotics tasks, including manipulation with a real-world robotic arm directly from images. We find that our method produces substantially better final performance than other model-based RL methods while being significantly more efficient than model-free RL.

Model Baseの手法で学習を行う際に、環境全体をモデル化するのでなく、局所的なパートだけモデル化して(このとき戦略も線形化する)、戦略の勾配を推定するという手法。これにより環境全体をモデル化する必要なしにModel Baseによる効率的な学習が可能になる。


A challenge in creating a dataset for machine reading comprehension (MRC) is to collect questions that require a sophisticated understanding of language to answer beyond using superficial cues. In this work, we investigate what makes questions easier across recent 12 MRC datasets with three question styles (answer extraction, description, and multiple choice). We propose to employ simple heuristics to split each dataset into easy and hard subsets and examine the performance of two baseline models for each of the subsets. We then manually annotate questions sampled from each subset with both validity and requisite reasoning skills to investigate which skills explain the difference between easy and hard questions. From this study, we observed that (i) the baseline performances for the hard subsets remarkably degrade compared to those of entire datasets, (ii) hard questions require knowledge inference and multiple-sentence reasoning in comparison with easy questions, and (iii) multiple-choice questions tend to require a broader range of reasoning skills than answer extraction and description questions. These results suggest that one might overestimate recent advances in MRC.

文書読解で使用されているデータセットについて、その難易度を調査した研究。質問文と最も距離が近い文に回答があるなどの簡単な質問と、複数文にまたがるなどの難しい質問とを分け、その比率を調査している。難しい質問ではモデルの精度が低下し、比率によっては既存モデルの過剰評価に繋がっていると


Abstractive text summarization aims to shorten long text documents into a human readable form that contains the most important facts from the original document. However, the level of actual abstraction as measured by novel phrases that do not appear in the source document remains low in existing approaches. We propose two techniques to improve the level of abstraction of generated summaries. First, we decompose the decoder into a contextual network that retrieves relevant parts of the source document, and a pretrained language model that incorporates prior knowledge about language generation. Second, we propose a novelty metric that is optimized directly through policy learning to encourage the generation of novel phrases. Our model achieves results comparable to state-of-the-art models, as determined by ROUGE scores and human evaluations, while achieving a significantly higher level of abstraction as measured by n-gram overlap with the source document.

抽象型要約の抽象度をさらに高める試み。BiLSTMがベースだが、生成に学習済み言語モデルも活用している。EncoderへのAttention、DecoderへのAttention、Decoder State、そして言語モデルの状態を結合して出力単語を予測する。原文にないフレーズを増やしつつ、ROUGE/人手評価の結果を維持できたという


Convolutional neural networks (CNNs) excel in a wide variety of computer vision applications, but their high performance also comes at a high computational cost. Despite efforts to increase efficiency both algorithmically and with specialized hardware, it remains difficult to deploy CNNs in embedded systems due to tight power budgets. Here we explore a complementary strategy that incorporates a layer of optical computing prior to electronic computing, improving performance on image classification tasks while adding minimal electronic computational cost or processing time. We propose a design for an optical convolutional layer based on an optimized diffractive optical element and test our design in two simulations: a learned optical correlator and an optoelectronic two-layer CNN. We demonstrate in simulation and with an optical prototype that the classification accuracies of our optical systems rival those of the analogous electronic implementations, while providing substantial savings on computational cost.

光学的にニューラルネットを実装する手法の研究。今までできていなかったCNNを、しかも自然光(インコヒーレント)で実装している。位相マスクを使い一画像内に複数の変換結果を並べることで、畳み込みによるチャンネル分の画像を表現している。これでCNNの一層目を実装し、以後は通常のCNNへ入力する


Modern neural networks are very powerful predictive models, but they are often incapable of recognizing when their predictions may be wrong. Closely related to this is the task of out-of-distribution detection, where a network must determine whether or not an input is outside of the set on which it is expected to safely perform. To jointly address these issues, we propose a method of learning confidence estimates for neural networks that is simple to implement and produces intuitively interpretable outputs. We demonstrate that on the task of out-of-distribution detection, our technique surpasses recently proposed techniques which construct confidence based on the network's output distribution, without requiring any additional labels or access to out-of-distribution examples. Additionally, we address the problem of calibrating out-of-distribution detectors, where we demonstrate that misclassified in-distribution examples can be used as a proxy for out-of-distribution examples.

予測を行う際、確信度も出力する手法の提案。これで想定外データに対し確信度が低いことを伝えられるようにする。予測と別に確信度(スカラー)を出力するようにし、確信度に応じラベルから受け取れる情報量が変化する。確信度が低いと答えを教えてもらえるがペナルティもかかるというトレードオフがある


Many NLP applications can be framed as a graph-to-sequence learning problem. Previous work proposing neural architectures on this setting obtained promising results compared to grammar-based approaches but still rely on linearisation heuristics and/or standard recurrent networks to achieve the best performance. In this work, we propose a new model that encodes the full structural information contained in the graph. Our architecture couples the recently proposed Gated Graph Neural Networks with an input transformation that allows nodes and edges to have their own hidden representations, while tackling the parameter explosion problem present in previous work. Experimental results show that our model outperforms strong baselines in generation from AMR graphs and syntax-based neural machine translation.

Gated Graph Neural Net(GGNN)を使いグラフから直接文章を生成する seq2seqなどの場合、グラフを一度系列化してからモデルに投入するためグラフが持つ表現が損なわれてしまう そこでGGNNを使いグラフを直接encodeし、文章をdecodeする AMRグラフからの文章生成と依存構造付き文章の翻訳タスクで有効性を確認


Text generation is a fundamental building block in natural language processing tasks. Existing sequential models performs autoregression directly over the text sequence and have difficulty generating long sentences of complex structures. This paper advocates a simple approach that treats sentence generation as a tree-generation task. By explicitly modelling syntactic structures in a constituent syntactic tree and performing top-down, breadth-first tree generation, our model fixes dependencies appropriately and performs implicit global planning. This is in contrast to transition-based depth-first generation process, which has difficulty dealing with incomplete texts when parsing and also does not incorporate future contexts in planning. Our preliminary results on two generation tasks and one parsing task demonstrate that this is an effective strategy.

テキスト生成にて、直接文生成ではなく構文木を予測して生成を行う手法。構文木はBi-directional RNNで表現し、上位レイヤから親、同レイヤから隣接ノードの潜在表現を取ってノードの潜在表現を作成する。そこからの隣接ノード生成は別のRNNで行う。学習は階層が浅いものから深いものへ切り替えていく


Many recent papers address reading comprehension, where examples consist of (question, passage, answer) tuples. Presumably, a model must combine information from both questions and passages to predict corresponding answers. However, despite intense interest in the topic, with hundreds of published papers vying for leaderboard dominance, basic questions about the difficulty of many popular benchmarks remain unanswered. In this paper, we establish sensible baselines for the bAbI, SQuAD, CBT, CNN, and Who-did-What datasets, finding that question- and passage-only models often perform surprisingly well. On $14$ out of $20$ bAbI tasks, passage-only models achieve greater than $50\%$ accuracy, sometimes matching the full model. Interestingly, while CBT provides $20$-sentence stories only the last is needed for comparably accurate prediction. By comparison, SQuAD and CNN appear better-constructed.

読解問題を機械学習で解くためのデータセットについて、その正当性を調査した論文。具体的には、文書だけ、質問だけでどれくらい回答できるのか調べている。SQuADではどちらかだけだとかなり落ちるが、bAbIでは落差が少なく、CBTに至っては質問だけ使うほうが精度が高いという結果。


In this paper we introduce a simple approach for exploration in reinforcement learning (RL) that allows us to develop theoretically justified algorithms in the tabular case but that is also extendable to settings where function approximation is required. Our approach is based on the successor representation (SR), which was originally introduced as a representation defining state generalization by the similarity of successor states. Here we show that the norm of the SR, while it is being learned, can be used as a reward bonus to incentivize exploration. In order to better understand this transient behavior of the norm of the SR we introduce the substochastic successor representation (SSR) and we show that it implicitly counts the number of times each state (or feature) has been observed. We use this result to introduce an algorithm that performs as well as some theoretically sample-efficient approaches. Finally, we extend these ideas to a deep RL algorithm and show that it achieves state-of-the-art performance in Atari 2600 games when in a low sample-complexity regime.

強化学習で、難しい探査問題に対してドメイン固有の知識に依存してるため、一様ランダムな探索ばかり行われていた。表現学習に密接な関係にある劣確率的代替表現(substochastic successor representation)を疑似カウントとするカウントベース探索を行うことでより大きいドメインの問題にアプローチできる研究。SSRは代替状態の類似性によって状態の一般化をする代替表現に少し変更を加えたもの。


Aspect Term Extraction (ATE), a key sub-task in Aspect-Based Sentiment Analysis, aims to extract explicit aspect expressions from online user reviews. We present a new framework for tackling ATE. It can exploit two useful clues, namely opinion summary and aspect detection history. Opinion summary is distilled from the whole input sentence, conditioned on each current token for aspect prediction, and thus the tailor-made summary can help aspect prediction on this token. Another clue is the information of aspect detection history, and it is distilled from the previous aspect predictions so as to leverage the coordinate structure and tagging schema constraints to upgrade the aspect prediction. Experimental results over four benchmark datasets clearly demonstrate that our framework can outperform all state-of-the-art methods.

系列レベリングの手法でaspect termを抽出する時、opinion termを考慮していない。opinion termの情報を利用するため、attentionモデルで抽出したaspect履歴を学習しながら、文ごとにopinion要約を生成する。aspect履歴とopinion要約をあわせて、aspect termを予測のする仕組みを提案した。


This paper presents a simple method for "do as I do" motion transfer: given a source video of a person dancing, we can transfer that performance to a novel (amateur) target after only a few minutes of the target subject performing standard moves. We approach this problem as video-to-video translation using pose as an intermediate representation. To transfer the motion, we extract poses from the source subject and apply the learned pose-to-appearance mapping to generate the target subject. We predict two consecutive frames for temporally coherent video results and introduce a separate pipeline for realistic face synthesis. Although our method is quite simple, it produces surprisingly compelling results (see video). This motivates us to also provide a forensics tool for reliable synthetic content detection, which is able to distinguish videos synthesized by our system from real data. In addition, we release a first-of-its-kind open-source dataset of videos that can be legally used for training and motion transfer.

Everybody Dance Nowというすごいタイトルの論文。動画を与えることで、画像を同じように動かすという内容。仕組み的には、フレームレベルでのイメージ変換として動いている。ポーズ推定のネットワークと、ポーズを取らせるGAN(ポーズと画像のペアについて、画像がGによる生成か判断)の2つで構成される


This paper describes the pricing strategy model deployed at Airbnb, an online marketplace for sharing home and experience. The goal of price optimization is to help hosts who share their homes on Airbnb set the optimal price for their listings. In contrast to conventional pricing problems, where pricing strategies are applied to a large quantity of identical products, there are no "identical" products on Airbnb, because each listing on our platform offers unique values and experiences to our guests. The unique nature of Airbnb listings makes it very difficult to estimate an accurate demand curve that's required to apply conventional revenue maximization pricing strategies. Our pricing system consists of three components. First, a binary classification model predicts the booking probability of each listing-night. Second, a regression model predicts the optimal price for each listing-night, in which a customized loss function is used to guide the learning. Finally, we apply additional personalization logic on top of the output from the second model to generate the final price suggestions. In this paper, we focus on describing the regression model in the second stage of our pricing system. We also describe a novel set of metrics for offline evaluation. The proposed pricing strategy has been deployed in production to power the Price Tips and Smart Pricing tool on Airbnb. Online A/B testing results demonstrate the effectiveness of the proposed strategy model.

Airbnbで使用している価格決定モデルについての論文。予約される確率と適正価格の予測を分けており、予約確率により提案価格を上下させている。価格の予測は、予約されなかった場合に提案モデルが引く価格を予測していた割合(=価格を下げれば予約されたケース)と、予約された場合に提案モデルが低い価格を予測していた割合(=もっと上げていればよかったケース)の2つをバランスさせる形で行う。


Real-time detection of irregularities in visual data is very invaluable and useful in many prospective applications including surveillance, patient monitoring systems, etc. With the surge of deep learning methods in the recent years, researchers have tried a wide spectrum of methods for different applications. However, for the case of irregularity or anomaly detection in videos, training an end-to-end model is still an open challenge, since often irregularity is not well-defined and there are not enough irregular samples to use during training. In this paper, inspired by the success of generative adversarial networks (GANs) for training deep models in unsupervised or self-supervised settings, we propose an end-to-end deep network for detection and fine localization of irregularities in videos (and images). Our proposed architecture is composed of two networks, which are trained in competing with each other while collaborating to find the irregularity. One network works as a pixel-level irregularity Inpainter, and the other works as a patch-level Detector. After an adversarial self-supervised training, in which I tries to fool D into accepting its inpainted output as regular (normal), the two networks collaborate to detect and fine-segment the irregularity in any given testing video. Our results on three different datasets show that our method can outperform the state-of-the-art and fine-segment the irregularity.

異常を消した画像を生成するGeneratorと、異常を検知するDiscriminatorを戦わせる形で異常検知を行う手法。実際検知を行う際は、実画像とGの生成結果の差(異常があれば消されるので差が大きくなる)と領域単位のDの判定結果を合わせて行う。


Current state-of-the-art machine translation systems are based on encoder-decoder architectures, that first encode the input sequence, and then generate an output sequence based on the input encoding. Both are interfaced with an attention mechanism that recombines a fixed encoding of the source tokens based on the decoder state. We propose an alternative approach which instead relies on a single 2D convolutional neural network across both sequences. Each layer of our network re-codes source tokens on the basis of the output sequence produced so far. Attention-like properties are therefore pervasive throughout the network. Our model yields excellent results, outperforming state-of-the-art encoder-decoder systems, while being conceptually simpler and having fewer parameters.

DenseNetで翻訳を行うという斬新な手法。入力マップは、ターゲット長Ｘソース長Ｘ(ターゲットembeddingサイズ＋ソースembeddingサイズ)となる。畳み込みは、未来の情報に依存しないようマスクをかけて実施。最終的にソース長部分を集約する際はAverage(+Attention)で実施。


Deep Learning has enabled remarkable progress over the last years on a variety of tasks, such as image recognition, speech recognition, and machine translation. One crucial aspect for this progress are novel neural architectures. Currently employed architectures have mostly been developed manually by human experts, which is a time-consuming and error-prone process. Because of this, there is growing interest in automated neural architecture search methods. We provide an overview of existing work in this field of research and categorize them according to three dimensions: search space, search strategy, and performance estimation strategy.

機械学習によるネットワークの構造探索手法(NAS)に関するサーベイ。探索空間・探索方法・アーキテクチャの評価方法と3つの観点に分けて手法をまとめている。探索以外の変形(mutation)については弱めだがきちんと載っている。


Attention mechanisms in sequence to sequence models have shown great ability and wonderful performance in various natural language processing (NLP) tasks, such as sentence embedding, text generation, machine translation, machine reading comprehension, etc. Unfortunately, existing attention mechanisms only learn either high-level or low-level features. In this paper, we think that the lack of hierarchical mechanisms is a bottleneck in improving the performance of the attention mechanisms, and propose a novel Hierarchical Attention Mechanism (Ham) based on the weighted sum of different layers of a multi-level attention. Ham achieves a state-of-the-art BLEU score of 0.26 on Chinese poem generation task and a nearly 6.5% averaged improvement compared with the existing machine reading comprehension models such as BIDAF and Match-LSTM. Furthermore, our experiments and theorems reveal that Ham has greater generalization and representation ability than existing attention mechanisms.

Attentionを再帰的に複数回かけるMulti-Level Attentionの、各段階における重み付き和をとるHierarchical Attentionを提案。5段階程度の階層で、ベースラインのモデルをブーストできることを確認。


Reinforcement learning algorithms rely on carefully engineering environment rewards that are extrinsic to the agent. However, annotating each environment with hand-designed, dense rewards is not scalable, motivating the need for developing reward functions that are intrinsic to the agent. Curiosity is a type of intrinsic reward function which uses prediction error as reward signal. In this paper: (a) We perform the first large-scale study of purely curiosity-driven learning, i.e. without any extrinsic rewards, across 54 standard benchmark environments, including the Atari game suite. Our results show surprisingly good performance, and a high degree of alignment between the intrinsic curiosity objective and the hand-designed extrinsic rewards of many game environments. (b) We investigate the effect of using different feature spaces for computing prediction error and show that random features are sufficient for many popular RL game benchmarks, but learned features appear to generalize better (e.g. to novel game levels in Super Mario Bros.). (c) We demonstrate limitations of the prediction-based rewards in stochastic setups. Game-play videos and code are at this https URL

内発的報酬のみでどこまでプレイできるかを検証した研究。内発的報酬は行動による状態変化に対して与えられており、本研究では「状態」の表現方法についてピクセル/固定CNN/VAE/IDFの4つを使用している。タスクが進むか、どれが有効かはかなりタスクに依存している。


We explore a new way to evaluate generative models using insights from evaluation of competitive games between human players. We show experimentally that tournaments between generators and discriminators provide an effective way to evaluate generative models. We introduce two methods for summarizing tournament outcomes: tournament win rate and skill rating. Evaluations are useful in different contexts, including monitoring the progress of a single model as it learns during the training process, and comparing the capabilities of two different fully trained models. We show that a tournament consisting of a single model playing against past and future versions of itself produces a useful measure of training progress. A tournament containing multiple separate models (using different seeds, hyperparameters, and architectures) provides a useful relative comparison between different trained GANs. Tournament-based rating methods are conceptually distinct from numerous previous categories of approaches to evaluation of generative models, and have complementary advantages and disadvantages.

GeneratorとDeiscriminatorでトーナメント戦を行うことで、GANの評価を行う試み。各GとDは、異なる学習過程、また異なるハイパーパラメーター設定のものを選出する。これにより、どの段階でGがDに勝てるようになったのかなどを計測できる。勝率以外にレーティング(Glicko2)を導入など本物さながら


State-of-the-art visual perception models for a wide range of tasks rely on supervised pretraining. ImageNet classification is the de facto pretraining task for these models. Yet, ImageNet is now nearly ten years old and is by modern standards "small". Even so, relatively little is known about the behavior of pretraining with datasets that are multiple orders of magnitude larger. The reasons are obvious: such datasets are difficult to collect and annotate. In this paper, we present a unique study of transfer learning with large convolutional networks trained to predict hashtags on billions of social media images. Our experiments demonstrate that training for large-scale hashtag prediction leads to excellent results. We show improvements on several image classification and object detection tasks, and report the highest ImageNet-1k single-crop, top-1 accuracy to date: 85.4% (97.6% top-5). We also perform extensive experiments that provide novel empirical data on the relationship between large-scale pretraining and transfer learning performance.

転移学習に関する研究で、どう事前学習するといいのかを検証した研究(Instagramのデータを使用)。事前学習の規模は転移性能への貢献がある、ラベル空間は近いほうがいい、また視覚的多様性を増やしたほうが良いとしている。また、現在のネットワークがImageNetのタスク/サイズに適合しすぎているとも。


Manually labeling datasets with object masks is extremely time consuming. In this work, we follow the idea of Polygon-RNN to produce polygonal annotations of objects interactively using humans-in-the-loop. We introduce several important improvements to the model: 1) we design a new CNN encoder architecture, 2) show how to effectively train the model with Reinforcement Learning, and 3) significantly increase the output resolution using a Graph Neural Network, allowing the model to accurately annotate high-resolution objects in images. Extensive evaluation on the Cityscapes dataset shows that our model, which we refer to as Polygon-RNN++, significantly outperforms the original model in both automatic (10% absolute and 16% relative improvement in mean IoU) and interactive modes (requiring 50% fewer clicks by annotators). We further analyze the cross-domain scenario in which our model is trained on one dataset, and used out of the box on datasets from varying domains. The results show that Polygon-RNN++ exhibits powerful generalization capabilities, achieving significant improvements over existing pixel-wise methods. Using simple online fine-tuning we further achieve a high reduction in annotation time for new datasets, moving a step closer towards an interactive annotation tool to be used in practice.

セグメンテーションにおけるアノテーションを支援する手法の提案。セグメンテーションとその頂点を提案、人が頂点を修正できるようにする。CNNで画像特長、RNNで頂点予測という構成で頂点予測に強化学習(厳密一致でなくても報酬を与える)、頂点のリファインにGNNと凝った構成


Recurrent neural networks (RNNs) sequentially process data by updating their state with each new data point, and have long been the de facto choice for sequence modeling tasks. However, their inherently sequential computation makes them slow to train. Feed-forward and convolutional architectures have recently been shown to achieve superior results on some sequence modeling tasks such as machine translation, with the added advantage that they concurrently process all inputs in the sequence, leading to easy parallelization and faster training times. Despite these successes, however, popular feed-forward sequence models like the Transformer fail to generalize in many simple tasks that recurrent models handle with ease, e.g. copying strings or even simple logical inference when the string or formula lengths exceed those observed at training time. We propose the Universal Transformer (UT), a parallel-in-time self-attentive recurrent sequence model which can be cast as a generalization of the Transformer model and which addresses these issues. UTs combine the parallelizability and global receptive field of feed-forward sequence models like the Transformer with the recurrent inductive bias of RNNs. We also add a dynamic per-position halting mechanism and find that it improves accuracy on several tasks. In contrast to the standard Transformer, under certain assumptions, UTs can be shown to be Turing-complete. Our experiments show that UTs outperform standard Transformers on a wide range of algorithmic and language understanding tasks, including the challenging LAMBADA language modeling task where UTs achieve a new state of the art, and machine translation where UTs achieve a 0.9 BLEU improvement over Transformers on the WMT14 En-De dataset.

Transformerは閉じた系列の予測には強いものの、RNNのように連続的な系列の予測には弱いという弱点があった(学習時にないサイズの系列の場合パフォーマンスが落ちる)。そこでEncoder/Decoderブロックの処理を再帰的に何回か繰り返す処理を提案(回数を動的に決める手法も提案)


A neural network (NN) is a parameterised function that can be tuned via gradient descent to approximate a labelled collection of data with high precision. A Gaussian process (GP), on the other hand, is a probabilistic model that defines a distribution over possible functions, and is updated in light of data via the rules of probabilistic inference. GPs are probabilistic, data-efficient and flexible, however they are also computationally intensive and thus limited in their applicability. We introduce a class of neural latent variable models which we call Neural Processes (NPs), combining the best of both worlds. Like GPs, NPs define distributions over functions, are capable of rapid adaptation to new observations, and can estimate the uncertainty in their predictions. Like NNs, NPs are computationally efficient during training and evaluation but also learn to adapt their priors to data. We demonstrate the performance of NPs on a range of learning tasks, including regression and optimisation, and compare and contrast with related models in the literature.

確率過程をニューラルネットで実装する試み。問題設定としては、ある入力と出力のペア(x, y)が与えられたときに、その背後にある相関関係を推定し別途与えられるx'からy'を予測するというもの。全体としては、与えられたペアを潜在表現にするencoder、それを集計するaggregator、集計されたものを基に復元を行うdecoderというシンプルな構成。


Multimodal research is an emerging field of artificial intelligence, and one of the main research problems in this field is multimodal fusion. The fusion of multimodal data is the process of integrating multiple unimodal representations into one compact multimodal representation. Previous research in this field has exploited the expressiveness of tensors for multimodal representation. However, these methods often suffer from exponential increase in dimensions and in computational complexity introduced by transformation of input into tensor. In this paper, we propose the Low-rank Multimodal Fusion method, which performs multimodal fusion using low-rank tensors to improve efficiency. We evaluate our model on three different tasks: multimodal sentiment analysis, speaker trait analysis, and emotion recognition. Our model achieves competitive results on all these tasks while drastically reducing computational complexity. Additional experiments also show that our model can perform robustly for a wide range of low-rank settings, and is indeed much more efficient in both training and inference compared to other methods that utilize tensor representations.

マルチモーダルデータを使った予測などに使うテンソルを低ランクへの分解を行うことで計算量の削減と並列化を実現 従来手法における各モーダルのベクトルの次元数が増えると計算コストが指数的に増える問題を緩和した 推論速度を比べると従来手法の2～3倍ほど


Aspect term extraction is one of the important subtasks in aspect-based sentiment analysis. Previous studies have shown that using dependency tree structure representation is promising for this task. However, most dependency tree structures involve only one directional propagation on the dependency tree. In this paper, we first propose a novel bidirectional dependency tree network to extract dependency structure features from the given sentences. The key idea is to explicitly incorporate both representations gained separately from the bottom-up and top-down propagation on the given dependency syntactic tree. An end-to-end framework is then developed to integrate the embedded representations and BiLSTM plus CRF to learn both tree-structured and sequential features to solve the aspect term extraction problem. Experimental results demonstrate that the proposed model outperforms state-of-the-art baseline models on four benchmark SemEval datasets.

文中のaspect termを抽出するため、文の構文木を利用するend-to-endモデルを提案した。 構文木のbottom-upとtop-down２つの方向で、樹形図の文法依頼を学習する。


In this paper, we study the problem of geometric reasoning in the context of question-answering. We introduce Dynamic Spatial Memory Network (DSMN), a new deep network architecture designed for answering questions that admit latent visual representations. DSMN learns to generate and reason over such representations. Further, we propose two synthetic benchmarks, FloorPlanQA and ShapeIntersection, to evaluate the geometric reasoning capability of QA systems. Experimental results validate the effectiveness of our proposed DSMN for visual thinking tasks.

状態の画像と説明文を読み込んで、質問に回答するタスクの提案。 具体的には「部屋の南東の角にキューブがある」といった説明と画像が与えられ、部屋に対する質問に対して回答を行う。 このタスクのために、説明文や画像を読みこんでMemoryに保持しておき、回答に利用するモデルDSMNを提案した。


In recent years, Opinion mining has been an active research area in Text mining and analysis, natural language processing. Opinion mining is the computational study of people’s opinion expressed in written language or text towards entities and their aspects. With the growth of internet, social networking sites, blogs, discussion forums, e-commerce websites have gained a tremendous importance and have provided platform for people to express and share their opinion on entities and their aspects. As opinionated web content is increasing rapidly in the form of reviews, comments, blogs, status updates, tweets, etc. it is practically impossible for people or organization to analyze all opinions at a time to make good decisions. Hence, there is a need for effective automated system to evaluate opinions and generate accurate results. This paper describes opinion mining and focuses on the sub topic aspect-based opinion mining, tasks in aspect-based opinion mining, current stateof-the-art methods used for aspect-based opinion mining, advantages and disadvantages of these methods and latest research challenges in aspect-based opinion mining. Our experimental results based on some of the aspect extraction techniques, gives an idea of which aspect extraction techniques are efficient and yield accurate results in practical opinion mining applications. General Terms Data Mining, Text Mining, Natural Language Processing


Aspect extraction aims to extract fine-grained opinion targets from opinion texts. Recent work has shown that the syntactical approach, which employs rules about grammar dependency relations between opinion words and aspects, performs quite well. This approach is highly desirable in practice because it is unsupervised and domain independent. However, the rules need to be carefully selected and tuned manually so as not to produce too many errors. Although it is easy to evaluate the accuracy of each rule automatically, it is not easy to select a set of rules that produces the best overall result due to the overlapping coverage of the rules. In this paper, we propose a novel method to select an effective set of rules. To our knowledge, this is the first work that selects rules automatically. Our experiment results show that the proposed method can select a subset of a given rule set to achieve significantly better results than the full rule set and the existing state-of-the-art CRF-based supervised method.

文法の依頼関係でopinion termとaspect termの抽出方法はよく使われている。これは教師なし、且つドメインに依存しない方法だから、文法のルールをきちんと選択する必要がある。本研究はgreed searchに基づいて、最適な文法ルールのサブセットを見つけることを狙う。


While large-scale knowledge graphs provide vast amounts of structured facts about entities, a short textual description can often be useful to succinctly characterize an entity and its type. Unfortunately, many knowledge graph entities lack such textual descriptions. In this paper, we introduce a dynamic memory-based network that generates a short open vocabulary description of an entity by jointly leveraging induced fact embeddings as well as the dynamic context of the generated sequence of words. We demonstrate the ability of our architecture to discern relevant information for more accurate generation of type description by pitting the system against several strong baselines.

Entityに対する事実情報をMemory機構に保持しておき、それを使ってDescriptionを生成するモデル。 先行研究の手法では、一般的なDescription(例えばhuman beingなど)が生成されやすい。 そこで入力文書からFact項を抜き出しそれをMemory機構に保持し、動的に更新をかけていくことでMemoryの情報を使ってより具体的な文が生成されることを狙う。


Aspect Term Extraction (ATE) detects opinionated aspect terms in sentences or text spans, with the end goal of performing aspect-based sentiment analysis. The small amount of available datasets for supervised ATE and the fact that they cover only a few domains raise the need for exploiting other data sources in new and creative ways. Publicly available review corpora contain a plethora of opinionated aspect terms and cover a larger domain spectrum. In this paper, we first propose a method for using such review corpora for creating a new dataset for ATE. Our method relies on an attention mechanism to select sentences that have a high likelihood of containing actual opinionated aspects. We thus improve the quality of the extracted aspects. We then use the constructed dataset to train a model and perform ATE with distant supervision. By evaluating on human annotated datasets, we prove that our method achieves a significantly improved performance over various unsupervised and supervised baselines. Finally, we prove that sentence selection matters when it comes to creating new datasets for ATE. Specifically, we show that, using a set of selected sentences leads to higher ATE performance compared to using the whole sentence set.

Aspect term extractionというタスクにおいて、一つの文はaspect termが含まない場合もある。このような文はaspect term extractionのノイズになる。この論文はattentionモデルを使って、一つのレビューの各文にattention scoreにつける。文のattention scoreは閾値より低かったら、この文はaspect termが含まない文として削除される。文をフィルタリングして、新しいデータセットを作る。実験によって、新しいデータセットはaspect term extraction taskによい効果がある。


Designing convolutional neural networks (CNN) for mobile devices is challenging because mobile models need to be small and fast, yet still accurate. Although significant efforts have been dedicated to design and improve mobile CNNs on all dimensions, it is very difficult to manually balance these trade-offs when there are so many architectural possibilities to consider. In this paper, we propose an automated mobile neural architecture search (MNAS) approach, which explicitly incorporate model latency into the main objective so that the search can identify a model that achieves a good trade-off between accuracy and latency. Unlike previous work, where latency is considered via another, often inaccurate proxy (e.g., FLOPS), our approach directly measures real-world inference latency by executing the model on mobile phones. To further strike the right balance between flexibility and search space size, we propose a novel factorized hierarchical search space that encourages layer diversity throughout the network. Experimental results show that our approach consistently outperforms state-of-the-art mobile CNN models across multiple vision tasks. On the ImageNet classification task, our MnasNet achieves 75.2% top-1 accuracy with 78ms latency on a Pixel phone, which is 1.8x faster than MobileNetV2 [29] with 0.5% higher accuracy and 2.3x faster than NASNet [36] with 1.2% higher accuracy. Our MnasNet also achieves better mAP quality than MobileNets for COCO object detection. Code is at this https URL

強化学習によるネットワーク構造探索を、モバイル用モデルの探索に活用した研究。精度/実行速度をバランスさせる指標を開発し、それを報酬として学習させる。探索空間についてはブロックの構成を決めてしまう一方、ブロック内のセル構造は位置により変えられるようにしている


Aspect Term Extraction (ATE) identifies opinionated aspect terms in texts and is one of the tasks in the SemEval Aspect Based Sentiment Analysis (ABSA) contest. The small amount of available datasets for supervised ATE and the costly human annotation for aspect term labelling give rise to the need for unsupervised ATE. In this paper, we introduce an architecture that achieves top-ranking performance for supervised ATE. Moreover, it can be used efficiently as feature extractor and classifier for unsupervised ATE. Our second contribution is a method to automatically construct datasets for ATE. We train a classifier on our automatically labelled datasets and evaluate it on the human annotated SemEval ABSA test sets. Compared to a strong rule-based baseline, we obtain a dramatically higher F-score and attain precision values above 80%. Our unsupervised method beats the supervised ABSA baseline from SemEval, while preserving high precision scores.

この論文はSupervised aspect term extractionというタスクにたいして、２つの問題を解決したい。一つ目はラベル付きデータの数が少ない。二つ目は人工的にアノテーションする時間がかかる、尚且お金がかかる。解決策としては、まずラベルなしデータを使うこと。このようなデータにたいして、自動的にアノテーションする。この論文はrule-based手法を基ついて、自動的にアノテーションするプロセスを提出した。


Neural networks can learn to represent and manipulate numerical information, but they seldom generalize well outside of the range of numerical values encountered during training. To encourage more systematic numerical extrapolation, we propose an architecture that represents numerical quantities as linear activations which are manipulated using primitive arithmetic operators, controlled by learned gates. We call this module a neural arithmetic logic unit (NALU), by analogy to the arithmetic logic unit in traditional processors. Experiments show that NALU-enhanced neural networks can learn to track time, perform arithmetic over images of numbers, translate numerical language into real-valued scalars, execute computer code, and count objects in images. In contrast to conventional architectures, we obtain substantially better generalization both inside and outside of the range of numerical values encountered during training, often extrapolating orders of magnitude beyond trained numerical ranges.

ニューラルネットでは数値スケール(1と20が20倍違うなど)が上手く学習できない。この改善としてスケールシフト(数値演算)が表現できるユニットを提案(加減算のNACと、掛け算のNALU)。実験から非線形変換が数値表現の獲得に影響を及ぼしていたので線形変換で作成。画像/言語の数値変換などで効果を確認


The growing scale of face recognition datasets empowers us to train strong convolutional networks for face recognition. While a variety of architectures and loss functions have been devised, we still have a limited understanding of the source and consequence of label noise inherent in existing datasets. We make the following contributions: 1) We contribute cleaned subsets of popular face databases, i.e., MegaFace and MS-Celeb-1M datasets, and build a new large-scale noise-controlled IMDb-Face dataset. 2) With the original datasets and cleaned subsets, we profile and analyze label noise properties of MegaFace and MS-Celeb-1M. We show that a few orders more samples are needed to achieve the same accuracy yielded by a clean subset. 3) We study the association between different types of noise, i.e., label flips and outliers, with the accuracy of face recognition models. 4) We investigate ways to improve data cleanliness, including a comprehensive user study on the influence of data labeling strategies to annotation accuracy. The IMDb-Face dataset has been released on this https URL.

顔認識におけるラベルのノイズの影響について。MegaFaceやMS-Celeb-1Mなどの大規模なデータセットは、量が多いため検索エンジン等を利用し収集とアノテーションをしている。そのため誤ったラベルが多く、これを修正した場合実データの20~30%程度で同等の精度が出せるという。また、大規模ながら人手でアノテーションしたデータセットIMDb-Faceを公開。