Recurrent neural networks (RNNs) have achieved state-of-the-art performances in many natural language processing tasks, such as language modeling and machine translation. However, when the vocabulary is large, the RNN model will become very big (e.g., possibly beyond the memory capacity of a GPU device) and its training will become very inefficient. In this work, we propose a novel technique to tackle this challenge. The key idea is to use 2-Component (2C) shared embedding for word representations. We allocate every word in the vocabulary into a table, each row of which is associated with a vector, and each column associated with another vector. Depending on its position in the table, a word is jointly represented by two components: a row vector and a column vector. Since the words in the same row share the row vector and the words in the same column share the column vector, we only need 2 \sqrt{|V|} vectors to represent a vocabulary of |V| unique words, which are far less than the |V| vectors required by existing approaches. Based on the 2-Component shared embedding, we design a new RNN algorithm and evaluate it using the language modeling task on several benchmark datasets. The results show that our algorithm significantly reduces the model size and speeds up the training process, without sacrifice of accuracy (it achieves similar, if not better, perplexity as compared to state-of-the-art language models). Remarkably, on the One-Billion-Word benchmark Dataset, our algorithm achieves comparable perplexity to previous language models, whilst reducing the model size by a factor of 40-100, and speeding up the training process by a factor of 2. We name our proposed algorithm \emph{LightRNN} to reflect its very small model size and very high training speed.

RNNにおいてaccuracyを犠牲にせずに時間/空間計算量を削減可能なLightRNNの提案。embedding空間を1単語1vectorで表現するのではなく、tableに分解して表現(Figure1 参照)。これにより単語数がVの場合、2\sqrt{V}個のembedding vectorで空間を表現出来る。したがって、このtableを構築する部分がLightRNNの勘所。構築したtableを使用し、1単語毎にrow, colmn別にhidden stateを計算する。


Real-world objects occur in specific contexts. Such context has been shown to facilitate detection by constraining the locations to search. But can context directly benefit object detection? To do so, context needs to be learned independently from target features. This is impossible in traditional object detection where classifiers are trained on images containing both target features and surrounding context. In contrast, humans can learn context and target features separately, such as when we see highways without cars. Here we show for the first time that human-derived scene expectations can be used to improve object detection performance in machines. To measure contextual expectations, we asked human subjects to indicate the scale, location and likelihood at which cars or people might occur in scenes without these objects. Humans showed highly systematic expectations that we could accurately predict using scene features. This allowed us to predict human expectations on novel scenes without requiring manual annotation. On augmenting deep neural networks with predicted human expectations, we obtained substantial gains in accuracy for detecting cars and people (1-3%) as well as on detecting associated objects (3-20%). In contrast, augmenting deep networks with other conventional features yielded far smaller gains. This improvement was due to relatively poor matches at highly likely locations being correctly labelled as target and conversely strong matches at unlikely locations being correctly rejected as false alarms. Taken together, our results show that augmenting deep neural networks with human-derived context features improves their performance, suggesting that humans learn scene context separately unlike deep networks.

人間のシーン情報などのコンテキスト情報の活用の仕方を取り入れて、物体認識の性能を向上させるための提案を行った。HOG, Gaussian blurなどの従来手法を積極的に取り入れ、人間(車)が写ってそうなシーンか、写っていそうな場所であるかのaverage likelihoodを算出。算出した数値とCNNの特徴量を合わせて使用。CNN単体では確信度が低い物体やシーンに対して効果的という結果が得られた。


Neural language models learn word representations that capture rich linguistic and conceptual information. Here we investigate the embeddings learned by neural machine translation models. We show that translation-based embeddings outperform those learned by cutting-edge monolingual models at single-language tasks requiring knowledge of conceptual similarity and/or syntactic role. The findings suggest that, while monolingual models learn information about how concepts are related, neural-translation models better capture their true ontological status.

ニューラル機械翻訳モデルによってword embeddingsを獲得した論文。Skip-gramのような単一言語モデルより翻訳モデルの方が概念間の類似度を捉えられるのではないかということを検証している。単一言語モデルと比較した結果、ニューラル翻訳モデルは概念間の類似度をより良く捉えられ、単一言語モデルは関連性をより良く捉えられることが示唆された。


State of the art deep reinforcement learning algorithms take many millions of interactions to attain human-level performance. Humans, on the other hand, can very quickly exploit highly rewarding nuances of an environment upon first discovery. In the brain, such rapid learning is thought to depend on the hippocampus and its capacity for episodic memory. Here we investigate whether a simple model of hippocampal episodic control can learn to solve difficult sequential decision-making tasks. We demonstrate that it not only attains a highly rewarding strategy significantly faster than state-of-the-art deep reinforcement learning algorithms, but also achieves a higher overall reward on some of the more challenging domains.

強化学習において、価値観数をパラメトリックに推定するDQNなどの方法は学習の初期段階で非効率的であり、少ない事例に対する汎化能力は低い。これに対して、提案手法はある状態の価値関数を、K-nearest-neighborsのような事例ベースのノンパラメトリックな手法で与えている。このとき、neighborsを決める類似度は変分オートエンコーダのような教師なし学習で得た表現をもとに計算する。論文ではさらに、実際の脳では、ノンパラメトリックな学習を行う海馬がエピソード的で急速な学習を行い、パラメトリックな学習を行う大脳皮質が汎用的で長期的な学習を行うと主張している。


We propose an extension to neural network language models to adapt their prediction to the recent history. Our model is a simplified version of memory augmented networks, which stores past hidden activations as memory and accesses them through a dot product with the current hidden activation. This mechanism is very efficient and scales to very large memory sizes. We also draw a link between the use of external memory in neural network and cache models used with count based language models. We demonstrate on several language model datasets that our approach performs significantly better than recent memory augmented networks.

Neural Cache Modelという汎用的なcache modelの提案。既存のneural network language modelに適用可能。pre-trainingされたモデルにattachするだけで良く、fine tune不要。cache対象はhidden stateとwordの対。neural cache language modelでは、現在見ている単語から推定される次の単語の確率と現在までのcacheから推定される確率の加重平均により次の単語の推定が行われる。


Though tremendous strides have been made in object recognition, one of the remaining open challenges is detecting small objects. We explore three aspects of the problem in the context of finding small faces: the role of scale invariance, image resolution, and contextual reasoning. While most recognition approaches aim to be scale-invariant, the cues for recognizing a 3px tall face are fundamentally different than those for recognizing a 300px tall face. We take a different approach and train separate detectors for different scales. To maintain efficiency, detectors are trained in a multi-task fashion: they make use of features extracted from multiple layers of single (deep) feature hierarchy. While training detectors for large objects is straightforward, the crucial challenge remains training detectors for small objects. We show that context is crucial, and define templates that make use of massively-large receptive fields (where 99% of the template extends beyond the object of interest). Finally, we explore the role of scale in pre-trained deep networks, providing ways to extrapolate networks tuned for limited scales to rather extreme ranges. We demonstrate state-of-the-art results on massively-benchmarked face datasets (FDDB and WIDER FACE). In particular, when compared to prior art on WIDER FACE, our results reduce error by a factor of 2 (our models produce an AP of 82% while prior art ranges from 29-64%).

small objectをdetectionする際の難しさの解析と改善を行った論文。題材は顔検出。templateのscale、画像解像度、contextの必要性について解析を行っている。小さな画像を引き伸ばすと認識精度は下がり、人間も機械も小さな物体を探すにはcontextをかなり必要としている事を検証。また、one-size-fits-allのようなtemplateのサイズを1つに固定すると認識に限界があるため、入力画像を2倍、0.5倍し、originalとともに独立の3つのネットワークで特徴抽出しdetectionを行い、その後マージするという方法を取っている。SoTA。


Mini-batch stochastic gradient descent and variants thereof have become standard for large-scale empirical risk minimization like the training of neural networks. These methods are usually used with a constant batch size chosen by simple empirical inspection. The batch size significantly influences the behavior of the stochastic optimization algorithm, though, since it determines the variance of the gradient estimates. This variance also changes over the optimization process; when using a constant batch size, stability and convergence is thus often enforced by means of a (manually tuned) decreasing learning rate schedule. We propose a practical method for dynamic batch size adaptation. It estimates the variance of the stochastic gradients and adapts the batch size to decrease the variance proportionally to the value of the objective function, removing the need for the aforementioned learning rate decrease. In contrast to recent related work, our algorithm couples the batch size to the learning rate, directly reflecting the known relationship between the two. On popular image classification benchmarks, our batch size adaptation yields faster optimization convergence, while simultaneously simplifying learning rate tuning. A TensorFlow implementation is available.

batch sizeを調整するCABSというルールを提案。mini batch sizeは確率的勾配のバリアンスに影響し、収束速度に大きく関係するため、重要なパラメタである。learning rateが大きい（小さい）場合はbatch sizeを大きく（小さく）する必要があるが（(19)式の関係）、本手法ではその関係性に基いて自動でbatch sizeの調整を行う。従来手法と比較してSGDでの収束速度が向上。


We introduce SceneNet RGB-D, expanding the previous work of SceneNet to enable large scale photorealistic rendering of indoor scene trajectories. It provides pixel-perfect ground truth for scene understanding problems such as semantic segmentation, instance segmentation, and object detection, and also for geometric computer vision problems such as optical flow, depth estimation, camera pose estimation, and 3D reconstruction. Random sampling permits virtually unlimited scene configurations, and here we provide a set of 5M rendered RGB-D images from over 15K trajectories in synthetic layouts with random but physically simulated object poses. Each layout also has random lighting, camera trajectories, and textures. The scale of this dataset is well suited for pre-training data-driven computer vision techniques from scratch with RGB-D inputs, which previously has been limited by relatively small labelled datasets in NYUv2 and SUN RGB-D. It also provides a basis for investigating 3D scene labelling tasks by providing perfect camera poses and depth data as proxy for a SLAM system. We host the dataset at this http URL

シーン認識のための学習データセットSceneNet RGB-Dの公開。物理シミュレーターでシーン(部屋の中にものが散らばった環境)を作り、そこでカメラの軌跡を設定し映像を作製、その映像のRGB+Depthをデータ化、という感じで生成


Synthesizing high-quality images from text descriptions is a challenging problem in computer vision and has many practical applications. Samples generated by existing text-to-image approaches can roughly reflect the meaning of the given descriptions, but they fail to contain necessary details and vivid object parts. In this paper, we propose Stacked Generative Adversarial Networks (StackGAN) to generate 256x256 photo-realistic images conditioned on text descriptions. We decompose the hard problem into more manageable sub-problems through a sketch-refinement process. The Stage-I GAN sketches the primitive shape and colors of the object based on the given text description, yielding Stage-I low-resolution images. The Stage-II GAN takes Stage-I results and text descriptions as inputs, and generates high-resolution images with photo-realistic details. It is able to rectify defects in Stage-I results and add compelling details with the refinement process. To improve the diversity of the synthesized images and stabilize the training of the conditional-GAN, we introduce a novel Conditioning Augmentation technique that encourages smoothness in the latent conditioning manifold. Extensive experiments and comparisons with state-of-the-arts on benchmark datasets demonstrate that the proposed method achieves significant improvements on generating photo-realistic images conditioned on text descriptions.

テキストから画像を生成する話で、単純なEncode情報+GANからさらに解像度を上げるためのGANを重ねて(Stackして)画像の鮮明度を上げるという話。


Current vector-space models of lexical semantics create a single “prototype” vector to represent the meaning of a word. However, due to lexical ambiguity, encoding word meaning with a single vector is problematic. This paper presents a method that uses clustering to produce multiple “sense-specific” vectors for each word. This approach provides a context-dependent vector representation of word meaning that naturally accommodates homonymy and polysemy. Experimental comparisons to human judgements of semantic similarity for both isolated words as well as words in sentential contexts demonstrate the superiority of this approach over both prototype and exemplar based vector-space models.

単語のあいまい性を考慮して、語義ごとのベクトル表現を生成することを提案している。具体的には、語義ごとのベクトルを生成するために、クラスタリング(movMF)を使う手法を提案している。評価については人間が判断した意味類似度と比較している。


We present two simple modifications to the models in the popular Word2Vec tool, in order to generate embeddings more suited to tasks involving syntax. The main issue with the original models is the fact that they are insensitive to word order. While order independence is useful for inducing semantic representations, this leads to suboptimal results when they are used to solve syntax-based problems. We show improvements in part-ofspeech tagging and dependency parsing using our proposed models.

品詞タグ付や係り受け解析のようなsyntaxが重要なタスクに適したword embeddingを生成する手法を提案。Mikolovらのモデルでは語順を無視していたのに対して、提案モデルではそのあたりを考慮。結果として品詞タグ付と係り受け解析の性能が向上した。比較相手はSENNA。


Predictive models deployed in the world may assign incorrect labels to instances with high confidence. Such errors or unknown unknowns are rooted in model incompleteness, and typically arise because of the mismatch between training data and the cases seen in the open world. As the models are blind to such errors, input from an oracle is needed to identify these failures. In this paper, we formulate and address the problem of optimizing the discovery of unknown unknowns of any predictive model under a fixed budget, which limits the number of times an oracle can be queried for true labels. We propose a model-agnostic methodology which uses feedback from an oracle to both identify unknown unknowns and to intelligently guide the discovery. We employ a two-phase approach which first organizes the data into multiple partitions based on instance similarity, and then utilizes an explore-exploit strategy for discovering unknown unknowns across these partitions. We demonstrate the efficacy of our framework by varying the underlying causes of unknown unknowns across various applications. To the best of our knowledge, this paper presents the first algorithmic approach to the problem of discovering unknown unknowns of predictive models.

Active Learningのようにラベル無しデータ(known unknowns)を選択するだけでなく、そもそもラベル自体を知らないデータ(unknown unknowns)についても、効率よくオラクルに問い合わせるための手法を提案している。例えば、白い猫画像と黒い犬画像ばかりを学習しているとき、テスト対象の白い犬画像を、白の特徴量に基いて"猫"として高い信頼度で誤判別してしまうところ、本手法によりそれを判別するためのデータの不完全性を検知することが可能となる。


Many Collaborative Filtering (CF) algorithms are item-based in the sense that they analyze item-item relations in order to produce item similarities. Recently, several works in the field of Natural Language Processing (NLP) suggested to learn a latent representation of words using neural embedding algorithms. Among them, the Skip-gram with Negative Sampling (SGNS), also known as word2vec, was shown to provide state-of-the-art results on various linguistics tasks. In this paper, we show that item-based CF can be cast in the same framework of neural word embedding. Inspired by SGNS, we describe a method we name item2vec for item-based CF that produces embedding for items in a latent space. The method is capable of inferring item-item relations even when user information is not available. We present experimental results that demonstrate the effectiveness of the item2vec method and show it is competitive with SVD.

Skip-gramをちょっと変えたものを協調フィルタリングに使った話。Skip-gramのようなモデルを使ってItemに対するベクトルを生成し、Item間の類似度から推薦する。SVDを用いた手法に匹敵する結果を得られた。特にunpopularなitemに関してはSVDより良い結果となった。


We describe an approach for unsupervised learning of a generic, distributed sentence encoder. Using the continuity of text from books, we train an encoder-decoder model that tries to reconstruct the surrounding sentences of an encoded passage. Sentences that share semantic and syntactic properties are thus mapped to similar vector representations. We next introduce a simple vocabulary expansion method to encode words that were not seen as part of training, allowing us to expand our vocabulary to a million words. After training our model, we extract and evaluate our vectors with linear models on 8 tasks: semantic relatedness, paraphrase detection, image-sentence ranking, question-type classification and 4 benchmark sentiment and subjectivity datasets. The end result is an off-the-shelf encoder that can produce highly generic sentence representations that are robust and perform well in practice. We will make our encoder publicly available.

encoder-decoderモデルを使って文の分散表現を学習しようという話。学習方法はSkip-gramに似ていて、ある文から周りの文を予測することを通じて文の分散表現を学習する。得られたベクトルについては8つのタスクで評価しており、まずまずの結果となっていた。


Many machine learning algorithms require the input to be represented as a fixed-length feature vector. When it comes to texts, one of the most common fixed-length features is bag-of-words. Despite their popularity, bag-of-words features have two major weaknesses: they lose the ordering of the words and they also ignore semantics of the words. For example, "powerful," "strong" and "Paris" are equally distant. In this paper, we propose Paragraph Vector, an unsupervised algorithm that learns fixed-length feature representations from variable-length pieces of texts, such as sentences, paragraphs, and documents. Our algorithm represents each document by a dense vector which is trained to predict words in the document. Its construction gives our algorithm the potential to overcome the weaknesses of bag-of-words models. Empirical results show that Paragraph Vectors outperform bag-of-words models as well as other techniques for text representations. Finally, we achieve new state-of-the-art results on several text classification and sentiment analysis tasks.

いわゆるDoc2vecの基となった論文。bag-of-wordsでは語順や単語の意味を無視するという弱点があったのでそれを克服するためにParagrapheベクトルを提案。評価分析と情報検索で評価したところ、state-of-the-artより良い結果となった。


This paper explores a simple and efficient baseline for text classification. Our experiments show that our fast text classifier fastText is often on par with deep learning classifiers in terms of accuracy, and many orders of magnitude faster for training and evaluation. We can train fastText on more than one billion words in less than ten minutes using a standard multicore~CPU, and classify half a million sentences among~312K classes in less than a minute.

fastTextを使ってテキスト分類をしてみた論文。具体的には、タグ予測と評価分析を行っている。accuracyでは深層学習モデルの分類器と同等で学習は数十倍速いという結果になった。


Continuous word representations, trained on large unlabeled corpora are useful for many natural language processing tasks. Many popular models to learn such representations ignore the morphology of words, by assigning a distinct vector to each word. This is a limitation, especially for morphologically rich languages with large vocabularies and many rare words. In this paper, we propose a new approach based on the skip-gram model, where each word is represented as a bag of character n-grams. A vector representation is associated to each character n-gram, words being represented as the sum of these representations. Our method is fast, allowing to train models on large corpus quickly. We evaluate the obtained word representations on five different languages, on word similarity and analogy tasks.

FastTextの論文。現在の単語表現を獲得するモデルの多くは単語の形態素を無視している。この論文ではこれら形態素を考慮するために各単語を文字ngramで表現し、それらのベクトル表現を学習している。その評価は単語類似度とアナロジータスクで行った。


Most contemporary approaches to instance segmentation use complex pipelines involving conditional random fields, recurrent neural networks, object proposals, or template matching schemes. In our paper, we present a simple yet powerful end-to-end convolutional neural network to tackle this task. Our approach combines intuitions from the classical watershed transform and modern deep learning to produce an energy map of the image where object instances are unambiguously represented as basins in the energy map. We then perform a cut at a single energy level to directly yield connected components corresponding to object instances. Our model more than doubles the performance of the state-of-the-art on the challenging Cityscapes Instance Level Segmentation task.

instance segmentationは複雑なpiplineで構成されたものが多かったが、本稿ではシンプルなネットワーク構成で当該タスクの実行を可能とした。watershedアルゴリズム（領域分割アルゴリズム）とDeep Learningモデルのあわせ技で、watershed transformも含めてend-to-endで学習を行う。RNNなどのiterationが必要なアルゴリズムを使用していないため、１画像中のobject数が多くても高速に推定が可能。従来のwatershed transformアルゴリズムとの違いとしては、各instanceのエネルギー分布の高さが大体同じになるように学習を行っている。入力はRGB。SoTA。


Attention-based neural encoder-decoder frameworks have been widely adopted for image captioning. Most methods force visual attention to be active for every generated word. However, the decoder likely requires little to no visual information from the image to predict non-visual words such as "the" and "of". Other words that may seem visual can often be predicted reliably just from the language model e.g., "sign" after "behind a red stop" or "phone" following "talking on a cell". In this paper, we propose a novel adaptive attention model with a visual sentinel. At each time step, our model decides whether to attend to the image (and if so, to which regions) or to the visual sentinel. The model decides whether to attend to the image and where, in order to extract meaningful information for sequential word generation. We test our method on the COCO image captioning 2015 challenge dataset and Flickr30K. Our approach sets the new state-of-the-art by a significant margin.

image caption generateタスクにおいて、attentionの際にwhereだけでなくwhenにも着目した論文。“the”や“of”はvisualizeさせる意味はないため、次の単語を予測する際にattentionを行う必要があるかどうかを判断するvisual sentinel(sentinel gate)をネットワークに追加。whenはsentinel gateが、 whereはspatial attentionが担う。Spatial Attention Modelの提案とvisual sentinelを構成要素として追加したAdaptive Attention Modelの提案を行っている。SoTA。


The move from hand-designed features to learned features in machine learning has been wildly successful. In spite of this, optimization algorithms are still designed by hand. In this paper we show how the design of an optimization algorithm can be cast as a learning problem, allowing the algorithm to learn to exploit structure in the problems of interest in an automatic way. Our learned algorithms, implemented by LSTMs, outperform generic, hand-designed competitors on the tasks for which they are trained, and also generalize well to new tasks with similar structure. We demonstrate this on a number of tasks, including simple convex problems, training neural networks, and styling images with neural art.

ニューラルネットの学習ってなんだかんだ手作業orルールベースだよね、ということで、「学習のさせ方」のノウハウを学習するという研究。ターゲットのネットワークの重みを、RNN(2層LSTM)を使って推定させる。AdamやRMSpropより優秀な結果。TensorFlow実装有り。


This paper studies the problem of embedding very large information networks into low-dimensional vector spaces, which is useful in many tasks such as visualization, node classification, and link prediction. Most existing graph embedding methods do not scale for real world information networks which usually contain millions of nodes. In this paper, we propose a novel network embedding method called the "LINE," which is suitable for arbitrary types of information networks: undirected, directed, and/or weighted. The method optimizes a carefully designed objective function that preserves both the local and global network structures. An edge-sampling algorithm is proposed that addresses the limitation of the classical stochastic gradient descent and improves both the effectiveness and the efficiency of the inference. Empirical experiments prove the effectiveness of the LINE on a variety of real-world information networks, including language networks, social networks, and citation networks. The algorithm is very efficient, which is able to learn the embedding of a network with millions of vertices and billions of edges in a few hours on a typical single machine. The source code of the LINE is available online.

大規模グラフを想定したノードの埋め込み表現を得る手法LINEの提案。ノードi,jの接続確率p(i,j)が埋め込みベクトルを用いてp(i,j)~exp((v_i,v_j))なるモデルを考え。最近接、第2最近接までの接続確率分布を学習し埋め込みベクトルを得る。negative samplingと非同期SGDアップデートで学習の高速化。共起語ネットワーク、filckr,youtube社会ネットワーク, citationネットワークで評価し既存手法(skip-gram,deepwalk含む)より高性能、高速。スケーラビリティーもよいので大規模グラフに適用化。 WWW '15


Recurrent neural networks are a powerful tool for modeling sequential data, but the dependence of each timestep's computation on the previous timestep's output limits parallelism and makes RNNs unwieldy for very long sequences. We introduce quasi-recurrent neural networks (QRNNs), an approach to neural sequence modeling that alternates convolutional layers, which apply in parallel across timesteps, and a minimalist recurrent pooling function that applies in parallel across channels. Despite lacking trainable recurrent layers, stacked QRNNs have better predictive accuracy than stacked LSTMs of the same hidden size. Due to their increased parallelism, they are up to 16 times faster at train and test time. Experiments on language modeling, sentiment classification, and character-level neural machine translation demonstrate these advantages and underline the viability of QRNNs as a basic building block for a variety of sequence tasks.

RNNでは1wordごとに処理するので並列処理できないし、前の隠れ層からの入力を受け続けることで隠れ層はいろんな単語の情報がミックスされた謎の何かになる。そこでCNNにより並列処理＋隠れ層を前回独立にキープして出力を計算するブロックを発明。その名はQRNN。Chainer実装有。


We present DeepWalk, a novel approach for learning latent representations of vertices in a network. These latent representations encode social relations in a continuous vector space, which is easily exploited by statistical models. DeepWalk generalizes recent advancements in language modeling and unsupervised feature learning (or deep learning) from sequences of words to graphs. DeepWalk uses local information obtained from truncated random walks to learn latent representations by treating walks as the equivalent of sentences. We demonstrate DeepWalk's latent representations on several multi-label network classification tasks for social networks such as BlogCatalog, Flickr, and YouTube. Our results show that DeepWalk outperforms challenging baselines which are allowed a global view of the network, especially in the presence of missing information. DeepWalk's representations can provide F_1 scores up to 10% higher than competing methods when labeled data is sparse. In some experiments, DeepWalk's representations are able to outperform all baseline methods while using 60% less training data. DeepWalk is also scalable. It is an online learning algorithm which builds useful incremental results, and is trivially parallelizable. These qualities make it suitable for a broad class of real world applications such as network classification, and anomaly detection.

与えられたグラフ上のノードの埋め込み表現を得る方法を提案。着目するノードを始点とするランダムウォークをコンテキストとして始点ノードを予測するskip-gramによって埋め込み表現を得る。大規模グラフをターゲットとしデータセットとしてブロガー、flickr、youtubeのソーシャルネットワークを使って評価。グラフ中のいくつかのノードを訓練データとして学習し、訓練データ以外のノードラベルを埋め込みベクトルを入力とするlogistic regressionで推定した結果を評価。(当時のまだDNN的手法が一般的でない)既存手法をうわまわる精度。 KDD 2014


Transferring knowledge from prior source tasks in solving a new target task can be useful in several learning applications. The application of transfer poses two serious challenges which have not been adequately addressed. First, the agent should be able to avoid negative transfer, which happens when the transfer hampers or slows down the learning instead of helping it. Second, the agent should be able to selectively transfer, which is the ability to select and transfer from different and multiple source tasks for different parts of the state space of the target task. We propose A2T (Attend, Adapt and Transfer), an attentive deep architecture which adapts and transfers from these source tasks. Our model is generic enough to effect transfer of either policies or value functions. Empirical evaluations on different learning algorithms show that A2T is an effective architecture for transfer by being able to avoid negative transfer while transferring selectively from multiple source tasks in the same domain.

ドメインが同じ転移学習を凸結合によって行う手法の提案。凸結合の係数を、どのタスクにどのくらい注目すべきかというスコアを出力するattention networkを使って求める。強化学習の方策や価値関数を転移学習させて実験した。転移させない学習と比べて高速に学習できたが、転移学習における他の結合方法との比較はないように見える。


A number of recent approaches to policy learning in 2D game domains have been successful going directly from raw input images to actions. However when employed in complex 3D environments, they typically suffer from challenges related to partial observability, combinatorial exploration spaces, path planning, and a scarcity of rewarding scenarios. Inspired from prior work in human cognition that indicates how humans employ a variety of semantic concepts and abstractions (object categories, localisation, etc.) to reason about the world, we build an agent-model that incorporates such abstractions into its policy-learning framework. We augment the raw image input to a Deep Q-Learning Network (DQN), by adding details of objects and structural elements encountered, along with the agent's localisation. The different components are automatically extracted and composed into a topological representation using on-the-fly object detection and 3D-scene reconstruction.We evaluate the efficacy of our approach in Doom, a 3D first-person combat game that exhibits a number of challenges discussed, and show that our augmented framework consistently learns better, more effective policies.

DQNを3次元に適用すると、次元が増えた分探索が疎になり学習が非常に難しい。そこで、SLAMによる自己位置推定と、Faster-RCNNによる物体検出で俯瞰図(=神の視点)を作って補助情報として入れてやる手法の提案。単純なDQNより学習の初速が速く、総報酬も2倍超となった。


Recent captioning models are limited in their ability to scale and describe concepts unseen in paired image-text corpora. We propose the Novel Object Captioner (NOC), a deep visual semantic captioning model that can describe a large number of object categories not present in existing image-caption datasets. Our model takes advantage of external sources -- labeled images from object recognition datasets, and semantic knowledge extracted from unannotated text. We propose minimizing a joint objective which can learn from these diverse data sources and leverage distributional semantic embeddings, enabling the model to generalize and describe novel objects outside of image-caption datasets. We demonstrate that our model exploits semantic information to generate captions for hundreds of object categories in the ImageNet object recognition dataset that are not observed in MSCOCO image-caption training data, as well as many categories that are observed very rarely. Both automatic evaluations and human judgements show that our model considerably outperforms prior work in being able to describe many more categories of objects.

Novel Object Captioningタスクで、(1) 画像とキャプションが対となっていないデータの活用/学習、(2) 複数ソースからのデータの活用、(3)pre-training済みのembedding空間の活用を行えるようにした。これにより、zero-shotなデータに対してもcaption生成が可能。手法としては、visual recognition network(画像のみのencode)、caption model(caption generation)、language model(sentence generation)を別々のデータソースで学習。そのパラメタを共有するというもの。従来手法と比較してzero-shotデータに対してMS COCOで10%、ImageNetで20%のF1を改善。


In this paper we advance the state-of-the-art for crowd counting in high density scenes by further exploring the idea of a fully convolutional crowd counting model introduced by (Zhang et al., 2016). Producing an accurate and robust crowd count estimator using computer vision techniques has attracted significant research interest in recent years. Applications for crowd counting systems exist in many diverse areas including city planning, retail, and of course general public safety. Developing a highly generalised counting model that can be deployed in any surveillance scenario with any camera perspective is the key objective for research in this area. Techniques developed in the past have generally performed poorly in highly congested scenes with several thousands of people in frame (Rodriguez et al., 2011). Our approach, influenced by the work of (Zhang et al., 2016), consists of the following contributions: (1) A training set augmentation scheme that minimises redundancy among training samples to improve model generalisation and overall counting performance; (2) a deep, single column, fully convolutional network (FCN) architecture; (3) a multi-scale averaging step during inference. The developed technique can analyse images of any resolution or aspect ratio and achieves state-of-the-art counting performance on the Shanghaitech Part B and UCF CC 50 datasets as well as competitive performance on Shanghaitech Part A.

FCNベースの人混みの中の人物数カウントモデルの提案。実際に提案しているのはaugmentation手法。入力画像のcropを行う際に4象限っぽく、4つにcropした画像を入力とし、入力データとして重なりをなくした。これによりoverfitのリスクを減らしている。また、test時の入力画像の大きさを50%downsamplingしても精度が大きく変わらず、計算速度は4倍早くなる事を指摘。SoTA。


Community based question answering services have arisen as a popular knowledge sharing pattern for netizens. With abundant interactions among users, individuals are capable of obtaining satisfactory information. However, it is not effective for users to attain answers within minutes. Users have to check the progress over time until the satisfying answers submitted. We address this problem as a user personalized satisfaction prediction task. Existing methods usually exploit manual feature selection. It is not desirable as it requires careful design and is labor intensive. In this paper, we settle this issue by developing a new multiple instance deep learning framework. Specifically, in our settings, each question follows a weakly supervised learning multiple instance learning assumption, where its obtained answers can be regarded as instance sets and we define the question resolved with at least one satisfactory answer. We thus design an efficient framework exploiting multiple instance learning property with deep learning to model the question answer pairs. Extensive experiments on large scale datasets from Stack Exchange demonstrate the feasibility of our proposed framework in predicting askers personalized satisfaction. Our framework can be extended to numerous applications such as UI satisfaction Prediction, multi armed bandit problem, expert finding and so on.

コミュニティ内での質問に対して、質問者が満足する適切な回答を素早く返す事モデルを作れないか、というのを研究した論文。深層学習とmultiple instance learning(弱教師あり学習)の枠組を手法として用いている。従来手法では特徴量をハンドメイキングしていたが、本手法では特徴量抽出は自動で行われるため、従来手法に比べ、性能だけでなく説明力も向上。


Recent advances in deep learning have shown exciting promise in filling large holes in natural images with semantically plausible and context aware details, impacting fundamental image manipulation tasks such as object removal. While these learning-based methods are significantly more effective in capturing high-level features than prior techniques, they can only handle very low-resolution inputs due to memory limitations and difficulty in training. Even for slightly larger images, the inpainted regions would appear blurry and unpleasant boundaries become visible. We propose a multi-scale neural patch synthesis approach based on joint optimization of image content and texture constraints, which not only preserves contextual structures but also produces high-frequency details by matching and adapting patches with the most similar mid-layer feature correlations of a deep classification network. We evaluate our method on the ImageNet and Paris Streetview datasets and achieved state-of-the-art inpainting accuracy. We show our approach produces sharper and more coherent results than prior methods, especially for high-resolution images.

画像の補完を、高解像度で行う手法の提案。単に画像全体だけでなく、周辺のテクスチャのパターンとの差異も見ることで従来手法より良い(loss的には微減という感じだが)補完を実現。


An important aspect of developing conversational agents is to give a bot the ability to improve through communicating with humans and to learn from the mistakes that it makes. Most research has focused on learning from fixed training sets of labeled data rather than interacting with a dialogue partner in an online fashion. In this paper we explore this direction in a reinforcement learning setting where the bot improves its question-answering ability from feedback a teacher gives following its generated responses. We build a simulator that tests various aspects of such learning in a synthetic environment, and introduce models that work in this regime. Finally, real experiments with Mechanical Turk validate the approach.

対話中の学習を可能にするため、Memory Networkと強化学習を組み合わせる手法の提案。正しい回答「だけ」を模倣するよう学習するモデル(RBI)と、返答から報酬を推定するモデル(FP)を検証。双方有効なことを確認。


We introduce the multiresolution recurrent neural network, which extends the sequence-to-sequence framework to model natural language generation as two parallel discrete stochastic processes: a sequence of high-level coarse tokens, and a sequence of natural language tokens. There are many ways to estimate or learn the high-level coarse tokens, but we argue that a simple extraction procedure is sufficient to capture a wealth of high-level discourse semantics. Such procedure allows training the multiresolution recurrent neural network by maximizing the exact joint log-likelihood over both sequences. In contrast to the standard log- likelihood objective w.r.t. natural language tokens (word perplexity), optimizing the joint log-likelihood biases the model towards modeling high-level abstractions. We apply the proposed model to the task of dialogue response generation in two challenging domains: the Ubuntu technical support domain, and Twitter conversations. On Ubuntu, the model outperforms competing approaches by a substantial margin, achieving state-of-the-art results according to both automatic evaluation metrics and a human evaluation study. On Twitter, the model appears to generate more relevant and on-topic responses according to automatic evaluation metrics. Finally, our experiments demonstrate that the proposed model is more adept at overcoming the sparsity of natural language and is better able to capture long-term structure.

RNNはLSTMとかの構造をいじるのが主流になっているけれど、入力を工夫した方がいいんじゃない？ということで、入力列を通常の文字列とそこから「粗い(Coarse)」情報を抽出したものとを並列で入力するモデルを提案している(=multi resolution)。


Conversational modeling is an important task in natural language understanding and machine intelligence. Although previous approaches exist, they are often restricted to specific domains (e.g., booking an airline ticket) and require hand-crafted rules. In this paper, we present a simple approach for this task which uses the recently proposed sequence to sequence framework. Our model converses by predicting the next sentence given the previous sentence or sentences in a conversation. The strength of our model is that it can be trained end-to-end and thus requires much fewer hand-crafted rules. We find that this straightforward model can generate simple conversations given a large conversational training dataset. Our preliminary results suggest that, despite optimizing the wrong objective function, the model is able to converse well. It is able extract knowledge from both a domain specific dataset, and from a large, noisy, and general domain dataset of movie subtitles. On a domain-specific IT helpdesk dataset, the model can find a solution to a technical problem via conversations. On a noisy open-domain movie transcript dataset, the model can perform simple forms of common sense reasoning. As expected, we also find that the lack of consistency is a common failure mode of our model.

sequence2sequenceモデルを対話に使ったシンプルなモデルで対話を実現した。評価についてはある質問に対する答え方を既存の対話システム(CleverBot)と比較することで行った。提案モデルでは200の質問の内97が好ましく、CleverBotは200の内60が好ましいという結果となった。


In this work, we explore the problem of generating fantastic special-effects for the typography. It is quite challenging due to the model diversities to illustrate varied text effects for different characters. To address this issue, our key idea is to exploit the analytics on the high regularity of the spatial distribution for text effects to guide the synthesis process. Specifically, we characterize the stylized patches by their normalized positions and the optimal scales to depict their style elements. Our method first estimates these two features and derives their correlation statistically. They are then converted into soft constraints for texture transfer to accomplish adaptive multi-scale texture synthesis and to make style element distribution uniform. It allows our algorithm to produce artistic typography that fits for both local texture patterns and the global spatial distribution in the example. Experimental results demonstrate the superiority of our method for various text effects over conventional style transfer methods. In addition, we validate the effectiveness of our algorithm with extensive artistic typography library generation.

StyleTransferをテキストに応用し、かっこいいフォントを作るというもの。フォントにかかるエフェクトは細かく、単純にStyleTransferをかけても上手くいかない。そこで、既存のフォントの特性を数学的に定義していって、それを適用していくという手法を取っている。


Machine comprehension (MC), answering a query about a given context paragraph, requires modeling complex interactions between the context and the query. Recently, attention mechanisms have been successfully extended to MC. Typically these methods use attention to focus on a small portion of the context and summarize it with a fixed-size vector, couple attentions temporally, and/or often form a uni-directional attention. In this paper we introduce the Bi-Directional Attention Flow (BIDAF) network, a multi-stage hierarchical process that represents the context at different levels of granularity and uses bi-directional attention flow mechanism to obtain a query-aware context representation without early summarization. Our experimental evaluations show that our model achieves the state-of-the-art results in Stanford Question Answering Dataset (SQuAD) and CNN/DailyMail cloze test.

SQuADで最高精度を更新した論文。文字・単語・フレーズ、その上に文書/クエリの関連(Attention)、さらにそれらの関連、出力、という階層型のモデル。Attentionとencodeの役割分担が肝とのこと。


To enhance developer productivity, all modern integrated development environments (IDEs) include code suggestion functionality that proposes likely next tokens at the cursor. While current IDEs work well for statically-typed languages, their reliance on type annotations means that they do not provide the same level of support for dynamic programming languages as for statically-typed languages. Moreover, suggestion engines in modern IDEs do not propose expressions or multi-statement idiomatic code. Recent work has shown that language models can improve code suggestion systems by learning from software repositories. This paper introduces a neural language model with a sparse pointer network aimed at capturing very long-range dependencies. We release a large-scale code suggestion corpus of 41M lines of Python code crawled from GitHub. On this corpus, we found standard neural language models to perform well at suggesting local phenomena, but struggle to refer to identifiers that are introduced many tokens in the past. By augmenting a neural language model with a pointer network specialized in referring to predefined classes of identifiers, we obtain a much lower perplexity and a 5 percentage points increase in accuracy for code suggestion compared to an LSTM baseline. In fact, this increase in code suggestion accuracy is due to a 13 times more accurate prediction of identifiers. Furthermore, a qualitative analysis shows this model indeed captures interesting long-range dependencies, like referring to a class member defined over 60 tokens in the past.

Pythonのコード補完をRNN言語モデルで行う話。Attentionをclassやfunctionの宣言にはることで、予測精度を上げている。なるべく品質の高いPythonコードを使用するため、GitHubでStar100以上のリポジトリ、かつfolkが多いものを学習データとして使用。


The use of Convolutional Neural Networks (CNN) in natural image classification systems has produced very impressive results. Combined with the inherent nature of medical images that make them ideal for deep-learning, further application of such systems to medical image classification holds much promise. However, the usefulness and potential impact of such a system can be completely negated if it does not reach a target accuracy. In this paper, we present a study on determining the optimum size of the training data set necessary to achieve high classification accuracy with low variance in medical image classification systems. The CNN was applied to classify axial Computed Tomography (CT) images into six anatomical classes. We trained the CNN using six different sizes of training data set (5, 10, 20, 50, 100, and 200) and then tested the resulting system with a total of 6000 CT images. All images were acquired from the Massachusetts General Hospital (MGH) Picture Archiving and Communication System (PACS). Using this data, we employ the learning curve approach to predict classification accuracy at a given training sample size. Our research will present a general methodology for determining the training data set size necessary to achieve a certain target classification accuracy that can be easily applied to other problems within such systems.

CT画像の異常状態の分類器において何枚の画像が必要か、を実験した論文。結果としては100枚以上からは精度の向上はあまり見られなかった。使用したNetworkはGoogLeNet。


Unsupervised word representations are very useful in NLP tasks both as inputs to learning algorithms and as extra word features in NLP systems. However, most of these models are built with only local context and one representation per word. This is problematic because words are often polysemous and global context can also provide useful information for learning word meanings. We present a new neural network architecture which 1) learns word embeddings that better capture the semantics of words by incorporating both local and global document context, and 2) accounts for homonymy and polysemy by learning multiple embeddings per word. We introduce a new dataset with human judgments on pairs of words in sentential context, and evaluate our model on it, showing that our model outperforms competitive baselines and other neural language models.

GloVeの基となった論文(多分)。従来の単語表現にはlocal contextしか使っていない、一つの単語につき一つの表現という問題があった。提案モデルでは、local contextとglobal contextを組み合わせて使用し、また一単語につき複数のembeddingsを学習することで性能向上を図った。


We present results that show it is possible to build a competitive, greatly simplified, large vocabulary continuous speech recognition system with whole words as acoustic units. We model the output vocabulary of about 100,000 words directly using deep bi-directional LSTM RNNs with CTC loss. The model is trained on 125,000 hours of semi-supervised acoustic training data, which enables us to alleviate the data sparsity problem for word models. We show that the CTC word models work very well as an end-to-end all-neural speech recognition model without the use of traditional context-dependent sub-word phone units that require a pronunciation lexicon, and without any language model removing the need to decode. We demonstrate that the CTC word models perform better than a strong, more complex, state-of-the-art baseline with sub-word units.

音声認識システムにおいて語彙数が多いときは単語より小さい音声単位を出力せざるを得ないと考えられていた。しかしこの論文では１０万語彙で単語を直接出力するシステムが可能であることを示した。そのためモデルは従来よりも単純化できて、双方向LSTMとCTS lossを使いend-to-end trainingされている。12万5千時間のsemi-supervisedなデータセットが単語単位モデルにおけるデータ不足の問題を解消した。


We present a comprehensive study of evaluation methods for unsupervised embedding techniques that obtain meaningful representations of words from text. Different evaluations result in different orderings of embedding methods, calling into question the common assumption that there is one single optimal vector representation. We present new evaluation techniques that directly compare embeddings with respect to specific queries. These methods reduce bias, provide greater insight, and allow us to solicit data-driven relevance judgments rapidly and accurately through crowdsourcing.

word embeddingの評価手法に関するサーベイ論文。既存の評価手法を解説しつつ、クラウドソーシングを用いた新しい評価手法を提案し、比較を行っている。結果として、既存の評価手法と新しい評価手法の結果が類似していることがわかった。


While recent deep neural networks have achieved a promising performance on object recognition, they rely implicitly on the visual contents of the whole image. In this paper, we train deep neural net- works on the foreground (object) and background (context) regions of images respectively. Consider- ing human recognition in the same situations, net- works trained on the pure background without ob- jects achieves highly reasonable recognition performance that beats humans by a large margin if only given context. However, humans still outperform networks with pure object available, which indicates networks and human beings have different mechanisms in understanding an image. Furthermore, we straightforwardly combine multiple trained networks to explore different visual cues learned by different networks. Experiments show that useful visual hints can be explicitly learned separately and then combined to achieve higher performance, which verifies the advantages of the proposed framework.

object recognitionでは人間がAlexnetに勝つ。この論文は、objectをマスクして背景だけにしてobjectを推測させると、人間よりAlexnetの方が上ということを確かめた。またobjectだけにすると人間が勝つことも。deep CNNは認識において背景情報をやはりかなり利用していることが分かる。


We propose a new self-supervised CNN pre-training technique based on a novel auxiliary task called "odd-one-out learning". In this task, the machine is asked to identify the unrelated or odd element from a set of otherwise related elements. We apply this technique to self-supervised video representation learning where we sample subsequences from videos and ask the network to learn to predict the odd video subsequence. The odd video subsequence is sampled such that it has wrong temporal order of frames while the even ones have the correct temporal order. Therefore, to generate a odd-one-out question no manual annotation is required. Our learning machine is implemented as multi-stream convolutional neural network, which is learned end-to-end. Using odd-one-out networks, we learn temporal representations for videos that generalizes to other related tasks such as action recognition. On action classification, our method obtains 60.3\% on the UCF101 dataset using only UCF101 data for training which is approximately 10% better than current state-of-the-art self-supervised learning methods. Similarly, on HMDB51 dataset we outperform self-supervised state-of-the art methods by 12.7% on action classification task.

時間順序をひっくり返したビデオクリップをひっくり返していないものと区別させるタスクで学習させるのでアノーテーション不要。UCF101データセットでの（アノーテーションありの）行動分類タスクにこのself-supervisedなtrainingを併用した場合、およそ10%のperformace向上をするらしい。


Recent methods for learning vector space representations of words have succeeded in capturing fine-grained semantic and syntactic regularities using vector arithmetic, but the origin of these regularities has remained opaque. We analyze and make explicit the model properties needed for such regularities to emerge in word vectors. The result is a new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods. Our model efficiently leverages statistical information by training only on the nonzero elements in a word-word cooccurrence matrix, rather than on the entire sparse matrix or on individual context windows in a large corpus. The model produces a vector space with meaningful substructure, as evidenced by its performance of 75% on a recent word analogy task. It also outperforms related models on similarity tasks and named entity recognition.

単語ベクトルを学習するためのモデルであるGloVeの論文。グローバルな行列分解のモデルとlocal context windowのモデルを組み合わせてよい単語ベクトルを学習した。評価はアナロジータスクとNERで行っている。


Continuous space language models have recently demonstrated outstanding results across a variety of tasks. In this paper, we examine the vector-space word representations that are implicitly learned by the input-layer weights. We find that these representations are surprisingly good at capturing syntactic and semantic regularities in language, and that each relationship is characterized by a relation-specific vector offset. This allows vector-oriented reasoning based on the offsets between words. For example, the male/female relationship is automatically learned, and with the induced vector representations, “King - Man + Woman” results in a vector very close to “Queen.” We demonstrate that the word vectors capture syntactic regularities by means of syntactic analogy questions (provided with this paper), and are able to correctly answer almost 40% of the questions. We demonstrate that the word vectors capture semantic regularities by using the vector offset method to answer SemEval-2012 Task 2 questions. Remarkably, this method outperforms the best previous systems.

word2vecの基となった論文の一つ。#59が基になっている。おそらくベクトルの足し引きで意味のある結果を得られた初めての論文。RNNLMで言語モデルを学習させた時の入力層の重みが良い単語表現だったという話。syntacticとsemanticな単語の関係について評価を行っており、従来手法(LSAと2つの先行論文の手法)より良い結果だった。


Deep reinforcement learning (RL) can acquire complex behaviors from low-level inputs, such as images. However, real-world applications of such methods require generalizing to the vast variability of the real world. Deep networks are known to achieve remarkable generalization when provided with massive amounts of labeled data, but can we provide this breadth of experience to an RL agent, such as a robot? The robot might continuously learn as it explores the world around it, even while it is deployed and performing useful tasks. However, this learning requires access to a reward function, to tell the agent whether it is succeeding or failing at its task. Such reward functions are often hard to measure in the real world, especially in domains such as robotics and dialog systems, where the reward could depend on the unknown positions of objects or the emotional state of the user. On the other hand, it is often quite practical to provide the agent with reward functions in a limited set of situations, such as when a human supervisor is present, or in a controlled laboratory setting. Can we make use of this limited supervision, and still benefit from the breadth of experience an agent might collect in the unstructured real world? In this paper, we formalize this problem setting as semi-supervised reinforcement learning (SSRL), where the reward function can only be evaluated in a set of “labeled” MDPs, and the agent must generalize its behavior to the wide range of states it might encounter in a set of “unlabeled” MDPs, by using experience from both settings. Our proposed method infers the task objective in the unlabeled MDPs through an algorithm that resembles inverse RL, using the agent’s own prior experience in the labeled MDPs as a kind of demonstration of optimal behavior. We evaluate our method on challenging, continuous control tasks that require control directly from images, and show that our approach can improve the generalization of a learned deep neural network policy by using experience for which no reward function is available. We also show that our method outperforms direct supervised learning of the reward.

報酬ありMDPと報酬なしMDP、両方使い強化学習するsemi-supervised reinforcement learning(半教師あり強化学習)を考え、その学習アルゴリズムsemi-supervised skill generalization(S3G)を提案。報酬ありなしの情報を両方使い汎化できていることが確かめられた。


Class imbalance is a common problem in the case of real-world object detection and classification tasks. Data of some classes is abundant making them an over-represented majority, and data of other classes is scarce, making them an under-represented minority. This imbalance makes it challenging for a classifier to appropriately learn the discriminating boundaries of the majority and minority classes. In this work, we propose a cost sensitive deep neural network which can automatically learn robust feature representations for both the majority and minority classes. During training, our learning procedure jointly optimizes the class dependent costs and the neural network parameters. The proposed approach is applicable to both binary and multi-class problems without any modification. Moreover, as opposed to data level approaches, we do not alter the original data distribution which results in a lower computational cost during the training process. We report the results of our experiments on six major image classification datasets and show that the proposed approach significantly outperforms the baseline algorithms. Comparisons with popular data sampling techniques and cost sensitive classifiers demonstrate the superior performance of our proposed method.

不均衡データに対して頑健に学習（特徴量抽出）を行うために、Cost-Sensitive Learning手法をnetworkの中に組み込み、自動でclass依存性のcost matrixを学習しようという提案。cost matrix用のlossは混合分布をベースとて算出、epoch毎に1度更新。


In this work we propose a novel interpretation of residual networks showing that they can be seen as a collection of many paths of differing length. Moreover, residual networks seem to enable very deep networks by leveraging only the short paths during training. To support this observation, we rewrite residual networks as an explicit collection of paths. Unlike traditional models, paths through residual networks vary in length. Further, a lesion study reveals that these paths show ensemble-like behavior in the sense that they do not strongly depend on each other. Finally, and most surprising, most paths are shorter than one might expect, and only the short paths are needed during training, as longer paths do not contribute any gradient. For example, most of the gradient in a residual network with 110 layers comes from paths that are only 10-34 layers deep. Our results reveal one of the key characteristics that seem to enable the training of very deep networks: Residual networks avoid the vanishing gradient problem by introducing short paths which can carry gradient throughout the extent of very deep networks.

ResNetの学習の振る舞いを調査した研究。勾配消失が何故防げているのかを解き明かしている。 ultra-deepなシングルネットワークだが、実は複数のネットワークが並行し、上層で結合しているように（アンサンブルのように）振る舞っている。また、gradientとして貢献しているのはshort path（勾配の伝搬を上手く行っているのがshort path）。deep pathはtraining時は重要ではない。という解釈がされている。


We present a simple, highly modularized network architecture for image classification. Our network is constructed by repeating a building block that aggregates a set of transformations with the same topology. Our simple design results in a homogeneous, multi-branch architecture that has only a few hyper-parameters to set. This strategy exposes a new dimension, which we call "cardinality" (the size of the set of transformations), as an essential factor in addition to the dimensions of depth and width. On the ImageNet-1K dataset, we empirically show that even under the restricted condition of maintaining complexity, increasing cardinality is able to improve classification accuracy. Moreover, increasing cardinality is more effective than going deeper or wider when we increase the capacity. Our models, named ResNeXt, are the foundations of our entry to the ILSVRC 2016 classification task in which we secured 2nd place. We further investigate ResNeXt on an ImageNet-5K set and the COCO detection set, also showing better results than its ResNet counterpart. The code and models are publicly available online.

ResNetの拡張版のResNeXtを提案。block内にinceptionモデルような構造を持たせた、“cardinality”という構造に変更。単純に層を深く、広くするよりも効果的にaccuracyの向上が行える。パラメタ数や時間計算量も通常のResNetと同程度。


We propose a technique for making Convolutional Neural Network (CNN)-based models more transparent by visualizing input regions that are 'important' for predictions -- or visual explanations. Our approach, called Gradient-weighted Class Activation Mapping (Grad-CAM), uses class-specific gradient information to localize important regions. These localizations are combined with existing pixel-space visualizations to create a novel high-resolution and class-discriminative visualization called Guided Grad-CAM. These methods help better understand CNN-based models, including image captioning and visual question answering (VQA) models. We evaluate our visual explanations by measuring their ability to discriminate between classes, to inspire trust in humans, and their correlation with occlusion maps. Grad-CAM provides a new way to understand CNN-based models. We have released code, an online demo hosted on CloudCV, and a full version of this extended abstract.

CNNモデルの出力結果に対する可視化手法であるGrad-CAMの提案。入力画像に対する注目領域の可視化を行う。image captioningやvisual question answering(VQA)モデルにも適用可能。 行われている操作は以下のとおり。 (1) guided backpropagationで得られたサリエンシーマップを作成 (2) target categoryを1、それ以外を0としてBPしたgradとfeature mapの値の積からblue heatmapを作成 (3) (1)と(2)の積をして、最終的なサリエンシーマップを作成


Deep Neural Networks often require good regularizers to generalize well. Dropout is one such regularizer that is widely used among Deep Learning practitioners. Recent work has shown that Dropout can also be viewed as performing Approximate Bayesian Inference over the network parameters. In this work, we generalize this notion and introduce a rich family of regularizers which we call Generalized Dropout. One set of methods in this family, called Dropout++, is a version of Dropout with trainable parameters. Classical Dropout emerges as a special case of this method. Another member of this family selects the width of neural network layers. Experiments show that these methods help in improving generalization performance over Dropout.

ニューラルネットの重みをベイズ推定するBNNは、すべての重みが互いに独立なのは過程として無理がある。そこで、重みの相関を表現するゲートを設けるという話。ノードの接続を調整するという面でDropOutの拡張、Dropout++と命名。


A goal of statistical language modeling is to learn the joint probability function of sequences of words in a language. This is intrinsically difficult because of the curse of dimensionality: a word sequence on which the model will be tested is likely to be different from all the word sequences seen during training. Traditional but very successful approaches based on n-grams obtain generalization by concatenating very short overlapping sequences seen in the training set. We propose to fight the curse of dimensionality by learning a distributed representation for words which allows each training sentence to inform the model about an exponential number of semantically neighboring sentences. The model learns simultaneously (1) a distributed representation for each word along with (2) the probability function for word sequences, expressed in terms of these representations. Generalization is obtained because a sequence of words that has never been seen before gets high probability if it is made of words that are similar (in the sense of having a nearby representation) to words forming an already seen sentence. Training such large models (with millions of parameters) within a reasonable time is itself a significant challenge. We report on experiments using neural networks for the probability function, showing on two text corpora that the proposed approach significantly improves on state-of-the-art n-gram models, and that the proposed approach allows to take advantage of longer contexts.

Word2vecの基となった論文の基となった論文。ニューラルネットワークを使った言語モデル(NNLM)を作る話。NNLMでは各単語の分散表現と単語列の確率を同時に学習することができる。単語の分散表現を使うことで、以前に現れたことのない単語列でもそれまでに現れた単語列と似た単語列で構成されていれば高い確率を付与することができるようになった。その結果として当時(2003年)のstate-of-the-artよりパープレキシティで10%〜20%の改善を行えた。ただ課題として計算速度の向上が挙げられてる。


We propose a method for finding alternate features missing in the Lasso optimal solution. In ordinary Lasso problem, one global optimum is obtained and the resulting features are interpreted as task-relevant features. However, this can overlook possibly relevant features not selected by the Lasso. With the proposed method, we can provide not only the Lasso optimal solution but also possible alternate features to the Lasso solution. We show that such alternate features can be computed efficiently by avoiding redundant computations. We also demonstrate how the proposed method works in the 20 newsgroup data, which shows that reasonable features are found as alternate features.

Lasso最適解では無視されてしまう、モデルの結果の性能に影響を与える重要な変数を抽出する手法の提案。


We investigate conditional adversarial networks as a general-purpose solution to image-to-image translation problems. These networks not only learn the mapping from input image to output image, but also learn a loss function to train this mapping. This makes it possible to apply the same generic approach to problems that traditionally would require very different loss formulations. We demonstrate that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks. As a community, we no longer hand-engineer our mapping functions, and this work suggests we can achieve reasonable results without hand-engineering our loss functions either.

最近流行のconditional GAN(条件指定の画像生成)。ある画像(イラスト)から画像(写真)を作り出す、画像翻訳ともいえるもの。特徴量比較でなく、GANの仕組みで「ペアかどうか」を真偽判定させる形で学習。少ないデータ(数百件・数時間 on GPU)でも学習可能なことを確認


Speech is one of the most effective ways of communication among humans. Even though audio is the most common way of transmitting speech, very important information can be found in other modalities, such as vision. Vision is particularly useful when the acoustic signal is corrupted. Multi-modal speech recognition however has not yet found wide-spread use, mostly because the temporal alignment and fusion of the different information sources is challenging. This paper presents an end-to-end audiovisual speech recognizer (AVSR), based on recurrent neural networks (RNN) with a connectionist temporal classification (CTC) loss function. CTC creates sparse "peaky" output activations, and we analyze the differences in the alignments of output targets (phonemes or visemes) between audio-only, video-only, and audio-visual feature representations. We present the first such experiments on the large vocabulary IBM ViaVoice database, which outperform previously published approaches on phone accuracy in clean and noisy conditions.

LipNetに先を越された感はあるが、音声認識に画像特徴量を組み合わせる試み。ノイズありの環境では、あらかじめノイズありの音声で学習＋口のあたりの画像特徴量を併用するのが良い結果になるとの結果。ノイズなしで学習させた場合、画像を組み合わせても精度が出なくなるのは重要な示唆。


We introduce a simple permutation equivariant layer for deep learning with set structure.This type of layer, obtained by parameter-sharing, has a simple implementation and linear-time complexity in the size of each set. We use deep permutation-invariant networks to perform point-could classification and MNIST-digit summation, where in both cases the output is invariant to permutations of the input. In a semi-supervised setting, where the goal is make predictions for each instance within a set, we demonstrate the usefulness of this type of layer in set-outlier detection as well as semi-supervised learning with clustering side-information.

データセットの中の各データを「点」と考えると、データセットは各点を関連付ける「構造」(点をつなぐ構造=グラフ構造)を持っていると考えることができる。この構造として何パターンかのシンプルな定義を行い、データセットへ適用してみることで「構造からの外れ値」の検出などを行っている。


The recently introduced continuous Skip-gram model is an efficient method for learning high-quality distributed vector representations that capture a large number of precise syntactic and semantic word relationships. In this paper we present several extensions that improve both the quality of the vectors and the training speed. By subsampling of the frequent words we obtain significant speedup and also learn more regular word representations. We also describe a simple alternative to the hierarchical softmax called negative sampling. An inherent limitation of word representations is their indifference to word order and their inability to represent idiomatic phrases. For example, the meanings of "Canada" and "Air" cannot be easily combined to obtain "Air Canada". Motivated by this example, we present a simple method for finding phrases in text, and show that learning good vector representations for millions of phrases is possible.

目的はskip-gramの高速化とより良いword vectorを得ること。またフレーズに対する学習を行うことも目的としている。高速化のために階層的ソフトマックス、ネガティブサンプリング、サブサンプリングを用いている。結果として、高速化しつつ良いword vectorを得られている。skip-gramに初めてネガティブサンプリングを使った論文のようだが説明が簡素なので、ネガティブサンプリングについて理解するには#41のほうが良い。


We propose two novel model architectures for computing continuous vector representations of words from very large data sets. The quality of these representations is measured in a word similarity task, and the results are compared to the previously best performing techniques based on different types of neural networks. We observe large improvements in accuracy at much lower computational cost, i.e. it takes less than a day to learn high quality word vectors from a 1.6 billion words data set. Furthermore, we show that these vectors provide state-of-the-art performance on our test set for measuring syntactic and semantic word similarities.

Word2vecの本家の論文。モデルの詳細についてはほとんど記述されていないし目的ともしていない。それより、既存モデル(NNLM、RNNLM)と提案モデル(Skip-Gram、CBOW)について計算量や性能の比較を行っている。結果として、既存手法より性能が高く計算量は小さかった。また、ベクトルの足し引きで興味深い結果(ex: vector("biggest")-vector("big")+vector("small")=vector("smallest"))を得られることがわかった。


Deep reinforcement learning agents have achieved state-of-the-art results by directly maximising cumulative reward. However, environments contain a much wider variety of possible training signals. In this paper, we introduce an agent that also maximises many other pseudo-reward functions simultaneously by reinforcement learning. All of these tasks share a common representation that, like unsupervised learning, continues to develop in the absence of extrinsic rewards. We also introduce a novel mechanism for focusing this representation upon extrinsic rewards, so that learning can rapidly adapt to the most relevant aspects of the actual task. Our agent significantly outperforms the previous state-of-the-art on Atari, averaging 880\% expert human performance, and a challenging suite of first-person, three-dimensional \emph{Labyrinth} tasks leading to a mean speedup in learning of 10× and averaging 87\% expert human performance on Labyrinth.

教師なしの補助タスクを同時に行う強化学習の手法UNsupervised REinforcement and Auxiliary Learning (UNREAL)を提案。画像入力3D迷路で従来手法に対し10倍の学習速度、人間の87%のスコア、Atariで人間の9倍のスコア。


The goal of this work is to recognise phrases and sentences being spoken by a talking face, with or without the audio. Unlike previous works that have focussed on recognising a limited number of words or phrases, we tackle lip reading as an open-world problem - unconstrained natural language sentences, and in the wild videos. Our key contributions are: (1) a 'Watch, Listen, Attend and Spell' (WLAS) network that learns to transcribe videos of mouth motion to characters; (2) a curriculum learning strategy to accelerate training and to reduce overfitting; (3) a 'Lip Reading Sentences' (LRS) dataset for visual speech recognition, consisting of over 100,000 natural sentences from British television. The WLAS model trained on the LRS dataset surpasses the performance of all previous work on standard lip reading benchmark datasets, often by a significant margin. This lip reading performance beats a professional lip reader on videos from BBC television, and we also demonstrate that visual information helps to improve speech recognition performance even when the audio is available.

DNNによる読唇術の続編。画像に加え、音声を用い(MFCCをLSTMでencode)、これらにAttentionを張った文字ベースのLSTM(Attend and Spell)で挑戦。※音声は使わない版も検証。BBCニュースの映像で検証したところ、プロを上回ることに成功。


Part of the appeal of Visual Question Answering (VQA) is its promise to answer new questions about previously unseen images. Most current methods demand training questions that illustrate every possible concept, and will therefore never achieve this capability, since the volume of required training data would be prohibitive. Answering general questions about images requires methods capable of Zero-Shot VQA, that is, methods able to answer questions beyond the scope of the training questions. We propose a new evaluation protocol for VQA methods which measures their ability to perform Zero-Shot VQA, and in doing so highlights significant practical deficiencies of current approaches, some of which are masked by the biases in current datasets. We propose and evaluate several strategies for achieving Zero-Shot VQA, including methods based on pretrained word embeddings, object classifiers with semantic embeddings, and test-time retrieval of example images. Our extensive experiments are intended to serve as baselines for Zero-Shot VQA, and they also achieve state-of-the-art performance in the standard VQA evaluation setting.

画像を見て質問に答えるタスクでは、学習した画像についてだけ答えられる、良くある答え(「2つ」とか)を多めに繰り出して精度が上がっているなど明らかな過適合が見られた。そこで真実見たことない画像(Zero-Shot)に回答可能かをテストするためのデータとベースラインモデルの提案


We introduce recurrent neural network grammars, probabilistic models of sentences with explicit phrase structure. We explain efficient inference procedures that allow application to both parsing and language modeling. Experiments show that they provide better parsing in English than any single previously published supervised generative model and better language modeling than state-of-the-art sequential RNNs in English and Chinese.

RNNを発展させたRecurrnt Neural Network Grammars（RNND）を用いて 構文解析、文生成を行った。


The ability to track a moving vehicle is of crucial importance in numerous applications. The task has often been approached by the importance sampling technique of particle filters due to its ability to model non-linear and non-Gaussian dynamics, of which a vehicle travelling on a road network is a good example. Particle filters perform poorly when observations are highly informative. In this paper, we address this problem by proposing particle filters that sample around the most recent observation. The proposal leads to an order of magnitude improvement in accuracy and efficiency over conventional particle filters, especially when observations are infrequent but low-noise.

粒子フィルタは移動物体の位置推定問題で多用されている。 本手法は従来の粒子フィルタの改良版。位置推定プロセスにおける直近の観測を元に提案分布を生成するのが従来手法との大きな違い。これにより、粒子の使用がより効率的になる。特に、観測ノイズに関連した遷移ノイズが高い状況で効果的。従来手法の性能を大きく改善。


Previous research has shown that computation of convolution in the frequency domain provides a significant speedup versus traditional convolution network implementations. However, this performance increase comes at the expense of repeatedly computing the transform and its inverse in order to apply other network operations such as activation, pooling, and dropout. We show, mathematically, how convolution and activation can both be implemented in the frequency domain using either the Fourier or Laplace transformation. The main contributions are a description of spectral activation under the Fourier transform and a further description of an efficient algorithm for computing both convolution and activation under the Laplace transform. By computing both the convolution and activation functions in the frequency domain, we can reduce the number of transforms required, as well as reducing overall complexity. Our description of a spectral activation function, together with existing spectral analogs of other network functions may then be used to compose a fully spectral implementation of a convolution network.

ラプラス変換やフーリエ変換を応用した計算量削減手法を、さらに効率的に改良した手法を提案。 従来はF->C->F^{-1}->A->F->C->F^{-1}->A、というように活性化関数の前に逆変換が必要であったが、提案手法ではF->C->A->F->C->A->F^{-1}という操作が可能となり、空間/時間計算量ともに削減。 従来はスペクトル表現では有限空間がサポートされていなかったために起きていたが、spectral activation functionによって有限空間でのactivationが可能となった。


In recent years deep reinforcement learning (RL) systems have attained superhuman performance in a number of challenging task domains. However, a major limitation of such applications is their demand for massive amounts of training data. A critical present objective is thus to develop deep RL methods that can adapt rapidly to new tasks. In the present work we introduce a novel approach to this challenge, which we refer to as deep meta-reinforcement learning. Previous work has shown that recurrent networks can support meta-learning in a fully supervised context. We extend this approach to the RL setting. What emerges is a system that is trained using one RL algorithm, but whose recurrent dynamics implement a second, quite separate RL procedure. This second, learned RL algorithm can differ from the original one in arbitrary ways. Importantly, because it is learned, it is configured to exploit structure in the training domain. We unpack these points in a series of seven proof-of-concept experiments, each of which examines a key aspect of deep meta-RL. We consider prospects for extending and scaling up the approach, and also point out some potentially important implications for neuroscience.

強化学習は、精度は高いが大量のデータを必要とする。人間ならもうちょい効率よくやるのに・・・ということで、様々なタスクのモデル(RNN)の学習を強化学習でやることで、各タスク間の共通構造などを習得させられないか(習得できれば、他の新しいタスクの時に上手くやれる)という試み。


Training time on large datasets for deep neural networks is the principal workflow bottleneck in a number of important applications of deep learning, such as object classification and detection in automatic driver assistance systems (ADAS). To minimize training time, the training of a deep neural network must be scaled beyond a single machine to as many machines as possible by distributing the optimization method used for training. While a number of approaches have been proposed for distributed stochastic gradient descent (SGD), at the current time synchronous approaches to distributed SGD appear to be showing the greatest performance at large scale. Synchronous scaling of SGD suffers from the need to synchronize all processors on each gradient step and is not resilient in the face of failing or lagging processors. In asynchronous approaches using parameter servers, training is slowed by contention to the parameter server. In this paper we compare the convergence of synchronous and asynchronous SGD for training a modern ResNet network architecture on the ImageNet classification problem. We also propose an asynchronous method, gossiping SGD, that aims to retain the positive features of both systems by replacing the all-reduce collective operation of synchronous training with a gossip aggregation algorithm. We find, perhaps counterintuitively, that asynchronous SGD, including both elastic averaging and gossiping, converges faster at fewer nodes (up to about 32 nodes), whereas synchronous SGD scales better to more nodes (up to about 100 nodes).

DeepLearningを行う上でボトルネックとなる学習時間を短縮するために、分散処理アルゴリズムとしてgossiping SGDという手法を提案。従来の同期アプローチでは障害（1ノードの計算失敗など）に弱く、非同期アプローチではparameterサーバを使用しているためパラメタが競合状態となり、学習が遅くなる。 提案手法では、同期アプローチのall-reduce collective operationをgossip aggregationアルゴリズムに置き換える事により、従来の同期/非同期アプローチの良いところを継承した非同期アプローチを構築。 直感に反するが、非同期アプローチでは32ノード, 同期アプローチでは100ノードまでスケール可能。


Transfer and multi-task learning have traditionally focused on either a single source-target pair or very few, similar tasks. Ideally, the linguistic levels of morphology, syntax and semantics would benefit each other by being trained in a single model. We introduce a joint many-task model together with a strategy for successively growing its depth to solve increasingly complex tasks. Higher layers include shortcut connections to lower-level task predictions to reflect linguistic hierarchies. We use a simple regularization term to allow for optimizing all model weights to improve one task's loss without exhibiting catastrophic interference of the other tasks. Our single end-to-end model obtains state-of-the-art or competitive results on five different tasks from tagging, parsing, relatedness, and entailment tasks.

言語を扱う複数のタスクは、相互に有用な知識を持つはずだから、組み合わせたほうがいい精度が出るのでは、という話。品詞づけ・文節判定・係り受け・文意関係(補強・反対・普通)・文関係の度合い、といった複数のタスクをこなす一つのネットワークを構築し、最高精度を達成。


Several deep learning models have been proposed for question answering. However, due to their single-pass nature, they have no way to recover from local maxima corresponding to incorrect answers. To address this problem, we introduce the Dynamic Coattention Network (DCN) for question answering. The DCN first fuses co-dependent representations of the question and the document in order to focus on relevant parts of both. Then a dynamic pointing decoder iterates over potential answer spans. This iterative procedure enables the model to recover from initial local maxima corresponding to incorrect answers. On the Stanford question answering dataset, a single DCN model improves the previous state of the art from 71.0% F1 to 75.9%, while a DCN ensemble obtains 80.4% F1.

一度だけ文書を読んで質問に答えるより、質問がわかってから見返せたほうがいいよね？ということで文書のencodeだけでなく、質問も掛け合わせたものを利用する(Coattention)、また一度で回答するのでなく何回か見直すことで局所最適を避けるという手法の提案。これでSOTAを更新


We present a method for performing hierarchical object detection in images guided by a deep reinforcement learning agent. The key idea is to focus on those parts of the image that contain richer information and zoom on them. We train an intelligent agent that, given an image window, is capable of deciding where to focus the attention among five different predefined region candidates (smaller windows). This procedure is iterated providing a hierarchical image analysis.We compare two different candidate proposal strategies to guide the object search: with and without overlap. Moreover, our work compares two different strategies to extract features from a convolutional neural network for each region proposal: a first one that computes new feature maps for each region proposal, and a second one that computes the feature maps for the whole image to later generate crops for each region proposal. Experiments indicate better results for the overlapping candidate proposal strategy and a loss of performance for the cropped image features due to the loss of spatial resolution. We argue that, while this loss seems unavoidable when working with large amounts of object candidates, the much more reduced amount of region proposals generated by our reinforcement learning agent allows considering to extract features for each location without sharing convolutional computation among regions.

物体認識を行う際、写真のどこに注目するかを決めて(右上・左上、など)切り取り、拡大する。そこからさらにどこに注目するか決め・・・と再帰的に繰り返すことで認識精度を上げるという研究。この挙動を強化学習で学習させる。重複には強くなったが、切り取りによる領域縮小で精度が下がったとのこと


Sequence models can be trained using supervised learning and a next-step prediction objective. This approach, however, suffers from known failure modes. For example, it is notoriously difficult to ensure multi-step generated sequences have coherent global structure. Motivated by the fact that reinforcement learning (RL) can be used to impose arbitrary properties on generated data by choosing appropriate reward functions, in this paper we propose a novel approach for sequence training which combines Maximum Likelihood (ML) and RL training. We refine a sequence predictor by optimizing for some imposed reward functions, while maintaining good predictive properties learned from data. We propose efficient ways to solve this by augmenting deep Q-learning with a cross-entropy reward and deriving novel off-policy methods for RNNs from stochastic optimal control (SOC). We explore the usefulness of our approach in the context of music generation. An LSTM is trained on a large corpus of songs to predict the next note in a musical sequence. This Note-RNN is then refined using RL, where the reward function is a combination of rewards based on rules of music theory, as well as the output of another trained Note-RNN. We show that by combining ML and RL, this RL Tuner method can not only produce more pleasing melodies, but that it can significantly reduce unwanted behaviors and failure modes of the RNN.

音楽を生成するRNNを、強化学習で学習させるという方法。actionは音符を選ぶことで、Rewardは実際の曲的に出現しうるか＋音楽理論に沿っているか(いくつかの特徴量で設定)で与える。これにより、これまでより格段に音楽的に好ましくない性質は低減し、好ましい性質は高くすることができた。


We build a machine solver for word problems on the physics of a free falling object under constant acceleration of gravity.  Each problem consists of a formulation part, describing the setting, and a question part asking for the value of an unknown.  Our solver consists of two long short-term memory recurrent neural networks and a numerical integrator. The first neural network (the labeler) labels each word of the problem, identifying the physical parameters and the question part of the problem. The second neural network (the  classifier) identifies what is being asked in the question. Using the information extracted by both networks, the numerical integrator computes the solution.  We observe that the classifier is resilient to errors made by the labeler, which does a better job of identifying the physics parameters than the question. Training, validation and test sets of problems are generated from a grammar, with validation and test problems structurally different from the training problems. The overall accuracy of the solver on the test cases is 99.8%.

"椅子が51mphで28度の角度で打ち上げられた。惑星ワトソンでは重力加速度を98m^2/sとせよ。最大の高さに達するにはかかる時間を求めよ。"のような力学の問題が与えられたとき回答を返すシステムを構築。具体的には2次元空間中で重力しか力のかからない自由粒子に関する自然言語で与えられる問題を想定。入力文にラベル付与するLSTMとラベル付与済の文からタスク抽出するLSTMとODEソルバーからなるシステム。テストケースで99.8%の正解率。


The word2vec software of Tomas Mikolov and colleagues (this https URL ) has gained a lot of traction lately, and provides state-of-the-art word embeddings. The learning models behind the software are described in two research papers. We found the description of the models in these papers to be somewhat cryptic and hard to follow. While the motivations and presentation may be obvious to the neural-networks language-modeling crowd, we had to struggle quite a bit to figure out the rationale behind the equations. This note is an attempt to explain equation (4) (negative sampling) in "Distributed Representations of Words and Phrases and their Compositionality" by Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado and Jeffrey Dean.

ネガティブサンプリングの式の導出を詳しく解説している論文


Enabling a computer to understand a document so that it can answer comprehension questions is a central, yet unsolved goal of NLP. A key factor impeding its solution by machine learned systems is the limited availability of human-annotated data. Hermann et al. (2015) seek to solve this problem by creating over a million training examples by pairing CNN and Daily Mail news articles with their summarized bullet points, and show that a neural network can then be trained to give good performance on this task. In this paper, we conduct a thorough examination of this new reading comprehension task. Our primary aim is to understand what depth of language understanding is required to do well on this task. We approach this from one side by doing a careful hand-analysis of a small subset of the problems and from the other by showing that simple, carefully designed systems can obtain accuracies of 73.6% and 76.6% on these two datasets, exceeding current state-of-the-art results by 7-10% and approaching what we believe is the ceiling for performance on this task.

DeepMindの教師なしによるQA回答(下)の追試。これは、文章中の固有表現をentityタグに置換して学習し、QAをentityの穴埋め問題として解くという手法。


The word2vec model and application by Mikolov et al. have attracted a great amount of attention in recent two years. The vector representations of words learned by word2vec models have been shown to carry semantic meanings and are useful in various NLP tasks. As an increasing number of researchers would like to experiment with word2vec or similar techniques, I notice that there lacks a material that comprehensively explains the parameter learning process of word embedding models in details, thus preventing researchers that are non-experts in neural networks from understanding the working mechanism of such models. This note provides detailed derivations and explanations of the parameter update equations of the word2vec models, including the original continuous bag-of-word (CBOW) and skip-gram (SG) models, as well as advanced optimization techniques, including hierarchical softmax and negative sampling. Intuitive interpretations of the gradient equations are also provided alongside mathematical derivations. In the appendix, a review on the basics of neuron networks and backpropagation is provided. I also created an interactive demo, wevi, to facilitate the intuitive understanding of the model.

word2vecのパラメータ更新式の詳細な導出と説明を行っている論文。対象のモデルはCBOWとSkip-Gram。パラメータ学習の過程を詳細に解説した資料がなかったから書いたそうだ。


Deep learning methods exhibit promising performance for predictive modeling in healthcare, but two important challenges remain: - Data insufficiency: Often in healthcare predictive modeling, the sample size is insufficient for deep learning methods to achieve satisfactory results.  - Interpretation: The representations learned by deep learning models should align with medical knowledge. To address these challenges, we propose a GRaph-based Attention Model, GRAM that supplements electronic health records (EHR) with hierarchical information inherent to medical ontologies.  Based on the data volume and the ontology structure, GRAM represents a medical concept as a combination of its ancestors in the ontology via an attention mechanism.  We compared predictive performance (i.e. accuracy, data needs, interpretability) of GRAM to various methods including the recurrent neural network (RNN) in two sequential diagnoses prediction tasks and one heart failure prediction task. Compared to the basic RNN, GRAM achieved 10% higher accuracy for predicting diseases rarely observed in the training data and 3% improved area under the ROC curve for predicting heart failure using an order of magnitude less training data. Additionally, unlike other methods, the medical concept representations learned by GRAM are well aligned with the medical ontology. Finally, GRAM exhibits intuitive attention behaviors by adaptively generalizing to higher level concepts when facing data insufficiency at the lower level concepts.

診断記録のsequenceから病気をあてるタスク。各診断記録にのっている専門用語の集合を1つの入力とするRNNを構築する。各専門用語のベクタをそのまま入力に使わず、オントロジーを使い上位概念のベクタも考慮しattentionで上位概念を考慮した専門用語のベクタ作り入力とする手法GRAMを提案。単純にRNNを使った場合と比較しaccuracyで10%,AUCで3%の性能向上。


We introduce a new, efficient, principled and backpropagation-compatible algorithm for learning a probability distribution on the weights of a neural network, called Bayes by Backprop. It regularises the weights by minimising a compression cost, known as the variational free energy or the expected lower bound on the marginal likelihood. We show that this principled kind of regularisation yields comparable performance to dropout on MNIST classification. We then demonstrate how the learnt uncertainty in the weights can be used to improve generalisation in non-linear regression problems, and how this weight uncertainty can be used to drive the exploration-exploitation trade-off in reinforcement learning.

NNモデル（の重み）の不確実性を、重み上の確率分布を用いて扱う。確率分布の学習方法として、誤差逆伝搬法を模したBayes by  backpropを提案。分類や回帰、バンディットなど色々なタスクで成功。特に、Thompson Samplingなどと組み合わせて探索ができるところが良さそう。


In this work, we study the problem of feature representation learning for graph-structured data. Many of the existing work in the area are task-specific and based on supervised techniques. We study a method for obtaining a generic feature representation for a graph using an unsupervised approach. The neural encoder-decoder model is a method that has been used in the natural language processing domain to learn feature representations of sentences. In our proposed approach, we train the encoder-decoder model to predict the random walk sequence of neighboring regions in a graph given a random walk along a particular region. The goal is to map subgraphs — as represented by their random walks — that are structurally and functionally similar to nearby locations in feature space. We evaluate the learned graph vectors using several real-world datasets on the graph classification task. The proposed model is able to achieve good results against state-of- the-art techniques.

与えられたグラフの埋め込み表現を抽出する手法を提案。生成したグラフ上のランダムウォークのシーケンスに対しseq2vecと同様にベクタ表現を得る。分子構造データベースを使い識別タスクで他のグラフ埋め込み手法と比較。SoTa


Experience replay lets online reinforcement learning agents remember and reuse experiences from the past. In prior work, experience transitions were uniformly sampled from a replay memory. However, this approach simply replays transitions at the same frequency that they were originally experienced, regardless of their significance. In this paper we develop a framework for prioritizing experience, so as to replay important transitions more frequently, and therefore learn more efficiently. We use prioritized experience replay in Deep Q-Networks (DQN), a reinforcement learning algorithm that achieved human-level performance across many Atari games. DQN with prioritized experience replay achieves a new state-of-the-art, outperforming DQN with uniform replay on 41 out of 49 games.

Experience replayにおいて効率的にサンプルを学習するため優先度によってサンプリングすることを提案。優先度としてTD-errorを使用、importace samplingになるので重みを調節し、学習終了付近でのバイアスの影響をなくすため一様サンンプリング近づくようなannealingをしている。DQN,Double-DQNで評価しともにbaselineより良好。人工的なimbalanced dataを使い通常の識別学習への適用も評価している。


Learning to solve complex sequences of tasks--while both leveraging transfer and avoiding catastrophic forgetting--remains a key obstacle to achieving human-level intelligence. The progressive networks approach represents a step forward in this direction: they are immune to forgetting and can leverage prior knowledge via lateral connections to previously learned features. We evaluate this architecture extensively on a wide variety of reinforcement learning tasks (Atari and 3D maze games), and show that it outperforms common baselines based on pretraining and finetuning. Using a novel sensitivity measure, we demonstrate that transfer occurs at both low-level sensory and high-level control layers of the learned policy.

Progressive netというfine tuningのためのネットワークの提案，pre trained networkの各層の出力をfine tuneしたいnetworkの各層に重みづけして足し合わせる．fine tuneの際pre trained networkの重みについては変更せずfine tuneに入力するときの重みを学習することで，pre trainした結果を忘れることもなく、また複数のpre trained networkを重ねあわせることもできる。この手法自体の適用先は強化学習に限らないが強化学習タスクで評価。Fisher情報行列を評価してpre trainded networkのどの素性が寄与しているかを解析もしている。#1 はこの結果のロボティクスへの応用.


When encountering novel object, humans are able to infer a wide range of physical properties such as mass, friction and deformability by interacting with them in a goal driven way. This process of active interaction is in the same spirit of a scientist performing an experiment to discover hidden facts. Recent advances in artificial intelligence have yielded machines that can achieve superhuman performance in Go, Atari, natural language processing, and complex control problems, but it is not clear that these systems can rival the scientific intuition of even a young child. In this work we introduce a basic set of tasks that require agents to estimate hidden properties such as mass and cohesion of objects in an interactive simulated environment where they can manipulate the objects and observe the consequences. We found that state of art deep reinforcement learning methods can learn to perform the experiments necessary to discover such hidden properties. By systematically manipulating the problem difficulty and the cost incurred by the agent for performing experiments, we found that agents learn different strategies that balance the cost of gathering information against the cost of making mistakes in different situations.

人が物体を目の前にしたときに、触ったり持ち上げたりすることで物体の特性を知るように、強化学習でその過程を再現しようという試み。具体的には、ものの重さを当てるタスク、ブロックが積まれている中で何個あるか当てるというタスクにチャレンジしている。


We investigate deep generative models that can exchange multiple modalities bi-directionally, e.g., generating images from corresponding texts and vice versa. Recently, some studies handle multiple modalities on deep generative models, such as variational autoencoders (VAEs). However, these models typically assume that modalities are forced to have a conditioned relation, i.e., we can only generate modalities in one direction. To achieve our objective, we should extract a joint representation that captures high-level concepts among all modalities and through which we can exchange them bi-directionally. As described herein, we propose a joint multimodal variational autoencoder (JMVAE), in which all modalities are independently conditioned on joint representation. In other words, it models a joint distribution of modalities. Furthermore, to be able to generate missing modalities from the remaining modalities properly, we develop an additional method, JMVAE-kl, that is trained by reducing the divergence between JMVAE's encoder and prepared networks of respective modalities. Our experiments show that our proposed method can obtain appropriate joint representation from multiple modalities and that it can generate and reconstruct them more properly than conventional VAEs. We further demonstrate that JMVAE can generate multiple modalities bi-directionally.

画像の特徴と、モダリティ(男性・女性など)の画像的特徴をそれぞれ学習させ、その結果を結合させる=元の画像に近く、かつ指定したモダリティの画像的特徴にも近い画像を生成する、という研究。画像のVAEにモダリティのVAEをJointさせると言うことで、JMVAEと命名。


Adaptive gradient methods for stochastic optimization adjust the learning rate for each parameter locally. However, there is also a global learning rate which must be tuned in order to get the best performance. In this paper, we present a new algorithm that adapts the learning rate locally for each parameter separately, and also globally for all parameters together. Specifically, we modify Adam, a popular method for training deep learning models, with a coefficient that captures properties of the objective function. Empirically, we show that our method, which we call Eve, outperforms Adam and other popular methods in training deep neural networks, like convolutional neural networks for image classification, and recurrent neural networks for language tasks.

Adamの変形で、目的関数からのフィードバック、具体的には目的関数の出力増減率の移動平均を加味してやることで性能が向上したというもの。論理コードがあるため、簡単に実装できそう。


We present a novel framework for generating pop music. Our model is a hierarchical Recurrent Neural Network, where the layers and the structure of the hierarchy encode our prior knowledge about how pop music is composed. In particular, the bottom layers generate the melody, while the higher levels produce the drums and chords. We conduct several human studies that show strong preference of our generated music over that produced by the recent method by Google. We additionally show two applications of our framework: neural dancing and karaoke, as well as neural story singing.

ポップ・ミュージックを生成する研究。音楽理論に基づいた設計を行っていて、音符(キー)、その長さ(キー＋長さでメロディー)、さらに和音、ドラムを重ねている。これで、GoogleのMagentaより圧倒的に評価の良い音楽を生成。また、曲に合わせたダンスと歌詞の生成も行っている。


We present the Neural Physics Engine (NPE), a framework for learning simulators of intuitive physics that naturally generalize across variable object count and different scene configurations. We propose a factorization of a physical scene into composable object-based representations and a neural network architecture whose compositional structure factorizes object dynamics into pairwise interactions. Like a symbolic physics engine, the NPE is endowed with generic notions of objects and their interactions; realized as a neural network, it can be trained via stochastic gradient descent to adapt to specific object properties and dynamics of different worlds. We evaluate the efficacy of our approach on simple rigid body dynamics in two-dimensional worlds. By comparing to less structured architectures, we show that the NPE's compositional representation of the structure in physical interactions improves its ability to predict movement, generalize across variable object count and different scene configurations, and infer latent properties of objects such as mass.

NNで物理法則を表現しようという話。物理法則による変化を状態遷移モデルとして捉えて、状態を「着目物体」と「その周辺の物体(=コンテキスト)」で表現し、それらの状態を入力にしたLSTMで遷移を表現している。Matter.jsを使用した実験結果があり予測結果をgifで確認できる。