This paper proposes an unsupervised approach for segmenting a multiauthor document into authorial components .

The key novelty is that we utilize the sequential patterns hidden among document elements when determining their authorships .

For this purpose , we adopt Hidden Markov Model ( HMM ) and construct a sequential probabilistic model to capture the dependencies of sequential sentences and their authorships .

An unsupervised learning method is developed to initialize the HMM parameters .

Experimental results on benchmark datasets have demonstrated the significant benefit of our idea and our approach has outperformed the state-of-the-arts on all tests .

As an example of its applications , the proposed approach is applied for attributing authorship of a document and has also shown promising results .

Authorship analysis is a process of inspecting documents in order to extract authorial information about these documents .

It is considered as a general concept that embraces several types of authorship subjects , including authorship verif ication , plagiarism detection and author attribution .

Authorship verification ( Brocardo et al . , 2013 ; Potha and Stamatatos , 2014 ) decides whether a given document is written by a specific author .

Plagiarism detection ( Stein et al . , 2011 ; Kestemont et al . , 2011 ) seeks to expose the similarity between two texts .

However , it is un-able to determine if they are written by the same author .

In author attribution ( Juola , 2006 ; Savoy , 2015 ) , a real author of an anonymous document is predicted using labeled documents of a set of candidate authors .

Another significant subject in authorship analysis , which has received comparatively less attention from research community , is authorship-based document decomposition ( ABDD ) .

This subject is to group the sentences of a multi-author document to different classes , of which each contains the sentences written by only one author .

Many applications can take advantage of such a subject , especially those in forensic investigation , which aim to determine the authorship of sentences in a multi-author document .

Furthermore , this kind of subject is beneficial for detecting plagiarism in a document and defining contributions of authors in a multi-author document for commercial purpose .

ABDD can also be applied to identify which source ( regarded as an ' author ' in this paper ) a part of a document is copied from when the document is formed by taking contents from various sources .

In despite of the benefits of ABDD , there has been little research reported on this subject .

Koppel et al . ( 2011 ) are the first researchers who implemented an unsupervised approach for ABDD .

However , their approach is restricted to Hebrew documents only .

The authors of Akiva and Koppel ( 2013 ) addressed the drawbacks of the above approach by proposing a generic unsupervised approach for ABDD .

Their approach utilized distance measurements to increase the precision and accuracy of clustering and classification phases , respectively .

The accuracy of their approach is highly dependent on the number of au-thors .

When the number of authors increases , the accuracy of the approach is significantly dropped .

Giannella ( 2015 ) presented an improved approach for ABDD when the number of authors of the document is known or unknown .

In his approach , a Bayesian segmentation algorithm is applied , which is followed by a segment clustering algorithm .

However , the author tested his approach by using only documents with a few transitions among authors .

Furthermore , the accuracy of the approach is very sensitive to the setting of its parameters .

In Aldebei et al . ( 2015 ) , the authors presented an unsupervised approach ABDD by exploiting the differences in the posterior probabilities of a Naive-Bayesian model in order to increase the precision and the classification accuracy , and to be less dependent on the number of authors in comparing with the approach in Akiva and Koppel ( 2013 ) .

Their work was tested on documents with up to 400 transitions among authors and the accuracy of their approach was not sensitive to the setting of parameters , in contrast with the approach in Giannella ( 2015 ) .

However , the performance of their approach greatly depends on a threshold , of which the optimal value for an individual document is not easy to find .

Some other works have focused on segmenting a document into components according to their topics .

For applications where the topics of documents are unavailable , these topicbased solutions will fail .

In this paper , the ABDD approach is independent of documents ' topics .

All of the existing works have assumed that the observations ( i.e. , sentences ) are independent and identically distributed ( i.i.d . ) .

No consideration has been given to the contextual information between the observations .

However , in some cases , the i.i.d . assumption is deemed as a poor one ( Rogovschi et al . , 2010 ) .

In this paper , we will relax this assumption and consider sentences of a document as a sequence of observations .

We make use of the contextual information hidden between sentences in order to identify the authorship of each sentence in a document .

In other words , the authorships of the " previous " and " subsequent " sentences have relationships with the authorship of the current sentence .

There-fore , in this paper , a well-known sequential model , Hidden Markov Model ( HMM ) , is used for modelling the sequential patterns of the document in order to describe the authorship relationships .

The contributions of this article are summarized as follows .

We capture the dependencies between consecutive elements in a document to identify different authorial components and construct an HMM for classification .

It is for the first time the sequential patterns hidden among document elements is considered for such a problem .

2 . To build and learn the HMM model , an unsupervised learning method is first proposed to estimate its initial parameters , and it does not require any information of authors or document 's context other than how many authors have contributed to write the document .

3 . Different from the approach in Aldebei et al . ( 2015 ) , the proposed unsupervised approach no longer relies on any predetermined threshold for ABDD .

Comprehensive experiments are conducted to demonstrate the superior performance of our ideas on both widely-used artificial benchmark datasets and an authentic scientific document .

As an example of its applications , the proposed approach is also applied for attributing authorship on a popular dataset .

The proposed approach can not only correctly determine the author of a disputed document but also provide a way for measuring the confidence level of the authorship decision for the first time .

The rest of this article is organised as follows .

Section 3 presents the details of our proposed approach , including the processes for initialization and learning of HMM parameters , and the Viterbi decoding process for classification .

Experiments are conducted in Section 4 , followed by the conclusion in Section 5 .

We have developed an unsupervised approach for decomposing a multi-author document based on authorship .

Different from the stateof-the-art approaches , we have innovatively made use of the sequential information hidden among document elements .

For this purpose , we have used HMM and constructed a sequential probabilistic model , which is used to find the best sequence of authors that represents the sentences of the document .

An unsupervised learning method has also been developed to estimate the initial parameter values of HMM .

Comparative experiments conducted on benchmark datasets have demonstrated the effectiveness of our ideas with superior perfor-mance achieved on both artificial and authentic documents .

An application of the proposed approach on authorship attribution has also achieved perfect results of 100 % accuracies together with confidence measurement for the first time .

However , it is un-able to determine if they are written by the same author .

An unsupervised learning method is developed to initialize the HMM parameters .


We investigate two systems for automatic disfluency detection on English and Mandarin conversational speech data .

The first system combines various lexical and prosodic features in a Conditional Random Field model for detecting edit disfluencies .

The second system combines acoustic and language model scores for detecting filled pauses through constrained speech recognition .

We compare the contributions of different knowledge sources to detection performance between these two languages .

Speech disfluencies are common phenomena in spontaneous speech .

They consist of spoken words and phrases that represent self-correction , hesitation , and floor-grabbing behaviors , but do not add semantic information ; removing them yields the intended , fluent utterance .

The presence of disfluencies in conversational speech data can cause problems for both downstream processing ( parsing and other natural language processing tasks ) and human readability of speech transcripts .

There has been much research effort on automatic disfluency detection in recent years ( Shriberg and Stolcke , 1997 ; Snover et al . , 2004 ; Liu et al . , 2006 ; Lin and Lee , 2009 ; Schuler et al . , 2010 ; Georgila et al . , 2010 ; Zwarts and Johnson , 2011 ) , particularly from the DARPA EARS ( Effective , Affordable , Reusable Speech-to-Text ) MDE ( MetaData Extraction ) ( DARPA Information Processing Technology Office , 2003 ) program , which focused on the automatic transcription of sizable amounts of speech data and rendering such transcripts in readable form , for both conversational telephone speech ( CTS ) and broadcast news ( BN ) .

However , the EARS MDE effort was focused on English only , and there has n't been much research on the effectiveness of similar automatic disfluency detection approaches for multiple languages .

This paper presents three main innovations .

First , we extend the EARS MDE-style disfluency detection approach combining lexical and prosodic features using a Conditional Random Field ( CRF ) model , which was employed for detecting disfluency on English conversational speech data ( Liu et al . , 2005 ) , to Mandarin conversational speech , as presented in Section 2 .

Second , we implement an automatic filled pause detection approach through constrained speech recognition , as presented in Section 3 .

Third , for both disfluency detection systems , we compare side-by-side contributions of different knowledge sources to detection performance for two languages , English and Mandarin , as presented in Section 4 .

In conclusion , we have presented two automatic disfluency detection systems , one combining various lexical and prosodic features , and the other combining LVCSR acoustic and language model knowledge sources .

We observed significant improvements in combining lexical and prosodic features over just employing word n-gram features , for both languages .

When combining AM and LM knowledge sources for FP detection in constrained speech recognition , we found increasing LM weight improved both false alarm and miss rates for Mandarin but degraded the miss rate for English .


We propose a method to generate large-scale encyclopedic knowledge , which is valuable for much NLP research , based on the Web .

We first search the Web for pages containing a term in question .

Then we use linguistic patterns and HTML structures to extract text fragments describing the term .

Finally , we organize extracted term descriptions based on word senses and domains .

In addition , we apply an automatically generated encyclopedia to a question answering system targeting the Japanese Information-Technology Engineers Examination .

Reflecting the growth in utilization of the World Wide Web , a number of Web-based language processing methods have been proposed within the natural language processing ( NLP ) , information retrieval ( IR ) and artificial intelligence ( AI ) communities .

A sample of these includes methods to extract linguistic resources ( Fujii and Ishikawa , 2000 ; Resnik , 1999 ; Soderland , 1997 ) , retrieve useful information in response to user queries ( Etzioni , 1997 ; McCallum et al . , 1999 ) and mine / discover knowledge latent in the Web ( Inokuchi et al . , 1999 ) .

In this paper , mainly from an NLP point of view , we explore a method to produce linguistic resources .

Specifically , we enhance the method proposed by Fujii and Ishikawa ( 2000 ) , which extracts encyclopedic knowledge ( i.e. , term descriptions ) from the Web .

In brief , their method searches the Web for pages containing a term in question , and uses linguistic expressions and HTML layouts to extract fragments describing the term .

They also use a language model to discard non-linguistic fragments .

In addition , a clustering method is used to divide descriptions into a specific number of groups .

On the one hand , their method is expected to enhance existing encyclopedias , where vocabulary size is relatively limited , and therefore the quantity problems has been resolved .

On the other hand , encyclopedias extracted from the Web are not comparable with existing ones in terms of quality .

In hand-crafted encyclopedias , term descriptions are carefully organized based on domains and word senses , which are especially effective for human usage .

However , the output of Fujii 's method is simply a set of unorganized term descriptions .

Although clustering is optionally performed , resultant clusters are not necessarily related to explicit criteria , such as word senses and domains .

To sum up , our belief is that by combining extraction and organization methods , we can enhance both quantity and quality of Web-based encyclopedias .

Motivated by this background , we introduce an organization model to Fujii 's method and reformalize the whole framework .

In other words , our proposed method is not only extraction but generation of encyclopedic knowledge .

Section 2 explains the overall design of our encyclopedia generation system , and Section 3 elaborates on our organization model .

Section 4 then explores a method for applying our resultant encyclopedia to NLP research , specifically , question answering .

Section 5 performs a number of experiments to evaluate our methods .

The World Wide Web has been an unprecedentedly enormous information source , from which a number of language processing methods have been explored to extract , retrieve and discover various types of information .

In this paper , we aimed at generating encyclopedic knowledge , which is valuable for many applications including human usage and natural language understanding .

For this purpose , we reformalized an existing Web-based extraction method , and proposed a new statistical organization model to improve the quality of extracted data .

Given a term for which encyclopedic knowledge ( i.e. , descriptions ) is to be generated , our method sequentially performs a ) retrieval of Web pages contain-ing the term , b ) extraction of page fragments describing the term , and c ) organizing extracted descriptions based on domains ( and consequently word senses ) .

In addition , we proposed a question answering system , which answers interrogative questions associated with what , by using a Web-based encyclopedia as a knowledge base .

For the purpose of evaluation , we used as test inputs technical terms collected from the Class II IT engineers examination , and found that the encyclopedia generated through our method was of operational quality and quantity .

We also used test questions from the Class II examination , and evaluated the Web-based encyclopedia in terms of question answering .

We found that our Webbased encyclopedia improved the system coverage obtained solely with an existing dictionary .

In addition , when we used both resources , the performance was further improved .

Future work would include generating information associated with more complex interrogations , such as ones related to how and why , so as to enhance Webbased natural language understanding .


Research on modeling time series text corpora has typically focused on predicting what text will come next , but less well studied is predicting when the next text event will occur .

In this paper we address the latter case , framed as modeling continuous inter-arrival times under a log-Gaussian Cox process , a form of inhomogeneous Poisson process which captures the varying rate at which the tweets arrive over time .

In an application to rumour modeling of tweets surrounding the 2014 Ferguson riots , we show how interarrival times between tweets can be accurately predicted , and that incorporating textual features further improves predictions .

Twitter is a popular micro-blogging service which provides real-time information on events happening across the world .

Evolution of events over time can be monitored there with applications to disaster management , journalism etc .

For example , Twitter has been used to detect the occurrence of earthquakes in Japan through user posts ( Sakaki et al . , 2010 ) .

Modeling the temporal dynamics of tweets provides useful information about the evolution of events .

Inter-arrival time prediction is a type of such modeling and has application in many settings featuring continuous time streaming text corpora , including journalism for event monitoring , real-time disaster monitoring and advertising on social media .

For example , journalists track several rumours related to an event .

Predicted arrival times of tweets can be applied for ranking rumours according to their activity and narrow the interest to investigate a rumour with a short interarrival time over that of a longer one .

Modeling the inter-arrival time of tweets is a challenging task due to complex temporal patterns exhibited .

Tweets associated with an event stream arrive at different rates at different points in time .

For example , Figure 1a shows the arrival times ( denoted by black crosses ) of tweets associated with an example rumour around Ferguson riots in 2014 .

Notice the existence of regions of both high and low density of arrival times over a one hour interval .

We propose to address inter-arrival time prediction problem with log-Gaussian Cox process ( LGCP ) , an inhomogeneous Poisson process ( IPP ) which models tweets to be generated by an underlying intensity function which varies across time .

Moreover , it assumes a non-parametric form for the intensity function allowing the model complexity to depend on the data set .

We also provide an approach to consider textual content of tweets to model inter-arrival times .

We evaluate the models using Twitter rumours from the 2014 Ferguson unrest , and demonstrate that they provide good predictions for inter-arrival times , beating the baselines e.g. homogeneous Poisson Process , Gaussian Process regression and univariate Hawkes Process .

Even though the central application is rumours , one could apply the proposed approaches to model the arrival times of tweets corresponding to other types of memes , e.g. discussions about politics .

This paper makes the following contributions : 1 . Introduces log-Gaussian Cox process to predict tweet arrival times .

2 . Demonstrates how incorporating text improves results of inter-arrival time prediction .

This paper introduced the log-Gaussian Cox processes for the problem of predicting the interarrival times of tweets .

We showed how text from posts helps to achieve significant improvements .

Evaluation on a set of rumours from Ferguson riots showed efficacy of our methods comparing to baselines .

The proposed approaches are generalizable to problems other than rumours , e.g. disaster management and advertisement campaigns .


Hypertext documents , such as web pages and academic papers , are of great importance in delivering information in our daily life .

Although being effective on plain documents , conventional text embedding methods suffer from information loss if directly adapted to hyper-documents .

In this paper , we propose a general embedding approach for hyper-documents , namely , hyperdoc2vec , along with four criteria characterizing necessary information that hyper-document embedding models should preserve .

Systematic comparisons are conducted between hyperdoc2vec and several competitors on two tasks , i.e. , paper classification and citation recommendation , in the academic paper domain .

Analyses and experiments both validate the superiority of hyperdoc2vec to other models w.r.t .

The ubiquitous World Wide Web has boosted research interests on hypertext documents , e.g. , personal webpages ( Lu and Getoor , 2003 ) , Wikipedia pages ( Gabrilovich and Markovitch , 2007 ) , as well as academic papers ( Sugiyama and Kan , 2010 ) .

Unlike independent plain documents , a hypertext document ( hyper-doc for short ) links to another hyper-doc by a hyperlink or citation mark in its textual content .

Given this essential distinction , hyperlinks or citations are worth specific modeling in many tasks such as link-based classification ( Lu and Getoor , 2003 ) , web retrieval ( Page et al . , 1999 ) , entity linking ( Cucerzan , 2007 ) , and citation recommendation ( He et al . , 2010 ) .

To model hypertext documents , various efforts ( Cohn and Hofmann , 2000 ; Kataria et al . , 2010 ; Perozzi et al . , 2014 ; Zwicklbauer et al . , 2016 ; Wang et al . , 2016 ) have been made to depict networks of hyper-docs as well as their content .

Among potential techniques , distributed representation ( Mikolov et al . , 2013 ;

Le and Mikolov , 2014 ) tends to be promising since its validity and effectiveness are proven for plain documents on many natural language processing ( NLP ) tasks .

Conventional attempts on utilizing embedding techniques in hyper-doc-related tasks generally fall into two types .

The first type ( Berger et al . , 2017 ; Zwicklbauer et al . , 2016 ) simply downcasts hyper-docs to plain documents and feeds them into word2vec

( Mikolov et al . , 2013 ) ( w2v for short ) or doc2vec ( Le and Mikolov , 2014 ) ( d2v for short ) .

These approaches involve downgrading hyperlinks and inevitably omit certain information in hyper-docs .

However , no previous work investigates the information loss , and how it affects the performance of such downcasting-based adaptations .

The second type designs sophisticated embedding models to fulfill certain tasks , e.g. , citation recommendation ( Huang et al . , 2015b ) , paper classification ( Wang et al . , 2016 ) , and entity linking ( Yamada et al . , 2016 ) , etc .

These models are limited to specific tasks , and it is yet unknown whether embeddings learned for those particular tasks can generalize to others .

Based on the above facts , we are interested in two questions :

• What information should hyper-doc embedding models preserve , and what nice property should they possess ?

• Is there a general approach to learning taskindependent embeddings of hyper-docs ?

To answer the two questions , we formalize the hyper-doc embedding task , and propose four criteria , i.e. , content awareness , context awareness , newcomer friendliness , and context intent aware-ness , to assess different models .

Then we discuss simple downcasting-based adaptations of existing approaches w.r.t .

the above criteria , and demonstrate that none of them satisfy all four .

To this end , we propose hyperdoc2vec ( h-d2v for short ) , a general embedding approach for hyperdocs .

Different from most existing approaches , h-d2v learns two vectors for each hyper-doc to characterize its roles of citing others and being cited .

Owning to this , h-d2v is able to directly model hyperlinks or citations without downgrading them .

To evaluate the learned embeddings , we employ two tasks in the academic paper domain1 , i.e. , paper classification and citation recommendation .

Experimental results demonstrate the superiority of h-d2v .

Comparative studies and controlled experiments also confirm that h-d2v benefits from satisfying the above four criteria .

We summarize our contributions as follows :

• We propose four criteria to assess different hyper-document embedding models .

• We propose hyperdoc2vec , a general embedding approach for hyper-documents .

• We systematically conduct comparisons with competing approaches , validating the superiority of h-d2v in terms of the four criteria .

We focus on the hyper-doc embedding problem .

We propose that hyper-doc embedding algorithms should be content aware , context aware , newcomer friendly , and context intent aware .

To meet all four criteria , we propose a general approach , hyperdoc2vec , which assigns two vectors to each hyper-doc and models citations in a straightforward manner .

In doing so , the learned embeddings satisfy all criteria , which no existing model is able to .

For evaluation , paper classification and citation recommendation are conducted on three academic paper datasets .

Results confirm the effectiveness of our approach .

Further analyses also demonstrate that possessing the four properties helps h-d2v outperform other models .


Recurrent neural networks ( RNNs ) have achieved impressive results in a variety of linguistic processing tasks , suggesting that they can induce non-trivial properties of language .

We investigate here to what extent RNNs learn to track abstract hierarchical syntactic structure .

We test whether RNNs trained with a generic language modeling objective in four languages ( Italian , English , Hebrew , Russian ) can predict long-distance number agreement in various constructions .

We include in our evaluation nonsensical sentences where RNNs can not rely on semantic or lexical cues ( " The colorless green ideas ideas ideas ideas ideas ideas ideas ideas ideas ideas ideas ideas ideas ideas ideas ideas ideas I ate with the chair sleep sleep sleep sleep sleep sleep sleep sleep sleep sleep sleep sleep sleep sleep sleep sleep sleep furiously " ) , and , for Italian , we compare model performance to human intuitions .

Our language-model-trained RNNs make reliable predictions about long-distance agreement , and do not lag much behind human performance .

We thus bring support to the hypothesis that RNNs are not just shallowpattern extractors , but they also acquire deeper grammatical competence .

Recurrent neural networks ( RNNs ; Elman , 1990 ) are general sequence processing devices that do not explicitly encode the hierarchical structure that is thought to be essential to natural language ( Everaert et al . , 2015 ) .

Early work using artificial languages showed that they may nevertheless be able to approximate context-free languages ( Elman , 1991 ) .

More recently , RNNs have achieved impressive results in large-scale tasks such as language modeling for speech recognition and machine translation , and are by now standard tools for sequential natural language tasks ( e.g. , Mikolov et al . , 2010 ; Graves , 2012 ; Wu et al . , 2016 ) .

This suggests that RNNs may learn to track grammatical structure even when trained on noisier natural data .

The conjecture is supported by the success of RNNs as feature extractors for syntactic parsing ( e.g. , Cross and Huang , 2016 ; Kiperwasser and Goldberg , 2016 ; Zhang et al . , 2017 ) .

Linzen et al . ( 2016 ) directly evaluated the extent to which RNNs can approximate hierarchical structure in corpus-extracted natural language data .

They tested whether RNNs can learn to predict English subject-verb agreement , a task thought to require hierarchical structure in the general case ( " the girl girl girl girl girl girl girl girl girl girl girl girl girl girl girl girl girl the boys like .

Their experiments confirmed that RNNs can , in principle , handle such constructions .

However , in their study RNNs could only succeed when provided with explicit supervision on the target task .

Linzen and colleagues argued that the unsupervised language modeling objective is not sufficient for RNNs to induce the syntactic knowledge necessary to cope with long-distance agreement .

is is is is is is is is is is is is is is is is is

The current paper reevaluates these conclusions .

We strengthen the evaluation paradigm of Linzen and colleagues in several ways .

Most importantly , their analysis did not rule out the possibility that RNNs might be relying on semantic or collocational / frequency-based information , rather than purely on syntactic structure .

In " dogs dogs dogs dogs dogs about what typically barks ( dogs , not neighbourhoods ) , without relying on more abstract structural cues .

In a follow-up study to Linzen and colleagues ' , Bernardy and Lappin ( 2017 ) observed that RNNs are better at long-distance agreement when they construct rich lexical representations of words , which suggests effects of this sort might indeed be at play .

We introduce a method to probe the syntactic abilities of RNNs that abstracts away from potential lexical , semantic and frequency-based confounds .

We extend the previous work in three additional ways .

First , alongside English , which has few morphological cues to agreement , we examine Italian , Hebrew and Russian , which have richer morphological systems .

Second , we go beyond subject-verb agreement and develop an automated method to harvest a variety of long-distance number agreement constructions from treebanks .

Finally , for Italian , we collect human judgments for the tested sentences , providing an important comparison point for RNN performance .

1We focus on the more interesting unsupervised setup , where RNNs are trained to perform generic , large-scale language modeling ( LM ): they are not given explicit evidence , at training time , that they must focus on long-distance agreement , but they are rather required to track a multitude of cues that might help with word prediction in general .

RNNs trained with a LM objective solve the long-distance agreement problem well , even on nonce sentences .

The pattern is consistent across languages , and , crucially , not far from human performance in Italian .

Moreover , RNN performance on language modeling ( measured in terms of perplexity ) is a good predictor of long-distance agreement accuracy .

This suggests that the ability to capture structural generalizations is an important aspect of what makes the best RNN architectures so good at language modeling .

Since our positive results contradict , to some extent , those of Linzen et al . ( 2016 ) , we also replicate their relevant experiment using our best RNN ( an LSTM ) .

We outperform their models , suggesting that a careful architecture / hyperparameter search is crucial to obtain RNNs that are not only good at language modeling , but able to extract syntactic generalizations .

2 Constructing a long-distance agreement benchmark

We construct our number agreement test sets as follows .

Original sentences are automatically extracted from a dependency treebank .

They are then converted into nonce sentences by substituting all content words with random words with the same morphology , resulting in grammatical but nonsensical sequences .

An LM is evaluated on its predictions for the target ( second ) word in the dependency , in both the original and nonce sentences .

Agreement relations , such as subject-verb agreement in English , are an ideal test bed for the syntactic abilities of LMs , because the form of the second item ( the target ) is predictable from the first item ( the cue ) .

Crucially , the cue and the target are linked by a structural relation , where linear order in the word sequence does not matter ( Everaert et al . , 2015 )

In all these cases , the number of the main verb " thinks " is determined by its subject ( " girl " ) , and this relation depends on the syntactic structure of the sentence , not on the linear sequence of words .

As the last sentence shows , the word directly preceding the verb can even be a noun with the opposite number ( " friends " ) , but this does not influence the structurally-determined form of the verb .

When the cue and the target are adjacent ( " the girl girl girl girl girl girl girl girl girl girl girl girl girl girl girl girl girl thinks thinks thinks thinks thinks thinks thinks thinks thinks thinks thinks thinks thinks thinks thinks thinks thinks . . .

" ) , an LM can predict the target without access to syntactic structure : it can simply extract the relevant morphosyntactic features of words ( e.g. , number ) and record the co-occurrence frequencies of patterns such as N P lur V P lur ( Mikolov et al . , 2013 ) .

Thus , we focus here on long-distance agreement , where an arbitrary num- ber of words can occur between the elements of the agreement relation .

We limit ourselves to number agreement ( plural or singular ) , as it is the only overt agreement feature shared by all of the languages we study .

We started by collecting pairs of part-of-speech ( POS ) tags connected by a dependency arc .

Independently of which element is the head of the relation , we refer to the first item as the cue and to the second as the target .

We additionally refer to the POS sequence characterizing the entire pattern as a construction , and to the elements in the middle as context .

For each candidate construction , we collected all of the contexts in the corpus that intervene between the cue and the target ( we define contexts as the sequence of POS tags of the top-level nodes in the dependency subtrees ) .

For example , for the English subject-verb agreement construction shown in Fig . 1a , the context is defined by VERB ( head of the relative clause ) and ADV ( adverbial modifier of the target verb ) , which together dominate the sequence " the boys like often " .

For the Russian adjective-noun agreement construction in Fig . 1b , the context is NOUN , because in the dependency grammar we use the noun " moment " is the head of the prepositional phrase " at that moment " , which modifies the adjective " deep " .

The candidate agreement pair and the context form a construction , which is characterized by a sequence of POS tags , e.g. , NOUN VERB ADV VERB or VERB NOUN CCONJ VERB ( Fig . 1c ) .

is is is is is is is is is is is is is is is is is "

and " girls girls girls girls girls girls girls girls girls girls girls girls girls girls girls girls girls who stayed at home were were were

Conversely , standard syntactic structures might be split between different constructions , e.g. , relative clause contexts occur in both NOUN VERB VERB and NOUN VERB ADV VERB constructions ( the latter is illustrated by the English example in Fig . 1a ) .

Construction contexts can contain a variable numbers of words .

Since we are interested in challenging cases , we only considered cases in which at least three tokens intervened between the cue and the target .

In the next step , we excluded constructions in which the candidate cue and target did not agree in number in all of the instances of the construction in the treebank ( if both the cue and the target were morphologically annotated for number ) .

This step retained English subject-verb constructions , for example , but excluded verb-object constructions , since any form of a verb can appear both with singular and plural objects .

To focus on robust agreement patterns , we only kept constructions with at least 10 instances of both plural and singular agreement .

When applied to the treebanks we used ( see Section 3 ) , this step resulted in between two ( English ) and 21 ( Russian ) constructions per lan-guage .

English has the poorest morphology and consequently the lowest number of patterns with identifiable morphological agreement .

Only the VP-conjunction construction ( Fig . 1c ) was identified in all four languages .

Subject-verb agreement constructions were extracted in all languages but Russian ; Russian has relatively flexible word order and a noun dependent preceding a head verb is not necessarily its subject .

The full list of extracted constructions in English and Italian is given in Tables 2 and 3 , respectively .

For the other languages , see the Supplementary Material ( SM ) .

Our " original " sentence test set included all sentences from each construction where all words from the cue and up to and including the target occurred in the LM vocabulary ( Section 3 ) , and where the singular / plural counterpart of the target occurred in the treebank and in the language model vocabulary ( this is required by the evaluation procedure outlined below ) .

The total counts of constructions and original sentences in our test sets are provided in Table 1 .

The average number of context words separating the cue and the target ranged from 3.6 ( Hebrew ) to 4.5 ( Italian ) .

We generated nine nonce variants of each original sentence as follows .

Each content word ( noun , verb , adjective , proper noun , numeral , adverb ) in the sentence was substituted by another random content word from the treebank with matching POS and morphological features .

To avoid forms that are ambiguous between several POS , which are particularly frequent in English ( e.g. , plural noun and singular verb forms ) , we excluded the forms that appeared with a different POS more than 10 % of the time in the treebank .

Function words ( determiners , pronouns , adpositions , particles ) and punctuation were left intact .

For example , we generated the nonce ( 1b ) from the original sentence ( 1a ):

( ( e.g. , " it stays the shuttle " in ( 1b ) ) .

For each sentence in our test set , we retrieved from our treebank the form that is identical to the agreement target in all morphological features except number ( e.g. , " finds " instead of " find " in ( 1b ) ) .

Given a sentence with prefix p up to and excluding the target , we then compute the probabilities P ( t 1 |p ) and P ( t 2 |p ) for the singular and plural variants of the target , t 1 and t 2 , based on the language model .

Following Linzen et al . ( 2016 ) , we say that the model identified the correct target if it assigned a higher probability to the form with the correct number .

In ( 1b ) , for example , the model should assign a higher probability to " finds " than " find".3 3 Experimental setup Treebanks .

We extracted our test sets from the Italian , English , Hebrew and Russian Universal Dependency treebanks ( UD , v2.0 , Nivre et al . , 2016 ) .

The English and Hebrew treebanks were post-processed to obtain a richer morphological annotation at the word level ( see SM for details ) .

We ran an extensive analysis of the abilities of RNNs trained on a generic language-modeling task to predict long-distance number agreement .

Results were consistent across four languages and a number of constructions .

They were above strong baselines even in the challenging case of nonsense sentences , and not far from human performance .

We are not aware of other collections of human long-distance agreement judgments on nonsensical sentences , and we thus consider our publicly available data set an important contribution of our work , of interest to students of human language processing in general .

The constructions we considered are quite infrequent ( according to a rough estimate based on the treebanks , the language in which they are most common is Hebrew , and even there they occur with average 0.8 % sentence frequency ) .

Moreover , they vary in the contexts that separate the cue and the target .

So , RNNs are not simply memorizing frequent morphosyntactic sequences ( which would already be impressive , for systems learning from raw text ) .

We tentatively conclude that LMtrained RNNs can construct abstract grammatical representations of their input .

This , in turn , suggests that the input itself contains enough information to trigger some form of syntactic learning in a system , such as an RNN , that does not contain an explicit prior bias in favour of syntactic structures .

In future work , we would like to better understand what kind of syntactic information RNNs are encoding , and how .

On the one hand , we plan to adapt methods to inspect information flow across RNN states ( e.g. , Hupkes et al . , 2017 ) .

On the other , we would like to expand our empirical investigation by focusing on other long-distance phenomena , such as overt case assignment ( Blake , 2001 ) or parasitic gap licensing ( Culicover and Postal , 2001 ) .

While it is more challenging to extract reliable examples of such phenomena from corpora , their study would probe more sophisticated syntactic capabilities , possibly even shedding light on the theoretical analysis of the underlying linguistic structures .

Finally , it may be useful to complement the corpus-driven approach used in the current paper with constructed evaluation sentences that isolate particular syntactic phenomena , independent of their frequency in a natural corpus , as is common in psycholinguistics ( Enguehard et al . , 2017 ) .

Their experiments confirmed that RNNs can , in principle , handle such constructions .

However , in their study RNNs could only succeed when provided with explicit supervision on the target task .

We investigate here to what extent RNNs learn to track abstract hierarchical syntactic structure .

RNNs trained with a LM objective solve the long-distance agreement problem well , even on nonce sentences .

The pattern is consistent across languages , and , crucially , not far from human performance in Italian .

Results were consistent across four languages and a number of constructions .

They were above strong baselines even in the challenging case of nonsense sentences , and not far from human performance .

We tentatively conclude that LMtrained RNNs can construct abstract grammatical representations of their input .


Current system combination methods usually use confusion networks to find consensus translations among different systems .

Requiring one-to-one mappings between the words in candidate translations , confusion networks have difficulty in handling more general situations in which several words are connected to another several words .

Instead , we propose a lattice-based system combination model that allows for such phrase alignments and uses lattices to encode all candidate translations .

Experiments show that our approach achieves significant improvements over the state-ofthe-art baseline system on Chinese-to-English translation test sets .

System combination aims to find consensus translations among different machine translation systems .

It has been proven that such consensus translations are usually better than the output of individual systems ( Frederking and Nirenburg , 1994 ) .

In recent several years , the system combination methods based on confusion networks developed rapidly ( Bangalore et al . , 2001 ; Matusov et al . , 2006 ; Sim et al . , 2007 ; Rosti et al . , 2007a ; Rosti et al . , 2007b ; Rosti et al . , 2008 ; He et al . , 2008 ) , which show state-of-the-art performance in benchmarks .

A confusion network consists of a sequence of sets of candidate words .

Each candidate word is associated with a score .

The optimal consensus translation can be obtained by selecting one word from each set to maximizing the overall score .

To construct a confusion network , one first need to choose one of the hypotheses ( i.e. , candidate translations ) as the backbone ( also called " skeleton " in the literature ) and then decide the word alignments of other hypotheses to the backbone .

Hypothesis alignment plays a crucial role in confusionnetwork-based system combination because it has a direct effect on selecting consensus translations .

However , a confusion network is restricted in such a way that only 1-to-1 mappings are allowed in hypothesis alignment .

This is not the fact even for word alignments between the same languages .

It is more common that several words are connected to another several words .

For example , " be capable of " and " be able to " have the same meaning .

Although confusion-network-based approaches resort to inserting null words to alleviate this problem , they face the risk of producing degenerate translations such as " be capable to " and " be able of " .

In this paper , we propose a new system combination method based on lattices .

As a more general form of confusion network , a lattice is capable of describing arbitrary mappings in hypothesis alignment .

In a lattice , each edge is associated with a sequence of words rather than a single word .

Therefore , we select phrases instead of words in each candidate set and minimize the chance to produce unexpected translations such as " be capable to " .

We compared our approach with the state-of-the-art confusion-network-based system ( He et al . , 2008 ) and achieved a significant absolute improvement of 1.23 BLEU points on the NIST 2005 Chinese-to-English test set and 0.93 BLEU point on the NIST 2008 Chinese-to-English test set .

The alignment model plays an important role in system combination .

Because of the expression limitation of confusion networks , only 1-to-1 mappings are employed in the confusion-network-based model .

This paper proposes a lattice-based system combination model .

As a general form of confusion networks , lattices can express n-to-n mappings .

a confusion-network-based model processes words only .

As a result , phrase pairs must be extracted before constructing a lattice .

On NIST MT05 test set , the lattice-based system gave better results with an absolute improvement of 1.23 BLEU points over the confusionnetwork-based system ( He et al . , 2008 ) and 3.73 BLEU points over the best single system .

On NIST MT08 test set , the lattice-based system outperformed the confusion-network-based system by 0.93 BLEU point and outperformed the best single system by 3.0 BLEU points .


The growing demand for structured knowledge has led to great interest in relation extraction , especially in cases with limited supervision .

However , existing distance supervision approaches only extract relations expressed in single sentences .

In general , cross-sentence relation extraction is under-explored , even in the supervised-learning setting .

In this paper , we propose the first approach for applying distant supervision to crosssentence relation extraction .

At the core of our approach is a graph representation that can incorporate both standard dependencies and discourse relations , thus providing a unifying way to model relations within and across sentences .

We extract features from multiple paths in this graph , increasing accuracy and robustness when confronted with linguistic variation and analysis error .

Experiments on an important extraction task for precision medicine show that our approach can learn an accurate cross-sentence extractor , using only a small existing knowledge base and unlabeled text from biomedical research articles .

Compared to the existing distant supervision paradigm , our approach extracted twice as many relations at similar precision , thus demonstrating the prevalence of cross-sentence relations and the promise of our approach .

The accelerating pace in technological advance and scientific discovery has led to an explosive growth in knowledge .

The ensuing information overload creates new urgency in assimilating frag-mented knowledge for integration and reasoning .

A salient case in point is precision medicine ( Bahcall , 2015 ) .

The cost of sequencing a person 's genome has fallen below $ 10001 , enabling individualized diagnosis and treatment of complex genetic diseases such as cancer .

The availability of measurement for 20,000 human genes makes it imperative to integrate all knowledge about them , which grows rapidly and is scattered in millions of articles in PubMed2 .

Traditional extraction approaches require annotated examples , which makes it difficult to scale to the explosion of extraction demands .

Consequently , there has been increasing interest in indirect supervision ( Banko et al . , 2007 ; Poon and Domingos , 2009 ; Toutanova et al . , 2015 ) , with distant supervision ( Craven et al . , 1998 ; Mintz et al . , 2009 ) emerging as a particularly promising paradigm for augmenting existing knowledge bases from unlabeled text ( Poon et al . , 2015 ; Parikh et al . , 2015 ) .

This progress is exciting , but distantsupervision approaches have so far been limited to single sentences , thus missing out on relations crossing the sentence boundary .

Consider the following example:"The p56Lck inhibitor Dasatinib was shown to enhance apoptosis induction by dexamethasone in otherwise GC-resistant CLL cells .

This finding concurs with the observation by Sade showing that Notch-mediated resistance of a mouse lymphoma cell line could be overcome by inhibiting p56Lck . "

Together , the two sentences convey the fact that the drug Dasatinib could overcome resistance conferred by mutations to the Notch gene , which can not be inferred from either sentence alone .

The impact of missed opportunities is especially pronounced in the long tail of knowledge .

Such information is crucial for integrative reasoning as it includes the newest findings in specialized domains .

In this paper , we present DISCREX , the first approach for distant supervision to relation extraction beyond the sentence boundary .

The key idea is to adopt a document-level graph representation that augments conventional intra-sentential dependencies with new dependencies introduced for adjacent sentences and discourse relations .

It provides a unifying way to derive features for classifying relations between entity pairs .

As we augment this graph with new arcs , the number of possible paths between entities grow .

We demonstrate that feature extraction along multiple paths leads to more robust extraction , allowing the learner to find structural patterns even when the language varies or the parser makes an error .

The cross-sentence scenario presents a new challenge in candidate selection .

This motivates our concept of minimal-span candidates in Section 3.2 .

Excluding non-minimal candidates substantially improves classification accuracy .

There is a long line of research on discourse phenomena , including coreference ( Haghighi and Klein , 2007 ; Poon and Domingos , 2008 ; Rahman and Ng , 2009 ; Raghunathan et al . , 2010 ) , narrative structures ( Chambers and Jurafsky , 2009 ; Cheung et al . , 2013 ) , and rhetorical relations ( Marcu , 2000 ) .

For the most part , this work has not been connected to relation extraction .

Our proposed extraction framework makes it easy to integrate such discourse relations .

Our experiments evaluated the impact of coreference and discourse parsing , a preliminary step toward in-depth integration with discourse research .

We conducted experiments on extracting druggene interactions from biomedical literature , an important task for precision medicine .

By bootstrapping from a recently curated knowledge base ( KB ) with about 162 known interactions , our DIS-CREX system learned to extract inter-sentence drug-gene interactions at high precision .

Crosssentence extraction doubled the yield compared to single-sentence extraction .

Overall , by applying distant supervision , we extracted about 64,000 distinct interactions from about one million PubMed Central full-text articles , attaining two orders of magnitude increase compared to the original KB .

We present the first approach for applying distant supervision to cross-sentence relation extraction , by adopting a document-level graph representation that incorporates both intra-sentential dependencies and inter-sentential relations such as adjacency and discourse relations .

We conducted both automatic and manual evaluation on extracting drug-gene interactions from biomedical literature .

With cross-sentence extraction , our DIS-CREX system doubled the yield of unique interactions , while maintaining the same accuracy .

Using distant supervision , DISCREX improved the coverage of the Gene Drug Knowledge Database ( GDKD ) by two orders of magnitude , without requiring annotated examples .

Future work includes : further exploration of features ; improved integration with coreference and discourse parsing ; combining distant supervision with active learning and crowd sourcing ; evaluate the impact of extractions to precision medicine ; applications to other domains .


Convolution tree kernel has shown promising results in semantic role classification .

However , it only carries out hard matching , which may lead to over-fitting and less accurate similarity measure .

To remove the constraint , this paper proposes a grammardriven convolution tree kernel for semantic role classification by introducing more linguistic knowledge into the standard tree kernel .

The proposed grammar-driven tree kernel displays two advantages over the previous one : 1 ) grammar-driven approximate substructure matching and 2 ) grammardriven approximate tree node matching .

The two improvements enable the grammardriven tree kernel explore more linguistically motivated structure features than the previous one .

Experiments on the CoNLL-2005 SRL shared task show that the grammardriven tree kernel significantly outperforms the previous non-grammar-driven one in SRL .

Moreover , we present a composite kernel to integrate feature-based and tree kernel-based methods .

Experimental results show that the composite kernel outperforms the previously best-reported methods .

Given a sentence , the task of Semantic Role Labeling ( SRL ) consists of analyzing the logical forms expressed by some target verbs or nouns and some constituents of the sentence .

In particular , for each predicate ( target verb or noun ) all the constituents in the sentence which fill semantic arguments ( roles ) of the predicate have to be recognized .

Typical semantic roles include Agent , Patient , Instrument , etc . and also adjuncts such as Locative , Temporal , Manner , and Cause , etc .

Generally , semantic role identification and classification are regarded as two key steps in semantic role labeling .

Semantic role identification involves classifying each syntactic element in a sentence into either a semantic argument or a non-argument while semantic role classification involves classifying each semantic argument identified into a specific semantic role .

This paper focuses on semantic role classification task with the assumption that the semantic arguments have been identified correctly .

Both feature-based and kernel-based learning methods have been studied for semantic role classification ( Carreras and Màrquez , 2004 ; Carreras and Màrquez , 2005 ) .

In feature-based methods , a flat feature vector is used to represent a predicateargument structure while , in kernel-based methods , a kernel function is used to measure directly the similarity between two predicate-argument structures .

As we know , kernel methods are more effective in capturing structured features .

Moschitti ( 2004 ) and Che et al . ( 2006 ) used a convolution tree kernel ( Collins and Duffy , 2001 ) for semantic role classification .

The convolution tree kernel takes sub-tree as its feature and counts the number of common sub-trees as the similarity between two predicate-arguments .

This kernel has shown very promising results in SRL .

However , as a general learning algorithm , the tree kernel only carries out hard matching between any two sub-trees without considering any linguistic knowledge in kernel design .

This makes the kernel fail to handle similar phrase structures ( e.g. , " buy a car " vs. " buy a red car " ) and near-synonymic grammar tags ( e.g. , the POS variations between " high / JJ degree / NN " 1 and " higher / JJR degree / NN " ) 2 .

To some degree , it may lead to over-fitting and compromise performance .

This paper reports our preliminary study in addressing the above issue by introducing more linguistic knowledge into the convolution tree kernel .

To our knowledge , this is the first attempt in this research direction .

In detail , we propose a grammar-driven convolution tree kernel for semantic role classification that can carry out more linguistically motivated substructure matching .

Experimental results show that the proposed method significantly outperforms the standard convolution tree kernel on the data set of the CoNLL-2005 SRL shared task .

The remainder of the paper is organized as follows : Section 2 reviews the previous work and Section 3 discusses our grammar-driven convolution tree kernel .

Section 4 shows the experimental results .

In this paper , we propose a novel grammar-driven convolution tree kernel for semantic role classification .

More linguistic knowledge is considered in the new kernel design .

The experimental results verify that the grammar-driven kernel is more effective in capturing syntactic structure features than the previous convolution tree kernel because it allows grammar-driven approximate matching of substructures and node features .

We also discuss the criteria to determine the optional nodes in a CFG rule in defining our grammar-driven convolution tree kernel .

The extension of our work is to improve the performance of the entire semantic role labeling system using the grammar-driven tree kernel , including all four stages : pruning , semantic role identification , classification and post inference .

In addition , a more interesting research topic is to study how to integrate linguistic knowledge and tree kernel methods to do feature selection for tree kernelbased NLP applications ( Suzuki et al . , 2004 ) .

In detail , a linguistics and statistics-based theory that can suggest the effectiveness of different substructure features and whether they should be generated or not by the tree kernels would be worked out .


We explore the problem of resolving the second person English pronoun you in multi-party dialogue , using a combination of linguistic and visual features .

First , we distinguish generic and referential uses , then we classify the referential uses as either plural or singular , and finally , for the latter cases , we identify the addressee .

In our first set of experiments , the linguistic and visual features are derived from manual transcriptions and annotations , but in the second set , they are generated through entirely automatic means .

Results show that a multimodal system is often preferable to a unimodal one .

The English pronoun you is the second most frequent word in unrestricted conversation ( after I and right before it ) .

Despite this , with the exception of Gupta et al . ( 2007b ; 2007a ) , its resolution has received very little attention in the literature .

This is perhaps not surprising since the vast amount of work on anaphora and reference resolution has focused on text or discourse -mediums where second-person deixis is perhaps not as prominent as it is in dialogue .

For spoken dialogue pronoun resolution modules however , resolving you is an essential task that has an important impact on the capabilities of dialogue summarization systems .

Besides being important for computational implementations , resolving you is also an interesting and challenging research problem .

As for third person pronouns such as it , some uses of you are not strictly referential .

These include discourse marker uses such as you know in example ( 1 ) , and generic uses like ( 2 ) , where you does not refer to the addressee as it does in ( 3 ) .

( 1 ) It 's not just , you know , noises like something hitting .

( 2 ) Often , you need to know specific button sequences to get certain functionalities done .

However , unlike it , you is ambiguous between singular and plural interpretations -an issue that is particularly problematic in multi-party conversations .

While you clearly has a plural referent in ( 4 ) , in ( 3 ) the number of its referent is ambiguous .

I do n't know if you guys have any questions .

When an utterance contains a singular referential you , resolving the you amounts to identifying the individual to whom the utterance is addressed .

This is trivial in two-person dialogue since the current listener is always the addressee , but in conversations with multiple participants , it is a complex problem where different kinds of linguistic and visual information play important roles ( Jovanovic , 2007 ) .

One of the issues we investigate here is how this applies to the more concrete problem of resolving the second person pronoun you .

We approach this issue as a three-step problem .

Using the AMI Meeting Corpus ( McCowan et al . , 2005 ) of multi-party dialogues , we first discriminate between referential and generic uses of you .

Then , within the referential uses , we distinguish between singular and plural , and finally , we resolve the singular referential instances by identifying the intended addressee .

We use multimodal features : initially , we extract discourse features from manual transcriptions and use visual information derived from manual annotations , but then we move to a fully automatic approach , using 1-best transcriptions produced by an automatic speech recognizer ( ASR ) and visual features automatically extracted from raw video .

In the next section of this paper , we give a brief overview of related work .

We describe our data in Section 3 , and explain how we extract visual and linguistic features in Sections 4 and 5 respectively .

Section 6 then presents our experiments with manual transcriptions and annotations , while Section 7 , those with automatically extracted information .

We have investigated the automatic resolution of the second person English pronoun you in multi-party dialogue , using a combination of linguistic and visual features .

We conducted a first set of experiments where our features were derived from manual transcriptions and annotations , and then a second set where they were generated by entirely automatic means .

To our knowledge , this is the first attempt at tackling this problem using automatically extracted multimodal information .

Our experiments showed that visual information can be highly predictive in resolving the addressee of singular referential uses of you .

Visual features significantly improved the performance of both our manual and automatic systems , and the latter achieved an encouraging 75 % accuracy .

We also found that our visual features had predictive power for distinguishing between generic and referential uses of you , and between referential singulars and plurals .

Indeed , for the latter task , they significantly improved the manual system 's performance .

The listeners ' gaze features were useful here : in our data set it was apparently the case that the speaker would often use the whiteboard / projector screen when addressing the group , thus drawing the listeners ' gaze in this direction .

Future work will involve expanding our dataset , and investigating new potentially predictive features .

In the slightly longer term , we plan to integrate the resulting system into a meeting assistant whose purpose is to automatically extract useful information from multi-party meetings .


There is a growing research interest in opinion retrieval as on-line users ' opinions are becoming more and more popular in business , social networks , etc .

Practically speaking , the goal of opinion retrieval is to retrieve documents , which entail opinions or comments , relevant to a target subject specified by the user 's query .

A fundamental challenge in opinion retrieval is information representation .

Existing research focuses on document-based approaches and documents are represented by bag-of-word .

However , due to loss of contextual information , this representation fails to capture the associative information between an opinion and its corresponding target .

It can not distinguish different degrees of a sentiment word when associated with different targets .

This in turn seriously affects opinion retrieval performance .

In this paper , we propose a sentence-based approach based on a new information representation , namely topic-sentiment word pair , to capture intra-sentence contextual information between an opinion and its target .

Additionally , we consider inter-sentence information to capture the relationships among the opinions on the same topic .

Finally , the two types of information are combined in a unified graph-based model , which can effectively rank the documents .

Compared with existing approaches , experimental results on the COAE08 dataset showed that our graph-based model achieved significant improvement .

In recent years , there is a growing interest in sharing personal opinions on the Web , such as product reviews , economic analysis , political polls , etc .

These opinions can not only help independent users make decisions , but also obtain valuable feedbacks ( Pang et al . , 2008 ) .

Opinion oriented research , including sentiment classifica-tion , opinion extraction , opinion question answering , and opinion summarization , etc . are receiving growing attention ( Wilson , et al . , 2005 ; Liu et al . , 2005 ; Oard et al . , 2006 ) .

However , most existing works concentrate on analyzing opinions expressed in the documents , and none on how to represent the information needs required to retrieve opinionated documents .

In this paper , we focus on opinion retrieval , whose goal is to find a set of documents containing not only the query keyword(s ) but also the relevant opinions .

This requirement brings about the challenge on how to represent information needs for effective opinion retrieval .

In order to solve the above problem , previous work adopts a 2-stage approach .

In the first stage , relevant documents are determined and ranked by a score , i.e. tf-idf value .

In the second stage , an opinion score is generated for each relevant document ( Macdonald and Ounis , 2007 ; Oard et al . , 2006 ) .

The opinion score can be acquired by either machine learning-based sentiment classifiers , such as SVM ( Zhang and Yu , 2007 ) , or a sentiment lexicons with weighted scores from training documents ( Amati et al . , 2007 ;

Hannah et al . , 2007 ; Na et al . , 2009 ) .

Finally , an overall score combining the two is computed by using a score function , e.g. linear combination , to re-rank the retrieved documents .

Retrieval in the 2-stage approach is based on document and document is represented by bag-of-word .

This representation , however , can only ensure that there is at least one opinion in each relevant document , but it can not determine the relevance pairing of individual opinion to its target .

In general , by simply representing a document in bag-of-word , contextual information i.e. the corresponding target of an opinion , is neglected .

This may result in possible mismatch between an opinion and a target and in turn affects opinion retrieval performance .

By the same token , the effect to documents consisting of mul-tiple topics , which is common in blogs and on-line reviews , is also significant .

In this setting , even if a document is regarded opinionated , it can not ensure that all opinions in the document are indeed relevant to the target concerned .

Therefore , we argue that existing information representation i.e. bag-of-word , can not satisfy the information needs for opinion retrieval .

In this paper , we propose to handle opinion retrieval in the granularity of sentence .

It is observed that a complete opinion is always expressed in one sentence , and the relevant target of the opinion is mostly the one found in it .

Therefore , it is crucial to maintain the associative information between an opinion and its target within a sentence .

We define the notion of a topic-sentiment word pair , which is composed of a topic term ( i.e. the target ) and a sentiment word ( i.e. opinion ) of a sentence .

Word pairs can maintain intra-sentence contextual information to express the potential relevant opinions .

In addition , inter-sentence contextual information is also captured by word pairs to represent the relationship among opinions on the same topic .

In practice , the inter-sentence information reflects the degree of a word pair .

Finally , we combine both intra-sentence and inter-sentence contextual information to construct a unified undirected graph to achieve effective opinion retrieval .

The rest of the paper is organized as follows .

In Section 2 , we describe the motivation of our approach .

Section 3 presents a novel unified graph-based model for opinion retrieval .

We evaluated our model and the results are presented in Section 4 .

We review related works on opinion retrieval in Section 5 .

Finally , in Section 6 , the paper is concluded and future work is suggested .

In this work we focus on the problem of opinion retrieval .

Different from existing approaches , which regard document relevance as the key indicator of opinion relevance , we propose to explore the relevance of individual opinion .

To do that , opinion retrieval is performed in the granularity of sentence .

We define the notion of word pair , which can not only maintain the association between the opinion and the corresponding target in the sentence , but it can also build up the relationship among sentences through the same word pair .

Furthermore , we convert the relationships between word pairs and sentences into a unified graph , and use the HITS algorithm to achieve document ranking for opinion retrieval .

Finally , we compare our approach with existing methods .

Experimental results show that our proposed model performs well on COAE08 dataset .

The novelty of our work lies in using word pairs to represent the information needs for opinion retrieval .

On the one hand , word pairs can identify the relevant opinion according to intra-sentence contextual information .

On the other hand , word pairs can measure the degree of a relevant opinion by taking inter-sentence contextual information into consideration .

With the help of word pairs , the information needs for opinion retrieval can be represented appropriately .

In the future , more research is required in the following directions :

( 1 ) Since word pairs can indicate relevant opinions effectively , it is worth further study on how they could be applied to other opinion oriented applications , e.g. opinion summarization , opinion prediction , etc . ( 2 ) The characteristics of blogs will be taken into consideration , i.e. , the post time , which could be helpful to create a more time sensitivity graph to filter out fake opinions .

( 3 ) Opinion holder is another important role of an opinion , and the identification of opinion holder is a main task in NTCIR .

It would be interesting to study opinion holders , e.g. its seniority , for opinion retrieval .

A fundamental challenge in opinion retrieval is information representation .

In this paper , we propose to handle opinion retrieval in the granularity of sentence .

Experimental results show that our proposed model performs well on COAE08 dataset .


We describe a syntax-based algorithm that automatically builds Finite State Automata ( word lattices ) from semantically equivalent translation sets .

These FSAs are good representations of paraphrases .

They can be used to extract lexical and syntactic paraphrase pairs and to generate new , unseen sentences that express the same meaning as the sentences in the input sets .

Our FSAs can also predict the correctness of alternative semantic renderings , which may be used to evaluate the quality of translations .

In the past , paraphrases have come under the scrutiny of many research communities .

Information retrieval researchers have used paraphrasing techniques for query reformulation in order to increase the recall of information retrieval engines ( Sparck Jones and Tait , 1984 ) .

Natural language generation researchers have used paraphrasing to increase the expressive power of generation systems ( Iordanskaja et al . , 1991 ;

And researchers in multi-document text summarization ( Barzilay et al . , 1999 ) , information extraction ( Shinyama et al . , 2002 ) , and question answering ( Lin and Pantel , 2001 ; Hermjakob et al . , 2002 ) have focused on identifying and exploiting paraphrases in the context of recognizing redundancies , alternative formulations of the same meaning , and improving the performance of question answering systems .

In previous work ( Barzilay and McKeown , 2001 ; Lin and Pantel , 2001 ; Shinyama et al . , 2002 ) , paraphrases are represented as sets or pairs of semantically equivalent words , phrases , and patterns .

Although this is adequate in the context of some applications , it is clearly too weak from a generative perspective .

Assume , for example , that we know that text pairs ( stock market rose , stock market gained ) and ( stock market rose , stock prices rose ) have the same meaning .

If we memorized only these two pairs , it would be impossible to infer that , in fact , consistent with our intuition , any of the following sets of phrases are also semantically equivalent : { stock market rose , stock market gained , stock prices rose , stock prices gained } and { stock market , stock prices } in the context of rose or gained ; { market rose } , { market gained } , { prices rose } and { prices gained } in the context of stock ; and so on .

In this paper , we propose solutions for two problems : the problem of paraphrase representation and the problem of paraphrase induction .

We propose a new , finite-statebased representation of paraphrases that enables one to encode compactly large numbers of paraphrases .

We also propose algorithms that automatically derive such representations from inputs that are now routinely released in conjunction with large scale machine translation evaluations ( DARPA , 2002 ) : multiple English translations of many foreign language texts .

For instance , when given as input the 11 semantically equivalent English translations in Figure 1 , our algorithm automatically induces the FSA in Figure 2 , which represents compactly 49 distinct renderings of the same semantic meaning .

Our FSAs capture both lexical paraphrases , such as { fighting , bat-tle } , { died , were killed } and structural paraphrases such as { last week 's fighting , the battle of last week } .

The contexts in which these are correct paraphrases are also conveniently captured in the representation .

In previous work , Langkilde and Knight ( 1998 ) used word lattices for language generation , but their method involved hand-crafted rules .

Bangalore et al . ( 2001 ) and Barzilay and Lee ( 2002 ) both applied the technique of multi-sequence alignment ( MSA ) to align parallel corpora and produced similar FSAs .

For their purposes , they mainly need to ensure the correctness of consensus among different translations , so that different constituent orderings in input sentences do not pose a serious prob- lem .

In contrast , we want to ensure the correctness of all paths represented by the FSAs , and direct application of MSA in the presence of different constituent orderings can be problematic .

For example , when given as input the same sentences in Figure 1 , one instantiation of the MSA algorithm produces the FSA in Figure 3 , which contains many " bad " paths such as the battle of last week 's fighting took at least 12 people lost their people died in the fighting last week 's fighting ( See Section 4.2.2 for a more quantitative analysis . ) .

It 's still possible to use MSA if , for example , the input is pre-clustered to have the same constituent ordering ( Barzilay and Lee ( 2003 ) ) .

But we chose to approach this problem from another direction .

As a result , we propose a new syntax-based algorithm to produce FSAs .

In this paper , we first introduce the multiple translation corpus that we use in our experiments ( see Section 2 ) .

We then present the algorithms that we developed to induce finite-state paraphrase representations from such data ( see Section 3 ) .

An important part of the paper is dedicated to evaluating the quality of the finite-state representations that we derive ( see Section 4 ) .

Since our representations encode thousands and sometimes millions of equivalent verbalizations of the same meaning , we use both manual and automatic evaluation techniques .

Some of the automatic evaluations we perform are novel as well .

In this paper , we presented a new syntax-based algorithm that learns paraphrases from a newly available dataset .

The multiple translation corpus that we use in this paper is the first instance in a series of similar corpora that are built and made publicly available by LDC in the context of a series of DARPA-sponsored MT evaluations .

The algorithm we proposed constructs finite state representations of paraphrases that are useful in many contexts : to induce large lists of lexical and structural paraphrases ; to generate semantically equivalent renderings of a given meaning ; and to estimate the quality of machine translation systems .

More experiments need to be carried out in order to assess extrinsically whether the FSAs we produce can be used to yield higher agreement scores between human and automatic assessments of translation quality .

In our future work , we wish to experiment with more flexible merging algorithms and to integrate better the top-down and bottom-up processes that are used to in-duce FSAs .

We also wish to extract more abstract paraphrase patterns from the current representation .

Such patterns are more likely to get reused -which would help us get reliable statistics for them in the extraction phase , and also have a better chance of being applicable to unseen data .


While sophisticated neural-based techniques have been developed in reading comprehension , most approaches model the answer in an independent manner , ignoring its relations with other answer candidates .

This problem can be even worse in open-domain scenarios , where candidates from multiple passages should be combined to answer a single question .

In this paper , we formulate reading comprehension as an extract-then-select twostage procedure .

We first extract answer candidates from passages , then select the final answer by combining information from all the candidates .

Furthermore , we regard candidate extraction as a latent variable and train the two-stage process jointly with reinforcement learning .

As a result , our approach has improved the state-ofthe-art performance significantly on two challenging open-domain reading comprehension datasets .

Further analysis demonstrates the effectiveness of our model components , especially the information fusion of all the candidates and the joint training of the extract-then-select procedure .

Teaching machines to read and comprehend human languages is a long-standing objective in natural language processing .

In order to evaluate this ability , reading comprehension ( RC ) is designed to answer questions through reading relevant passages .

In recent years , RC has attracted intense interest .

Various advanced neural models have been proposed along with newly released datasets ( Hermann et al . , 2015 ; Rajpurkar et al . , 2016 ; Dunn et al . , 2017 ; Dhingra et al . , 2017b ; He et al . , 2017 ) .

In this paper , we formulate the problem of RC as a two-stage process , which first generates candidates with an extraction model , then selects the final answer by combining the information from all the candidates .

Furthermore , we treat candidate extraction as a latent variable and jointly train these two stages with RL .

Experiments on public open-domain RC datasets Quasar-T and SearchQA show the necessity of introducing the selection model and the effectiveness of fusing candidates information when modeling .

Moreover , our joint training strategy leads to significant improvements in performance .


We present two discriminative methods for name transliteration .

The methods correspond to local and global modeling approaches in modeling structured output spaces .

Both methods do not require alignment of names in different languages -their features are computed directly from the names themselves .

We perform an experimental evaluation of the methods for name transliteration from three languages ( Arabic , Korean , and Russian ) into English , and compare the methods experimentally to a state-of-theart joint probabilistic modeling approach .

We find that the discriminative methods outperform probabilistic modeling , with the global discriminative modeling approach achieving the best performance in all languages .

Name transliteration is an important task of transcribing a name from alphabet to another .

For example , an Arabic ,"وﻟﻴﺎم" Korean " 윌리엄 " , and Russian " Вильям " all correspond to English " William " .

We address the problem of transliteration in the general setting : it involves trying to recover original English names from their transcription in a foreign language , as well as finding an acceptable spelling of a foreign name in English .

We apply name transliteration in the context of cross-lingual information extraction .

Name extractors are currently available in multiple languages .

Our goal is to make the extracted names understandable to monolingual English speakers by transliterating the names into English .

The extraction context of the transliteration application imposes additional complexity constraints on the task .

In particular , we aim for the transliteration speed to be comparable to that of extraction speed .

Since most current extraction systems are fairly fast ( > 1 Gb of text per hour ) , the complexity requirement reduces the range of techniques applicable to the transliteration .

More precisely , we can not use WWW and the web count information to hone in on the right transliteration candidate .

Instead , all relevant transliteration information has to be represented within a compact and self-contained transliteration model .

We present two methods for creating and applying transliteration models .

In contrast to most previous transliteration approaches , our models are discriminative .

Using an existing transliteration dictionary D ( a set of name pairs { ( f , e ) } ) , we learn a function that directly maps a name f from one language into a name e in another language .

We do not estimate either direct conditional p(e|f ) or reverse conditional p(f|e ) or joint p(e , f ) probability models .

Furthermore , we do away with the notion of alignment : our transliteration model does not require and is not defined of in terms of aligned e and f.

Instead , all features used by the model are computed directly from the names f and e without any need for their alignment .

The two discriminative methods that we present correspond to local and global modeling paradigms for solving complex learning problems with structured output spaces .

In the local setting , we learn linear classifiers that predict a letter e i from the previously predicted letters e 1 … e i-1 and the original name f.

In the global setting , we learn a function W mapping a pair ( f , e ) into a score W(f , e)∈ R. The function W is linear in features computed from the pair ( f , e ) .

We describe the pertinent feature spaces as well as pre-sent both training and decoding algorithms for the local and global settings .

We perform an experimental evaluation for three language pairs ( transliteration from Arabic , Korean , and Russian into English ) comparing our methods to a joint probabilistic modeling approach to transliteration , which was shown to deliver superior performance .

We show experimentally that both discriminative methods outperform the probabilistic approach , with global discriminative modeling achieving the best performance in all languages .

We presented two novel discriminative approaches to name transliteration that do not employ the notion of alignment .

We showed experimentally that the approaches lead to superior experimental results in all languages , with the global discriminative modeling approach achieving the best performance .

The results are somewhat surprising , for the notion of alignment seems very intuitive and useful for transliteration .

We will investigate whether similar alignment-free methodology can be extended to full-text translation .

It will also be interesting to study the relationship between our discriminative alignment-free methods and recently proposed discriminative alignment-based methods for transliteration and translation ( Taskar et al . 2005a ; Moore 2005 ) .

We also showed that for name transliteration , global discriminative modeling is superior to local classifier-based discriminative modeling .

This may have resulted from poor calibration of scores and probabilities produced by individual classifiers .

We plan to further investigate the relationship between the local and global approaches to complex learning problems in natural language .


Motivated by the success of T5 ( Text-To-Text Transfer Transformer ) in pre-trained natural language processing models , we propose a unified-modal SpeechT5 framework that explores the encoder-decoder pre-training for self-supervised speech / text representation learning .

The SpeechT5 framework consists of a shared encoder-decoder network and six modal-specific ( speech / text ) pre / post-nets .

After preprocessing the input speech / text through the pre-nets , the shared encoder-decoder network models the sequence-to-sequence transformation , and then the post-nets generate the output in the speech / text modality based on the output of the decoder .

Leveraging large-scale unlabeled speech and text data , we pre-train SpeechT5 to learn a unified-modal representation , hoping to improve the modeling capability for both speech and text .

To align the textual and speech information into this unified semantic space , we propose a cross-modal vector quantization approach that randomly mixes up speech / text states with latent units as the interface between encoder and decoder .

Extensive evaluations show the superiority of the proposed SpeechT5 framework on a wide variety of spoken language processing tasks , including automatic speech recognition , speech synthesis , speech translation , voice conversion , speech enhancement , and speaker identification .

We release our code and model at https://github.com / microsoft/ SpeechT5 .

Starting with ELMo ( Peters et al . , 2018 ) and BERT ( Devlin et al . , 2019 ) , substantial work has shown that pre-trained models can significantly improve in various natural language processing ( NLP ) tasks Figure 1 :

An illustration of the SpeechT5 framework , which treats spoken language processing tasks as a speech / text to speech / text format , including automatic speech recognition ( ASR ) , speech translation ( ST ) , speech identification ( SID ) , text to speech ( TTS ) , voice conversion ( VC ) , and speech enhancement ( SE ) .

( Radford et al . , 2019 ; CONNEAU and Lample , 2019 ;

Yang et al . , 2019 ; Dong et al . , 2019 ; Lewis et al . , 2020 ) .

Following the pre-training techniques in NLP , self-supervised speech representation learning has also been investigated and shown promising results , benefiting from richly learned representations ( Chung and Glass , 2018 ; Chuang et al . , 2020 ; Song et al . , 2019 ; Baevski et al . , 2020 ; Wang et al . , 2021 ; Hsu et al . , 2021 ; Chung et al . , 2021a ) , such as wav2vec 2.0 ( Baevski et al . , 2020 ) and HuBERT ( Hsu et al . , 2021 ) .

However , previous speech pre-training work suffers from two problems : ( 1 ) most of them learn the speech representation with only unlabeled speech data but ignore the importance of textual data to spoken language tasks ( e.g. , automatic speech recognition ) which require the modality transformation ; ( 2 ) most of these models solely rely on a pre-trained speech encoder for various downstream tasks , leaving the decoder not pre-trained for the sequence-to-sequence generation tasks .

How to design a unified encoder-decoder model that can take advantage of both unlabeled speech and text data to improve various spoken language processing tasks is not well explored .

Inspired by the T5 method ( Raffel et al . , 2020 ) , we attempt to formulate each spoken language processing task as a speech / text to speech / text problem via an encoder-decoder framework , which enables us to use the same pre-trained model with bimodal data across diverse tasks , as shown in Figure 1 .

To achieve this , we propose a unifiedmodal pre-training framework , SpeechT5 , containing an encoder-decoder backbone network and modal-specific pre / post-nets .

With the pre-nets , the input speech / text is embedded in a shared space , and the encoder-decoder backbone network models the sequence-to-sequence conversion , from which the model-specific post-nets generate the speech / text output .

Particularly , SpeechT5 is mainly pre-trained with a denoising sequence-tosequence method by leveraging large-scale unlabeled text and speech corpus .

To align the textual and acoustic information into a unified semantic space , the proposed SpeechT5 model ( 1 ) maps text and speech representations into a shared vector quantization space , and ( 2 ) randomly mixes up the quantized latent representations and the contextual states , which can better guide the quantizer to learn the cross-modal features .

We fine-tune SpeechT5 on a wide variety of downstream spoken language processing tasks , including automatic speech recognition ( ASR ) , textto-speech ( TTS ) , speech translation ( ST ) , voice conversion ( VC ) , speech enhancement ( SE ) , and speaker identification ( SID ) .

Massive experiments show that the proposed SpeechT5 model achieves a significant improvement on these spoken language processing tasks compared with the state-of-theart baselines .

Specifically , the proposed SpeechT5 outperforms wav2vec 2.0 ( Baevski et al . , 2020 ) and HuBERT ( Hsu et al . , 2021 ) with the BASE model on the ASR task and also performs better than the state-of-the-art voice Transformer network ( Huang et al . , 2021 ) on the VC task .

Besides , SpeechT5 is significantly superior to SpeechNet ( Chen et al . , 2021b ) and pre-trained models from SUPERB ( Yang et al . , 2021 ) and achieves the stateof-the-art performance ( i.e. , 96.49 % ) on the SID task .

We further provide an empirical comparison of the pre-training tasks and modules , and the ablation study demonstrates the effectiveness of the proposed joint speech-text pre-training method .

The contributions of this paper are summarized as follows .

• To the best of our knowledge , this is the first work to investigate a unified encoder-decoder framework for various spoken language processing tasks .

• We propose a cross-modal vector quantization approach , which learns the implicit alignment between acoustic and textual representation with large-scale unlabeled speech and text data .

• Extensive experiments on spoken language processing tasks demonstrate the effectiveness and superiority of the proposed SpeechT5 model .

In this paper , we have proposed SpeechT5 as a pretrained encoder-decoder model for various spoken language processing tasks .

We convert all spoken language processing tasks into a speech / text to speech / text format and propose a novel joint pretraining method to utilize cross-modal information by leveraging the unlabeled speech and text data .

The proposed unified encoder-decoder model can support generation tasks such as speech translation and voice conversion .

Massive experiments show that SpeechT5 significantly outperforms all baselines in several spoken language processing tasks .

In the future , we are going to pre-train the SpeechT5 with a larger model and more unlabeled data .

We are also interested in extending the proposed SpeechT5 framework to address multilingual spoken language processing tasks for future work .


Recent studies on extractive text summarization formulate it as a combinatorial optimization problem such as a Knapsack Problem , a Maximum Coverage Problem or a Budgeted Median Problem .

These methods successfully improved summarization quality , but they did not consider the rhetorical relations between the textual units of a source document .

Thus , summaries generated by these methods may lack logical coherence .

This paper proposes a single document summarization method based on the trimming of a discourse tree .

First , we propose rules for transforming a rhetorical structure theorybased discourse tree into a dependency-based discourse tree , which allows us to take a treetrimming approach to summarization .

Second , we formulate the problem of trimming a dependency-based discourse tree as a Tree Knapsack Problem , then solve it with integer linear programming ( ILP ) .

Evaluation results showed that our method improved ROUGE scores .

State-of-the-art extractive text summarization methods regard a document ( or a document set ) as a set of textual units ( e.g. sentences , clauses , phrases ) and formulate summarization as a combinatorial optimization problem , i.e. selecting a subset of the set of textual units that maximizes an objective without violating a length constraint .

For example , Mc-Donald ( 2007 ) formulated text summarization as a Knapsack Problem , where he selects a set of textual units that maximize the sum of significance scores of each unit .

Filatova et al . ( 2004 ) proposed a summarization method based on a Maximum Coverage Problem , in which they select a set of textual units that maximizes the weighted sum of the conceptual units ( e.g. unigrams ) contained in the set .

Although , their greedy solution is only an approximation , Takamura et al . ( 2009a ) extended it to obtain the exact solution .

More recently , Takamura et al . ( 2009b ) regarded summarization as a Budgeted Median Problem and obtain exact solutions with integer linear programming .

These methods successfully improved ROUGE ( Lin , 2004 ) scores , but they still have one critical shortcoming .

Since these methods are based on subset selection , the summaries they generate can not preserve the rhetorical structure of the textual units of a source document .

Thus , the resulting summary may lack coherence and may not include significant textual units from a source document .

One powerful and potential way to overcome the problem is to include discourse tree constraints in the summarization procedure .

Marcu ( 1998 ) regarded a document as a Rhetorical Structure Theory ( RST ) ( William Charles , Mann and Sandra Annear , Thompson , 1988 ) -based discourse tree ( RST-DT ) and selected textual units according to a preference ranking derived from the tree structure to make a summary .

Daumé et al . ( 2002 ) proposed a document compression method that directly models the probability of a summary given an RST-DT by using a noisy-channel model .

These methods generate well-organized summaries , however , since they do not formulate summarizations as combinatorial op-Figure 1 : Example RST-DT from ( Marcu , 1998 ) .

timization problems , the optimality of the generated summaries is not guaranteed .

In this paper , we propose a single document summarization method based on the trimming of a discourse tree based on the Tree Knapsack Problem .

If a discourse tree explicitly represents parent-child relationships between textual units , we can apply the well-known tree-trimming approach to a discourse tree and reap the benefit of combinatorial optimization methods .

In other words , to apply the treetrimming approach , we need a tree whose all nodes represent textual units .

Unfortunately , the RST-DT does not allow it , because textual units in the RST-DT are located only on leaf nodes and parent-child relationship between textual units are represented implicitly at higher positions in a tree .

Therefore , we first propose rules that transform an RST-DT into a dependency-based discourse tree ( DEP-DT ) that explicitly defines the parent-child relationships .

Second , we treat it as a rooted subtree selection , in other words , a Tree Knapsack Problem and formulate the problem as an ILP .

This paper proposed rules for transforming an RST-DT to a DEP-DT to obtain the parent-child relationships between EDUs .

We treated a single document summarization method as a Tree Knapsack Problem , i.e. the summarizer selects the best rooted subtree from a DEP-DT .

To demonstrate the effectiveness of our method , we conducted an experimental evaluation using 30 documents selected from the RST Discourse Treebank Corpus .

The results showed that our method achieved the highest ROUGE-1,2 scores .