docs/parser/test/docling_output/355.pdf.json

{
  "name": "355.pdf",
  "metadata": {
    "source": "docling-granite",
    "title": "Neural Modeling of Multi-Predicate Interactions for Japanese Predicate Argument Structure Analysis",
    "authors": [],
    "emails": [],
    "sections": [
      {
        "heading": "Anonymous ACL submission",
        "text": ""
      },
      {
        "heading": "1 Introduction",
        "text": "Predicate argument structure (PAS) analysis is a basic semantic analysis task, in which systems are required to identify semantic units of a sentence, such as who did what to whom . In pro-drop languages such as Japanese, Chinese and Italian, arguments are often omitted in text, and such argument omission is regarded as one of the most problematic issues of PAS analysis (Iida and Poesio, 2011; Sasano and Kurohashi, 2011).\n\nAs an approach to the argument omission problem, in Japanese PAS analysis, joint modeling of interactions between multiple predicates has been gaining popularity and achieved the state-of-theart result (Ouchi et al., 2015; Shibata et al., 2016). This approach is based on the linguistic intuition that the predicates in a sentence are semantically related to each other and the interaction information can be a clue for PAS analysis. However, to model such multi-predicate interactions , this ap- Figure 1: An example of Japanese PAS. The upper edges denote dependency relations, and the under edges denote case arguments. ' NOM ' and ' ACC ' represent the nominative and accusative arguments, respectively. ' ϕ i ' is a zero pronoun , referring to the antecedent ' 男 i (man i )'.\n\nproach heavily relies on syntactic information predicted by parsers and suffers from the error propagation caused by the pipeline processing.\n\nTo remedy this problem, we propose a neural model which automatically induces features sensitive to multi-predicate interactions from word sequence information of a sentence. This model takes as input all predicates and their argument candidates in a sentence at a time, and captures the interactions using grid-type recurrent neural networks (Grid-RNN) without syntactic information.\n\nIn this paper, we firstly introduce a basic model using RNNs, which independently estimates arguments of each predicate without considering the multi-predicate interactions (Sec. 3). Then, extending this model, we propose a neural model using Grid-RNNs (Sec. 4).\n\nPerforming experiments on the NAIST Text Corpus (Iida et al., 2007), we demonstrate that our neural models exceed the accuracy of the stateof-the-art Japanese PAS analyzer (Ouchi et al., 2015). In particular, the neural model using GridRNNs achieves the best result, which suggests that our grid-type neural architecture effectively captures multi-predicate interactions. 1\n\n1 Our source code is publicly available at http:xxx\n\nFigure 2: An overview of the neural models: (i) single-sequence and (ii) multi-sequence models."
      },
      {
        "heading": "2 Japanese Predicate Argument Structure Analysis",
        "text": ""
      },
      {
        "heading": "2.1 Task Description",
        "text": "In Japanese PAS analysis, we identify arguments taking part in the three major case roles, nominative ( NOM ), accusative ( ACC ) and dative ( DAT ) cases, for each predicate. Arguments can be divided into the following three categories according to the positions relative to their predicates (Hayashibe et al., 2011; Ouchi et al., 2015):\n\nDep : The arguments that have direct syntactic dependency with the predicate.\n\nZero : The arguments referred to by zero pronouns within the same sentence, which have no direct syntactic dependency with the predicate.\n\nInter-Zero : The arguments referred to by zero pronouns out of the same sentence.\n\nFor example, in Figure 1, the nominative argument ' 警察 (police)' for the predicate ' 逮捕した (arrested)' is regarded as a Dep argument since the argument has a direct syntactic dependency with the predicate. In contrast, the nominative argument ' 男 i (man i )' for the predicate ' 逃⾛した (escaped)' is regarded as a Zero argument since the argument has no direct syntactic dependency with the predicate.\n\nIn this paper, we focus on the analysis for these intra-sentential arguments, i.e., Dep and Zero . In order to identify inter-sentential arguments ( Inter-Zero ), it is required to search a much broader space, such as the whole document, resulting in a much harder analysis than intrasentential arguments. 2 Thus, Ouchi et al. (2015)\n\n2 The F-measure remains 10-20% (Taira et al., 2008; Imamura et al., 2009; Sasano and Kurohashi, 2011).\n\nand Shibata et al. (2016) focused on only intrasentential argument analysis. Following this trend, we focus on intra-sentential argument analysis."
      },
      {
        "heading": "2.2 Challenging Problem",
        "text": "Arguments are often omitted in Japanese sentences. In Figure 1, ϕ i represents the omitted argument, called zero pronoun . This zero pronoun ϕ i refers to ' 男 i (man i )'. In Japanese PAS analysis, when an argument of the target predicate is omitted, we have to identify the antecedent of the omitted argument ( Zero argument).\n\nThe analysis for such Zero arguments is much more difficult than that for Dep arguments because of the lack of direct syntactic dependencies. For Dep arguments, the syntactic dependency between an argument and its predicate is a strong clue. In the sentence in Figure 1, for the predicate ' 逮捕し た (arrested)', the nominative argument is ' 警察 (police)'. This argument can easily be identified by relying on the syntactic dependency. In contrast, since the nominative argument ' 男 i (man i )' has no syntactic dependency with its predicate ' 逃 ⾛した (escaped)', we have to use other information for such zero argument identification.\n\nAs an solution to this problem, we exploit two kinds of information: (i) context in the entire sentence and (ii) multi-predicate interactions. For the former, we introduce single-sequence model , which induces context-sensitive representations from a sequence of argument candidates of a predicate. For the latter, we introduce multi-sequence model , which induces predicate-sensitive representations from multiple sequences of argument candidates of all predicates in a sentence (shown in Figure 2).\n\nFigure 3: The overall architecture of the single sequence model. This model consists of three components: (i) Input Layer, (ii) RNN Layer and (iii) Output Layer."
      },
      {
        "heading": "3 Single-Sequence Model",
        "text": "The single-sequence model exploits stacked bidirectional RNNs (Bi-RNN) (Schuster and Paliwal, 1997; Graves et al., 2005, 2013; Zhou and Xu, 2015). Figure 3 shows the overall architecture, which consists of the following three components:\n\nInput Layer: Map each word to a feature vector representation.\n\nRNN Layer: Produce high-level feature vectors using Bi-RNNs.\n\nOutput Layer: Compute the probability of each case label for each word using the softmax function.\n\nIn the following subsections, we describe each of them in more detail."
      },
      {
        "heading": "3.1 Input Layer",
        "text": "Given an input sentence w 1:T = ( w 1 , · · · , w T ) and a predicate p , each word w t is mapped to a feature representation x t , which is the concatenation ( ⊕ ) of three types of vectors:\n\nx _ t = x _ t ^ a r g oplus x _ t ^ p r e d oplus x _ t ^ m a r k quad ( 1 ) quad ≥slant 1\n\nwhere each vector is based on the following atomic features inspired by Zhou and Xu (2015):\n\nARG : Word index of each word.\n\nPRED : Word index of the target predicate and words around the predicate.\n\nMARK : Binary index that represents whether the word is the predicate or not.\n\nFigure 4: An example of the feature extraction. The underlined word is the target predicate. From the sentence ' 彼⼥はパンを⾷べた。 (She ate a bread.)', the three types of features are extracted for the target predicate ' ⾷べた (ate)'.\n\nFigure 5: An example of the process of creating a feature vector. The extracted features are mapped to each vector, and all the vectors are concatenated into one feature vector.\n\nFigure 4 presents an example of the atomic features. As the ARG feature, we extract a word index x word ∈ V of each word. Similarly, as the PRED feature, we extract each word index x word of the C words taking the target predicate at the center, where C is the window size. The MARK feature x mark ∈ { 0 , 1 } is a binary value that represents whether the word is the predicate or not.\n\nThen, using feature indices, we extract feature vector representations from each embedding matrix. Figure 5 shows the process of creating the feature vector x 1 for the word w 1 ' 彼⼥ (she)'. We set two embedding matrices: (i) word embedding matrix E word ∈ R d word ×|V| , and (ii) mark embedding matrix E mark ∈ R d mark × 2 . From each embedding matrix, we extract corresponding column vectors and concatenate them as a feature vector representation x t based on Eq. 1.\n\nEach feature vector x t is multiplied with a parameter matrix W x :\n\nh _ t ^ ( 0 ) = W _ x x _ t quad ( 2 ) quad stackrel i n n s e\n\nThe vector h (0) t is given to the first layer of the RNN layers as input."
      },
      {
        "heading": "3.2 RNN Layer",
        "text": "In the RNN layers, feature vectors are updated recurrently using Bi-RNNs. Bi-RNNs process an input sequence from the left-to-right manner in odd-numbered layers and the opposite in evennumbered layers. By stacking these layers, we can construct the deeper network structures.\n\nStacked Bi-RNNs consist of L layers, and the hidden state in the layer ℓ ∈ (1 , · · · , L ) is calculated as follows:\n\nh _ t ^ ( ell ) = begincases g ^ ( ell ) ( h _ t ^ ( ell - 1 ) , h _ t - 1 ^ ( ell ) ) & ( ell = odd ) \\ g ^ ( ell ) ( h _ t ^ ( ell - 1 ) , h _ t + 1 ^ ( ell ) ) & ( ell = even ) endcases quad in the more\n\nBoth of the odd and even-numbered layers receive h ( ℓ -1) t , the t -th hidden state of the ℓ -1 layer, as the first input of the function g ( ℓ ) , which is an arbitrary function 3 . As the second input of g ( ℓ ) , odd-numbered layers receive h ( ℓ ) t -1 while even-numbered layers receive h ( ℓ ) t +1 . By calculating the hidden states until the L -th layer, we obtain a hidden state sequence h ( L ) 1:T = ( h ( L ) 1 , · · · , h ( L ) T ) . Using each vector h ( L ) t , we calculate the probability of case labels for each word in the output layer."
      },
      {
        "heading": "3.3 Output Layer",
        "text": "In the output layer, multi-class classification is performed using the softmax function:\n\ny _ t = softmax ( W _ y h _ t ^ ( L ) )\n\nwhere h ( L ) t is a vector representation propagated from the last RNN layer (Fig 3). Each element of y t is a probability value corresponding to each label. The label with the maximum probability among them is output as a result. In this task, there are five labels: NOM , ACC , DAT , PRED , null . The labels NOM , ACC and DAT indicate the nominative, accusative and dative case, respectively. PRED is the label for the predicate. null represents a word that does not play any case role.\n\n3 In this work, we use the Gated Recurrent Unit (GRU) (Cho et al., 2014) as the function g ( ℓ ) ."
      },
      {
        "heading": "4 Multi-Sequence Model",
        "text": "While the single-sequence model assumes the independence between predicates, the multisequence model assumes the multi-predicate interactions . To capture such interactions between all predicates in a sentence, we extend the singlesequence model to the multi-sequence model using Grid-RNNs (Graves and Schmidhuber, 2009; Kalchbrenner et al., 2016). Figure 6 presents the overall architecture of the multi-sequence model, which consists of three components:\n\nInput Layer: Map words to M sequences of feature vectors for M predicates.\n\nGrid Layer: Update the hidden states over different sequences using Grid-RNNs.\n\nOutput Layer: Compute the probability of each case label for each word using the softmax function.\n\nIn the following subsections, we describe them in more detail."
      },
      {
        "heading": "4.1 Input Layer",
        "text": "The multi-sequence model takes as input a sentence w 1:T = ( w 1 , · · · , w T ) and all predicates { p m } M 1 in the sentence. For each predicate p m , the input layer creates a sequence of feature vectors X m = ( x m, 1 , · · · , x m, T ) by mapping each input word w t to a feature vector x m,t based on Eq 1. That is, for M predicates, M sequences of feature vectors { X m } M 1 are created.\n\nThen, using Eq. 2, each feature vector x m,t is mapped to h (0) m,t , and a feature sequence is created for a predicate p m , i.e., H (0) m = ( h (0) m, 1 , · · · , h (0) m, T ). Consequently, for M predicates, we obtain M feature sequences { H (0) m } M 1 ."
      },
      {
        "heading": "4.2 Grid Layer",
        "text": ""
      },
      {
        "heading": "Inter-Sequence Connections",
        "text": "In the grid layers, we use Grid-RNNs to propagate the feature information over the different sequences ( inter-sequence connections ). The right figure in Figure 6 shows an odd-numbered layer of the Grid layers. The hidden state is recurrently calculated from upper-left ( m = 1 , t = 1 ) to lowerright ( m = M,t = T ).\n\nFormally, in the ℓ -th layer, the hidden state h ( ℓ ) m,t is calculated as follows:\n\nh _ m , t ^ ( ell ) = begincases g ^ ( ell ) ( h _ m , t ^ ( ell - 1 ) oplus h _ m - 1 , t ^ ( ell ) , h _ m , t - 1 ^ ( ell ) ) & ( ell = odd ) \\ g ^ ( ell ) ( h _ m , t ^ ( ell - 1 ) oplus h _ m + 1 , t ^ ( ell ) , h _ m , t + 1 ^ ( ell ) ) & ( ell = even ) endcases\n\nFigure 6: The overall architecture of the multi-sequence model: an example of three sequences.\n\nThis equation is similar to Eq. 3. The main difference is that the hidden state of a neighboring sequence, h ( ℓ ) m -1 ,t (or h ( ℓ ) m +1 ,t ), is concatenated ( ⊕ ) with the hidden state of the previous ( ℓ -1 ) layer, h ( ℓ -1) m,t , and is taken as input of the function g ( ℓ ) .\n\nIn the right figure in Figure 6, the blue curve lines represent the inter-sequence connections. Taking as input the hidden states of neighboring sequences, the network propagates feature information over multiple sequences (predicates). By calculating the hidden states until the L -th layer, we obtain M sequences of the hidden states, i.e., { H ( L ) m } M 1 , in which H ( L ) m = ( h ( L ) m, 1 , · · · , h ( L ) m, T ) ."
      },
      {
        "heading": "Residual Connections",
        "text": "As more layers are stacked, it gets more difficult to learn the model parameters due to some problems such as gradient vanishment (Pascanu et al., 2013). In this work, we integrate residual connections (He et al., 2015; Wu et al., 2016) with our networks to connect between layers. Specifically, the input vector h ( ℓ -1) m,t of the ℓ -th layer is added to the output vector h ( ℓ ) m,t . Residual connections can also be applied to the single-sequence model, and thus we perform the experiments on both models with/without residual connections."
      },
      {
        "heading": "4.3 Output Layer",
        "text": "Like the single-sequence model, using the softmax function, we calculate the probability of case labels of each word w t for each predicate p m :\n\ny _ m , t = softmax ( W _ y h _ m , t ^ ( L ) )\n\nwhere h ( L ) m,t is a hidden state vector calculated in the last Grid Layer."
      },
      {
        "heading": "5 Related Work",
        "text": ""
      },
      {
        "heading": "5.1 Japanese PAS Analysis Approaches",
        "text": "Existing approaches for Japanese PAS analysis are divided into two categories: (i) pointwise approach and (ii) joint approach . In the pointwise approach, we estimate the score of each argument candidate for one predicate, and select the argument candidate with the maximum score as an argument (Taira et al., 2008; Imamura et al., 2009; Hayashibe et al., 2011; Iida et al., 2016). In the joint approach, we select the score of all the predicate-argument combinations in one sentence, and select the combination with the highest score (Yoshikawa et al., 2011; Sasano and Kurohashi, 2011; Ouchi et al., 2015; Shibata et al., 2016). Compared with the pointwise approach, the joint approach has achieved better results."
      },
      {
        "heading": "5.2 Multi-Predicate Interactions",
        "text": "Ouchi et al. (2015) reported that it is beneficial to Japanese PAS analysis to capture the interactions between all predicates in a sentence. This is based on the linguistic intuition that the predicates in a sentence are semantically related to each other, and the interaction information can be a clue for PAS analysis.\n\nSimilarly, in semantic role labeling (SRL), Yang and Zong (2014) also reported that their reranking model capturing the multi-predicate interactions is effective for the English constituentbased SRL task (Carreras and M` arquez, 2005). Taking a step further in this direction, we propose the neural architecture that effectively models the multi-predicate interactions."
      },
      {
        "heading": "5.3 Neural Approaches",
        "text": ""
      },
      {
        "heading": "Japanese PAS",
        "text": "In recent years, several attempts have been made to apply neural networks for Japanese PAS analysis (Shibata et al., 2016; Iida et al., 2016) 4 . Shibata et al. (2016) used a feed-forward neural network for the score calculation part of the joint model proposed by Ouchi et al. (2015). Iida et al. (2016) used multi-column convolutional neural networks for the zero anaphora resolution task.\n\nBoth models exploited syntactic and selectional preference information as atomic features of neural networks. Using neural networks, the good performance was realized with mitigating the cost of manually designing combination features. In this work, we demonstrate that even without such syntactic information, our neural models realize the state-of-the-art performance by using word sequence information of a sentence."
      },
      {
        "heading": "English SRL",
        "text": "Some neural models achieved high performance without syntactic information in English SRL. Collobert et al. (2011) worked on the English constituent-based SRL task (Carreras and M` arquez, 2005). Their model exploited a convolutional neural network and achieved 74.15% in F-measure without syntactic information. Zhou and Xu (2015) also worked on the same task using bidirectional RNNs with CRF and achieved the state-of-the-art result, 81.07% in F-measure. Our models can be regarded as an extension from their model.\n\nThe main differences between Zhou and Xu (2015) and our work are: (i) the constituent vs dependency-based argument identification and (ii) the multi-predicate consideration. In the constituent-based SRL, since systems are required to identify the spans of arguments for each predicate, Zhou and Xu (2015) used CRF to capture the IOB label dependencies. In contrast, in Japanese dependency-based PAS analysis, since arguments are infrequently adjacent to each other, we replaced the CRF with the softmax function. Also, while the model of Zhou and Xu (2015) predicts arguments for each predicate independently, our multi-sequence model jointly predicts arguments for all predicates in a sentence at a time by considering the multi-predicate interactions.\n\n4 These previous studies used unpublished datasets and evaluated the performance with different experimental settings, so we cannot compare their models with ours."
      },
      {
        "heading": "6 Experiments",
        "text": ""
      },
      {
        "heading": "6.1 Experimental Settings",
        "text": ""
      },
      {
        "heading": "Dataset",
        "text": "We use the NAIST Text Corpus 1.5, which consists of 40,000 sentences of Japanese newspaper text (Iida et al., 2007). In the experiments, we adopt the standard data splits (Taira et al., 2008; Imamura et al., 2009; Ouchi et al., 2015):\n\nTrain: Articles: Jan 1-11,\n\nEditorials: Jan-Aug\n\nDev: Articles: Jan 12-13, Editorials: Sept\n\nTest: Articles: Jan 14-17, Editorials: Oct-Dec\n\nWe use the word boundaries annotated to the NAIST Text Corpus and the target predicates that have at least one argument in the same sentence. We do not use any external resources."
      },
      {
        "heading": "Learning",
        "text": "We train the model parameters by minimizing the cross-entropy loss function:\n\nL ( θ ) = - ∑ _ n ∑ _ t log P ( y _ t | x _ t ) + ( λ )/( 2 ) | | θ | | ^ 2 \\\\\n\nwhere θ is a set of model parameters, and the hyper-parameter λ is the coefficient governing the L2 weight decay."
      },
      {
        "heading": "Implementation Details",
        "text": "We implement our neural models using Theano (Bastien et al., 2012). The number of epochs is set to 50, and we report the result of the test set in the epoch with the best F-measure of the development set. Parameter optimization is done by stochastic gradient descent method (SGD) using mini-batch, whose size is selected from { 2 , 4 , 8 } . The learning rate is automatically adjusted using Adam (Kingma and Ba, 2014). For the L2 weight decay, the hyper-parameter λ in Eq. 4 is selected from { 0 . 001 , 0 . 0005 , 0 . 0001 } .\n\nIn the neural models, the number of the RNN and Grid layers are selected from { 2 , 4 , 6 , 8 } . The window size C for the PRED feature (Sec. 3.1) is set to 5 . Words with frequency 2 or more are mapped to each word index, and the remaining words are mapped to the unknown word index. The dimensions d word and d mark of the embeddings are set to 32 . In the single-sequence model, the parameters of GRUs are set to 32 × 32 . In the multi-sequence model, the parameters of GRUs related to the input values are set to 64 × 32 , and\n\nDep Zero All Imamura+ 09 85.06 41.65 78.15 Ouchi+ 15 86.07 44.09 79.23 Single-Seq 88.10 46.10 81.15 Multi-Seq 88.17 47.12 81.42\n\n†\n\n†\n\n†\n\nTable 1: F-measures in the test set. Single-Seq is the single-sequence model, and Multi-Seq is the multi-sequence model. Imamura+ 09 is the model of Imamura et al. (2009) reimplemented by Ouchi et al. (2015), and Ouchi+ 15 is the ALLCases Joint Model of Ouchi et al. (2015). The mark † denotes the significantly better results with the significance level p < 0 . 05 comparing between Single-Seq and Multi-Seq .\n\nothers are 32 × 32 . The initial values of all the parameters are sampled according to the uniform distribution from [ -√ 6 √ row + col , √ 6 √ row + col ] , where row and col are the number of rows and columns of each matrix, respectively."
      },
      {
        "heading": "Baseline Models",
        "text": "We compare our models with the models in the previous works (Sec. 5.1) that use the NAIST Text Corpus 1.5. As a baseline of the pointwise approach, we use the pointwise model 5 of Imamura et al. (2009). In addition, as a baseline of the joint approach, we use the model of Ouchi et al. (2015), which achieved the best result on the NAIST Text Corpus 1.5."
      },
      {
        "heading": "6.2 Results",
        "text": ""
      },
      {
        "heading": "Neural Models vs Baseline Models",
        "text": "Table 1 presents F-measures of our neural sequence models with 8 RNN or Grid layers and the baseline models on the test set, in which as the significent test, we used the bootstrap resampling method. In all the metrics, both of the single-sequence ( Single-Seq ) and multisequence model ( Multi-Seq ) outperformed the baseline models. This confirms that our neural sequence models realize high-performance even without syntactic information by learning contextual information effective for PAS analysis from a word sequence of the sentence.\n\nIn particular, for zero arguments ( Zero ), our models achieved a considerable improvement compared with the state-of-the-art model of Ouchi et al. (2015), i.e., the single model improved around 2.0 points and the multi-sequence model improved around 3.0 points in F-measure. These results suggest that it is beneficial to Japanese PAS analysis, particularly to the zero argument identification, to model the context in the entire sentence using RNNs.\n\n5 We compared the results of the model reimplemented by Ouchi et al. (2015).\n\nL Single-Seq.+ res. Single-Seq.- res. Multi-Seq.+ res. Multi-Seq.- res. Dep 87.34 87.10 87.43 87.73 2 Zero 47.98 47.90 47.66 46.93 All 80.62 80.24 80.71 80.68 Dep 87.27 87.41 87.60 87.09 4 Zero 50.43 50.83 48.10 48.58 All 80.92 80.99 80.99 80.59 Dep 87.73 87.11 88.04 87.39 6 Zero 48.81 49.51 48.98 48.91 All 81.05 80.63 81.19 80.68 Dep 87.98 87.23 87.65 87.07 8 Zero 47.40 48.38 49.34 48.23 All 81.31 80.33 81.33 80.40\n\nTable 2: Performance comparison for different numbers of layers on the development set in Fmeasures. L is the number of the RNN or Grid layers. + res. or -res. indicates whether the model has the residual connections ( + ) or not ( -)."
      },
      {
        "heading": "Effects of Multiple Predicate Consideration",
        "text": "As Table 1 shows, the multi-sequence model significantly outperformed the single-sequence model in F-measure in total (81.42% vs 81.15%). This result demonstrates that the grid-type neural architecture can effectively capture the multipredicate interactions by connecting between the sequences of the argument candidates for all predicates in a sentence.\n\nCompared with the single-sequence model for different argument types, the multi-sequence model achieved slightly but significantly better result for the direct dependency arguments ( Dep ) (88.10% vs 88.17%). In addition, for zero arguments ( Zero ), which have no syntactic dependency with its predicate, the multi-sequence model significantly outperformed the single-sequence model by around 1.0 points in F-measure (46.10% vs 47.12%). This shows that capturing the multipredicate interactions is particularly effective for zero arguments, which is consistent with the results of Ouchi et al. (2015).\n\nDep.NOM.NAIST Text Corpus 1.5 Dep.ACC.NAIST Text Corpus 1.5 Dep.DAT.NAIST Text Corpus 1.5 Zero.NOM.NAIST Text Corpus 1.5 Zero.ACC.NAIST Text Corpus 1.5 Zero.DAT.NAIST Text Corpus 1.5 Imamura+ 09 86.50 92.84 30.97 45.56 21.38 0.83 Ouchi+ 15 88.13 92.74 38.39 48.11 24.43 4.80 Single-Seq 88.32 93.89 65.91 49.51 35.07 9.83 Multi-Seq 88.75 93.68 64.38 50.65 32.35 7.52 NAIST Text Corpus 1.4 β NAIST Text Corpus 1.4 β NAIST Text Corpus 1.4 β NAIST Text Corpus 1.4 β NAIST Text Corpus 1.4 β NAIST Text Corpus 1.4 β Taira+ 08 * 75.53 88.20 89.51 30.15 11.41 3.66 Imamura+ 09 * 87.0 93.9 80.8 50.0 30.8 0.0 Sasano+ 11 * - - - 39.5 17.5 8.9\n\nTable 3: Performance comparison for different case roles on the test set in F-measures. NOM , ACC or DAT is the nominal, accusative or dative case, respectively. The mark * indicates that the model uses external resources."
      },
      {
        "heading": "Effects of Network Depth",
        "text": "Table 2 presents F-measures of the neural sequence models with different network depths and with/without residual connections. The performance tends to get better as the RNN or Grid layers get deeper with residual connections. In particular, the two models with 8 layers and residual connections achieved considerable improvements of around 1.0 point in F-measure compared the models without residual connections, which means that the residual connections contribute to the effective parameter learning of deeper models."
      },
      {
        "heading": "Comparison per Case Role",
        "text": "Table 3 shows F-measures for each case role. For reference, we show the results of the previous studies using NAIST Text Corpus 1.4 β with external resources as well. 6\n\nComparing between the models using the NAIST Text Corpus 1.5, the single-sequence and multi-sequence models outperformed the baseline models in all the metrics. In particular, for the dative case, the two neural models achieved much higher results by around 30 points. This suggests that although dative arguments appear infrequently compared with the other two case arguments, the neural models can robustly learn it.\n\nIn addition, for zero arguments ( Zero ), the neural models achieved better results than the baseline models. Especially, for zero arguments of the nominative case ( NOM ), the multi-sequence model achieved a considerable improvement of around 2.5 points in F-measure compared with the state-of-the-art model of Ouchi et al. (2015). To achieve high accuracies for the analysis of such zero arguments, it is necessary to capture long distance dependencies (Iida et al., 2005; Sasano and Kurohashi, 2011; Iida et al., 2015). Therefore, the improvements of the results suggest that the neural models effectively capture long distance dependencies using RNNs that can encode the context in the entire sentence.\n\n6 The major difference between NAIST Text Corpus 1.4 β and 1.5 is the revision of the annotation criterion for the dative case ( DAT ) (corresponding to Japanese case marker ' に '). Argument and adjunct usages of the case marker ' に ' are not distinguished in 1.4 β , making the identification of the dative case seemingly easy (Ouchi et al., 2015)."
      },
      {
        "heading": "7 Conclusion",
        "text": "In this work, we introduced neural sequence models that automatically induce effective feature representations from word sequence information of a sentence for Japanese PAS analysis. The experiments on NAIST Text Corpus 1.5 demonstrated that the models achieve the state-of-the-art result without syntactic information. In particular, our multi-sequence model improved the performance for zero argument identification, one of the problematic issues in Japanese PAS analysis, by considering the multi-predicate interactions using Grid-RNNs.\n\nSince our neural models are applicable to SRL, applying our models for multilingual SRL tasks is an interesting line of the future research. In addition, in this work, the model parameters were learned without any external resources. For future work, we plan to explore effective methods for exploiting large-scale unlabeled data to learn the neural models."
      }
    ],
    "references": [
      {
        "title": "Theano: new features and speed improvements",
        "author": [
          "Fr´ ed´ eric Bastien",
          "Pascal Lamblin",
          "Razvan Pascanu",
          "James Bergstra",
          "Ian J. Goodfellow",
          "Arnaud Bergeron",
          "Nicolas Bouchard",
          "Yoshua Bengio."
        ],
        "venue": "Deep Learning and Unsupervised Feature Learning NIPS 2012 Workshop.",
        "citeRegEx": "Bastien et al\\.,? 2012",
        "shortCiteRegEx": "Bastien et al\\.",
        "year": 2012
      },
      {
        "title": "Introduction to the conll-2005 shared task: Semantic role labeling",
        "author": [
          "Xavier Carreras",
          "Llu´ ıs M` arquez."
        ],
        "venue": "In Proceedings of CoNLL . pages 152-164.",
        "citeRegEx": "Carreras and arquez.,? 2005",
        "shortCiteRegEx": "Carreras and arquez.",
        "year": 2005
      },
      {
        "title": "Learning phrase representations using rnn encoder-decoder for statistical machine translation",
        "author": [
          "Kyunghyun Cho",
          "Bart van Merrienboer",
          "Caglar Gulcehre",
          "Dzmitry Bahdanau",
          "Fethi Bougares",
          "Holger Schwenk",
          "Yoshua Bengio."
        ],
        "venue": "In Proceedings of EMNLP . pages 1724-1734.",
        "citeRegEx": "Cho et al\\.,? 2014",
        "shortCiteRegEx": "Cho et al\\.",
        "year": 2014
      },
      {
        "title": "Natural language processing (almost) from scratch",
        "author": [
          "Ronan Collobert",
          "Jason Weston",
          "Leon Bottou",
          "Michael Karlen",
          "Koray Kavukcuoglu",
          "Pavel Kuksa."
        ],
        "venue": "Journal of Machine Learning Research .",
        "citeRegEx": "Collobert et al\\.,? 2011",
        "shortCiteRegEx": "Collobert et al\\.",
        "year": 2011
      },
      {
        "title": "Hybrid speech recognition with deep bidirectional lstm",
        "author": [
          "Alan Graves",
          "Navdeep Jaitly",
          "Abdel-rahman Mohamed."
        ],
        "venue": "In Proceedings of Automatic Speech Recognition and Understanding (ASRU), 2013 IEEE Workshop .",
        "citeRegEx": "Graves et al\\.,? 2013",
        "shortCiteRegEx": "Graves et al\\.",
        "year": 2013
      },
      {
        "title": "Bidirectional lstm networks for improved phoneme classification and recognition",
        "author": [
          "Alex Graves",
          "Santiago Fern´ andez",
          "J¨ urgen Schmidhuber."
        ],
        "venue": "In Proceedings of International Conference on Artificial Neural Networks . pages 799-804.",
        "citeRegEx": "Graves et al\\.,? 2005",
        "shortCiteRegEx": "Graves et al\\.",
        "year": 2005
      },
      {
        "title": "Offline handwriting recognition with multidimensional recurrent neural networks",
        "author": [
          "Alex Graves",
          "J¨ urgen Schmidhuber."
        ],
        "venue": "In Proceedings of NIPS . pages 545-552.",
        "citeRegEx": "Graves and Schmidhuber.,? 2009",
        "shortCiteRegEx": "Graves and Schmidhuber.",
        "year": 2009
      },
      {
        "title": "Japanese predicate argument structure analysis exploiting argument position and type",
        "author": [
          "Yuta Hayashibe",
          "Mamoru Komachi",
          "Yuji Matsumoto."
        ],
        "venue": "In Proceedings of IJCNLP . pages 201-209.",
        "citeRegEx": "Hayashibe et al\\.,? 2011",
        "shortCiteRegEx": "Hayashibe et al\\.",
        "year": 2011
      },
      {
        "title": "Deep residual learning for image recognition",
        "author": [
          "Kaiming He",
          "Xiangyu Zhang",
          "Shaoqing Ren",
          "Jian Sun."
        ],
        "venue": "arXiv preprint arXiv:1512.03385 .",
        "citeRegEx": "He et al\\.,? 2015",
        "shortCiteRegEx": "He et al\\.",
        "year": 2015
      },
      {
        "title": "Anaphora resolution by antecedent identification followed by anaphoricity determination",
        "author": [
          "Ryu Iida",
          "Kentaro Inui",
          "Yuji Matsumoto."
        ],
        "venue": "ACM Transactions on Asian Language Information Processing (TALIP) 4(4):417-434.",
        "citeRegEx": "Iida et al\\.,? 2005",
        "shortCiteRegEx": "Iida et al\\.",
        "year": 2005
      },
      {
        "title": "Annotating a japanese text corpus with predicate-argument and coreference relations",
        "author": [
          "Ryu Iida",
          "Mamoru Komachi",
          "Kentaro Inui",
          "Yuji Matsumoto."
        ],
        "venue": "In Proceedings of the Linguistic Annotation Workshop . pages 132-139.",
        "citeRegEx": "Iida et al\\.,? 2007",
        "shortCiteRegEx": "Iida et al\\.",
        "year": 2007
      },
      {
        "title": "A cross-lingual ilp solution to zero anaphora resolution",
        "author": [
          "Ryu Iida",
          "Massimo Poesio."
        ],
        "venue": "In Proceedings of ACL-HLT . pages 804-813.",
        "citeRegEx": "Iida and Poesio.,? 2011",
        "shortCiteRegEx": "Iida and Poesio.",
        "year": 2011
      },
      {
        "title": "Intrasentential zero anaphora resolution using subject sharing recognition",
        "author": [
          "Ryu Iida",
          "Kentaro Torisawa",
          "Chikara Hashimoto",
          "JongHoon Oh",
          "Julien Kloetzer."
        ],
        "venue": "In Proceedings of EMNLP . pages 2179-2189.",
        "citeRegEx": "Iida et al\\.,? 2015",
        "shortCiteRegEx": "Iida et al\\.",
        "year": 2015
      },
      {
        "title": "Intrasentential subject zero anaphora resolution using multi-column convolutional neural network",
        "author": [
          "Ryu Iida",
          "Kentaro Torisawa",
          "Jong-Hoon Oh",
          "Canasai Kruengkrai",
          "Julien Kloetzer."
        ],
        "venue": "In Proceedings of EMNLP . pages 1244-1254.",
        "citeRegEx": "Iida et al\\.,? 2016",
        "shortCiteRegEx": "Iida et al\\.",
        "year": 2016
      },
      {
        "title": "Discriminative approach to predicateargument structure analysis with zero-anaphora resolution",
        "author": [
          "Kenji Imamura",
          "Kuniko Saito",
          "Tomoko Izumi."
        ],
        "venue": "In Proceedings of ACL-IJCNLP . pages 8588.",
        "citeRegEx": "Imamura et al\\.,? 2009",
        "shortCiteRegEx": "Imamura et al\\.",
        "year": 2009
      },
      {
        "title": "Grid long short-term memory",
        "author": [
          "Nal Kalchbrenner",
          "Ivo Danihelka",
          "Alex Graves."
        ],
        "venue": "In Proceedings of ICLR .",
        "citeRegEx": "Kalchbrenner et al\\.,? 2016",
        "shortCiteRegEx": "Kalchbrenner et al\\.",
        "year": 2016
      },
      {
        "title": "Adam: A method for stochastic optimization",
        "author": [
          "D.P. Kingma",
          "J. Ba."
        ],
        "venue": "arXiv preprint arXiv: 1412.6980 .",
        "citeRegEx": "Kingma and Ba.,? 2014",
        "shortCiteRegEx": "Kingma and Ba.",
        "year": 2014
      },
      {
        "title": "Joint case argument identification for japanese predicate argument structure analysis",
        "author": [
          "Hiroki Ouchi",
          "Hiroyuki Shindo",
          "Kevin Duh",
          "Yuji Matsumoto."
        ],
        "venue": "In Proceedings of ACL-IJCNLP . pages 961970.",
        "citeRegEx": "Ouchi et al\\.,? 2015",
        "shortCiteRegEx": "Ouchi et al\\.",
        "year": 2015
      },
      {
        "title": "On the difficulty of training recurrent neural networks",
        "author": [
          "Razvan Pascanu",
          "Tomas Mikolov",
          "Yoshua Bengio."
        ],
        "venue": "In Proceedings of ICML .",
        "citeRegEx": "Pascanu et al\\.,? 2013",
        "shortCiteRegEx": "Pascanu et al\\.",
        "year": 2013
      },
      {
        "title": "A discriminative approach to japanese zero anaphora resolution with large-scale lexicalized case frames",
        "author": [
          "Ryohei Sasano",
          "Sadao Kurohashi."
        ],
        "venue": "In Proceedings of IJCNLP . pages 758-766.",
        "citeRegEx": "Sasano and Kurohashi.,? 2011",
        "shortCiteRegEx": "Sasano and Kurohashi.",
        "year": 2011
      },
      {
        "title": "Bidirectional recurrent neural networks",
        "author": [
          "Mike Schuster",
          "Kuldip K Paliwal."
        ],
        "venue": "IEEE Transactions on Signal Processing pages 2673-2681.",
        "citeRegEx": "Schuster and Paliwal.,? 1997",
        "shortCiteRegEx": "Schuster and Paliwal.",
        "year": 1997
      },
      {
        "title": "Neural network-based model for japanese predicate argument structure analysis",
        "author": [
          "Tomohide Shibata",
          "Daisuke Kawahara",
          "Sadao Kurohashi."
        ],
        "venue": "In Proceedings of ACL . pages 1235-1244.",
        "citeRegEx": "Shibata et al\\.,? 2016",
        "shortCiteRegEx": "Shibata et al\\.",
        "year": 2016
      },
      {
        "title": "A japanese predicate argument structure analysis using decision lists",
        "author": [
          "Hirotoshi Taira",
          "Sanae Fujita",
          "Masaaki Nagata."
        ],
        "venue": "In Proceedings of EMNLP . pages 523-532.",
        "citeRegEx": "Taira et al\\.,? 2008",
        "shortCiteRegEx": "Taira et al\\.",
        "year": 2008
      },
      {
        "title": "Google's neural machine translation system: Bridging the gap between human and machine translation",
        "author": [
          "Yonghui Wu",
          "Mike Schuster",
          "Zhifeng Chen",
          "Quoc V Le",
          "Mohammad Norouzi",
          "Wolfgang Macherey",
          "Maxim Krikun",
          "Yuan Cao",
          "Qin Gao",
          "Klaus Macherey",
          "et al."
        ],
        "venue": "arXiv preprint arXiv:1609.08144 .",
        "citeRegEx": "Wu et al\\.,? 2016",
        "shortCiteRegEx": "Wu et al\\.",
        "year": 2016
      },
      {
        "title": "Multipredicate semantic role labeling",
        "author": [
          "Haitong Yang",
          "Chengqing Zong."
        ],
        "venue": "In Proceedings of EMNLP . pages 363-373.",
        "citeRegEx": "Yang and Zong.,? 2014",
        "shortCiteRegEx": "Yang and Zong.",
        "year": 2014
      },
      {
        "title": "Jointly extracting japanese predicate-argument relation with markov logic",
        "author": [
          "Katsumasa Yoshikawa",
          "Masayuki Asahara",
          "Yuji Matsumoto."
        ],
        "venue": "In Proceedings of IJCNLP . pages 1125-1133.",
        "citeRegEx": "Yoshikawa et al\\.,? 2011",
        "shortCiteRegEx": "Yoshikawa et al\\.",
        "year": 2011
      },
      {
        "title": "End-to-end learning of semantic role labeling using recurrent neural networks",
        "author": [
          "Jie Zhou",
          "Wei Xu."
        ],
        "venue": "In Proceedings of ACL-IJCNLP .",
        "citeRegEx": "Zhou and Xu.,? 2015",
        "shortCiteRegEx": "Zhou and Xu.",
        "year": 2015
      }
    ],
    "referenceMentions": [
      {
        "referenceID": 11,
        "context": "In pro-drop languages such as Japanese, Chinese and Italian, arguments are often omitted in text, and such argument omission is regarded as one of the most problematic issues of PAS analysis (Iida and Poesio, 2011; Sasano and Kurohashi, 2011).",
        "startOffset": 192,
        "endOffset": 213
      },
      {
        "referenceID": 19,
        "context": "In pro-drop languages such as Japanese, Chinese and Italian, arguments are often omitted in text, and such argument omission is regarded as one of the most problematic issues of PAS analysis (Iida and Poesio, 2011; Sasano and Kurohashi, 2011).",
        "startOffset": 215,
        "endOffset": 241
      },
      {
        "referenceID": 17,
        "context": "As an approach to the argument omission problem, in Japanese PAS analysis, joint modeling of interactions between multiple predicates has been gaining popularity and achieved the state-of-theart result (Ouchi et al., 2015; Shibata et al., 2016).",
        "startOffset": 203,
        "endOffset": 221
      },
      {
        "referenceID": 21,
        "context": "As an approach to the argument omission problem, in Japanese PAS analysis, joint modeling of interactions between multiple predicates has been gaining popularity and achieved the state-of-theart result (Ouchi et al., 2015; Shibata et al., 2016).",
        "startOffset": 223,
        "endOffset": 243
      },
      {
        "referenceID": 10,
        "context": "Performing experiments on the NAIST Text Corpus (Iida et al., 2007), we demonstrate that our neural models exceed the accuracy of the stateof-the-art Japanese PAS analyzer (Ouchi et al., 2015).",
        "startOffset": 49,
        "endOffset": 66
      },
      {
        "referenceID": 17,
        "context": "Performing experiments on the NAIST Text Corpus (Iida et al., 2007), we demonstrate that our neural models exceed the accuracy of the stateof-the-art Japanese PAS analyzer (Ouchi et al., 2015).",
        "startOffset": 173,
        "endOffset": 191
      },
      {
        "referenceID": 7,
        "context": "Arguments can be divided into the following three categories according to the positions relative to their predicates (Hayashibe et al., 2011; Ouchi et al., 2015):\n\nDep : The arguments that have direct syntactic dependency with the predicate.",
        "startOffset": 118,
        "endOffset": 140
      },
      {
        "referenceID": 17,
        "context": "Arguments can be divided into the following three categories according to the positions relative to their predicates (Hayashibe et al., 2011; Ouchi et al., 2015):\n\nDep : The arguments that have direct syntactic dependency with the predicate.",
        "startOffset": 142,
        "endOffset": 160
      },
      {
        "referenceID": 22,
        "context": "(2015)\n\n2 The F-measure remains 10-20% (Taira et al., 2008; Imamura et al., 2009; Sasano and Kurohashi, 2011).",
        "startOffset": 40,
        "endOffset": 58
      },
      {
        "referenceID": 14,
        "context": "(2015)\n\n2 The F-measure remains 10-20% (Taira et al., 2008; Imamura et al., 2009; Sasano and Kurohashi, 2011).",
        "startOffset": 60,
        "endOffset": 80
      },
      {
        "referenceID": 19,
        "context": "(2015)\n\n2 The F-measure remains 10-20% (Taira et al., 2008; Imamura et al., 2009; Sasano and Kurohashi, 2011).",
        "startOffset": 82,
        "endOffset": 108
      },
      {
        "referenceID": 20,
        "context": "The single-sequence model exploits stacked bidirectional RNNs (Bi-RNN) (Schuster and Paliwal, 1997; Graves et al., 2005, 2013; Zhou and Xu, 2015).",
        "startOffset": 72,
        "endOffset": 98
      },
      {
        "referenceID": 5,
        "context": "The single-sequence model exploits stacked bidirectional RNNs (Bi-RNN) (Schuster and Paliwal, 1997; Graves et al., 2005, 2013; Zhou and Xu, 2015).",
        "startOffset": 100,
        "endOffset": 119
      },
      {
        "referenceID": 26,
        "context": "The single-sequence model exploits stacked bidirectional RNNs (Bi-RNN) (Schuster and Paliwal, 1997; Graves et al., 2005, 2013; Zhou and Xu, 2015).",
        "startOffset": 127,
        "endOffset": 144
      },
      {
        "referenceID": 2,
        "context": "3 In this work, we use the Gated Recurrent Unit (GRU) (Cho et al., 2014) as the function g ( ℓ ) .",
        "startOffset": 55,
        "endOffset": 71
      },
      {
        "referenceID": 6,
        "context": "To capture such interactions between all predicates in a sentence, we extend the singlesequence model to the multi-sequence model using Grid-RNNs (Graves and Schmidhuber, 2009; Kalchbrenner et al., 2016).",
        "startOffset": 147,
        "endOffset": 175
      },
      {
        "referenceID": 15,
        "context": "To capture such interactions between all predicates in a sentence, we extend the singlesequence model to the multi-sequence model using Grid-RNNs (Graves and Schmidhuber, 2009; Kalchbrenner et al., 2016).",
        "startOffset": 177,
        "endOffset": 202
      },
      {
        "referenceID": 18,
        "context": "As more layers are stacked, it gets more difficult to learn the model parameters due to some problems such as gradient vanishment (Pascanu et al., 2013).",
        "startOffset": 131,
        "endOffset": 151
      },
      {
        "referenceID": 8,
        "context": "In this work, we integrate residual connections (He et al., 2015; Wu et al., 2016) with our networks to connect between layers.",
        "startOffset": 49,
        "endOffset": 64
      },
      {
        "referenceID": 23,
        "context": "In this work, we integrate residual connections (He et al., 2015; Wu et al., 2016) with our networks to connect between layers.",
        "startOffset": 66,
        "endOffset": 81
      },
      {
        "referenceID": 22,
        "context": "In the pointwise approach, we estimate the score of each argument candidate for one predicate, and select the argument candidate with the maximum score as an argument (Taira et al., 2008; Imamura et al., 2009; Hayashibe et al., 2011; Iida et al., 2016).",
        "startOffset": 168,
        "endOffset": 186
      },
      {
        "referenceID": 14,
        "context": "In the pointwise approach, we estimate the score of each argument candidate for one predicate, and select the argument candidate with the maximum score as an argument (Taira et al., 2008; Imamura et al., 2009; Hayashibe et al., 2011; Iida et al., 2016).",
        "startOffset": 188,
        "endOffset": 208
      },
      {
        "referenceID": 7,
        "context": "In the pointwise approach, we estimate the score of each argument candidate for one predicate, and select the argument candidate with the maximum score as an argument (Taira et al., 2008; Imamura et al., 2009; Hayashibe et al., 2011; Iida et al., 2016).",
        "startOffset": 210,
        "endOffset": 232
      },
      {
        "referenceID": 13,
        "context": "In the pointwise approach, we estimate the score of each argument candidate for one predicate, and select the argument candidate with the maximum score as an argument (Taira et al., 2008; Imamura et al., 2009; Hayashibe et al., 2011; Iida et al., 2016).",
        "startOffset": 234,
        "endOffset": 251
      },
      {
        "referenceID": 25,
        "context": "In the joint approach, we select the score of all the predicate-argument combinations in one sentence, and select the combination with the highest score (Yoshikawa et al., 2011; Sasano and Kurohashi, 2011; Ouchi et al., 2015; Shibata et al., 2016).",
        "startOffset": 154,
        "endOffset": 176
      },
      {
        "referenceID": 19,
        "context": "In the joint approach, we select the score of all the predicate-argument combinations in one sentence, and select the combination with the highest score (Yoshikawa et al., 2011; Sasano and Kurohashi, 2011; Ouchi et al., 2015; Shibata et al., 2016).",
        "startOffset": 178,
        "endOffset": 204
      },
      {
        "referenceID": 17,
        "context": "In the joint approach, we select the score of all the predicate-argument combinations in one sentence, and select the combination with the highest score (Yoshikawa et al., 2011; Sasano and Kurohashi, 2011; Ouchi et al., 2015; Shibata et al., 2016).",
        "startOffset": 206,
        "endOffset": 224
      },
      {
        "referenceID": 21,
        "context": "In the joint approach, we select the score of all the predicate-argument combinations in one sentence, and select the combination with the highest score (Yoshikawa et al., 2011; Sasano and Kurohashi, 2011; Ouchi et al., 2015; Shibata et al., 2016).",
        "startOffset": 226,
        "endOffset": 246
      },
      {
        "referenceID": 21,
        "context": "In recent years, several attempts have been made to apply neural networks for Japanese PAS analysis (Shibata et al., 2016; Iida et al., 2016) 4 .",
        "startOffset": 101,
        "endOffset": 121
      },
      {
        "referenceID": 13,
        "context": "In recent years, several attempts have been made to apply neural networks for Japanese PAS analysis (Shibata et al., 2016; Iida et al., 2016) 4 .",
        "startOffset": 123,
        "endOffset": 140
      },
      {
        "referenceID": 10,
        "context": "We use the NAIST Text Corpus 1.5, which consists of 40,000 sentences of Japanese newspaper text (Iida et al., 2007).",
        "startOffset": 97,
        "endOffset": 114
      },
      {
        "referenceID": 22,
        "context": "In the experiments, we adopt the standard data splits (Taira et al., 2008; Imamura et al., 2009; Ouchi et al., 2015):\n\nTrain: Articles: Jan 1-11,\n\nEditorials: Jan-Aug\n\nDev: Articles: Jan 12-13, Editorials: Sept\n\nTest: Articles: Jan 14-17, Editorials: Oct-Dec\n\nWe use the word boundaries annotated to the NAIST Text Corpus and the target predicates that have at least one argument in the same sentence.",
        "startOffset": 55,
        "endOffset": 73
      },
      {
        "referenceID": 14,
        "context": "In the experiments, we adopt the standard data splits (Taira et al., 2008; Imamura et al., 2009; Ouchi et al., 2015):\n\nTrain: Articles: Jan 1-11,\n\nEditorials: Jan-Aug\n\nDev: Articles: Jan 12-13, Editorials: Sept\n\nTest: Articles: Jan 14-17, Editorials: Oct-Dec\n\nWe use the word boundaries annotated to the NAIST Text Corpus and the target predicates that have at least one argument in the same sentence.",
        "startOffset": 75,
        "endOffset": 95
      },
      {
        "referenceID": 17,
        "context": "In the experiments, we adopt the standard data splits (Taira et al., 2008; Imamura et al., 2009; Ouchi et al., 2015):\n\nTrain: Articles: Jan 1-11,\n\nEditorials: Jan-Aug\n\nDev: Articles: Jan 12-13, Editorials: Sept\n\nTest: Articles: Jan 14-17, Editorials: Oct-Dec\n\nWe use the word boundaries annotated to the NAIST Text Corpus and the target predicates that have at least one argument in the same sentence.",
        "startOffset": 97,
        "endOffset": 115
      },
      {
        "referenceID": 0,
        "context": "We implement our neural models using Theano (Bastien et al., 2012).",
        "startOffset": 45,
        "endOffset": 65
      },
      {
        "referenceID": 16,
        "context": "The learning rate is automatically adjusted using Adam (Kingma and Ba, 2014).",
        "startOffset": 56,
        "endOffset": 75
      },
      {
        "referenceID": 9,
        "context": "To achieve high accuracies for the analysis of such zero arguments, it is necessary to capture long distance dependencies (Iida et al., 2005; Sasano and Kurohashi, 2011; Iida et al., 2015).",
        "startOffset": 123,
        "endOffset": 140
      },
      {
        "referenceID": 19,
        "context": "To achieve high accuracies for the analysis of such zero arguments, it is necessary to capture long distance dependencies (Iida et al., 2005; Sasano and Kurohashi, 2011; Iida et al., 2015).",
        "startOffset": 142,
        "endOffset": 168
      },
      {
        "referenceID": 12,
        "context": "To achieve high accuracies for the analysis of such zero arguments, it is necessary to capture long distance dependencies (Iida et al., 2005; Sasano and Kurohashi, 2011; Iida et al., 2015).",
        "startOffset": 170,
        "endOffset": 187
      },
      {
        "referenceID": 17,
        "context": "Argument and adjunct usages of the case marker ' に ' are not distinguished in 1.4 β , making the identification of the dative case seemingly easy (Ouchi et al., 2015).",
        "startOffset": 147,
        "endOffset": 165
      }
    ],
    "year": null,
    "abstractText": "The accuracy of Japanese predicate argument structure (PAS) analysis has improved in recent years thanks to joint modeling of interactions between multiple predicates. However, this approach heavily relies on syntactic information predicted by parsers, and suffers from the error propagation. To remedy this problem, we introduce a model using grid-type recurrent neural networks (Grid-RNN), which automatically induces features sensitive to multi-predicate interactions from word sequence information of a sentence. The experiments on the NAIST Text Corpus show that our model exceeds the accuracy of the state-of-the-art Japanese PAS analyzer without syntactic information.",
    "creator": "Docling-granite"
  }
}