reflection interact to produce decisions in social dilem- mas (Rand, Greene & Nowak, 2012).

More specifically, Rand and colleagues argued that a cognitive conflict ex- ists between intuition and reflection (deliberation), such that "intuition supports cooperation . . . and . . .

reflec- tion can undermine these cooperative impulses" (p. 427).

As a theoretical explanation, Rand, Peysakhovich et al. ( 2014) stated that cooperation is typically advantageous in the context of daily life interactions, and therefore leads to the formation of cooperative intuitions.

These intu- itions are generalized to other contexts, such as social dilemma experiments, resulting in spontaneous coopera- tive impulses in these situations as well.

However, co- operation might not be advantageous at all in these new contexts, and reflection may override the initial coopera- tive impulse, adapting behavior towards the context spe- cific optimum (defection).


Empirical studies designed to test the prediction of intu- itive cooperation can be grouped into two categories: ex- perimental manipulations designed to influence the degree to which people rely on intuition versus reflection in their decision, and correlational studies focusing on the rela- tionship between process characteristics (such as response times) and decisions in social dilemmas.


Studies using experimental manipulations follow the idea that manipulations designed to foster the influence of intuition should increase cooperation, whereas manipula- 510 https://doi.org/10.1017/S1930297500006392 Published online by Cambridge University Press 511 tions designed to foster the influence of reflection should decrease cooperation.


In line with this reasoning, ask- ing participants to write about a situation where intuitive decision making had led them in the right direction in- creased contributions in a public goods game compared to a condition where participants had to write about a situ- ation where careful reasoning worked out well (Rand et al., 2012, Studies 8 & 9).


In addition, participants con- tributed more to a public good when they were forced to decide quickly (i.e., under time pressure) than when they were instructed to reflect and to decide slowly (Rand et al., 2012, Studies 6 & 7).


However, in another series of experi- ments no effect of time pressure on cooperation was found (Tinghög et al., 2013), leading to an ongoing controversy about the stability and potential moderators of this ef- fect (Rand, Greene & Nowak, 2013; Rand & Kraft-Todd, 2014; Rand, Newman & Wurzbacher, in press; Rand, Peysakhovich et al., 2014; Verkoeijen & Bouwmeester, 2014).


One possible caveat concerning time pressure may be that, rather than increasing the influence of intuition, time pressure may lead to an increase in guessing 1 or a de- crease in the amount of information searched (from givens or from memory)-neither of which necessarily foster co- operation.


In addition to experimental manipulations, research in- vestigating the link between intuition and cooperation has also examined the process characteristics of decisions (Rand et al., 2012).

These process characteristics are taken as indicators for the degree to which a decision was in- tuitive.

If the hypothesis put forward by Rand and col- leagues holds, there should be a positive relationship be- tween cooperation and indicators of intuition.

So far, stud- ies have focused on one specific attribute of the process, namely speed.

2 The basic assumption is that intuitive re- sponses are faster than reflective responses (Kahneman, 2011).


In line with their hypothesis that cooperation is in- tuitive, Rand and colleagues (2012) report that faster deci- sions were more cooperative than slower decisions across several studies.

For example, response time negatively predicted contributions to a public good (Studies 1 & 5).

Similarly, response time was negatively associated with the probability of cooperative choices in different variants of the prisoner's dilemma game (Studies 2-4).


Evidence on the relationship between response time and cooperation is, however, also not consistent in the litera- ture.

In line with Rand et al. ( 2012), several other stud- 1 Rand, Peysakhovich et al. ( 2014) however suggest that the effect of time pressure on cooperation in Rand et al. ( 2012) cannot be explained by more random behavior in the time pressure condition, as the average contribution in this condition is not closer to 50% (which should result from random behavior) compared with the time delay condition.

2 Recently, text analyses of open ended survey responses have also been used to investigate the relationship between intuition and coopera- tion in social dilemmas (Rand, Kraft-Todd & Gruber, 2014; Roberts et al., 2014).


ies reported a negative relationship between response time and cooperation in public goods games (Lotito, Migheli & Ortona, 2013; Nielsen, Tyran & Wengström, 2014).

Yet, the opposite pattern was found in a dictator game in which faster participants made more egoistic choices (Pi- ovesan & Wengström, 2009).

A more complex relation- ship between response time and cooperation was found in studies assessing the social value orientation of par- ticipants, where individualists had shorter response times than both cooperators and competitors (Dehue, McClin- tock & Liebrand, 1993; Fiedler, Glöckner, Nicklisch & Dickert, 2013; Liebrand & McClintock, 1988).

In their re- cent working paper, Evans, Dillon, and Rand (2014) find evidence for an inverted-U relationship between response time and cooperation in both the public goods game and the prisoner's dilemma game in two studies and a meta- analysis.

They conclude that "self-paced reaction times should not be treated as a direct proxy for the degree of reflection [vs. intuition], as other psychological vari- ables may also influence decision speed" (p. 29, brackets added).

Indeed, although it is commonly assumed that in- tuitive processes are fast, short response times could well be produced by mechanisms other than intuition (e.g., very limited information search or pure guessing).


Part of the difficulty, we conjecture, lies in the reliance on theoretical constructs that are insufficiently specified on the level of testable cognitive processes (Glöckner & Wit- teman, 2010).

Specifically, it is difficult to test a dissocia- tion between intuitive and deliberative processing because these are rather vague labels (Gigerenzer, 1998) that have been understood to mean quite different things (Keren & Schul, 2009).

For example, some have implied that intuition drives the use of heuristics (Gigerenzer, 2007; Kahneman, 2003)-and thus strategies for effort reduc- tion (Shah & Oppenheimer, 2008)-whereas others equate intuition with effortless integration of large amounts of information (Betsch & Glöckner, 2010; Hogarth, 2001; Seligman & Kahana, 2009) and thus quite the opposite of what effort-reducing heuristics do (Hilbig, Scholl & Pohl, 2010).

Also, in-depth analyses imply that there may be some gradual differences rather than distinct modes of thinking (Horstmann, Ahlgrimm & Glöckner, 2009).

Fi- nally, even though intuitions are often understood to be fast, one cannot assert that fast processes are intuitive and it has also been argued that some intuition can be no- tably slow and time-consuming (Dijksterhuis & Nordgren, 2006).

In summary, the intuitive-reflective-dichotomy makes it difficult to test the underlying theoretical idea of Rand et al. ( 2012).


Thus, rather than trying to test a dissociation between intuition and deliberation per se, it seems prudent to rely on specific process measures that can be used to test the proposed relationship.

That is, one can focus on process variables for which the assumptions made by Rand et al. https://doi.org/10.1017/S1930297500006392 Published online by Cambridge University Press 512 (2012) make clear-cut predictions, independent of the la- bels intuition and deliberation.


One such process charac- teristic is the cognitive conflict involved in making a de- cision.

If cooperation is the "effortless", "spontaneous", "initial" or "intuitive" response (Rand et al., 2012), de- fection should entail more cognitive conflict than cooper- ation.


To measure cognitive conflict one can employ a re- cently proposed and promising method: the analysis of re- sponse dynamics.

Response dynamics provide researchers with implicit measures of the growth of a preference lead- ing up to the decision (Koop & Johnson, 2011; Spivey & Dale, 2006).

From this method, measures for cognitive conflict can be derived.


In the following, we will briefly introduce the logic of response dynamics and report an ex- periment applying this type of measure to social dilemma choices.


The basic response dynamics paradigm for computerized experiments involves the continuous tracking of mouse movements while participants choose between options that are spatially separated on a screen (also referred to as mouse-tracking; Freeman & Ambady, 2010).

The as- sumption behind the paradigm is that cognitive processing is continuously revealed in motor movements (Spivey & Dale, 2006).

More specifically, it is assumed that mouse movements reveal the tentative commitments to the differ- ent choice options during the decision process (Freeman, Dale & Farmer, 2011).

In that sense, an option that is strongly considered at one point during the decision pro- cess exerts a "pull" of the mouse movement in the direc- tion of this option.

Therefore, the mouse movement as a whole (the so-called response trajectory) and different measures derived from it can be used to draw inferences about the cognitive processes that take place during the formation of the decision (Koop & Johnson, 2011).


One property of the response trajectory that is of par- ticular interest for our purposes is its curvature.

The basic idea is that the more seriously an ultimately non-chosen option is considered during the decision process, the more will the trajectory deviate in direction of this option.

In other words, the degree of curvature represents the spa- tial attraction towards the non-chosen option (Freeman & Ambady, 2010; Spivey & Dale, 2006), which can be taken as a proxy of the overall degree of cognitive conflict.

An early experiment by Spivey, Grosjean, and Knoblich (2005) validates this interpretation.

Participants were re- peatedly asked via headphones to click on one of two im- ages displayed on the screen (e.g., "Click the candy.").

The requested object ("candy") was paired with a distrac- tor that was either phonologically similar (e.g., "candle") or phonologically dissimilar (e.g., "jacket").

As expected, response trajectories were more curved towards the dis- tractor if the distractor was phonologically similar, sug- gesting that the increased curvature was caused by the increasing attraction of the distractor in this condition.


More recently, a validation study of the response dynam- ics paradigm was also provided for the area of preferential choice (Koop & Johnson, 2013, Experiment 1).

In a series of trials, participants had to indicate which of two images they preferred.

Importantly, images were selected from a well-normed set of stimuli (International Affective Pic- ture System; Lang, Bradley & Cuthbert, 2008), and the a priori difference in pleasantness of the pictures in a pair was varied continuously.

As expected, the curvature of the response trajectories increased with increasing similarity in pleasantness, validating the interpretation that curvature reveals the competition between choice options for the do- main of preferential choice.


Both behavioral and neurophysiological evidence has corroborated the assumptions behind the response dynam- ics approach (Spivey, 2007).

Following the pioneering work by Spivey et al. ( 2005), the paradigm has been suc- cessfully applied to a variety of cognitive domains, includ- ing language, social cognition, and learning (see Freeman et al., 2011, for a review).

More recently, response dy- namics have also been used to study judgment and de- cision making processes (Dshemuchadse, Scherbaum & Goschke, 2013; Koop, 2013; Koop & Johnson, 2011, 2013).

However, to the best of our knowledge, no study has considered response dynamics as process indicators in social dilemma decision making.


The experiment reported below takes advantage of the response dynamics paradigm to test a hypothesis derived from the theoretical framework by Rand et al. ( 2012) that considers cooperation as the spontaneous response in so- cial dilemmas.

Building on the previously outlined theo- retical and empirical arguments, we expect a main effect of decision (cooperation vs. defection) on cognitive con- flict (as indicated by the curvature of mouse trajectories) in social dilemmas.

More specifically, defection should entail more cognitive conflict (greater curvature) than co- operation.


In their studies, Rand et al. ( 2012) observed that the no- tion of intuitive cooperation appears to hold for some indi- viduals, whereas both intuition and reflection lead to little cooperation for other individuals.

Consequently, Rand et al. have stressed the importance of identifying individual difference factors that might moderate the effect.

Indeed, it has been emphasized that one should not only test for effects on the aggregate but also consider to what extent an effect holds per individual-both in judgment and de- cision making research and experimental psychology in general (Baron, 2010).

Optimally, then, such interindivid- https://doi.org/10.1017/S1930297500006392 Published online by Cambridge University Press 513 ual variation in the effect could also be explained by some moderating factor.

Therefore, an additional goal was to investigate whether there are between-participant differ- ences in the hypothesized relation between decision and cognitive conflict, and if so, to explain these through mod- erating factors.


Regarding the relationship of intuition and coopera- tion, two moderating factors have been uncovered so far, namely (a) the cooperativeness of interaction partners in daily life and (b) prior experience with laboratory exper- iments.

Faster responses were found to be positively re- lated to cooperation only for individuals who reported having many cooperative daily-life interaction partners (Rand et al., 2012).

Besides, time pressure increased co- operation only for individuals who had little prior expe- rience with laboratory experiments (Rand, Peysakhovich et al., 2014) and were thus arguably less often exposed to situations in which defection is positively reinforced.

An additional study that assessed both factors in combi- nation even showed evidence for an interaction, namely, that time pressure increased cooperation only when par- ticipants both had little prior experience with economic games and reported to have mainly cooperative interaction partners (Rand & Kraft-Todd, 2014).


Although these specific factors fit nicely with the rea- soning of Rand, Peysakhovich et al. ( 2014), they refer more to prior experiences than to individual differences in the sense of basic, dispositional tendencies such as person- ality traits.

However, it is straightforward to predict that personality traits reflecting cooperativeness and pro-social tendencies should explain the difference in cognitive con- flict when people defect versus cooperate.

Disposition- ally cooperative individuals should be particularly inclined to cooperate spontaneously and their decisions should be characterized by particularly strong conflict whenever they defect.

In other words, especially for highly cooperative individuals a large positive difference in cognitive conflict between defection and cooperation should be expected.

In turn, the overall hypothesized effect (that cooperation yields less conflict) may diminish or even reverse for dis- positionally uncooperative individuals.


One personality trait that specifically reflects disposi- tional cooperativeness is the Honesty-Humility (HH) fac- tor of the HEXACO personality model (Ashton & Lee, 2007; Ashton, Lee & de Vries, 2014).

HH is defined as the "tendency to be fair and genuine in dealing with oth- ers, in the sense of cooperating with others even when one might exploit them without suffering retaliation" (Ash- ton & Lee, 2007, p. 156).

In line with this definition, HH has been found to predict cooperation in a variety of paradigms from behavioral economics ranging from the dictator game (Hilbig & Zettler, 2009; Hilbig, Zettler, Leist & Heydasch, 2013) and social values (Ackermann, Fleiß & Murphy, in press; Hilbig, Glöckner & Zettler, Building on the theoretical definition of HH and corre- sponding empirical findings, HH should predict between- participant variation in the hypothesized effect of deci- sion on cognitive conflict, that is, function as a modera- tor.

Specifically, individuals high in HH should be partic- ularly inclined to cooperate spontaneously and should find it more difficult to counteract this impulse when ultimately deciding to defect.

In other words, the difference in con- flict between defection and cooperation decisions should be larger for individuals high in HH.

For their counterparts low in HH, this difference should be considerably smaller or it might even reverse.


The hypotheses outlined above were tested in a fully in- centivized lab experiment, where participants played sim- ple two-person social dilemma games with the two options cooperation and defection.

Depending on the combina- tion of players' choices, the individual payoff varied fol- lowing a specific payoff structure.

( See Table 1 for the general structure.) If both players cooperated, both re- ceived the same payoff called reward (R).

Similarly, if both players defected, both received an inferior payoff called punishment (P).

If one player cooperated but the other de- fected, the cooperative player received the sucker's payoff (S) while the defecting player received the payoff called temptation (T).

By modifying the order of the different payoffs, different types of social dilemmas were created such that different motivations for cooperation versus de- fection existed.


To increase variation in cooperation rates, three differ- ent types of social dilemma games were implemented.

First, the prisoner's dilemma game (PDG; Rapoport & Chammah, 1965) was used (T > R > P > S), in which https://doi.org/10.1017/S1930297500006392 Published online by Cambridge University Press 514 two mechanisms-temptation (T > R) and fear (P > S)- motivate defection.

Second, the chicken game (Rapoport & Chammah, 1966) was employed (T > R > S > P), where temptation still motivates defection (T > R), but fear is no longer present (P < S).

Third, the stag hunt game (Gächter, 2004) was used (R > T = P > S), where fear motivates de- fection (P > S), but temptation is no longer present (T < R).

All three game types have in common the previously out- lined characteristics of a social dilemma (Van Lange et al., 2013).

That is, in each game, there is a non-cooperative option that is (at least sometimes) more attractive for the individual than the cooperative option.

However, if both individuals decide to pursue this non-cooperative option, they end up worse off than if they had both cooperated (R > P for all three game types).


For the experiment, we constructed five variants of each of the three previously described game types.

Within each game type, the cooperation index (Rapoport & Chammah, 1965; Vlaev & Chater, 2006) was held constant and only the specific payoff values were varied, meaning that the relative attractiveness of cooperation versus defection did not differ across variants.

All payoff matrices used in the current experiment can be found in Appendix A.


Basic personality factors and the HH dimension in par- ticular were assessed using the German version (Mosha- gen, Hilbig & Zettler, 2014) of the 60-item HEXACO- Personality-Inventory-Revised (HEXACO-60; Ashton & Lee, 2009; for further information on the inventory and all items see http://hexaco.org).

To maintain the structure of the inventory, all 60 items-including those pertaining to personality factors not of interest here-were assessed and answered on a scale ranging from 1 (strongly disagree) to 5 (strongly agree).

However, in the analysis only the individual scores on the HH scale (i.e., individual means across the 10 items corresponding to this factor) were con- sidered.

3 The internal consistency and descriptive statis- tics of this scale (Cronbach's α = .78, M = 3.31, SD = 515 able responding (e.g., Moshagen, Hilbig & Musch, 2011).

Whether the cooperative choice option was presented left (as "Option A") or right (as "Option B") was counterbal- anced across participants to avoid presentation order ef- fects (Hehman, Stolier & Freeman, in press).

In addition to the choice and response time, the mouse trajectory (i.e., the x-, y-coordinates of the cursor) was recorded at a rate of 100 Hz (higher than or comparable to other mouse- tracking applications; Freeman & Ambady, 2010; Koop & Johnson, 2011).

Participants were not told that their mouse movements were recorded nor were they given spe- cial instructions regarding the movements of the mouse (Koop & Johnson, 2011, 2013).


Participants were recruited from a local participant pool.

In total, 116 individuals participated in the experiment.

Their age ranged from 18 to 36 years (M = 22.5, SD = 4.0), 79 were female, and 107 were students from various disciplines.

On average, participants earned 2.56 C (ap- prox. 3.50 USD) in the social dilemma game which lasted for about fifteen minutes.

The obtained sample size yields high statistical power (1−β = .94) to detect a small effect (d = 0.30) in a paired t-test given an α error probability of .05 (Faul, Erdfelder, Lang & Buchner, 2007).


To assess whether the variability in participants' coopera- tion/defection decisions was sufficient, we first computed the individual cooperation rate per participant across all games.

On average, participants cooperated in 58% of their decisions (SD = 19%).

Inspection of the individ- ual cooperation rates revealed that 115 participants (99% of participants) chose each option (cooperation and defec- tion) in at least one of the fifteen rounds.

The remaining participant always cooperated and was excluded from all further analyses.

6 In line with the notion that two mech- anisms foster defection in the PDG but only one mecha- nism does so in each of the other games, participants co- operated less in the PDG (25%) than in the other games, as confirmed by a generalized linear-mixed model using a binomial link function, 7 z = -18.63, p < .001.

In addition, the probability of cooperation was higher in the stag hunt game (86%) than in the chicken game (63%), z = 9.12, p < .001.


Next, we tested the first hypothesis that defection en- tails more cognitive conflict than cooperation.

As a first descriptive indicator of conflict, we inspected the average response trajectories.

Following previous studies employ- 516 ), we mapped all trajectories to one side, time-normalized trajectories into 101 time bins, 8 and aggregated them first within and then across participants separately for the dif- ferent decisions.

The resulting average trajectories for cooperation and defection are depicted in Figure 2.

In line with the hypothesis, trajectories of defection deci- sions were characterized by a greater curvature and thus greater attraction towards the non-chosen option (cooper- ation) than vice versa.


To statistically test this curvature pattern, we calculated the maximum deviation (MD, in pixels) for each single trajectory.

The MD is defined as the maximum perpen- dicular deviation between the actual trajectory and its ide- alized trajectory, which is the straight line connecting the trajectory's starting point and end point (Freeman & Am- bady, 2010; McKinstry, Dale & Spivey, 2008).

MD serves as an index of the maximum attraction towards the non- chosen option (Freeman & Ambady, 2010), and thus as an indicator of the maximum conflict during the decision.

Whereas the following analyses use the MD towards the non-chosen option as dependent variable, Appendix B re- ports additional analyses based on other measures.


Given the skewness of the MD distribution, we square root transformed the MD values and used these values for all following analyses (but replicated all results using the 8 Potential initial phases without mouse movements were removed be- fore time-normalization.

0 5 10 15 20 cooperate defect Decision Square root (maximum deviation) untransformed MD values, see Appendix B).


Next, we av- eraged the MD per decision and participant (Wojnowicz, Ferguson, Dale & Spivey, 2009).


As can be seen in Figure 3, the mean MD was higher when participants defected (M = 15.14, Mdn = 14.33, SD = 6.03) than when they cooper- ated (M = 12.92, Mdn = 12.31, SD = 4.98).

This difference was statistically significant in a paired t-test, t(114) = 3.40, p < .001, d = 0.32, hence confirming statistically the ob- served differences in the average response trajectories.


9 Furthermore, we examined whether the effect of deci- sion was robust across the different game types.


For each game type, the qualitative pattern was as expected, as the mean MD was higher for defection than cooperation (see Table 2, 428.30) = 0.16, p = .85.


Hence, defection decisions entailed more conflict than cooperation decisions regardless of the game type and thereby regardless of differences in the specific structure of the payoff matrix.


Moreover, it is noteworthy that even in the game encompassing the lowest coopera- tion rates (25% cooperation in PDG) the mean MD for de- fection was significantly higher than for cooperation, F(1, 89.31) = 6.54, p = .01.


Thus, even though defection was predominant on the choice level, cooperation still entailed less cognitive conflict than defection.


To investigate between-participant differences in the ef- fect of decision on cognitive conflict, we computed the dif- ference of mean MD between defection and cooperation per participant.

Directly replicating our previous analy- sis, the mean difference of 2.22 was significantly greater than zero, t(114) = 3.40, p < .001, d = 0.32.

However, as indicated by a SD of 6.99, there was also considerable variation in the size of the effect.

Specifically, although the difference was positive for the majority of participants (63%), a substantial proportion of participants showed the reversed difference (37%).


To account for this variation across individuals we tested the hypothesized moderating factors as outlined above.

First, we examined whether the overall coop- erativeness of participants in the current study (as indi- cated by their individual proportion of cooperative choices across all trials) predicted the MD difference between co- operation and defection, and, as a more distal factor, we asked whether the individual HH scores predicted the MD difference.

In line with expectations, both predictors sig- nificantly (positively) predicted the MD differences in sep- arate correlational analyses (r = .46, t(113) = 5.51, p < .001, for cooperativeness, and r = .20, t(113) = 2.18, p = Honesty-Humility (centered on mean) Square root (maximum deviation) cooperate defect .03, for HH).

As displayed in Figure 4, the relationship between mean MD and HH was as hypothesized: For in- dividuals high in HH, defection yielded more conflict than cooperation; for their counterparts low in HH there was no such difference or even a tendency for the reversed effect.


Finally, we asked whether the effect of HH was actually driven by the situational cooperativeness of participants in the current study.

To this end, we conducted a mediation analysis following the steps proposed by Baron and Kenny (1986).

As reported above, HH predicted the difference score, β = 0.20, t(113) = 2.18, p = .03.

In addition, HH also predicted cooperativeness, β = 0.34, t(113) = 3.79, p < .001.

When including both HH and cooperativeness in a regression to predict the difference score, only cooper- ativeness was a significant predictor, β = 0.44, t(112) = 4.98, p < .001, whereas HH was no longer significant, β = 0.05, t(112) = 0.58, p = .56-thus confirming a full medi- ation.

Correspondingly, the indirect effect of HH was sta- tistically significant, Sobel's Z = 2.98, p = .003.

11 That is, the situational cooperativeness of participants in the cur- rent study mediated the effect of HH on the difference in cognitive conflict between defection and cooperation.

In summary, these analyses help explain the substantial vari- ation in the main effect across individuals.


In the current experiment, we tested a hypothesis derived from the recently proposed assumption that cooperation is intuitive in social dilemmas, whereas defection requires effortful deliberation (Rand et al., 2012).

To this end, we presented a first application of the response dynam- ics paradigm (Koop & Johnson, 2011) to social dilem- mas.


Specifically, given that intuition and reflection are inherently difficult to tease apart because they represent a rather vague dichotomy, we tested a more specific impli- cation: that defection should entail more cognitive con- flict than cooperation as indicated by participants' mouse movements during the decision process.


In line with this hypothesis, response trajectories were more curved (to- wards the non-chosen option) when individuals defected than when they cooperated.

In other words, the coopera- tive option exerted more "pull" on mouse movements in case of defection than the non-cooperative option (defec- tion) did in case of cooperation.


This effect was robust across different types of social dilemmas and occurred even in the prisoner's dilemma where defection was pre- dominant on the choice level.


The effect of more con- flict in defection held for the majority of individuals, but there was also notable interindividual variation in the ef- fect.


This interindividual variation could be accounted for by individuals' situational cooperativeness which, in turn, was driven by their dispositional cooperativeness in terms of the basic personality factor Honesty-Humility as speci- fied in the HEXACO model (Ashton & Lee, 2007).


The current experiment extends previous research that has primarily focused on response times to examine the relationship between intuition and cooperation.

Specifi- cally, one main argument for the hypothesis that cooper- ation is intuitive stems from correlational studies demon- strating that shorter response times are associated with in- creased cooperation (Rand et al., 2012).

However, such a negative relationship between response times and coop- eration has not been consistently found in the literature, and recently it was argued that response times should not be taken as a direct indicator of intuitive versus reflective processing (Evans et al., 2014).

The current experiment provides a solution for this problem by considering an al- ternative process variable-cognitive conflict-for which specific predictions can be derived based on the hypoth- esis that cooperation is the "spontaneous", "initial", "ef- fortless" response in social dilemmas while defection re- quires effortful deliberation (Rand et al., 2012).

Indeed, cooperation decisions entailed less conflict.

This is also in line with growing evidence that prosocial behavior in general may stem from intuitive, reflexive, and automatic processes (see Zaki & Mitchell, 2013, for a short sum- mary).


Moreover, the current experiment further specifies ex- tant findings on interindividual variation in how much of a spontaneous, initial response cooperation tends to be (Rand, Peysakhovich et al., 2014).

While previous stud- ies have identified individual differences in terms of prior experiences as moderators (Rand et al., 2012; Rand & Kraft-Todd, 2014; Rand, Peysakhovich et al., 2014), we demonstrated the moderating influence of a basic person- ality trait.

The HH factor of the HEXACO personality model has recently been identified as the basic trait driving active cooperativeness in various domains (for overviews see Ashton et al., 2014; Hilbig et al., 2014).

Correspond- ingly, we found that particularly decisions of individuals high in HH were characterized by more cognitive conflict when defecting than when cooperating.

Furthermore, the effect of HH was mediated by the situational cooperative- ness of participants in the current study.


Finally, the current experiment demonstrates the use- fulness of the response dynamics paradigm for investigat- ing complex decision making tasks.

Whereas earlier stud- ies used the response dynamics paradigm primarily to in- vestigate basic cognitive tasks with objectively correct re- sponse options, the paradigm has recently been extended to study complex decision making tasks that also include preferences (Koop & Johnson, 2011).

In these studies, re- sponse dynamics have offered novel insights and availed tests for which existing response output data (i.e., choices and response times) were not sufficient-in areas such as risky decision making (Koop & Johnson, 2013, Experi- ment 2), moral decision making (Koop, 2013), and in- tertemporal choice (Dshemuchadse et al., 2013).

We add to this recent development and extend the application of response dynamics to social dilemma decision making.


Nevertheless, the application of the response dynam- ics paradigm to more complex tasks without objectively correct response options also bears some methodological challenges.

Previous research with basic cognitive tasks usually involved a correct or desired response option and the primary comparison dimension of the response tra- jectories could be experimentally manipulated (e.g., the phonological similarity of the distractor in the experiment by Spivey et al., 2005).

By contrast, the current experi- ment involved preferences, and the final choice (coopera- tion vs. defection) constituted the comparison dimension of interest.


This implies a loss of experimental control over the comparison dimension, because the frequency of each response type depended entirely on the participant.

This dependence could result in missing cells and other analytical challenges (Koop & Johnson, 2013).

To address this problem, we decided to not only rely on the classic social dilemma, the PDG, in which cooperation rates are typically low, but to also implement two additional social dilemmas that should render cooperation more likely.

This modification was successful, as observations for both re- sponses were obtained for all participants but one.

Nev- https://doi.org/10.1017/S1930297500006392 Published online by Cambridge University Press 519 ertheless, the comparison dimension of the response tra- jectories in our experiment was an observed predictor, not under full experimental control.


The amount of information that people needed to take into account when making their decision was also consid- erably larger in the current experiment than in previous applications of response dynamics.

Nevertheless, in line with previous applications of response dynamics we re- frained from instructing participants to move the mouse in a specific way (Koop & Johnson, 2011, 2013).

As a con- sequence, several response trajectories in our dataset con- tained analysis-irrelevant mouse movements (e.g., "read- ing" movements).

We accounted for this increased level of noise by performing the analyses both with the unfiltered dataset and after excluding trials with irrelevant mouse movements.

Importantly, the results remained stable re- gardless of the criterion used for excluding trials (see Ap- pendix B).

Future studies could find procedural solutions to reduce the noise in the mouse movements, for exam- ple, by simplifying the task or by modifying the temporal order in which the task information is presented-similar to solutions used in other decision making applications of response dynamics (Dshemuchadse et al., 2013; Koop, 2013).


Finally, although the different curvatures of response trajectories for cooperation and defection decisions sup- port the idea that cooperation is the spontaneous and less conflicting response in social dilemmas, they do not nec- essarily imply that there are actually two distinct sys- tems (intuition and reflection)-as assumed by Rand et al. ( 2012)-that interact to produce decisions in social dilemmas (Horstmann et al., 2009).

Instead, these re- sponse trajectory differences could also be explained by a single system assuming functional differences in the ev- idence accumulation process (see Dshemuchadse et al., 2013, for a similar argument in intertemporal choice).

Fu- ture studies should address this problem, following Koop and Johnson (2011, 2013), who emphasized the potential of response dynamics for distinguishing between different model classes.

A particularly promising approach is a re- cent study combining response dynamics and eye tracking that tested quantitative predictions derived from a simple evidence accumulation model for risky decision making (Koop & Johnson, 2013, Experiment 3).

Future studies could use a similar approach to discriminate between dif- ferent model classes and test their specific predictions for social dilemmas.


Kieslich and Hilbig (2014) employ a mouse-tracking technique to measure decision conflict in social dilemmas.

They report that defectors exhibit more conflict than do cooperators.

They infer that cooperation thus is the reflexive, default behavior.

We argue, however, that their analysis fails to discriminate between reflexive versus cognitively controlled behavioral responses.

This is because cognitive conflict can emanate from resisting impulse successfully—or unsuccessfully.


Kieslich and Hilbig (2014) use mouse-tracking to measure cognitive conflict in three different economic games—the prisoner’s dilemma, the chicken game, and the stag hunt game—each characterized by a social dilemma.

They report that defection is associated with higher degrees of conflict than is cooperation, and they reason that this result is consistent with their prediction that the reflexive, “default” behavior is to cooperate [1].

We argue, however, that their specific result—and their paradigm, more generally—fails to discriminate between reflexive versus cognitively controlled behavioral responses.


Kieslich and Hilbig (2014) base their prediction on results arising from a controversial research program by Rand and colleagues (see e.g., Rand et al., 2012; Rand et al., 2014) [2][3].

The message in this program is succinctly conveyed in the title of the first paper, “Spontaneous giving and calculated greed” (Rand et al., 2012) [2].

However, as Kieslich and Hilbig (2014) note, the empirical pattern is contested (e.g., Tinghög et al., 2013; Lohse et al., 2014; Verkoeijen & Bouwmeester, 2014) [4][5][6], and a recent re-examination of Rand et al. ( 2012) and Rand et al. ( 2014) reveals that the data sets contain no meaningful evidence for the notion that giving is “spontaneous” and greed “calculated” (Myrseth & Wollbrant, 2015) [7].

Efforts to shed further light on the cognitive processes behind the decision to defect or cooperate are thus well placed.


Adopting the stated assumption that cooperation represents the reflexive, default behavior, Kieslich and Hilbig (2014) formulate the following prediction: cooperation should be associated with less cognitive conflict than should defection.

Their line of reasoning is predicated on two tacit assumptions: (1) efforts to resist behavioral impulse involve more conflict than would no such efforts; and (2) efforts to resist impulse are successful.

The former is unproblematic, but the latter is clearly not.

Sometimes individuals succeed at self-control; other times they fail.

Thus, cognitive conflict may emanate both from the successful and unsuccessful resistance to impulse.

If we knew that conflict associated with successful self-control were stronger than that associated with self-control failure, then we could derive the prediction put forth by Kieslich and Hilbig (2014).

However, we do not know this, and the alternative possibility—that unsuccessful resistance to impulse is associated with more conflict—seems equally plausible.

In that event, and on the assumption that cooperation is the impulsive response, one might predict that cooperation is associated with more cognitive conflict [8].

The authors’ prediction, therefore, does not follow from their stated assumption; it might well be the opposite.


That said, Kieslich and Hilbig (2014) add to the literature by applying a measure of cognitive conflict hitherto not introduced in the study of social dilemmas. And they find that those who defect exhibit more conflict than do those who cooperate.

Might we work the other way, and deduce from this which behavioral response—to defect or to cooperate—represents the default? Unfortunately, we cannot.

Observing that defectors exhibit more conflict could mean (a) that cooperation is the default, and that successful self-control entails more conflict than does unsuccessful self-control, but it could also mean (b) that defection is the default, and self-control failure entails more conflict than does successful self-control.

Neither possibility seems profoundly more plausible than the other.

A visualization of possible interpretations of differences in conflict might prove instructive.


Myrseth and Fishbach (2009) present a two-stage model of self-control (see Figure 1 for an adaptation), which distinguishes between identification of self-control conflict, and the exercise of self-control [9].

This framework has recently been applied to the study of pro-social behavior in economic games (e.g., Martinsson et al., 2012; Kocher et al., 2013; Martinsson et al., 2014) [10][11][12], and it highlights when we would expect individuals to experience cognitive conflict.

Crucially, when individuals fail to identify a self-control conflict, they will experience none, and will follow their impulse.

However, they may also follow their impulse after having seen the conflict but failed in their efforts to resist.

Hence, an observed behavior could be the result of three distinct cognitive scenarios: (1) absence of conflict identification, accompanied by no experience of conflict; (2) conflict identification, conflict experience, and successful self-control; or, (3) conflict identification, conflict experience, and failed resistance.

Merely observing, then, that one of two behaviors is associated with more conflict, does not allow us to determine which is driven by impulse.


One might also consider whether the conclusions drawn by Kieslich and Hilbig (2014) are justified by the moderator, the Honesty-Humility factor of the HEXACO personality model (Ashton & Lee, 2007; Ashton et al., 2014) [13][14].

Unfortunately, they are not. Again Kieslich and Hilbig (2014) base their reasoning on an unnecessarily restrictive assumption: “Dispositionally cooperative individuals should be particularly inclined to cooperate spontaneously and their decisions should be characterized by particularly strong conflict whenever they defect” (p. 513).

How do we know that disposition here captures spontaneous inclination as opposed to deliberative goal pursuit? Might it not be equally plausible that dispositionally cooperative individuals are characterized by the same spontaneous inclinations as others, but differ in their deliberate pursuit of abstract goals? This second interpretation would also allow for the pattern observed by Kieslich and Hilbig, but it would change the meaning: we might observe among dispositionally cooperative individuals a stronger difference in cognitive conflict between defectors and cooperators because dispositionally cooperative individuals more likely would identify the self-control conflict between cooperation and defection.

Higher observed levels of conflict associated with defection would then mean that self-control failure involves more conflict than does successful self-control.


Although Kieslich and Hilbig fail to discriminate between default versus cognitively controlled behavioral responses, they do provide evidence against four out of a total of six possible cognitive scenarios.

As illustrated in Table 1, there might have been no difference in conflict between cooperators and defectors, and cooperation could have been associated with more conflict; the latter possibility could have resulted from cooperation as the default and self-control failure yielding more conflict than successful self-control, or from defection as the default and successful self-control yielding more conflict than self-control failure.

As for assigning merit to the two possibilities that both involve higher conflict associated with defection, we would have to determine how to interpret the moderator—whether it captures dispositional spontaneous inclination or deliberative goal-pursuit.

If it captures the former, then we could conclude that the default is to cooperate (among the dispositionally cooperative) and that more conflict is associated with successful self-control in efforts to defect. But if it measures the latter, then we would conclude that the default is to defect, and that more conflict is associated with failed resistance to the impulse to defect.


We conclude that the predictions formulated by Kieslich and Hilbig (2014) are premature, as is the inference that their empirical results “support the idea that cooperation is the spontaneous and less conflicting response in social dilemmas” (p. 519).

While their results indeed have narrowed the possibility set, they do not allow us to discriminate between the two possible reflexive, default behaviors.

Future research might clarify the meaning of cognitive conflict in economic games.


Myrseth and Wollbrant (2015) offer an alternative theoretical explanation for our finding that defection entails more cognitive conflict than cooperation [1].

Although we completely agree that different theoretical explanations for a result are possible, we maintain that the theoretical approach we tested (Rand et al., 2014) is parsimonious and falsifiable, excluding certain plausible results a priori [2].

By comparison, the alternative framework proposed by Myrseth and Wollbrant requires several debatable assumptions to account for our findings, rendering it the more complex theory.

Besides, their framework as a whole could have accounted for any possible finding in our experiment, making it impossible to falsify it with our data.

We thus conclude that the notion by Rand et al.—that there is a spontaneous disposition to cooperate—has more empirical content while requiring fewer assumptions.


In their comment on our original article [1], Myrseth and Wollbrant (2015, M&W in what follows) raise an important general issue concerning the conclusiveness of empirical findings.

Whereas their arguments do not question that the methodology and analyses presented in our article indeed test (and support) the hypothesis that defection entails more cognitive conflict than cooperation, M&W offer an alternative explanation for the empirical pattern based on their own theoretical framework.

They argue that different assumptions within their framework—assumptions that run contrary to the theoretical assumptions underlying our hypothesis—could predict the same empirical pattern.

Here we will very briefly reiterate our line of reasoning and then discuss M&W’s alternative theoretical framework.


From recent theoretical arguments by Rand and colleagues that cooperation is the effortless and spontaneous response in social dilemmas [3][2], we derived the prediction that defection should entail more cognitive conflict—as measured by the curvature of response trajectories—than cooperation.

Our findings mirrored the hypothesis, showing that the attraction of the non-chosen option was stronger when people defected—which held particularly for individuals with a stronger dispositional tendency to make cooperative choices.

More importantly, the theoretical framework by Rand et al. could not have accounted for the opposite pattern (more conflict for cooperative choices) or a null effect (equivalent conflict for defection and cooperation).

As such, the hypothesis derived from Rand et al. ’s framework is specific and was strictly tested in the sense that it could have been falsified.


However, as is well known from the philosophy of science, the fact that empirical data are in line with a theory (even though it could have failed) is merely necessary but not sufficient for the “truth” of said theory [4]—simply because it is usually possible to generate various theoretical explanations for the same finding.

Note that we meant to imply this in the discussion of our original article in stating that “although the different curvatures of response trajectories for cooperation and defection decisions support the idea that cooperation is the spontaneous and less conflicting response in social dilemmas, they do not necessarily imply that there are actually two distinct systems (intuition and reflection)—as assumed by Rand et al. ( 2012)—that interact to produce decisions in social dilemmas” [1].

Whereas M&W’s selective citation of part of this sentence lets our reasoning appear to be inductive, it was clearly not our intention to make conclusive inferences regarding the “truth” of the theoretical framework by Rand et al. We merely sought to test a specific prediction derived from it and found support (in that the empirical data was in line with—and hence did not falsify—the prediction).

Nonetheless, other theoretical models may just as well allow for the observed empirical pattern to occur and thus be viable alternatives.


Consequently, there is nothing inherently worrisome (or indeed surprising) about M&W’s assertion that our findings could also be accounted for by a set of assumptions that are, in essence, incompatible with the theory we intended to test.

So, it seems worthwhile to consider the alternative account offered.

As an alternative theory, M&W put forward a framework that conceptualizes decisions in social dilemmas as a self-control problem [5].

Specifically, decisions are assumed to be the outcome of a two-stage process (see Figure 1 in their comment): First, people may not identify a self-control conflict and follow their impulse (in M&W’s framework referred to as “reflexive, default behavior”).

If people do identify a self-control conflict, they may next try to resist their impulse.

They may either fail to do so, thus following their impulse (i.e., choosing the default), or exercise successful self-control, thus following their deliberative goal instead.

Which behavior constitutes the default (cooperation vs. defection) is assumed to be context-dependent [5], but it remains entirely open which context would apply to our experiment.


To derive predictions for cognitive conflict, M&W further make the following assumptions: Following the default because no self-control conflict was identified leads to a “no conflict experience”.

Identifying a self-control conflict leads to a “conflict experience” in which case either self-control failure (sometimes also called unsuccessful resistance to impulse by M&W) or successful self-control may lead to a relatively higher degree of conflict (both cases are considered to be equally plausible in M&W’s theoretical framework).

M&W argue that, according to their framework, six different scenarios are possible, and that two of these are compatible with the results of our experiment.

As one of these scenarios assumes that cooperation is the majority default (which is compatible with the theory by Rand et al.) but the other presumes that defection is the default, M&W conclude that our findings do not allow for any inference whether cooperation or defection is the default behavior.


However, the scenario in which defection is the default option does require a number of additional assumptions in order to fit our results.

First, M&W assume that self-control failure (leading to defection) is associated with more cognitive conflict than successful self-control (leading to cooperation).

However, M&W explicitly state that the opposite (more conflict for successful self-control than failed resistance) is equally plausible.


Moreover, according to M&W’s framework a defection choice can be reached by two routes when assuming that defection is the default: People can either directly follow their impulse to defect (without identifying a self-control conflict) or they can identify a self-control conflict and fail to resist (and thus choose defection).

Importantly, when directly following the impulse, defection should be accompanied by a “no conflict experience” (see Figure 1 in M&W).

Yet, this would run contrary to the results in our study that M&W are trying to explain (more conflict for defection than cooperation).

To account for this, M&W additionally assume “that the difference in conflict [between self-control failure and successful self-control] more than offsets the low levels of conflict exhibited by those individuals who, without resisting (see Figure 1), act on the[ir] impulse” (Footnote 2 as referred to in M&W’s Table 1, squared brackets added).

However, we believe that it is questionable to assume that the difference in conflict between self-control failure and successful self-control is so large that it offsets a “no conflict experience”—especially since M&W themselves consider a reversed pattern (more conflict for successful self-control than self-control failure) to be equally plausible.

Importantly, even if one assumes that the amount of cognitive conflict for successful self-control and self-control failure is comparable, the additional impulsive choices for defection would lead to the prediction that cooperation is associated with more conflict than defection (i.e., the prediction would run contrary to the observed empirical pattern).


In other words, the proposed alternative account by M&W requires additional untested and, in our view, debatable assumptions.

If, in line with Occam’s razor, the “goal of model selection is to choose the simplest (i.e., least complex) model that describes the data well (i.e., descriptive adequacy)” [6], it is our impression that M&W’s alternative theoretical explanation is more complex (i.e., less parsimonious) than the theory by Rand et al.


Besides, M&W do not make any predictions as to the overall relative frequency of cases in which a conflict is identified or not—although this is clearly relevant for the predicted results pattern.

Only when talking about our finding that individuals high in Honesty-Humility (HH) experience more conflict when defecting vs. cooperating than individuals low in HH, M&W assume that the likelihood of identifying a self-control conflict differs: M&W assume that individuals low in HH should have a low likelihood while individuals high in HH should have a high likelihood.

However, based on this assumption people low in HH should show less conflict when defecting even when compared to individuals high in HH that cooperate (since no conflict identification should be accompanied by no conflict experience).

Yet, this is incompatible with the results we reported (see Figure 4; Kieslich & Hilbig, 2014).


Finally, M&W argue that by altering the assumptions within their framework one can actually explain every possible empirical pattern in our experiment (more conflict for cooperation, more conflict for defection, and no difference between the two).

However, M&W do not provide any theoretical prediction as to when which of their assumptions should hold.

As a consequence, there is no empirical pattern in our experiment that could falsify M&W’s theoretical framework.

It thus does not have sufficient empirical content [7] (see also Glöckner & Betsch, 2011) in the context of our study, at least not without further theoretical specification.

( See Postscript by Jonathan Baron who points out that more specific predictions for M&W’s framework can be derived, which are only partly supported by our data.)


In sum, we agree with M&W that other theoretical explanations for our result are possible.

Nonetheless, we test a theoretical approach that is falsifiable, excluding certain plausible results a priori—as any useful theory should [8].

Consequently, its descriptive adequacy indeed speaks for the theory.

By comparison, the alternative framework proposed by M&W requires several additional (and debatable) assumptions to account for the findings (especially when assuming that defection is the default behavior).

Moreover, it could have accounted for any finding in our experiment.

We would of course welcome any suggestion for a critical experiment that can disentangle the different theoretical explanations and conclusively test them against each other.

For the time being, we conclude that the approach of Rand et al. and the explanation assuming a spontaneous disposition to cooperate has more empirical content while requiring fewer assumptions, and, consequently, being more parsimonious than M&W’s alternative framework.


IX intensity was the only significant independent predictor of DSS.

These data implicate that intensity of CA IX expression seems to be a powerful independent prognostic factor, not confounded by the mode of treatment or other established prognostic factors (Giatromanolaki et al, 2001).


This study could not show a correlation between the expression of CA IX and other clinical or histopathological factors.

Most of the patients in this study had received RT with or without chemotherapy before operation, which may have an influence on tumour size, stage and grade, nodal status and consequently, CA IX staining in the operative sample.

To compensate this problem, CA IX expression was compared with the preoperative biopsies and respective surgical specimens.

However, some of the available diagnostic biopsy samples were scarce, precluding adequate comparison of all sample pairs.


Carbonic anhydrase isozyme IX staining intensity in the operative samples was shown to be significantly different among the treatment categories.

The same was also true among patients with different response to therapy, which was analysed from the long-course RT group specimens.

Most tumours in this group that had been treated with long-course RT without chemotherapy were CA IX positive with moderate/strong staining intensity.

Instead, tumours that had been treated with chemoradiotherapy were mostly CA IX negative.

This may imply that chemotherapy can act


Tumour oxygenation is known to have an effect on patient prognosis.

In advanced cancer of the uterine cervix, Hockel et al (1996) showed that low oxygen pressure was associated with larger tumours and more frequent parametrial spread as compared with tumours of the same stage and higher oxygen pressure.

Patients with hypoxic tumours had poorer disease outcome (Hockel et al, 1996).

Hypoxia induces CA IX, especially in or near necrotic regions (Wykoff et al, 2000) and in areas with high proliferation (Saarnio et al, 1998).

Preoperative treatment can shrink the tumour and even complete remissions are seen (Ruo et al, 2002).

As the tumour shrinks, its oxygenation may improve.

Hence, preoperative therapy may reduce tumour hypoxia and thereby CA IX expression may be downregulated.

In this study, there were two histologically complete responses after preoperative therapy.

One complete response was unexpectedly seen after short-course RT (Marijnen et al, 2001), which remains to be explained but may have been related to exceptional radiosensitivity.


The present data are in alignment with the results reported in other human malignancies, where CA IX expression has been shown to be a prognostic marker as well (Table 4).

This includes non-small-cell lung cancer (studied by Swinson et al (2003) and Giatromanolaki et al (2001)), bladder cancer (Hoskin et al, 2003), invasive breast cancer (Chia et al, 2001; Brennan et al, 2006) and oligodendroglioma (Järvelä et al, 2008).

Interestingly, in renal cancer, low CA IX expression and absence of VHL mutation were related to a more advanced tumour and unfavourable outcome (Patard et al, 2008).

Currently, antibodies against CA IX are being studied in phase three trials in the treatment of renal cancer (Pastorekova et al, 2007).

Also, sulphonamides have been tested for therapeutic purposes against CA IX for several years (Pastorekova et al, 2007).

It is possible that this strategy will also be evaluated in the treatment of other types of tumours.


To our knowledge, this is the first study to report the important prognostic significance of CA IX in rectal cancer.

This study shows that strong staining intensity of CA IX is an adverse prognostic factor in rectal cancer.

Further studies are needed to evaluate its potential therapeutic implications.


Tumour regression grade was analysed after long-course RT in the HE-stained sections according to the modified Dworak and Rödel scales, using three categories: poor, moderate and excellent response (Dworak et al, 1997; Rödel et al, 2005).

The response was assessed as poor, if only minimal or no tumour regression was seen and there was a dominant tumour mass left (Dworak 0 - 1).

When only a few tumour cells or tumour cell groups were left in the primary tumour, lymph nodes or perirectal fat, the response was assessed as moderate (Dworak 2).

The response was defined as excellent, if there were only very few or no tumour cells left (Dworak 3 - 4).

In case of moderate and excellent response, a total of 2 - 8 separate histological slides were studied to confirm the regression grade.


Carbonic anhydrase isozyme IX expression was analysed in all preoperative diagnostic biopsies (control group excluded) avail- able for the study, as well as in all tumour samples obtained at operation.

In each case, the most representative blocks were selected, cut to 5 mm sections and subjected to immunohisto- chemical (IHC) staining with rabbit polyclonal antibody for CA IX (ab15086, Abcam, Cambridge, UK).

The slides were pre-treated in microwave oven twice for 7 min in 10 mM sodium citrate buffer, pH 6.

The antibody was diluted to a 1 : 8000 concentration in DAKO Antibody diluent-solution.

The staining was carried out using the PowerVision þ Poly-HRP IHC-kit (Immunovision Technologies, Vision BioSystems, Norwell, MA, USA).

The HE staining was performed according to the standard laboratory protocol.


The IHC stainings of the samples were evaluated by two observers (EK and JS), blinded to clinical and radiological information.

Light microscopes with Â 4 and Â 10 objectives (EK) and Â 5 and Â 10 objectives (JS) were used for evaluation.

Carbonic anhydrase isozyme IX staining was graded using three approaches: (1) general grouping of the cases into positive or negative, (2) proportion of positive staining and (3) staining intensity.


The slides were assessed as negative, if the proportion of positive carcinoma cells in the section was less than 10%.

For positive cases, the proportion of positively stained carcinoma cells was analysed using four categories: (i) 10 - 25%, (ii) 26 - 50% and (iii) over 50%.

The positive slides were further evaluated to determine the staining intensity.


In grading the staining intensity, three categories were used: 0 for negative, 1 for weak and 2 - 3 for moderate to strong staining intensity.

Renal cell carcinoma was used as a positive control for strong staining intensity and normal rectal mucosa as a negative control.

In weak staining intensity, there was a faint positive staining in the cytoplasm and occasional staining in the cell membranes impossible to detect at a small magnification (objectives Â 4 to Â 5).

The staining was assessed as moderate or strong, if the positive reaction in the cell membranes could easily be identified at a small magnification (objectives Â 4 to Â 5).

If there were areas of a variety of staining intensities, the predominant intensity was chosen.

In preoperative biopsies, only positive/negative staining and staining intensity were assessed, but not the proportion of staining, because of the small size of the samples.


To support the IHC reaction of CA IX in this study, a small tissue material from CRC patients (n ¼ 4) was further studied for CA IX protein.

RNA from these tumours and corresponding normal colorectal mucosa had been previously isolated for cDNA microarray analysis, and CA9 was shown to be the most upregulated gene (Talvinen et al, 2006).

The protein from these samples was isolated to confirm CA IX expression in western blotting analysis.

Total protein was quantified for each sample in duplicate by Bio-Rad Protein Assay Dye (Bio-Rad Laboratories, Hercules, CA, USA) and the quality was verified with Coomassie blue staining.

For western blotting, equal amounts of denaturated protein samples were size fractioned using 10% SDS - PAGE and electroblotted onto nitrocellulose membrane (Whatman Protran, Perkin Elmer, Boston, MA, USA).

Uniform loading and blotting was checked with Ponceau S staining.

Carbonic anhydrase isozyme IX primary antibody (ab15086, Abcam) was diluted 1 : 1000 for western detection.

Horseradish peroxidase conjugated anti-rabbit immunoglobulins (Dako, Glostrup, Denmark) and Pierce ECL Western Blotting Substrate (Thermo Scientific, Rockford, IL, USA) were used according to the manufacturers' instructions for visualisation of the signal.


All statistical analyses were run using SPSS (SPSS Inc., Chicago, IL, USA) and STATA (Stata Corp., College Station, TX, USA) software packages (SPSS for Windows, version 16.0.2 and STATA/SE 10.1).

Frequency tables were analysed using the w 2 -test, with the likelihood ratio (LR) or Fisher's exact test for categorical variables.

Differences in the means of continuous variables were analysed using Mann - Whitney's test or Kruskal - Wallis's test for two and multiple independent samples, respectively.


Inter-observer reproducibility of the CA IX assessments was tested using regular (Cohen's) k and weighted k. To CA IX in rectal cancer E Korkeila et al 875 British Journal of Cancer (2009) 100(6), 874 - 880 & 2009 Cancer Research UK Clinical Studies calculate the latter, the ICC (intra-class correlation coefficient) test was used, with parallel mode and two-way random model, using consistency assumption and average measures option to interpret the ICC (95% CI).

Concordance of CA IX expres- sion between preoperative biopsies and operative samples was analysed using non-parametric paired-samples test (Wilcoxon signed ranks test or McNemar test).

The inter-observer reproducibility of all CA IX assessments was almost perfect, with regular k having values k ¼ 0.988, k ¼ 0.931 and k ¼ 0.892, for CA IX þ /À, staining proportion and intensity, respectively.

Using weighted k (ICC), even higher values were obtained: ICC ¼ 0.994, 0.985 and 0.984, respectively.

This indicates that all three classifications of CA IX staining used in this study are highly reproducible.


Univariate survival analysis for DFS and DSS was based on the Kaplan-Meier method, where stratum-specific outcomes were compared using log-rank (Mantel-Cox) statistics.

To adjust for covariates, Cox proportional hazards regression model was used, covariates being entered in stepwise backward manner.

All statistical tests were two-sided and declared significant at P-value o0.05 level.


Carbonic anhydrase isozyme IX staining was positive in 49% of the diagnostic biopsies and 44% of the operative specimens.

Staining intensity was weak in 15% and moderate or strong in 29% of the operative samples.

The proportion of CA IX positive staining and staining intensity were directly related (Spearman R ¼ 0.917, P ¼ 0.0001).

A low proportion of CA IX-positive staining (10 - 25%) was associated with weak staining intensity and vice versa.

Negative, weak, moderate and strong staining intensity of CA IX has been illustrated in Figure 1.

The results of IHC analysis were confirmed by the demonstration of CA IX protein in western blotting analysis (Figure 2).


Biopsy samples (B) stained for CA IX were compared with the respective operative samples (S).

Using CA IX positive/negative ( þ /À) grading, 48 out of 78 (61%) B - S pairs were concordant and 30 out of 78 (39%) were discordant.

Of the discordant samples, positive CA IX expression in biopsies was downregulated in 18 out of 38 (48%) cases, and negative CA IX expression was upregulated in 12 out of 40 (30%) cases.

Using CA IX intensity evaluation 43 out of 78 (55%) B - S pairs were concordant and 35 out of 78 (45%) were discordant.

In pairwise comparison (Wilcoxon), CA IX expression pattern in B - S pairs was not significantly different, P ¼ 0.268 and 1.000 for CA IX þ /À and CA IX intensity, respectively.

Results of staining intensity in the biopsy specimens are shown in Table 2.


In Table 3, the intensity of CA IX staining is compared with the traditional prognostic factors and survival.

When the long-course RT group was divided into two treatment categories (with or without concomitant chemotherapy), there was a statistically significant difference in staining intensity (P ¼ 0.006) between the groups, showing more intense staining in the group that had not received concomitant chemotherapy.

Tumour regression grade was possible to be evaluated in 36 long-course RT group patients.

The staining was more intense in the poor response group than in the moderate/excellent response group (P ¼ 0.010).

On the other hand, CA IX expression pattern was not significantly associated with the size, nodal status or grade of the tumour, number of examined lymph nodes, resection type or its radicality, circum- ferential-, proximal- or distal margins, vessel invasion, tumour necrosis or patient gender.


In univariate survival (Kaplan - Meier) analysis, patients with negative or weak CA IX staining intensity had significantly longer DFS (P ¼ 0.001), as depicted in Figure 3A.

Also, DSS was significantly (P ¼ 0.009) longer among patients who had tumours with negative or weak CA IX staining intensity (Figure 3B).

To analyse CA IX staining intensity as independent predictor of DFS and DSS, the following variables were entered in the multivariate (Cox) proportional hazards regression model: sex, age (young/old with median age as cutoff), treatment series (short-course RT, long-course RT and control group), preoperative tumour assess- ment (T), blood vessel invasion (yes/no), number of metastatic lymph nodes (LNN) (with four positive LNN as cutoff) and CA IX staining intensity.

In this multivariate model, only two variables were independent predictors of DFS: (1) number of metastatic


In this study, CA IX expression pattern was analysed from the samples of 166 patients, of whom 29 had received pre- operative chemoradiotherapy, 8 patients long-course RT without chemotherapy, 75 short-course RT and 54 patients had no therapy before surgery.

The intensity of CA IX expression was shown to be a significant prognostic factor in this study.

When analysed with univariate (Kaplan - Meier) survival model, we found that the intensity of CA IX expression was significantly related to both DFS and DSS.

Patients whose tumours were negative or weakly positive in CA IX staining had better prognosis.

The difference in DFS between patients with negative/weak staining intensity and those with moderate/strong staining intensity was evident from the early postoperative period, and the difference increased over time to about 50% at 3 years.

This is important while considering potential interventions with adjuvant therapies.

Similar data have not been previously reported in rectal cancer.


When examined in multivariate (Cox) proportional hazards regression model controlled for several covariates, CA IX intensity together with the number of metastatic lymph nodes were shown to be independent predictors of DFS.

Carbonic anhydrase isozyme


We have read with great interest the article by Korkeila et al (2009) 'Expression of carbonic anhydrase IX suggests poor outcome in rectal cancer' in a recent issue of the British Journal of Cancer.

As the authors rightly state, CA-IX has been found to be of clinical value in different tumour types.

The authors, however, state that their study was 'designed to assess the prognostic and predictive value of CA-IX in rectal cancer treated by either short or long course of radiotherapy'.

Although this might be the case, their data analysis does not support this statement.


The prognostic value of CA-IX, if present, cannot be deduced from the applied patient cohort, as all patients received radio- and/or chemotherapy.

A pure prognostic factor can only be detected in an untreated patient population.

CA-IX is also known to be strongly induced by hypoxia through HIF1a upregulation [2].

The hypoxic phenotype that is represented by the high expression levels of CA-IX is very likely to cause a predictive value of CA-IX, predicting in this case resistance to radiotherapy and possibly also chemotherapy.

Earlier, we have found CA-IX to be predictive for different adjuvant treatments in breast cancer [3].


On the other hand, the authors also describe a control group that would seem to have received no neoadjuvant chemo- or radiotherapy.

If these patients also did not receive adjuvant therapy, which is regrettably unclear from the paper, the authors should be able to determine whether the clinical value of CA-IX is similar in treated and untreated rectal cancer patients.

This could add valuable information to this subject and would warrant publication.


In our opinion, the results shown so far only allow the statement 'Expression of Carbonic Anhydrase IX suggests poor response to therapy in rectal cancer'.


Sir,


In our study, there were two index patient groups: (i) 75 patients who had received preoperative short-course radiotherapy and (ii) 37 patients who had received long-course radiotherapy with or without chemotherapy.

The third group included 54 control patients, none of whom had received any preoperative treatment.

In this group there were 34 patients who had not received any postoperative adjuvant treatment (chemotherapy or chemoradiotherapy) either.

To test the effect of adjuvant treatment, we compared these 34 patients with all the other patients (n = 132).

There was no statistically significant difference (P = 0.604) in disease-free survival, and the same was true with disease-specific survival (P = 0.111) in Kaplan – Meier analysis.


To explore this issue further, we also analysed these 54 control group patients separately, i.e., 20 patients who had received and 34 who had not received any adjuvant chemotherapy.

Among those 34 patients who had not received adjuvant therapy, 17 tumours were CA IX-positive and 17 CA IX-negative.

Among those 20 who had been treated with adjuvant therapy, 11 tumours were CA IX-positive and 9 were CA IX-negative (P = 0.722).

Among the control group patients who had not received adjuvant therapy, 35% of the tumours had moderate or strong and 15% weak intensity in CA IX staining.

Among the patients who had been treated with adjuvant therapy, seven tumours had weak and four tumours had moderate or strong staining intensity (P = 0.225).

There were no statistically significant differences in DFS or DSS between the patients who had or had not received adjuvant therapy.


However, a direct comparison between the groups might be biased by the fact that the patients who had received preoperative therapy had more advanced tumours.

According to our practice, adjuvant therapy is given whenever indicated due to a high recurrence risk (node-positive tumours, perforated tumours or tumours that have caused bowel obstruction, tumours with nerve or vessel invasion).

All these patients who had received adjuvant therapy had tumours with a higher risk of recurrence.


In conclusion, we agree that 'Expression of carbonic anhydrase IX suggests poor response to therapy in rectal cancer' [1].

We were delighted to learn that our article 'Expression of carbonic anhydrase IX suggests poor outcome in rectal cancer' (Korkeila et al, 2009) had been read with great interest.

Span et al (2009) pointed out that the results presented in our study only allow the statement that 'Expression of carbonic anhydrase IX suggests poor response to therapy in rectal cancer', which we agree upon [2].


The world's tropical forests are essential places for environmental sustainability and the future of our planet.

They combine high biodiversity and significant carbon storage with ecological services [1,2].

Since the 1980s, the world's tropical forests have undergone substantial change.

Agricultural expansion worldwide happened at the expense of tropical forest areas [3].

In particular, the Brazilian Amazon rain forest has suffered significant deforestation.

According to Brazil's National Institute for Space Research (INPE), deforestation has reached 20% of the Amazon rain forest in the country [4].


Producing qualified assessments of land-use and land cover change in Amazonia is essential for evidence-based policies that can protect the forest [5,6].

Earth observation data is the primary source of assessments of deforestation in Amazonia.

Since the late 1980s, INPE produces annual estimates of clear-cut areas of forest with the PRODES project and daily indications of deforestation alerts with the DETER initiative [7].

PRODES and DETER are the authoritative sources of information that support Brazil's actions in protecting Amazonia [8,9].

As a companion to the monitoring of clear-cut areas, INPE and its partners also produce periodic maps of land-use change in deforested areas in Amazonia with the TerraClass project [10].

Universities and research groups complement INPE's work [11,12].

Altogether, there is a substantial amount of land-use change information on the Brazilian Amazonia derived from remote sensing data.


Despite the widespread data availability, the complexity of land-use transitions in Amazonia requires continuous improvements in image classification.

Law enforcement actions by the Brazilian federal government managed to reduce deforestation from 2.7 Mha (million hectares) in 2004 to an average of 0.6 Mha between 2009 and 2018.

Despite such reduction, deforestation in Amazonia is still at a relatively high level.

To understand the complex interplay between crop production, cattle ranching, and land speculation, ever more detailed data is required.


To improve monitoring of the Amazon forest from satellites, researchers are investigating the use of big data analytics [13,14].

Such methods rely on the increase of data provided by the new generation of satellites such as Sentinel-2 [15].

However, to use these large data sets in tropical forest areas, researchers need suitable methods of automated cloud detection in optical imagery.


Traditional alternatives for dealing with cloud cover include combining information from various dates and selecting a "best pixel" for an extended period [16].

These methods lead to the loss of temporal information required to identify crop types [14,17] and pasture management [18,19].

To capture temporal information, many researchers prefer methods that identify cloud-covered pixels and replace them with interpolated estimates [20].

When different satellites are combined to produce a denser time series [21], replacing cloudy pixels by interpolated values becomes feasible.

For this reason, automated cloud detection algorithms are a necessary complement to big Earth observation data analytics.


Cloud detection algorithms are an active research field [20,22-25].

Each algorithm has specific characteristics and ad hoc techniques; thus, comparing them on a theoretical basis is hard.

In practice, performance assessment is done by selecting representative images and assessing how well each algorithm performs in each image.

As an example, Baetens et al. [ 26] compared three cloud detection methods (MAJA, Sen2Cor, and Fmask) using 32 images from 10 different locations.

As a reference for comparison, the authors use a machine learning method to identify clouds for Sentinel-2 images.

Given the different land cover, the diversity of sensors, and the advances in detection methods, such comparisons serve as general guidance only.


In this paper, we approach the problem of comparing different cloud detection algorithms from a regional viewpoint.

Given the importance of monitoring land change in Amazonia, we consider cloud detection methods for Sentinel-2 MSI images in this region.

We consider four cloud detection algorithms: Fmask 4 [25], MAJA [23], Sen2Cor 2.8 [22], and s2cloudless [27].


Cloud formation in Amazonia is distinct from most continental areas [28].

The forest produces its own rain [29].

The rainforest generates the aerosols that make up the cloud condensation nucleus in the region [30].

The probabilities of cloud coverage in satellite imagery depend not only on the month of the year but also on the location inside Amazonia [31].


Cloud types are heterogeneous in the Amazon biome; the southern region of the Amazonia has high aerosol concentration, whereas the northern and northwestern regions have low aerosol concentration and high precipitation [30,32].

These characteristics indicate different processes in cloud formation in subregions of the Amazon biome.


During the wet season, the precipitating clouds in the Amazon basin are either low-level stratus type clouds (up to 2-5 km altitude) or high-level convective systems (more than 6 km altitude) [30].

The different land cover influences the amount and type of clouds.

Deep clouds are commonly found over the forest while shallow clouds are frequent over deforested areas [33,34].

Water bodies absorb visible and near-infrared radiation diminishing the reflectivity of the thin clouds above [35].

The high reflectivity of artificial surfaces induces commission errors in cloud detection over urban areas [36].

Such differences pose a challenge for cloud detection algorithms in Amazonia; they need to consider many types of cloud formations and associated shadows.

These singular characteristics suggest that it is useful to take images over Amazonia as a study case when comparing cloud detection methods.


The rest of this article is organized as follows.

We first introduce the study area and the sample regions.

Then we present the cloud detection algorithms and how we configured them.

Later we show how the classes resulting from each cloud detection algorithm compares to the others.

Finally, we introduce our results and then we discuss some implications of this work.


The Amazon forest covers half of Brazil (49.3%) and provides four-fifths of its groundwater (81%) with an average rainfall of approximately 2300 mm per year [37].

Persistent cloud cover in Amazonia is a significant limitation for deforestation monitoring by satellite.

Using the Landsat archive from 1984 to 1997, Asner [31] shows how the probability of cloud cover on Landsat images depends not only on the month of the year but also on its location inside Amazonia.

From June to August, the chance of finding one image with less than 30% cloud cover is 60-90% in southeastern Amazonia.

In the southwestern part, cloud cover is persistent all year round.

While the recent availability of medium-resolution (10-100 m) sensors with higher temporal frequency than Landsat has improved the chances of obtaining cloud-free pixels, cloud cover in rain forests such as Amazonia will always be a challenge for optical remote sensing.


This study uses images from the Sentinel-2A satellite, launched in 2015.

The satellite is part of the Copernicus Earth Observation program of the European Union, which is operated by the European Space Agency (ESA) and managed by the European Commission.

It carries the Multispectral Instrument (MSI), which detects 13 bands of the electromagnetic spectrum spanning from the visible to the short infrared (SWIR) wavelengths at spatial resolutions of 10 m, 20 m, and 60 m, with a revisit period of 10 days [15] (see Table 1).

MSI's three bands at 60 m resolution are dedicated to atmospheric correction and cloud screening, leaving ten bands for land observation [38].

Sentinel-2A data enables researchers to explore the changes on Earth's surface due to its open data access policy and its temporal, spatial, and spectral resolutions.

To assess cloud detection algorithms over Amazonia, we chose five areas representative of its climate heterogeneity.

We identify them using the tiling system of Sentinel-2:


round and low deforestation.

Tile T20NPH overlaps forest and natural savanna, where emerging mining activities are menacing indigenous territories (Figure 1).


The paper compares four algorithms: Fmask 4 [25], MAJA [23], Sen2Cor 2.8 [22], and S2cloudless [27].

Fmask 4 and s2cloudless are specific for cloud detection.

MAJA and Sen2Cor 2.8 are image processors; they generate cloud masks as part of image conversion from radiance at the top of the atmosphere to reflectance from ground targets.


To process Landsat 8 data, USGS uses a version of the Fmask method that requires the thermal band [40].

Fmask 4 is a version of Fmask that has been adjusted to be used with sensors without thermal bands.

ESA uses Sen2Cor to process Sentinel-2 images.

MAJA is developed by CNES and is used by applications such as Sen2Agri [41].

The Sentinel Hub uses S2cloudless for the fast generation of cloud masks [27].

These methods represent the latest generation of cloud detection algorithms for optical remote sensing images.


Fmask 4 [25] is the most recent version of Fmask [20].

Earlier versions of Fmask required a thermal band and worked only on Landsat images.

The latest version also works on Sentinel-2 images [25].

To distinguish between clouds and bright surfaces in Landsat 8 images, Fmask 4 uses the thermal band.


In the case of Sentinel-2 images, it takes the view angle parallax of the NIR bands [24].

To reduce false positives resulting from snow and built-up areas, Fmask 4 uses spectral and contextual features.

To distinguish land from water, it relies on global surface water map [42].


Fmask 4 matches clouds with their shadows based on similarity.

It iterates cloud height from a minimum to a maximum level; for each possible height, it computes the similarity between cloud and cloud shadows [43].

When processing Sentinel-2 images, its cloud and cloud shadow masks have a 20 m resolution [25].


Sen2Cor processes Sentinel-2 data to estimate Bottom-Of-Atmosphere (BOA) reflectances from Top-Of-Atmosphere (TOA) data [22].

It takes Level-1C images and adjusts for atmospheric effects, generating Level-2A surface reflectance products [22,39].

It generates two types of results: (1) atmospheric correction products, such as aerosol optical thickness, surface reflectance, and water vapor maps; and (2) cloud screening and scene Classification (SCL), which assigns a class to each pixel.

Sen2Cor provides two quality indicators: A cloud confidence map and a snow confidence map with values ranging from 0 to 100%.

The distinction between cloudy, clear and water pixels in the SCL and the output of the cloud confidence map are used to produce the cloud confidence information [39].

The current version of Sen2Cor (2.8) increases the accuracy of classification on water, urban, and bare areas while reducing false positives for snow.

Other improvements include cirrus detection, false cloud detection due to permanent bright targets, classification of water pixels inside of cloud borders, and discrimination between topographic and cloud shadow pixels [44].


Sentinel Hub's S2cloudless is a machine-learning-based cloud detector [27].

Its input is Level-1C top of atmosphere data from 10 Sentinel-2 bands (bands 1-5 and 8-12) combined with pairwise band differences and band ratios.

It uses the LightGBM algorithm [45] trained over multiple clouded and non-clouded samples over the world.

As training data, is uses cloud masks provided by MAJA as a proxy for ground truth.

S2cloudless trained its classification model with 15,000 Sentinel-2 tiles from 596 geographically unique areas in 77 different countries.


MAJA (MACCS-ATCOR Joint Algorithm) combines two methods: (a) the Multi-sensor Atmospheric Correction and Cloud Screening (MACCS) and (b) the Atmospheric and Topographic Correction (ATCOR).

It builds on these methods by including time-series of images to improve detection of reflectance changes due to clouds [46].

The method assumes that surface reflectances without clouds are stable in time, while clouds or cloud shadows result in quick variations [23].

MAJA uses multi-temporal images that contain the most recent cloud-free observation for each pixel.

At each new image, the algorithm updates this composite with the newly-available cloud-free pixels.

Thus, it processes the data for a given location in chronological order [26].

The algorithm needs to be initialized to fix cases where a given pixel has no cloud-free observations.

To cover these specific cases, MAJA also uses a mono-temporal criterion based only on spectral information [23].


To run Fmask 4, MAJA 3.2.2, and Sen2Cor 2.8, we used Linux Docker containers.

The Fmask 4 implementation uses the MATLAB code available at GitHub (https://github.com/GERSL/Fmask).

For Sen2Cor 2.8, we installed the version provided by ESA (https://step.esa.int/main/third-party- plugins-2/sen2cor/).

The MAJA implementation was obtained from CNES (https://logiciels.cnes.fr/ fr/content/maja).

We downloaded S2cloudless 1.4.0 from the Sentinel Hub (https://www.sentinel- hub.com/).

Run-time parameters are described below.


Fmask 4: Dilation parameters for cloud, cloud shadows, and snow were set to 3, 3, and 0 pixels, respectively.

The cloud probability threshold was 20%, following Qiu et al. [ 47].


Cloud probability threshold was set to 70%, using a four-pixel convolution for averaging cloud probabilities and dilation of two pixels, following the parameters set by Zupanc et al. [ 27].


Sen2Cor 2.8: The tests used the same configuration as that of the Land Cover maps of ESA's Climate Change Initiative (http://maps.elie.ucl.ac.be/CCI/viewer/download.php).


MAJA: The evaluation used the same configuration as that of the Sen2Agri application (http://www.esa-sen2agri.org).


To validate the resulting cloud masks, we used sample points tagged by remote sensing experts through visual interpretation, following the work of Foga et al. and Zhu et al. [ 36,40].

We selected a set of random locations inside each Sentinel-2 tile.

Since Sentinel-2 images at 10 m resolution have over 120 million pixels, standard statistical techniques indicate that using about 400 samples per image is enough to achieve a 95% confidence level with a 5% margin of error [48].


Five experts labeled those points in all 20 images.

The labels were "cloud", "cloud shadow", "clear", and "other".

The "other" label is a placeholder for samples that the experts could not tag.

Since the areas of cloud shadow are small compared to other labels, we tried to ensure there were at least 50 samples of cloud shadows.


Two different experts classified each point; only those where both experts agreed were selected.

Because of the need for agreement between experts, the final number of selected samples changes from image to image (see Table 2).


Since the algorithms tested use different labels, we recoded their results to match the labels in the validation sample set.

In particular, MAJA produces an 8-bit mask, so that many labels can be applied to a pixel, allowing combinations that are not available in the results of other algorithms.

For example, MAJA's mask allows tagging a pixel as a shadow projected on top of a cloud from another cloud in a neighboring image.

To make MAJA's more detailed results compatible with the output of the other methods, we prioritize clouds over cloud shadows and cloud shadows over clear pixels.

Table 3 shows how the original codes for each method were relabelled for compatibility.


To compare the results of the algorithms, we use the F1 score [49] and the user's, producer's, and overall accuracies [50].

The F1 score (Equation (1)) is the harmonic mean of the precision (Equation (2)) and recall rates (Equation (3)).

The producer's accuracy measures how well a certain label has been classified.

It is computed by dividing the correctly classified pixels in each class by the total number of pixels of the corresponding class.

The user's accuracy indicates the probability that prediction represents reality.

It is computed by dividing the correctly classified pixels in each label by the total number of pixels classified in that label.

The overall accuracy indicates the quality of the map classification.

It is calculated by dividing the total number of correctly classified pixels by the total number of reference pixels.


In our experiments, Fmask 4 has the best overall accuracy, followed by Sen2Cor, MAJA, and S2cloudless (see Table 4).

Fmask 4 consistently outperforms the other algorithms in overall, user's and producer's accuracies for all classes.

It also has the best results considering individual tiles.

For cloud shadow detection, Fmask 4 has a better performance than Sen2Cor.

Although MAJA has the ability to detect cloud shadows, in practice the methods extend its cloud mask to include shadows.

MAJA is a conservative method that uses dilation operators to improve the user's accuracy of the clear sky.

Therefore, no shadows are reported by MAJA.


This is observable in either the images themselves or Figure 2, in which MAJA consistently detects more cloud pixels than the other methods.

Furthermore, Fmask 4 and Sen2Cor also distinguish cirrus clouds from clouds, as shown in Figure 2.

For accuracy assessment, we merged both types of clouds for computing the information in Tables 4 and 5 to be able to compare Fmask 4 and Sen2Cor with the other methods.


When comparing the overall accuracy of Sen2Cor with that of MAJA, their design choices stand out.

MAJA has a better user's accuracy for clear sky pixels than Sen2Cor; for producer's accuracy of this class, Sen2Cor is superior.

Conversely, MAJA has a better producer's accuracy than Sen2Cor for cloud pixels; for the user's accuracy, the situation is inverted.

This situation also holds in individual tiles (see Table 5).

The designers of MAJA have chosen to maximize the probability that pixels labeled as a clear sky are correct.


The behavior of S2cloudless is erratic; sometimes it produces results visually similar to those of Fmask 4 or Sen2Cor (see Figure 3), while in some other occasions it misclassifies clear pixels as clouds (see Figure 2).

For example, for tile T21LXH on 28 March 2017 and tile T22MCA on 28 June 2018, S2cloudless has a particularly poor performance.

Figure 4 shows tile T21LXH on 28 March 2017, a case where S2cloudless performs differently from the other methods.


These results showed that the four algorithms produce their best results on images with few well-defined (crisp) clouds.

Except for s2cloudless, the algorithms agree on the shape and the number of areas classified as either cloud or clear.

However, they cannot adequately approximate the shape of clouds and their shadows on thin semi-transparent cirrus or tightly packed clouds (see Figure 5).

The accurate detection of cloud shadows is challenging because dark surfaces, such as wetlands, burned areas, and terrain shadows can be easily confused with cloud shadows [25].


The pictures shown in Figure 4 confirm the results discussed above.

While Fmask4 has the best performance, it is interesting to compare MAJA with Sen2Cor.

MAJA uses squircles (i.e., a shape between a circle and a rectangle) to fill in the cloud shape, ensuring the total coverage of each cloud in detriment of cloud shadows.

Thus, MAJA sometimes incorporates clear pixels in its cloud mask, as also reported by [26].

By contrast, Sen2Cor approximates the shape of the cloud from the inside, filling in the clouds' boundaries with saturated labels-particularly with thin cirrus clouds-which produces rough borders (see Figure 5).

Furthermore, Sen2Cor cannot detect small clouds which are correctly identified by Fmask 4 and MAJA (e.g., see the small clouds at the center of Figure 3).


The shadow masks produced by Fmask 4 are displaced regarding to the shadows visible in the images.

This is a consequence of the coarse spatial resolution of the digital elevation model used by the method.

As for their shapes, Fmask 4 matches well the cloud shadows respect to the clouds producing them, while in Sen2Cor the cloud shadows have smoother and different boundaries than their clouds (see Figure 5).

Our results confirm the work of Qiu et al. [ 25] and Baetens et al. [ 26], who report that Fmask 4 works better in detecting clouds and cloud shadows than Sen2Cor for Sentinel-2A images.


We could not compute the accuracies for cloud shadow detection for MAJA and S2cloudless.

This is expected from s2cloudless but it comes as a surprise in the case of MAJA.

An explanation is MAJA's greedy behavior regarding clouds; it tends to tag pixels as clouds in disregard of their shadows (see Figure 2).

An alternative explanation is due to our interpretation of the MAJA's bit mask where we prioritized clouds over cloud shadows (see Section 2.6).


Sen2Cor tags many pixels as saturated, defective, or unclassified which we labelled as other (see Table 3).

A visual inspection reveals most of the saturated pixels are the external border of clouds (see Figure 3).

On the other hand, Sen2Cor in tile T21LXH for 08 March 2017 (see Figure 4) mistakenly displays cloud and cloud shadow pixels along the riverbank, almost perfectly profiling the whole river; this could be caused by suspended matter in the water.

Sen2Cor problems to detect cloud cover over water were also reported by [25,51].

As discussed above, the shapes of clouds in the Sen2Cor mask are rougher compared to the smoother results using the other algorithms.

Despite these issues, Sen2Cor is a reliable method for cloud detection.

Its producer's accuracies for clear sky and clouds are respectively 89% and 88%.

If its errors in detecting cloud shadows can be tolerated by the user, its efficiency and ease of use may justify its choice for bulk processing.


S2cloudless erratically mixes land features and clouds, particularly on images with few clouds.

It does not spot cloud shadows.

We could not confirm the claims made by the authors of this method [27] about the good performance of this algorithm.

One explanation is that the clouds in tropical forests such as Amazonia are not adequately included in the S2cloudless training set.