BVendeville commited on
Commit
97b702e
·
verified ·
1 Parent(s): 7d929b7

annotation app code-885a43f88db4+git-f87f4b3-dirty

Browse files
Files changed (1) hide show
  1. guide.py +55 -29
guide.py CHANGED
@@ -17,25 +17,33 @@ EXAMPLE_SOURCE = ("Twelve randomised trials (n = 2,431) compared daily walking p
17
  "community, but we do not know whether it reduces fractures.")
18
 
19
 
20
- def _texts() -> str:
21
- return "\n".join(f"> **Text {L}.** {t}\n>" for L, t in EXAMPLE_TEXTS.items())
 
 
 
 
 
 
 
 
22
 
23
 
24
  GUIDE = {
25
  "simplicity": f"""
26
- #### What you do
27
 
28
  For each document you see three simplifications (A, B, C) of the same scientific text, written to help the general
29
  public (non-expert readers) understand it. You do **not** see the original scientific text in this part.
30
 
31
- #### The questions
32
 
33
  1. **Easiest to follow**: the text you understood with the least effort.
34
  2. **Hardest to follow**: the text that took the most effort to understand.
35
  3. **How much easier**: two short questions comparing the easiest with the middle one, and the middle one with the
36
  hardest: *negligible*, *small*, *moderate* or *large*.
37
 
38
- #### What to judge
39
 
40
  - Only how easy each text is to **read and understand**: the words, the sentences, how the ideas follow each other.
41
  - **Not** whether the content is correct or complete (you cannot check that here).
@@ -43,7 +51,7 @@ public (non-expert readers) understand it. You do **not** see the original scien
43
 
44
  There are no right answers: we want your experience as a reader. Once you submit a document you cannot go back to it.
45
 
46
- #### Example
47
 
48
  *Made up, not one of the study documents.*
49
 
@@ -57,46 +65,64 @@ There are no right answers: we want your experience as a reader. Once you submit
57
  *large*. (C's claim about broken bones is wrong, but that is not what this part asks about.)
58
  """,
59
  "validity": f"""
60
- #### What you do
61
 
62
  For each document you see the **original** scientific text and three simplifications of it (A, B, C), written to
63
- help the general public (non-expert readers) understand it. For each question, tick every text it applies to:
64
- none, one, two or all three.
 
 
 
 
 
 
 
 
 
65
 
66
- #### The questions
 
 
67
 
68
- 1. **Leaves out important information.** After reading the text, a non-expert reader would not know **what the
69
- research found** (its main result) or **how sure the researchers are** about it (for example "probably", or "the
70
- evidence is very uncertain"). Take both from the original's conclusions. Leaving out details (numbers, methods,
71
- secondary results) is normal in a simplification and is **not** an omission.
72
- 2. **Contradicts the source.** The text says the opposite of the original, or states something the original does
73
- not contain: a made-up fact, a wrong number.
74
- 3. **States a finding more broadly than the source supports.** The text generalises the original to the point where
75
- it becomes false: it drops *who* the finding applies to or *under what conditions* (for example "people" for
76
- "adults over 65 living at home"), or presents an uncertain result as established.
77
 
78
- Judge each text only against what the original says, not how easy it is to read.
79
 
80
- #### There may be no error at all
 
 
 
 
 
 
 
 
 
 
 
 
 
81
 
82
  Many texts may be fine on all three questions. Leaving every box unticked is a normal answer, not a missed one: tick
83
  a box only when you find a real problem. Once you submit a document you cannot go back to it.
84
 
85
- #### Example
86
 
87
  *Made up, not one of the study documents.*
88
 
89
  > **Original.** {EXAMPLE_SOURCE}
90
 
91
- {_texts()}
92
 
93
  **A possible answer:**
94
 
95
- - *Leaves out important information*: **none**. All three tell a non-expert reader that walking reduces falls, and
96
- A and B say how sure the researchers are. (C's problem with certainty is under question 3.)
97
- - *Contradicts the source*: **C**. It says walking helped prevent broken bones, which the original does not say:
98
- it says this is not known.
99
- - *More broadly than supported*: **C**. "Older people" drops "over 65" and "living at home", and "had fewer falls"
100
- presents a probable result as established. A and B keep who the finding applies to and how sure it is.
101
  """,
102
  }
 
17
  "community, but we do not know whether it reduces fractures.")
18
 
19
 
20
+ VALIDITY_TEXTS = {
21
+ "A": "We looked at 12 studies with 2,431 people over 65 living at home. Walking every day probably means fewer "
22
+ "falls after one year. We do not know if it prevents broken bones.",
23
+ "B": "Studies in people over 65 living at home suggest that walking every day probably helps prevent falls.",
24
+ "C": "Older people who walk every day have fewer falls. Walking also helps prevent broken bones.",
25
+ }
26
+
27
+
28
+ def _texts(texts=None) -> str:
29
+ return "\n".join(f"> **Text {L}.** {t}\n>" for L, t in (texts or EXAMPLE_TEXTS).items())
30
 
31
 
32
  GUIDE = {
33
  "simplicity": f"""
34
+ ### What you do
35
 
36
  For each document you see three simplifications (A, B, C) of the same scientific text, written to help the general
37
  public (non-expert readers) understand it. You do **not** see the original scientific text in this part.
38
 
39
+ ### The questions
40
 
41
  1. **Easiest to follow**: the text you understood with the least effort.
42
  2. **Hardest to follow**: the text that took the most effort to understand.
43
  3. **How much easier**: two short questions comparing the easiest with the middle one, and the middle one with the
44
  hardest: *negligible*, *small*, *moderate* or *large*.
45
 
46
+ ### What to judge
47
 
48
  - Only how easy each text is to **read and understand**: the words, the sentences, how the ideas follow each other.
49
  - **Not** whether the content is correct or complete (you cannot check that here).
 
51
 
52
  There are no right answers: we want your experience as a reader. Once you submit a document you cannot go back to it.
53
 
54
+ ### Example
55
 
56
  *Made up, not one of the study documents.*
57
 
 
65
  *large*. (C's claim about broken bones is wrong, but that is not what this part asks about.)
66
  """,
67
  "validity": f"""
68
+ ### What you do
69
 
70
  For each document you see the **original** scientific text and three simplifications of it (A, B, C), written to
71
+ help the general public (non-expert readers) understand it.
72
+
73
+ The three simplifications come from different sources. We want to compare these sources and find out whether, and
74
+ how often, they make mistakes when they simplify. Your answers tell us which simplifications have which kind of
75
+ mistake.
76
+
77
+ For each question below, tick every simplification it applies to: none, one, two or all three.
78
+
79
+ ### The questions
80
+
81
+ #### 1. Leaves out important information
82
 
83
+ The simplification omits critical information from the original. It is not misleading, but it leaves the reader
84
+ unaware of key points: for example a part of the research question, a conclusion, or what the results mean in
85
+ practice. Judge "important" from the point of view of a non-expert reader. This includes:
86
 
87
+ - an **empty** simplification;
88
+ - a simplification **so general** that it loses what the original found or explains (for example "Exercise is good");
89
+ - a simplification that covers **only one of two independent findings** of the original.
 
 
 
 
 
 
90
 
91
+ Leaving out details (numbers, methods, secondary results) is normal in a simplification and is **not** an omission.
92
 
93
+ #### 2. Contradicts the source
94
+
95
+ The simplification says the opposite of the original, or states something the original does not contain: a made-up
96
+ fact, a wrong number.
97
+
98
+ #### 3. States a finding more broadly than the source supports
99
+
100
+ The simplification generalises the original to the point where it becomes false: it drops *who* the finding applies
101
+ to or *under what conditions* (for example "people" for "adults over 65 living at home"), or presents an uncertain
102
+ result as established.
103
+
104
+ Judge each simplification only against what the original says, not how easy it is to read.
105
+
106
+ ### There may be no error at all
107
 
108
  Many texts may be fine on all three questions. Leaving every box unticked is a normal answer, not a missed one: tick
109
  a box only when you find a real problem. Once you submit a document you cannot go back to it.
110
 
111
+ ### Example
112
 
113
  *Made up, not one of the study documents.*
114
 
115
  > **Original.** {EXAMPLE_SOURCE}
116
 
117
+ {_texts(VALIDITY_TEXTS)}
118
 
119
  **A possible answer:**
120
 
121
+ - *Leaves out important information*: **B**. The original has two conclusions, about falls and about broken bones;
122
+ B keeps only the first, so the reader never learns that the effect on broken bones is unknown.
123
+ - *Contradicts the source*: **C**. It says walking helps prevent broken bones; the original says this is not known.
124
+ - *More broadly than supported*: **C**. "Older people" drops "over 65" and "living at home", and "have fewer falls"
125
+ presents a probable result as established.
126
+ - **A** has none of these problems: nothing to tick for it.
127
  """,
128
  }