Spaces:
Running
Running
File size: 7,735 Bytes
8bb074c 2a0c691 695bb7b 2a0c691 8bb074c 2a0c691 d129cb1 695bb7b c174a9f 2a0c691 8bb074c 695bb7b d129cb1 2a0c691 8bb074c 695bb7b 0badfc1 695bb7b 626d74b 695bb7b 8bb074c 695bb7b d129cb1 0badfc1 d129cb1 0badfc1 8bb074c 0badfc1 8bb074c 0badfc1 d129cb1 0badfc1 8bb074c 2a0c691 8bb074c d129cb1 2a0c691 c174a9f 2a0c691 8bb074c 2a0c691 859c546 2a0c691 8bb074c 2a0c691 8bb074c 2a0c691 d129cb1 2a0c691 d129cb1 8bb074c 2a0c691 8bb074c d129cb1 2a0c691 695bb7b 2a0c691 8bb074c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 | """Instructions and a worked example for each part of the study, shown in a panel participants can reopen at any
time. The example is made up (it is not one of the study documents), so it cannot prime any real answer.
A config can replace a text with ``study.guide_simplicity`` / ``study.guide_validity`` (Markdown)."""
EXAMPLE_TEXTS = {
"A": "We looked at 12 studies with 2,431 people over 65. Walking every day probably means fewer falls after "
"one year. We do not know if it prevents broken bones.",
"B": "Twelve randomised trials in adults aged over 65 found that daily walking programmes probably reduce the "
"incidence of falls at 12 months relative to usual care, whereas the effect on fractures remains very "
"uncertain owing to very low-certainty evidence.",
"C": "Older people who walked every day had fewer falls. Walking also helped prevent broken bones.",
}
EXAMPLE_SOURCE = ("Twelve randomised trials (n = 2,431) compared daily walking programmes with usual care in adults "
"over 65 living at home. Walking reduced falls at 12 months (risk ratio 0.78, 95% CI 0.66 to 0.92; "
"moderate-certainty evidence). Effects on fractures were uncertain (very low-certainty evidence). "
"Authors' conclusions: daily walking probably reduces falls in older adults living in the "
"community, but we do not know whether it reduces fractures.")
VALIDITY_TEXTS = {
"A": "We looked at 12 studies with 2,431 people over 65 living at home. Walking every day probably means fewer "
"falls after one year. We do not know if it prevents broken bones.",
"B": "Studies in people over 65 living at home suggest that walking every day probably helps prevent falls.",
"C": "Older people who walk every day probably have fewer falls. Walking also helps prevent broken bones.",
}
def _texts(texts=None) -> str:
return "\n".join(f"> **Text {L}.** {t}\n>" for L, t in (texts or EXAMPLE_TEXTS).items())
GUIDE = {
"simplicity": f"""
### What you do
For each document you see three simplifications (A, B, C) of the same scientific text, written to help the general
public (non-expert readers) understand it. You do **not** see the original scientific text in this part.
The three simplifications come from different sources. We want to compare these sources and find out how easy
their texts are to follow for a non-expert reader. Your answers tell us which simplifications readers find easier.
You can stop and come back at any time with your ID: your answers are saved after each document. Please take a break
whenever you feel tired, and try to finish within a few days so your judgements stay consistent.
### How to read
Please read each text through once, at your normal pace. If the easiest and the hardest are then obvious to you,
answer straight away: there is no need to re-read or compare them in detail.
### The questions
#### 1. Easiest to follow
The simplification you understood with the least effort.
#### 2. Hardest to follow
The simplification that took the most effort to understand. It must be a different one from question 1.
#### 3. How much easier
Two short questions appear once you have answered 1 and 2. They compare the easiest with the middle one, and the
middle one with the hardest: *negligible*, *small*, *moderate* or *large*. If two texts feel about the same, answer
*negligible*: that is a normal answer.
### What to judge
- Only how easy each simplification is to **read and understand** for a non-expert reader: the words, the sentences,
how the ideas follow each other, how clearly the text is organised.
- **Not** whether the content is correct or complete: this is not the subject of this study.
- **Not** simply length: a short text can still be hard to follow, and a longer one can be easy.
- **Do not look anything up** (dictionary, search engine, AI assistant such as ChatGPT): read each text as it is,
with what you already know. If a word or a sentence is unclear to you, that is exactly what we want to measure.
### There is no right answer
We want your own experience as a reader. Once you submit a document you cannot go back to it.
### How the questions fit together
*An illustration of the mechanics only: which text is easier is entirely up to you.*
Suppose that, for some document, you find **C** the easiest to follow and **A** the hardest. B is then the middle
one, and the two follow-up questions become:
- "How much easier is **C** than **B**?"
- "How much easier is **B** than **A**?"
For each, pick *negligible*, *small*, *moderate* or *large*, from your own impression.
""",
"validity": f"""
### What you do
For each document you see the **original** scientific text and three simplifications of it (A, B, C), written to
help the general public (non-expert readers) understand it.
The three simplifications come from different sources. We want to compare these sources and find out whether, and
how often, they make mistakes when they simplify. Your answers tell us which simplifications have which kind of
mistake.
You can stop and come back at any time with your ID: your answers are saved after each document. Please take a break
whenever you feel tired, and try to finish within a few days so your judgements stay consistent.
For each question below, tick every simplification it applies to: none, one, two or all three.
### The questions
#### 1. Leaves out important information
The simplification omits critical information from the original. It is not misleading, but it leaves the reader
unaware of key points: for example a part of the research question, a conclusion, or what the results mean in
practice. Judge "important" from the point of view of a non-expert reader. This includes:
- an **empty** simplification;
- a simplification **so general** that it loses what the original found or explains (for example "Exercise is good");
- a simplification that covers **only one of two independent findings** of the original.
Leaving out details (numbers, methods, secondary results) is normal in a simplification and is **not** an omission.
#### 2. Contradicts the source
The simplification says the opposite of the original, or states something the original does not contain: a made-up
fact, a wrong number.
#### 3. States a finding more broadly than the source supports
The simplification generalises the original to the point where it becomes false: it drops *who* the finding applies
to or *under what conditions* (for example "people" for "adults over 65 living at home"), or presents an uncertain
result as established.
Judge each simplification only against what the original says, not how easy it is to read.
### There may be no error at all
Many texts may be fine on all three questions. Leaving every box unticked is a normal answer, not a missed one: tick
a box only when you find a real problem. Once you submit a document you cannot go back to it.
### Example
*Made up, not one of the study documents.*
> **Original.** {EXAMPLE_SOURCE}
{_texts(VALIDITY_TEXTS)}
**A possible answer:**
- *Leaves out important information*: **B**. The original has two conclusions, about falls and about broken bones;
B keeps only the first, so the reader never learns that the effect on broken bones is unknown.
- *Contradicts the source*: **C**. It says walking helps prevent broken bones; the original says this is not known.
- *More broadly than supported*: **C**. "Older people" is broader than "adults over 65 living at home": the original
says nothing about people aged 60, for instance, or living in care homes.
- **A** has none of these problems: nothing to tick for it.
""",
}
|