Spaces:
Sleeping
Sleeping
Download guide.py from BVendeville/reading-study-validity: direct link, hf CLI and curl.
- Browser
- Download file 7.74 kB
-
https://huggingface.co/spaces/BVendeville/reading-study-validity/resolve/main/guide.py
- Command line
-
hf download hf://spaces/BVendeville/reading-study-validity/guide.py
-
curl -L -o guide.py https://huggingface.co/spaces/BVendeville/reading-study-validity/resolve/main/guide.py
7.74 kB
| """Instructions and a worked example for each part of the study, shown in a panel participants can reopen at any | |
| time. The example is made up (it is not one of the study documents), so it cannot prime any real answer. | |
| A config can replace a text with ``study.guide_simplicity`` / ``study.guide_validity`` (Markdown).""" | |
| EXAMPLE_TEXTS = { | |
| "A": "We looked at 12 studies with 2,431 people over 65. Walking every day probably means fewer falls after " | |
| "one year. We do not know if it prevents broken bones.", | |
| "B": "Twelve randomised trials in adults aged over 65 found that daily walking programmes probably reduce the " | |
| "incidence of falls at 12 months relative to usual care, whereas the effect on fractures remains very " | |
| "uncertain owing to very low-certainty evidence.", | |
| "C": "Older people who walked every day had fewer falls. Walking also helped prevent broken bones.", | |
| } | |
| EXAMPLE_SOURCE = ("Twelve randomised trials (n = 2,431) compared daily walking programmes with usual care in adults " | |
| "over 65 living at home. Walking reduced falls at 12 months (risk ratio 0.78, 95% CI 0.66 to 0.92; " | |
| "moderate-certainty evidence). Effects on fractures were uncertain (very low-certainty evidence). " | |
| "Authors' conclusions: daily walking probably reduces falls in older adults living in the " | |
| "community, but we do not know whether it reduces fractures.") | |
| VALIDITY_TEXTS = { | |
| "A": "We looked at 12 studies with 2,431 people over 65 living at home. Walking every day probably means fewer " | |
| "falls after one year. We do not know if it prevents broken bones.", | |
| "B": "Studies in people over 65 living at home suggest that walking every day probably helps prevent falls.", | |
| "C": "Older people who walk every day probably have fewer falls. Walking also helps prevent broken bones.", | |
| } | |
| def _texts(texts=None) -> str: | |
| return "\n".join(f"> **Text {L}.** {t}\n>" for L, t in (texts or EXAMPLE_TEXTS).items()) | |
| GUIDE = { | |
| "simplicity": f""" | |
| ### What you do | |
| For each document you see three simplifications (A, B, C) of the same scientific text, written to help the general | |
| public (non-expert readers) understand it. You do **not** see the original scientific text in this part. | |
| The three simplifications come from different sources. We want to compare these sources and find out how easy | |
| their texts are to follow for a non-expert reader. Your answers tell us which simplifications readers find easier. | |
| You can stop and come back at any time with your ID: your answers are saved after each document. Please take a break | |
| whenever you feel tired, and try to finish within a few days so your judgements stay consistent. | |
| ### How to read | |
| Please read each text through once, at your normal pace. If the easiest and the hardest are then obvious to you, | |
| answer straight away: there is no need to re-read or compare them in detail. | |
| ### The questions | |
| #### 1. Easiest to follow | |
| The simplification you understood with the least effort. | |
| #### 2. Hardest to follow | |
| The simplification that took the most effort to understand. It must be a different one from question 1. | |
| #### 3. How much easier | |
| Two short questions appear once you have answered 1 and 2. They compare the easiest with the middle one, and the | |
| middle one with the hardest: *negligible*, *small*, *moderate* or *large*. If two texts feel about the same, answer | |
| *negligible*: that is a normal answer. | |
| ### What to judge | |
| - Only how easy each simplification is to **read and understand** for a non-expert reader: the words, the sentences, | |
| how the ideas follow each other, how clearly the text is organised. | |
| - **Not** whether the content is correct or complete: this is not the subject of this study. | |
| - **Not** simply length: a short text can still be hard to follow, and a longer one can be easy. | |
| - **Do not look anything up** (dictionary, search engine, AI assistant such as ChatGPT): read each text as it is, | |
| with what you already know. If a word or a sentence is unclear to you, that is exactly what we want to measure. | |
| ### There is no right answer | |
| We want your own experience as a reader. Once you submit a document you cannot go back to it. | |
| ### How the questions fit together | |
| *An illustration of the mechanics only: which text is easier is entirely up to you.* | |
| Suppose that, for some document, you find **C** the easiest to follow and **A** the hardest. B is then the middle | |
| one, and the two follow-up questions become: | |
| - "How much easier is **C** than **B**?" | |
| - "How much easier is **B** than **A**?" | |
| For each, pick *negligible*, *small*, *moderate* or *large*, from your own impression. | |
| """, | |
| "validity": f""" | |
| ### What you do | |
| For each document you see the **original** scientific text and three simplifications of it (A, B, C), written to | |
| help the general public (non-expert readers) understand it. | |
| The three simplifications come from different sources. We want to compare these sources and find out whether, and | |
| how often, they make mistakes when they simplify. Your answers tell us which simplifications have which kind of | |
| mistake. | |
| You can stop and come back at any time with your ID: your answers are saved after each document. Please take a break | |
| whenever you feel tired, and try to finish within a few days so your judgements stay consistent. | |
| For each question below, tick every simplification it applies to: none, one, two or all three. | |
| ### The questions | |
| #### 1. Leaves out important information | |
| The simplification omits critical information from the original. It is not misleading, but it leaves the reader | |
| unaware of key points: for example a part of the research question, a conclusion, or what the results mean in | |
| practice. Judge "important" from the point of view of a non-expert reader. This includes: | |
| - an **empty** simplification; | |
| - a simplification **so general** that it loses what the original found or explains (for example "Exercise is good"); | |
| - a simplification that covers **only one of two independent findings** of the original. | |
| Leaving out details (numbers, methods, secondary results) is normal in a simplification and is **not** an omission. | |
| #### 2. Contradicts the source | |
| The simplification says the opposite of the original, or states something the original does not contain: a made-up | |
| fact, a wrong number. | |
| #### 3. States a finding more broadly than the source supports | |
| The simplification generalises the original to the point where it becomes false: it drops *who* the finding applies | |
| to or *under what conditions* (for example "people" for "adults over 65 living at home"), or presents an uncertain | |
| result as established. | |
| Judge each simplification only against what the original says, not how easy it is to read. | |
| ### There may be no error at all | |
| Many texts may be fine on all three questions. Leaving every box unticked is a normal answer, not a missed one: tick | |
| a box only when you find a real problem. Once you submit a document you cannot go back to it. | |
| ### Example | |
| *Made up, not one of the study documents.* | |
| > **Original.** {EXAMPLE_SOURCE} | |
| {_texts(VALIDITY_TEXTS)} | |
| **A possible answer:** | |
| - *Leaves out important information*: **B**. The original has two conclusions, about falls and about broken bones; | |
| B keeps only the first, so the reader never learns that the effect on broken bones is unknown. | |
| - *Contradicts the source*: **C**. It says walking helps prevent broken bones; the original says this is not known. | |
| - *More broadly than supported*: **C**. "Older people" is broader than "adults over 65 living at home": the original | |
| says nothing about people aged 60, for instance, or living in care homes. | |
| - **A** has none of these problems: nothing to tick for it. | |
| """, | |
| } | |