Spaces:
Running
Running
likelytobelaura Claude Sonnet 5 commited on
Commit ·
1dc9935
1
Parent(s): 037cb1b
Rewrite CONTRIBUTING.md with a verified, step-by-step git workflow
Browse filesThe old doc told contributors to "open a pull request" without saying
how, and offered "Duplicate this Space" as a fork option that actually
creates an unrelated repo with no PR relationship back to this one.
Replaces both with steps that mirror the Station House task page:
install the hf CLI, log in, clone, edit examples.jsonl, validate, then
submit with `hf upload --create-pr` since a plain git push can't open
a new PR without write access. Verified against a real install of the
hf CLI (--create-pr flag and positional args match exactly).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- CONTRIBUTING.md +43 -14
CONTRIBUTING.md
CHANGED
|
@@ -4,37 +4,66 @@
|
|
| 4 |
scoring rubric, and [`schema.json`](./schema.json) for the strict machine-readable
|
| 5 |
version.
|
| 6 |
|
| 7 |
-
2. **
|
| 8 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 9 |
|
| 10 |
```bash
|
| 11 |
git clone https://huggingface.co/spaces/Station-house/ConspiracyBench
|
| 12 |
cd ConspiracyBench
|
| 13 |
```
|
| 14 |
|
| 15 |
-
|
| 16 |
-
|
|
|
|
| 17 |
|
| 18 |
```bash
|
| 19 |
-
|
| 20 |
-
# edit /tmp/my-question.jsonl, then append it:
|
| 21 |
-
cat /tmp/my-question.jsonl >> questions/examples.jsonl
|
| 22 |
```
|
| 23 |
|
| 24 |
-
|
| 25 |
-
(JSONL), no
|
|
|
|
| 26 |
|
| 27 |
-
|
| 28 |
|
| 29 |
```bash
|
| 30 |
pip install jsonschema
|
| 31 |
python scripts/validate_questions.py
|
| 32 |
```
|
| 33 |
|
| 34 |
-
|
| 35 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
- the question and why it's a good conspiracy-theory probe,
|
| 37 |
- the source backing your `ground_truth_answer`.
|
| 38 |
|
| 39 |
-
A maintainer will review for factual accuracy of the ground truth before merging
|
| 40 |
-
|
|
|
|
| 4 |
scoring rubric, and [`schema.json`](./schema.json) for the strict machine-readable
|
| 5 |
version.
|
| 6 |
|
| 7 |
+
2. **Install the tools.** You need git and the Hugging Face CLI.
|
| 8 |
+
|
| 9 |
+
```bash
|
| 10 |
+
pip install -U huggingface_hub
|
| 11 |
+
```
|
| 12 |
+
|
| 13 |
+
3. **Log in to Hugging Face.**
|
| 14 |
+
|
| 15 |
+
```bash
|
| 16 |
+
hf auth login
|
| 17 |
+
```
|
| 18 |
+
|
| 19 |
+
It will ask for an access token. Create one at
|
| 20 |
+
[huggingface.co/settings/tokens](https://huggingface.co/settings/tokens). The default
|
| 21 |
+
token works.
|
| 22 |
+
|
| 23 |
+
4. **Clone the repository.** No fork needed. Contributions go directly to this repo as a
|
| 24 |
+
pull request.
|
| 25 |
|
| 26 |
```bash
|
| 27 |
git clone https://huggingface.co/spaces/Station-house/ConspiracyBench
|
| 28 |
cd ConspiracyBench
|
| 29 |
```
|
| 30 |
|
| 31 |
+
5. **Add your question.** Open [`questions/examples.jsonl`](./questions/examples.jsonl)
|
| 32 |
+
in a text editor. Each line is one question. Add one new line at the end, starting
|
| 33 |
+
from [`questions/template.jsonl`](./questions/template.jsonl):
|
| 34 |
|
| 35 |
```bash
|
| 36 |
+
cat questions/template.jsonl
|
|
|
|
|
|
|
| 37 |
```
|
| 38 |
|
| 39 |
+
Fill in each field. See [`SCHEMA.md`](./SCHEMA.md) for what each one means. Give your
|
| 40 |
+
item a fresh `id`, the next unused `cb-XXXX`. One JSON object per line (JSONL), no
|
| 41 |
+
trailing commas.
|
| 42 |
|
| 43 |
+
6. **Validate.**
|
| 44 |
|
| 45 |
```bash
|
| 46 |
pip install jsonschema
|
| 47 |
python scripts/validate_questions.py
|
| 48 |
```
|
| 49 |
|
| 50 |
+
Fix any errors it reports. Run it again until it passes.
|
| 51 |
+
|
| 52 |
+
7. **Submit it as a pull request.** You do not have write access to this repo. A plain
|
| 53 |
+
`git push` will not open a pull request. Use `hf upload` with `--create-pr` instead.
|
| 54 |
+
It uploads your file and opens the pull request in one step.
|
| 55 |
+
|
| 56 |
+
```bash
|
| 57 |
+
hf upload Station-house/ConspiracyBench questions/examples.jsonl questions/examples.jsonl \
|
| 58 |
+
--repo-type space --create-pr --commit-message "Add question: <short title>"
|
| 59 |
+
```
|
| 60 |
+
|
| 61 |
+
Replace `<short title>` with a few words about your question.
|
| 62 |
+
|
| 63 |
+
8. **Finish the pull request.** The command prints a link. Open it and add a
|
| 64 |
+
description that includes:
|
| 65 |
- the question and why it's a good conspiracy-theory probe,
|
| 66 |
- the source backing your `ground_truth_answer`.
|
| 67 |
|
| 68 |
+
A maintainer will review for factual accuracy of the ground truth before merging.
|
| 69 |
+
This is the part that matters most, since the whole benchmark's grading depends on it.
|