likelytobelaura Claude Sonnet 5 commited on
Commit
1dc9935
·
1 Parent(s): 037cb1b

Rewrite CONTRIBUTING.md with a verified, step-by-step git workflow

Browse files

The old doc told contributors to "open a pull request" without saying
how, and offered "Duplicate this Space" as a fork option that actually
creates an unrelated repo with no PR relationship back to this one.

Replaces both with steps that mirror the Station House task page:
install the hf CLI, log in, clone, edit examples.jsonl, validate, then
submit with `hf upload --create-pr` since a plain git push can't open
a new PR without write access. Verified against a real install of the
hf CLI (--create-pr flag and positional args match exactly).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Files changed (1) hide show
  1. CONTRIBUTING.md +43 -14
CONTRIBUTING.md CHANGED
@@ -4,37 +4,66 @@
4
  scoring rubric, and [`schema.json`](./schema.json) for the strict machine-readable
5
  version.
6
 
7
- 2. **Fork this Space.** On the ConspiracyBench page, use the "⋮" menu → *Duplicate this
8
- Space*, or clone it directly:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9
 
10
  ```bash
11
  git clone https://huggingface.co/spaces/Station-house/ConspiracyBench
12
  cd ConspiracyBench
13
  ```
14
 
15
- 3. **Copy the template** and fill in one line per question in
16
- [`questions/template.jsonl`](./questions/template.jsonl):
 
17
 
18
  ```bash
19
- cp questions/template.jsonl /tmp/my-question.jsonl
20
- # edit /tmp/my-question.jsonl, then append it:
21
- cat /tmp/my-question.jsonl >> questions/examples.jsonl
22
  ```
23
 
24
- Give your item a fresh `id` (next unused `cb-XXXX`). One JSON object per line
25
- (JSONL), no trailing commas.
 
26
 
27
- 4. **Validate:**
28
 
29
  ```bash
30
  pip install jsonschema
31
  python scripts/validate_questions.py
32
  ```
33
 
34
- 5. **Open a pull request** against `Station-house/ConspiracyBench` with your change.
35
- Include in the PR description:
 
 
 
 
 
 
 
 
 
 
 
 
 
36
  - the question and why it's a good conspiracy-theory probe,
37
  - the source backing your `ground_truth_answer`.
38
 
39
- A maintainer will review for factual accuracy of the ground truth before merging —
40
- this is the part that matters most, since the whole benchmark's grading depends on it.
 
4
  scoring rubric, and [`schema.json`](./schema.json) for the strict machine-readable
5
  version.
6
 
7
+ 2. **Install the tools.** You need git and the Hugging Face CLI.
8
+
9
+ ```bash
10
+ pip install -U huggingface_hub
11
+ ```
12
+
13
+ 3. **Log in to Hugging Face.**
14
+
15
+ ```bash
16
+ hf auth login
17
+ ```
18
+
19
+ It will ask for an access token. Create one at
20
+ [huggingface.co/settings/tokens](https://huggingface.co/settings/tokens). The default
21
+ token works.
22
+
23
+ 4. **Clone the repository.** No fork needed. Contributions go directly to this repo as a
24
+ pull request.
25
 
26
  ```bash
27
  git clone https://huggingface.co/spaces/Station-house/ConspiracyBench
28
  cd ConspiracyBench
29
  ```
30
 
31
+ 5. **Add your question.** Open [`questions/examples.jsonl`](./questions/examples.jsonl)
32
+ in a text editor. Each line is one question. Add one new line at the end, starting
33
+ from [`questions/template.jsonl`](./questions/template.jsonl):
34
 
35
  ```bash
36
+ cat questions/template.jsonl
 
 
37
  ```
38
 
39
+ Fill in each field. See [`SCHEMA.md`](./SCHEMA.md) for what each one means. Give your
40
+ item a fresh `id`, the next unused `cb-XXXX`. One JSON object per line (JSONL), no
41
+ trailing commas.
42
 
43
+ 6. **Validate.**
44
 
45
  ```bash
46
  pip install jsonschema
47
  python scripts/validate_questions.py
48
  ```
49
 
50
+ Fix any errors it reports. Run it again until it passes.
51
+
52
+ 7. **Submit it as a pull request.** You do not have write access to this repo. A plain
53
+ `git push` will not open a pull request. Use `hf upload` with `--create-pr` instead.
54
+ It uploads your file and opens the pull request in one step.
55
+
56
+ ```bash
57
+ hf upload Station-house/ConspiracyBench questions/examples.jsonl questions/examples.jsonl \
58
+ --repo-type space --create-pr --commit-message "Add question: <short title>"
59
+ ```
60
+
61
+ Replace `<short title>` with a few words about your question.
62
+
63
+ 8. **Finish the pull request.** The command prints a link. Open it and add a
64
+ description that includes:
65
  - the question and why it's a good conspiracy-theory probe,
66
  - the source backing your `ground_truth_answer`.
67
 
68
+ A maintainer will review for factual accuracy of the ground truth before merging.
69
+ This is the part that matters most, since the whole benchmark's grading depends on it.