# Development pilot dataset

The E00 development pilot uses the official
[GitLab Runner repository](https://gitlab.com/gitlab-org/gitlab-runner/), a Go
project hosted on GitLab. This dataset is for pipeline debugging and pilot
effect-size estimation only; it is not a confirmatory benchmark split.

## Pinned checkout

| Field | Value |
|---|---|
| Repository | `https://gitlab.com/gitlab-org/gitlab-runner.git` |
| Release | `v19.0.1` |
| Commit | `c2831b75a3ff0782dca8f64498cbc6f71c76819e` |
| Tracked files | 1,229 |
| Readable tracked bytes | approximately 13.35 MB |
| Go/Proto files | 845 |
| Go/Proto lines | 211,912 |
| Go/Proto bytes | 6,153,499 |

The model is loaded with a 262,144-token context. The Go/Proto subset alone
would fit only if the tokenizer averaged more than 23 bytes per token. This is
a conservative size sanity check, not the final tokenizer measurement. Before
confirmatory evaluation, repository size must be measured with the exact local
Qwen tokenizer file and the measurement script/version must be recorded.

## Task construction

Five real fixes are converted into localization tasks. For every task, the
searchable snapshot is the parent of the fix commit. The visible statement is
an issue-style description that does not name the gold file or symbol. Changed
production locations that already exist at the parent commit form the hidden
retrieval gold.

| Task | Gold commit | Topic | Difficulty |
|---|---|---|---|
| TASK_GR_001 | `991cf3c0` | proactive authentication during checkout lazy fetch | medium, two files |
| TASK_GR_002 | `fd33f65d` | proxy-exec inherited-pipe hang | medium, one file |
| TASK_GR_003 | `9968f744` | adaptive concurrency under network retries | hard, cross-module |
| TASK_GR_004 | `6cd0dce8` | S3 checksum defaults for custom endpoints | medium, domain terminology |
| TASK_GR_005 | `8c5b8c70` | autoscaler context-cause classification | medium, semantic |

These tasks have `validation_status = "retrieval_ready"`. They must not be used
for end-to-end repair claims until hidden regression tests are extracted,
verified to fail on the base commit, verified to pass on the gold commit, and
checked for flakiness.

## E00 treatments

- H000: deterministic literal occurrence ranking over issue-derived terms.
- H001: in-memory BM25 with a bounded fuzzy path-token bonus.
- H003: Qwen3 Embedding 0.6B with exact flat cosine ranking.

All treatments use the same 120-line chunks, 20-line overlap, and 16,000-character
ceiling in this pilot. This common chunking is a deliberate control. E00 has 15
cells: five tasks by three harnesses.