Eric Xu commited on
Commit
74851ea
·
1 Parent(s): 3ad352e

Remove all hardcoded paths, make skill portable

Browse files

- SKILL.md: replace ~/Private_Projects/... with $SGO_DIR pattern,
add auto-detection of install location
- Scripts: data dir now defaults to <project>/data/nemotron/
instead of ~/Data/nvidia/...
- AGENT.md: use relative paths, fix domain-specific language
- Synced installed skill at ~/.claude/skills/sgo/

Files changed (4) hide show
  1. AGENT.md +2 -2
  2. SKILL.md +47 -32
  3. scripts/persona_loader.py +2 -1
  4. scripts/setup_data.py +4 -2
AGENT.md CHANGED
@@ -35,7 +35,7 @@ If `.env` doesn't exist, copy `.env.example` and ask the user to fill it in. Do
35
 
36
  ### Check data
37
 
38
- If `~/Data/nvidia/Nemotron-Personas-USA/dataset_info.json` exists, the persona dataset is ready. If not, run:
39
 
40
  ```bash
41
  uv run python scripts/setup_data.py
@@ -116,7 +116,7 @@ uv run python scripts/evaluate.py \
116
 
117
  **Present results to the user**:
118
 
119
- 1. Overall score distribution (avg, swipe-right %, swipe-left %)
120
  2. Breakdown by each stratification dimension
121
  3. Top 5 attractions (aggregated)
122
  4. Top 5 concerns (aggregated)
 
35
 
36
  ### Check data
37
 
38
+ If `data/nemotron/dataset_info.json` exists (relative to the project root), the persona dataset is ready. If not, run:
39
 
40
  ```bash
41
  uv run python scripts/setup_data.py
 
116
 
117
  **Present results to the user**:
118
 
119
+ 1. Overall score distribution (avg, positive %, negative %)
120
  2. Breakdown by each stratification dimension
121
  3. Top 5 attractions (aggregated)
122
  4. Top 5 concerns (aggregated)
SKILL.md CHANGED
@@ -8,53 +8,63 @@ argument-hint: "[entity-path or description]"
8
 
9
  You are executing the SGO pipeline. This optimizes an entity the user controls (θ) against a population of evaluators (x), using LLM-based scoring and counterfactual probes to estimate a semantic gradient.
10
 
11
- **Repo**: https://github.com/xuy/semantic-gradient-optimization
12
- **Local**: `~/Private_Projects/semantic-gradient-optimization/`
13
 
14
- Read the full framework: `~/Private_Projects/semantic-gradient-optimization/README.md`
15
- Read the agent guide: `~/Private_Projects/semantic-gradient-optimization/AGENT.md`
 
 
 
 
 
 
 
 
 
 
 
16
 
17
  ## Quick Reference
18
 
19
  ```
20
  Pipeline: Build Entity → Build Cohort → Score f(θ,x) → Probe Counterfactuals → Act & Re-evaluate
21
- Scripts: ~/Private_Projects/semantic-gradient-optimization/scripts/
22
- Templates: ~/Private_Projects/semantic-gradient-optimization/templates/
 
 
23
  ```
24
 
25
  ---
26
 
27
- ## Interactive Flow
28
 
29
- Follow the AGENT.md guide, but here's the condensed version:
 
 
30
 
31
- ### Phase 0 — Setup
32
-
33
- 1. Check `~/Private_Projects/semantic-gradient-optimization/.env` exists with LLM API keys
34
- 2. Check `~/Data/nvidia/Nemotron-Personas-USA/dataset_info.json` exists (if not, run `setup_data.py`)
35
- 3. Ensure deps installed: `cd ~/Private_Projects/semantic-gradient-optimization && uv sync`
36
 
37
- ### Phase 1 — Entity (θ)
38
 
39
  Ask the user: **"What are you optimizing?"**
40
 
41
  Options: product, resume, pitch, policy, profile, or custom.
42
 
43
- - If they have a document, save it to `entities/<name>.md`
44
- - If not, use a template from `templates/` and fill it in together
45
- - Confirm: "Here's what evaluators will see. Anything to add or remove?"
46
 
47
- ### Phase 2 — Cohort ({xᵢ})
48
 
49
- Ask: **"Who is your target audience?"**
50
 
51
- Then ask: **"What dimensions matter for segmentation?"**
52
 
53
  Two paths:
54
 
55
  **A) Nemotron dataset** (preferred — census-grounded, 1M US personas):
56
  ```bash
57
- cd ~/Private_Projects/semantic-gradient-optimization
58
 
59
  # Filter
60
  uv run python scripts/persona_loader.py \
@@ -68,7 +78,7 @@ uv run python scripts/stratified_sampler.py \
68
  --output data/cohort.json
69
  ```
70
 
71
- **B) LLM-generated** (fallback — warn about mode collapse):
72
  ```bash
73
  uv run python scripts/generate_cohort.py \
74
  --description "..." \
@@ -78,9 +88,12 @@ uv run python scripts/generate_cohort.py \
78
 
79
  Show the cohort distribution table. Confirm with user.
80
 
81
- ### Phase 3 — Evaluate
 
 
82
 
83
  ```bash
 
84
  uv run python scripts/evaluate.py \
85
  --entity entities/<name>.md \
86
  --cohort data/cohort.json \
@@ -88,17 +101,17 @@ uv run python scripts/evaluate.py \
88
  --parallel 5
89
  ```
90
 
91
- Present: avg score, breakdown by segment, top attractions, top concerns, dealbreakers, most/least receptive.
92
 
93
  Ask: **"Anything surprising? Want to dig into a segment?"**
94
 
95
- ### Phase 4 — Counterfactual Probe
 
 
96
 
97
  Ask: **"What changes are you considering?"** and **"What won't you change?"**
98
 
99
- If unsure, propose changes based on top concerns from Phase 3.
100
-
101
- Save changes to `data/changes.json` (see `templates/changes.json` for format).
102
 
103
  ```bash
104
  uv run python scripts/counterfactual.py \
@@ -111,10 +124,12 @@ Present the semantic gradient: ranked changes by avg Δ, % helped, % hurt, demog
111
 
112
  Ask: **"Which change do you want to make first?"**
113
 
114
- ### Phase 5 — Iterate
 
 
115
 
116
  1. User updates entity → save as `entities/<name>_v2.md`
117
- 2. Re-evaluate: `--tag <new_tag>`
118
  3. Compare: `uv run python scripts/compare.py --runs <old> <new>`
119
  4. Show delta, new attractions, resolved concerns
120
  5. Ask: **"Another round, or are we good?"**
@@ -124,9 +139,9 @@ Ask: **"Which change do you want to make first?"**
124
  ## Key Principles
125
 
126
  - **Cohort is the control group** — keep it fixed across runs
127
- - **Census-grounded > LLM-generated** personas (see README § The Seeding Problem)
128
  - **Only probe actionable changes** — don't waste tokens on things they can't or won't change
129
- - **The gradient is semantic** — it's natural language, not numbers. Present the reasoning, not just the deltas.
130
  - **Each run is tagged** — results are comparable longitudinally
131
 
132
  ## Arguments
 
8
 
9
  You are executing the SGO pipeline. This optimizes an entity the user controls (θ) against a population of evaluators (x), using LLM-based scoring and counterfactual probes to estimate a semantic gradient.
10
 
11
+ **Repo**: https://github.com/xuy/sgo
 
12
 
13
+ ## First: find the install location
14
+
15
+ The SGO repo could be installed at:
16
+ - `~/.claude/skills/sgo/` (if installed as a skill)
17
+ - Wherever the user cloned it
18
+
19
+ Run `ls ~/.claude/skills/sgo/scripts/evaluate.py 2>/dev/null` to check. If not found, ask the user where they cloned SGO, or offer to clone it:
20
+
21
+ ```bash
22
+ git clone https://github.com/xuy/sgo.git ~/.claude/skills/sgo
23
+ ```
24
+
25
+ For the rest of this file, `$SGO_DIR` refers to the SGO install location. All paths are relative to it.
26
 
27
  ## Quick Reference
28
 
29
  ```
30
  Pipeline: Build Entity → Build Cohort → Score f(θ,x) → Probe Counterfactuals → Act & Re-evaluate
31
+ Scripts: $SGO_DIR/scripts/
32
+ Templates: $SGO_DIR/templates/
33
+ Data: $SGO_DIR/data/
34
+ Results: $SGO_DIR/results/
35
  ```
36
 
37
  ---
38
 
39
+ ## Phase 0 — Setup
40
 
41
+ 1. **Dependencies**: `cd $SGO_DIR && uv sync` (or `pip install datasets huggingface_hub openai python-dotenv`)
42
+ 2. **API key**: Check `$SGO_DIR/.env` exists. If not, `cp $SGO_DIR/.env.example $SGO_DIR/.env` and ask the user to fill in their LLM API key. Do NOT read `.env` — just confirm it's configured.
43
+ 3. **Persona data**: Run `uv run python $SGO_DIR/scripts/setup_data.py`. This downloads NVIDIA Nemotron-Personas-USA (~2GB) to `$SGO_DIR/data/nemotron/` on first run and skips if already cached.
44
 
45
+ ---
 
 
 
 
46
 
47
+ ## Phase 1 — Entity (θ)
48
 
49
  Ask the user: **"What are you optimizing?"**
50
 
51
  Options: product, resume, pitch, policy, profile, or custom.
52
 
53
+ - If they have a document, save it to `$SGO_DIR/entities/<name>.md`
54
+ - If not, use a template from `$SGO_DIR/templates/` and fill it in together
55
+ - Confirm: *"Here's what evaluators will see. Anything to add or remove?"*
56
 
57
+ ---
58
 
59
+ ## Phase 2 Cohort ({xᵢ})
60
 
61
+ Ask: **"Who is your target audience?"** and **"What dimensions matter for segmentation?"**
62
 
63
  Two paths:
64
 
65
  **A) Nemotron dataset** (preferred — census-grounded, 1M US personas):
66
  ```bash
67
+ cd $SGO_DIR
68
 
69
  # Filter
70
  uv run python scripts/persona_loader.py \
 
78
  --output data/cohort.json
79
  ```
80
 
81
+ **B) LLM-generated** (fallback — warn user about mode collapse risk):
82
  ```bash
83
  uv run python scripts/generate_cohort.py \
84
  --description "..." \
 
88
 
89
  Show the cohort distribution table. Confirm with user.
90
 
91
+ ---
92
+
93
+ ## Phase 3 — Evaluate
94
 
95
  ```bash
96
+ cd $SGO_DIR
97
  uv run python scripts/evaluate.py \
98
  --entity entities/<name>.md \
99
  --cohort data/cohort.json \
 
101
  --parallel 5
102
  ```
103
 
104
+ Present: avg score, breakdown by segment, top attractions, top concerns, dealbreakers, most/least receptive evaluators with quotes.
105
 
106
  Ask: **"Anything surprising? Want to dig into a segment?"**
107
 
108
+ ---
109
+
110
+ ## Phase 4 — Counterfactual Probe
111
 
112
  Ask: **"What changes are you considering?"** and **"What won't you change?"**
113
 
114
+ If unsure, propose changes based on top concerns from Phase 3. Save to `data/changes.json` (see `templates/changes.json` for format).
 
 
115
 
116
  ```bash
117
  uv run python scripts/counterfactual.py \
 
124
 
125
  Ask: **"Which change do you want to make first?"**
126
 
127
+ ---
128
+
129
+ ## Phase 5 — Iterate
130
 
131
  1. User updates entity → save as `entities/<name>_v2.md`
132
+ 2. Re-evaluate with same cohort: `--tag <new_tag>`
133
  3. Compare: `uv run python scripts/compare.py --runs <old> <new>`
134
  4. Show delta, new attractions, resolved concerns
135
  5. Ask: **"Another round, or are we good?"**
 
139
  ## Key Principles
140
 
141
  - **Cohort is the control group** — keep it fixed across runs
142
+ - **Census-grounded > LLM-generated** personas (see README)
143
  - **Only probe actionable changes** — don't waste tokens on things they can't or won't change
144
+ - **The gradient is semantic** — present the reasoning, not just the deltas
145
  - **Each run is tagged** — results are comparable longitudinally
146
 
147
  ## Arguments
scripts/persona_loader.py CHANGED
@@ -21,7 +21,8 @@ import argparse
21
  from pathlib import Path
22
  from datasets import load_from_disk
23
 
24
- DEFAULT_DATA_DIR = Path.home() / "Data" / "nvidia" / "Nemotron-Personas-USA"
 
25
 
26
  MBTI_TYPES = [
27
  "INTJ", "INTP", "ENTJ", "ENTP", "INFJ", "INFP", "ENFJ", "ENFP",
 
21
  from pathlib import Path
22
  from datasets import load_from_disk
23
 
24
+ PROJECT_ROOT = Path(__file__).resolve().parent.parent
25
+ DEFAULT_DATA_DIR = PROJECT_ROOT / "data" / "nemotron"
26
 
27
  MBTI_TYPES = [
28
  "INTJ", "INTP", "ENTJ", "ENTP", "INFJ", "INFP", "ENFJ", "ENFP",
scripts/setup_data.py CHANGED
@@ -1,7 +1,8 @@
1
  """
2
  Download and cache the Nemotron-Personas-USA dataset.
3
 
4
- Downloads 1M synthetic US personas (~2GB) from HuggingFace to ~/Data/nvidia/Nemotron-Personas-USA/.
 
5
  Only runs once — subsequent calls detect the cached dataset and skip.
6
 
7
  Usage:
@@ -13,7 +14,8 @@ import argparse
13
  from pathlib import Path
14
  from datasets import load_dataset, load_from_disk
15
 
16
- DEFAULT_DATA_DIR = Path.home() / "Data" / "nvidia" / "Nemotron-Personas-USA"
 
17
 
18
 
19
  def setup(data_dir: Path = DEFAULT_DATA_DIR):
 
1
  """
2
  Download and cache the Nemotron-Personas-USA dataset.
3
 
4
+ Downloads 1M synthetic US personas (~2GB) from HuggingFace.
5
+ Default location: <project_root>/data/nemotron/
6
  Only runs once — subsequent calls detect the cached dataset and skip.
7
 
8
  Usage:
 
14
  from pathlib import Path
15
  from datasets import load_dataset, load_from_disk
16
 
17
+ PROJECT_ROOT = Path(__file__).resolve().parent.parent
18
+ DEFAULT_DATA_DIR = PROJECT_ROOT / "data" / "nemotron"
19
 
20
 
21
  def setup(data_dir: Path = DEFAULT_DATA_DIR):