README.md

---
title: PixelPilotAI
emoji: 📷
colorFrom: purple
colorTo: blue
sdk: docker
pinned: false
---

# Photo Editing Recommendation Agent

Recommend global photo edits by retrieving similar expert-edited examples (MIT–Adobe FiveK), aggregating Expert A recipes, and applying them deterministically. This repo is structured so **dataset → embeddings → vector DB** and the **inference API + LLM** can be developed in parallel and merged cleanly.

## Project layout (merge-ready)

```
PhotoEditor/
├── .env                    # Copy from .env.example; set FIVEK_SUBSET_SIZE, Azure Search, etc.
├── .env.example
├── requirements.txt
├── photo_editor/            # Core package (shared by pipeline and future API)
│   ├── config/              # Settings from env (paths, Azure, subset size)
│   ├── dataset/             # FiveK paths, subset selection (filesAdobe.txt)
│   ├── lrcat/               # Lightroom catalog: Expert A recipe extraction
│   ├── images/              # DNG → RGB (rawpy, neutral development)
│   ├── embeddings/          # CLIP image embeddings (index + query)
│   └── vector_store/        # Azure AI Search index (upload + search)
├── scripts/
│   └── build_vector_index.py   # Build vector index: subset → embed → push to Azure
├── fivek_dataset/           # MIT–Adobe FiveK (file lists, raw_photos/, fivek.lrcat)
├── LLM.py                   # Existing Azure GPT-4o explanation layer (to be wired to RAG)
└── api/                     # (Future) FastAPI: /analyze-image, /apply-edits, /edit-and-explain
```

- **Inference merge**: The API will use `photo_editor.vector_store.AzureSearchVectorStore` for retrieval, `photo_editor.embeddings` for query embedding, and `LLM.py` (or a moved `photo_editor.llm`) for explanations. Apply-edits will use a separate editing engine (OpenCV/Pillow) consuming `EditRecipe` from `photo_editor.lrcat.schema`.

## Dataset → Vector DB (this slice)

1. **Subset**: First `FIVEK_SUBSET_SIZE` images from `fivek_dataset/filesAdobe.txt` (default 500; set in `.env`).
2. **Edits**: Expert A only; recipes read from `fivek.lrcat` (virtual copy "Copy 1").
3. **Embeddings**: Original DNG → neutral development → RGB → CLIP (`openai/clip-vit-base-patch32`).
4. **Vector DB**: Azure AI Search index (created if missing); each document = `id`, `image_id`, `embedding`, `recipe` (JSON).

### Setup

```bash
cp .env.example .env
# Edit .env: FIVEK_SUBSET_SIZE (e.g. 500), AZURE_SEARCH_*, optional paths
pip install -r requirements.txt
```

### Build the index

From the project root:

```bash
PYTHONPATH=. python scripts/build_vector_index.py
```

- Requires the FiveK `raw_photos` folder (DNGs + `fivek.lrcat`) under `fivek_dataset/`.
- If Azure Search is not configured in `.env`, the script still runs and skips upload (prints a reminder).

## How to run things

All commands below assume you are in the **project root** (`PhotoEditor/`) and have:

- created and edited `.env` (see config table below), and
- installed dependencies:

```bash
pip install -r requirements.txt
```

## Deploy (Streamlit Cloud + Hugging Face Spaces)

For cloud deploy, keep the repo minimal and include only runtime files:

- `app.py`
- `photo_editor/`
- `requirements.txt`
- `.streamlit/config.toml`
- `.env.example` (template only, no secrets)

Do not commit local artifacts or large datasets (`fivek_dataset/`, `renders/`, generated images/html/json, `.env`).

### Streamlit Community Cloud

1. Push this repo to GitHub.
2. In Streamlit Cloud, create a new app from the repo.
3. Set the app file path to `app.py`.
4. Add required secrets in the app settings (same keys as in `.env.example`, e.g. `AZURE_SEARCH_*`, `AZURE_OPENAI_*`).
5. Deploy.

### Hugging Face Spaces (Streamlit SDK)

1. Create a new Space and choose **Streamlit** SDK.
2. Point it to this repository (or push these files to the Space repo).
3. Ensure `app.py` is at repo root and `requirements.txt` is present.
4. Add secrets in Space Settings (same variables as `.env.example`).
5. Launch the Space.

Optional automation: sync supported secrets from local `.env` directly to your Space:

```bash
pip install huggingface_hub
HF_TOKEN=hf_xxx python scripts/sync_hf_secrets.py --space-id <username/space-name>
```

### Hugging Face Spaces (Docker SDK)

This repo now includes a production-ready `Dockerfile` that serves the app on port `7860`.

1. Create a new Space and choose **Docker** SDK.
2. Push this repository to that Space.
3. In Space Settings, add secrets (or sync them later with `scripts/sync_hf_secrets.py`).
4. Build and launch the Space.

Local Docker test:

```bash
docker build -t lumigrade-ai .
docker run --rm -p 7860:7860 --env-file .env lumigrade-ai
```

### 1. Run the Streamlit UI (full app)

Interactive app to upload an image (JPEG/PNG) or point to a DNG on disk, then run the full pipeline and see **original vs edited** plus the suggested edit parameters.

```bash
streamlit run app.py
```

This will:
- Check Azure Search + Azure OpenAI config from `.env`.
- For each run: retrieve similar experts → call LLM for summary + suggested edits → apply edits (locally or via external API) → show before/after.

### 2. Run the full pipeline from the terminal

Run the same pipeline as the UI, but from the CLI for a single image:

```bash
python scripts/run_pipeline.py <image_path> [--out output.jpg] [--api] [-v]
```

Examples:

```bash
# Run pipeline locally, save to result.jpg, print summary + suggested edits
python scripts/run_pipeline.py photo.jpg --out result.jpg -v

# Run pipeline but use an external editing API (requires EDITING_API_URL in .env)
python scripts/run_pipeline.py photo.jpg --out result.jpg --api -v
```

What `-v` prints:
- 📋 **Summary** of what the LLM thinks should be done.
- 📐 **Suggested edits**: the numeric recipe (exposure, contrast, temp, etc.) coming from Azure OpenAI for that image.
- 📎 **Expert used**: which FiveK expert image/recipe was used as reference.

### 3. Just retrieve similar experts (no LLM / no edits)

If you only want to see which FiveK images are closest to a given photo and inspect their stored recipes:

```bash
python scripts/query_similar.py <image_path> [--top-k 50] [--top-n 5]
```

Examples:

```bash
# Show the best 5 expert matches (default top-k=50 search space)
python scripts/query_similar.py photo.jpg --top-n 5

# Show only the single best match
python scripts/query_similar.py photo.jpg --top-n 1
```

Output:
- Ranks (`1.`, `2.`, …), image_ids, rerank scores.
- The stored **Expert A recipe** JSON for each match.

### 4. Get the exact Expert A recipe for a FiveK image

Given a FiveK `image_id` (with or without extension), extract the Expert A recipe directly from the Lightroom catalog:

```bash
python scripts/get_recipe_for_image.py <image_name> [-o recipe.json]
```

Examples:

```bash
# Print the recipe as JSON
python scripts/get_recipe_for_image.py a0001-jmac_DSC1459

# Save the recipe to a file
python scripts/get_recipe_for_image.py a0001-jmac_DSC1459 -o my_recipe.json
```

### 5. Apply a custom (LLM) recipe to a FiveK image

If you already have a JSON recipe (for example, something you crafted or got from the LLM) and want to apply it to a FiveK RAW image using the same rendering pipeline:

```bash
python scripts/apply_llm_recipe.py <image_id> <recipe.json> [--out path.jpg]
```

Example:

```bash
python scripts/apply_llm_recipe.py a0059-JI2E5556 llm_recipe_a0059.json --out renders/a0059-JI2E5556_LLM.jpg
```

This will:
- Load the DNG for `<image_id>`.
- Use `dng_to_rgb_normalized` to bake in exposure/brightness from the recipe.
- Apply the rest of the recipe (contrast, temperature, etc.) on top of the original Expert A baseline.
- Save the rendered JPEG.

## Config (.env)

| Variable | Description |
|--------|-------------|
| `FIVEK_SUBSET_SIZE` | Number of images to index (default 500). |
| `FIVEK_LRCAT_PATH` | Path to `fivek.lrcat` (default: `fivek_dataset/raw_photos/fivek.lrcat`). |
| `FIVEK_RAW_PHOTOS_DIR` | Root of range folders (e.g. `HQa1to700`, …). |
| `AZURE_SEARCH_ENDPOINT` | Azure AI Search endpoint URL. |
| `AZURE_SEARCH_KEY` | Azure AI Search admin key. |
| `AZURE_SEARCH_INDEX_NAME` | Index name (default `fivek-vectors`). |

## License / data

See `fivek_dataset/LICENSE.txt` and related notices for the MIT–Adobe FiveK dataset.