README.md

metadata

title: PixelPilotAI
emoji: 📷
colorFrom: purple
colorTo: blue
sdk: docker
pinned: false

Photo Editing Recommendation Agent

Recommend global photo edits by retrieving similar expert-edited examples (MIT–Adobe FiveK), aggregating Expert A recipes, and applying them deterministically. This repo is structured so dataset → embeddings → vector DB and the inference API + LLM can be developed in parallel and merged cleanly.

Project layout (merge-ready)

PhotoEditor/
├── .env                    # Copy from .env.example; set FIVEK_SUBSET_SIZE, Azure Search, etc.
├── .env.example
├── requirements.txt
├── photo_editor/            # Core package (shared by pipeline and future API)
│   ├── config/              # Settings from env (paths, Azure, subset size)
│   ├── dataset/             # FiveK paths, subset selection (filesAdobe.txt)
│   ├── lrcat/               # Lightroom catalog: Expert A recipe extraction
│   ├── images/              # DNG → RGB (rawpy, neutral development)
│   ├── embeddings/          # CLIP image embeddings (index + query)
│   └── vector_store/        # Azure AI Search index (upload + search)
├── scripts/
│   └── build_vector_index.py   # Build vector index: subset → embed → push to Azure
├── fivek_dataset/           # MIT–Adobe FiveK (file lists, raw_photos/, fivek.lrcat)
├── LLM.py                   # Existing Azure GPT-4o explanation layer (to be wired to RAG)
└── api/                     # (Future) FastAPI: /analyze-image, /apply-edits, /edit-and-explain

• Inference merge: The API will use photo_editor.vector_store.AzureSearchVectorStore for retrieval, photo_editor.embeddings for query embedding, and LLM.py (or a moved photo_editor.llm) for explanations. Apply-edits will use a separate editing engine (OpenCV/Pillow) consuming EditRecipe from photo_editor.lrcat.schema.

Dataset → Vector DB (this slice)

1. Subset: First FIVEK_SUBSET_SIZE images from fivek_dataset/filesAdobe.txt (default 500; set in .env).
2. Edits: Expert A only; recipes read from fivek.lrcat (virtual copy "Copy 1").
3. Embeddings: Original DNG → neutral development → RGB → CLIP (openai/clip-vit-base-patch32).
4. Vector DB: Azure AI Search index (created if missing); each document = id, image_id, embedding, recipe (JSON).

Setup

cp .env.example .env
# Edit .env: FIVEK_SUBSET_SIZE (e.g. 500), AZURE_SEARCH_*, optional paths
pip install -r requirements.txt

Build the index

From the project root:

PYTHONPATH=. python scripts/build_vector_index.py

• Requires the FiveK raw_photos folder (DNGs + fivek.lrcat) under fivek_dataset/.
• If Azure Search is not configured in .env, the script still runs and skips upload (prints a reminder).

How to run things

All commands below assume you are in the project root (PhotoEditor/) and have:

• created and edited .env (see config table below), and
• installed dependencies:

pip install -r requirements.txt

Deploy (Streamlit Cloud + Hugging Face Spaces)

For cloud deploy, keep the repo minimal and include only runtime files:

• app.py
• photo_editor/
• requirements.txt
• .streamlit/config.toml
• .env.example (template only, no secrets)

Do not commit local artifacts or large datasets (fivek_dataset/, renders/, generated images/html/json, .env).

Streamlit Community Cloud

1. Push this repo to GitHub.
2. In Streamlit Cloud, create a new app from the repo.
3. Set the app file path to app.py.
4. Add required secrets in the app settings (same keys as in .env.example, e.g. AZURE_SEARCH_*, AZURE_OPENAI_*).
5. Deploy.

Hugging Face Spaces (Streamlit SDK)

1. Create a new Space and choose Streamlit SDK.
2. Point it to this repository (or push these files to the Space repo).
3. Ensure app.py is at repo root and requirements.txt is present.
4. Add secrets in Space Settings (same variables as .env.example).
5. Launch the Space.

Optional automation: sync supported secrets from local .env directly to your Space:

pip install huggingface_hub
HF_TOKEN=hf_xxx python scripts/sync_hf_secrets.py --space-id <username/space-name>

Hugging Face Spaces (Docker SDK)

This repo now includes a production-ready Dockerfile that serves the app on port 7860.

1. Create a new Space and choose Docker SDK.
2. Push this repository to that Space.
3. In Space Settings, add secrets (or sync them later with scripts/sync_hf_secrets.py).
4. Build and launch the Space.

Local Docker test:

docker build -t lumigrade-ai .
docker run --rm -p 7860:7860 --env-file .env lumigrade-ai

1. Run the Streamlit UI (full app)

Interactive app to upload an image (JPEG/PNG) or point to a DNG on disk, then run the full pipeline and see original vs edited plus the suggested edit parameters.

streamlit run app.py

This will:

• Check Azure Search + Azure OpenAI config from .env.
• For each run: retrieve similar experts → call LLM for summary + suggested edits → apply edits (locally or via external API) → show before/after.

2. Run the full pipeline from the terminal

Run the same pipeline as the UI, but from the CLI for a single image:

python scripts/run_pipeline.py <image_path> [--out output.jpg] [--api] [-v]

Examples:

# Run pipeline locally, save to result.jpg, print summary + suggested edits
python scripts/run_pipeline.py photo.jpg --out result.jpg -v

# Run pipeline but use an external editing API (requires EDITING_API_URL in .env)
python scripts/run_pipeline.py photo.jpg --out result.jpg --api -v

What -v prints:

• 📋 Summary of what the LLM thinks should be done.
• 📐 Suggested edits: the numeric recipe (exposure, contrast, temp, etc.) coming from Azure OpenAI for that image.
• 📎 Expert used: which FiveK expert image/recipe was used as reference.

3. Just retrieve similar experts (no LLM / no edits)

If you only want to see which FiveK images are closest to a given photo and inspect their stored recipes:

python scripts/query_similar.py <image_path> [--top-k 50] [--top-n 5]

Examples:

# Show the best 5 expert matches (default top-k=50 search space)
python scripts/query_similar.py photo.jpg --top-n 5

# Show only the single best match
python scripts/query_similar.py photo.jpg --top-n 1

Output:

• Ranks (1., 2., …), image_ids, rerank scores.
• The stored Expert A recipe JSON for each match.

4. Get the exact Expert A recipe for a FiveK image

Given a FiveK image_id (with or without extension), extract the Expert A recipe directly from the Lightroom catalog:

python scripts/get_recipe_for_image.py <image_name> [-o recipe.json]

Examples:

# Print the recipe as JSON
python scripts/get_recipe_for_image.py a0001-jmac_DSC1459

# Save the recipe to a file
python scripts/get_recipe_for_image.py a0001-jmac_DSC1459 -o my_recipe.json

5. Apply a custom (LLM) recipe to a FiveK image

If you already have a JSON recipe (for example, something you crafted or got from the LLM) and want to apply it to a FiveK RAW image using the same rendering pipeline:

python scripts/apply_llm_recipe.py <image_id> <recipe.json> [--out path.jpg]

Example:

python scripts/apply_llm_recipe.py a0059-JI2E5556 llm_recipe_a0059.json --out renders/a0059-JI2E5556_LLM.jpg

This will:

• Load the DNG for <image_id>.
• Use dng_to_rgb_normalized to bake in exposure/brightness from the recipe.
• Apply the rest of the recipe (contrast, temperature, etc.) on top of the original Expert A baseline.
• Save the rendered JPEG.

Config (.env)

Variable	Description
FIVEK_SUBSET_SIZE	Number of images to index (default 500).
FIVEK_LRCAT_PATH	Path to fivek.lrcat (default: fivek_dataset/raw_photos/fivek.lrcat).
FIVEK_RAW_PHOTOS_DIR	Root of range folders (e.g. HQa1to700, …).
AZURE_SEARCH_ENDPOINT	Azure AI Search endpoint URL.
AZURE_SEARCH_KEY	Azure AI Search admin key.
AZURE_SEARCH_INDEX_NAME	Index name (default fivek-vectors).

License / data

See fivek_dataset/LICENSE.txt and related notices for the MIT–Adobe FiveK dataset.