README.md

---
title: Carbon Dioxide Removal Toolkit
emoji: 🌍
colorFrom: green
colorTo: blue
sdk: streamlit
sdk_version: "1.58.0"
app_file: app.py
pinned: false
---

# Carbon Dioxide Removal Optimizer

A Streamlit app for optimizing carbon removal portfolios across different methods and constraints.

---

## Table of contents

- [Data Maintenance Guide](#data-maintenance-guide)
  - [How it works](#how-it-works-overview)
  - [Option 1: Hugging Face](#option-1-edit-directly-on-hugging-face-no-local-setup-needed)
  - [Option 2: Local](#option-2-work-locally)
  - [Quick reference](#quick-reference-where-is-each-parameter)
  - [File structure](#file-structure-what-each-file-does)
  - [Understanding inputs_aggregated.csv](#understanding-inputs_aggregatedcsv)
- [Maintenance cases](#maintenance-cases)
  - [Before you start](#before-you-start)
  - [Case 1 - Modify existing coefficient values](#case-1--modify-existing-coefficient-values)
  - [Case 2 - Add an existing resource to an existing method](#case-2--add-an-existing-resource-to-an-existing-method)
  - [Case 3 - Add a new CDR method](#case-3--add-a-new-cdr-method)
  - [Case 4 - Add a new resource](#case-4--add-a-new-resource)
  - [Case 5 - Add a new resource group](#case-5--add-a-new-resource-group-new-category)
  - [Case 6 - Change whether a resource is optional](#case-6--change-whether-a-resource-is-optional-secondary-for-a-method)
  - [Case 7 - Rename a method or resource](#case-7--rename-a-method-or-resource)
  - [Case 8 - Edit tooltip texts](#case-8--edit-tooltip-texts-hover-descriptions)
  - [Case 10 - Change the display order](#case-10--change-the-display-order-of-methods-or-resources)
  - [After any change - checklist](#after-any-change--checklist)
- [Library Maintenance Guide](#library-maintenance-guide)
  - [What is a library?](#what-is-a-library-dependency)
  - [The key libraries](#the-key-libraries-and-what-they-do)
  - [Dependency chain](#dependency-chain-what-depends-on-what)
  - [When to update](#when-to-update)
  - [How to safely update](#how-to-safely-update-a-library)
  - [Install on a new machine](#how-to-install-all-libraries-on-a-new-machine)
  - [Check for outdated libraries](#checking-for-outdated-libraries)
  - [Check for vulnerabilities](#checking-for-known-security-vulnerabilities)
  - [Using AI to handle updates](#using-an-ai-assistant-to-handle-updates)
  - [Summary: update strategy](#summary-update-strategy)

---

## Data Maintenance Guide

This guide is for Carbon Gap staff who need to update the tool's data: **no coding experience required** for most tasks.

---

### How it works (overview)

All the data the tool uses (CDR methods, resources, coefficients) comes from a single file:

```
data/inputs_aggregated.csv
```

Every time the app starts, it computes a fingerprint (hash) of the CSV and compares it to the last known value. If they differ, meaning the CSV has been modified, it automatically regenerates all the data files used by the application before loading. You do not need to run anything manually.

There are two ways to update the data, depending on whether you are working directly on Hugging Face or on a local copy of the project:

### Option 1: Edit directly on Hugging Face (no local setup needed)

This is the simplest option for non-coders with admin access to the Hugging Face Space.

**Steps:**

1. Go to the Hugging Face **Space** (Carbon Dioxide Removal Toolkit) → click the **Files** tab
2. Open `data/inputs_aggregated.csv` and click **Edit** (pencil icon)
   *OR*
   Upload a new version of the file: click **Delete** `data/inputs_aggregated.csv`, then go to **data** folder → click **+ Contribute** → Upload Files
4. Commit the change directly on Hugging Face (checkbox *Commit directly to the main branch*): the Space will restart automatically
5. **Check the logs** to confirm the update ran:
   - In the Space, click: **Logs** (The button at the very top, next to *Running*)
   - You should see:
     ```
     inputs_aggregated.csv has changed - running data/update.py ...
     [... update details ...]
     Data files updated successfully.
     ```
6. If the logs show this message, the app is running with the new data. If not, something went wrong: try option 2 or contact a developer. You may also not have waited long enough or, conversely, you may have waited too long before checking the logs: check whether the app has successfully applied your changes.

> **Note:** with this option, only `inputs_aggregated.csv` needs to be uploaded. The app handles the rest automatically on restart.

### Option 2: Work locally

Use this option when making bigger changes where you want to preview the result before publishing.

**Step 1: Get the project on your machine**

If you do not already have git installed:
- Mac: install it via https://git-scm.com/download/mac
- Windows: install it via https://git-scm.com/download/win
- 
Then in a terminal, navigate to the folder where you want to put the project and run:
```
git clone <repository URL>
```
Example:
```
git clone https://huggingface.co/spaces/CarbonGap/CRRA_Optimization_Tool
```

**Step 1b: Set up the Python environment (first time only)**

If you just cloned the repo, the venv/ folder doesn't exist yet. Create it by running in the terminal:

```
python -m venv venv
```

Then activate it and install the dependencies:
```
source venv/bin/activate    # Mac / Linux
venv\Scripts\activate       # Windows
```

You should now see `(venv)` at the start of your terminal line.

```
pip install -r requirements.txt
```
You only need to do this once per machine. After that, just activate the environment (Step 3) each time.

**Step 2: Edit the CSV**

Open `data/inputs_aggregated.csv` in Excel or any spreadsheet editor. Make your changes and save.

> ⚠️ **Commas in names:** if a method name, resource name, or group name contains a comma, the CSV parser will misread it. Wrap the value in double quotes in the CSV (e.g. `"Biochar, pyrolysis"`) or, preferably, avoid commas entirely and use a dash or slash instead.

**Step 3: Open a terminal in the project folder**

You can use any of the following:

- In your code editor (VS Code, PyCharm, etc.): open the integrated terminal, it's usually already pointing to the project folder
- Mac: right-click the project folder → **New Terminal at Folder**
- Windows: open the folder, click in the address bar, type `cmd`, press Enter

Activate the Python environment:
```
source venv/bin/activate    # Mac / Linux
venv\Scripts\activate       # Windows
```

**Step 4: Start the app to verify**

```
streamlit run app.py
```

The app detects that the CSV has changed and automatically runs the update script before loading. Check the terminal output: it should say `Data files updated successfully.` Then browse the app to confirm the change looks correct.

> If the auto-update doesn't seem to work (no message in the terminal), you can trigger it manually as a fallback: `python data/update.py`

**Step 5: Commit and push**

In the terminal:
```
git add -A -- ':!venv/'
git commit -m "Your description here (e.g. Update coefficients for Afforestation)"
git push
```

Hugging Face will detect the push and rebuild the Space automatically.

---

### Quick reference: where is each parameter?

| I want to change… | File | Where exactly |
|---|---|---|
| A coefficient value (min / median / max) | `data/inputs_aggregated.csv` | Columns `min`, `median`, `max`: find the row by `Implementation` + `Resource` |
| The name of a CDR method | `data/inputs_aggregated.csv` | Column `Implementation |
| Which group a CDR method belongs to | `data/inputs_aggregated.csv` | Column `Method group` |
| The name of a resource | `data/inputs_aggregated.csv` | Column `Resource` |
| Which group a resource belongs to | `data/inputs_aggregated.csv` | Column `Resource group` |
| The unit of a resource group | `data/inputs_aggregated.csv` | Column `Unit`: must be consistent across all rows for that group |
| Whether a resource is optional for a method | `data/inputs_aggregated.csv` | Column `Is secondary`: `TRUE` = optional, `FALSE` = always active |
| The display order of methods or resources | `data/inputs_aggregated.csv` | Row order: in the app, groups are displayed in the order their first row is encountered in the CSV; within a group, methods or resources are displayed in the order their rows appear. Move rows up/down to reorder |
| The app's UI colours (background, buttons, links) | `.streamlit/config.toml` | `primaryColor`, `backgroundColor`, `secondaryBackgroundColor`, `textColor` |
| The chart colours (Plotly graphs) | `config/config.py` | `CRRA_COLORS` dictionary |
| The app's UI font | `.streamlit/config.toml` | `font` - accepted values: `"sans-serif"`, `"serif"`, `"monospace"` |
| The chart font (Plotly graphs) | `config/config.py` | `CRRA_FONT` variable |
| The app's UI colours and font (fine-tuned overrides) | `assets/styles.css` | `:root` block at the top of the file - change the hex values next to each variable name |
| The tooltip text on a method name | `data/tooltips.xlsx` | Sheet **Methods**: column `Tooltip` |
| The tooltip text on a resource name | `data/tooltips.xlsx` | Sheet **Resources**: column `Tooltip` |
| UI spacing, font sizes, button styles | `assets/styles.css` | Standard CSS: search for the element name |
| Python library versions | `requirements.txt` | One line per library: see Library Maintenance Guide below |

> After any change to `inputs_aggregated.csv`, start the app - it detects the change and updates automatically. After changes to `config/config.py`, `assets/styles.css`, or `data/tooltips.xlsx`, simply restart the app.

### File structure (what each file does)

| File | What it is | Do you edit it? |
|------|-----------|-----------------|
| `data/inputs_aggregated.csv` | **Main data file**: all methods, resources and coefficients | ✅ Yes |
| `data/update.py` | Script that regenerates everything from the CSV | ❌ No |
| `data/cost_sliders.json` | Auto-generated. Coefficients used by the app | ❌ No (auto) |
| `data/method_groups.json` | Auto-generated. CDR method categories | ❌ No (auto) |
| `data/secondary_resources.json` | Auto-generated. optional resource assignments | ❌ No (auto) |
| `data/inputs_template.xlsx` | Auto-generated. Excel template for users | ❌ No (auto) |
| `data/resource_template.xlsx` | Auto-generated. Excel template for users | ❌ No (auto) |
| `data/tooltips.xlsx` | Tooltip texts shown on hover in the app | ✅ Yes |

> ⚠️ **Never edit the auto-generated files directly**: your changes will be overwritten the next time `update.py` is run.

### Understanding `inputs_aggregated.csv`

Each row represents **one resource used by one CDR method**, with the following columns:

| Column | Description | Example |
|--------|-------------|---------|
| `Implementation` | Name of the CDR method | `Afforestation` |
| `Method group` | Category the method belongs to | `Forestry` |
| `Resource` | Name of the resource used | `Other land` |
| `Resource group` | Category the resource belongs to | `Land` |
| `Type` | Physical type of the resource | `Physical` |
| `Unit` | Unit of the coefficient (per MtCO₂) | `Mha/MtCO2` |
| `min` | Minimum coefficient value | `0.002` |
| `median` | Default (median) coefficient value | `0.094927` |
| `max` | Maximum coefficient value | `0.384615` |
| `Is secondary` | Whether this resource is optional for this method | `FALSE` |

> **Primary resources** (`Is secondary = FALSE`) are always active in the model.
> **Secondary resources** (`Is secondary = TRUE`) are optional: users can toggle them on in the app.

---

## Maintenance cases

### Before you start

Each case below covers two workflows:
- **Option 1 - Hugging Face:** edit directly in the browser, no local setup needed.
- **Option 2 - Local:** make changes on your machine, then push to Hugging Face.

For Option 2, make sure the project is cloned and the Python environment is set up (see [Option 2](#option-2-work-locally) at the top of this guide). Before running any command in the terminal, activate the environment:

```
source venv/bin/activate    # Mac / Linux
venv\Scripts\activate       # Windows
```

You should see `(venv)` at the start of your terminal line.

> **For all Option 2 cases below:** after editing the CSV, start the app (`streamlit run app.py`) to verify - it detects the change and runs the update automatically. If the auto-update doesn't trigger (no message in the terminal), run `python data/update.py` manually as a fallback.

---

### Case 1 - Modify existing coefficient values

**When to use:** The literature has been updated and you want to change the min/median/max values for an existing method-resource pair.

**Option 1 - On Hugging Face**

1. In the Space, click the **Files** tab.
2. Open `data/inputs_aggregated.csv` → click the **pencil icon** to edit inline.
3. Find the row with the correct `Implementation` and `Resource`. Edit the `min`, `median`, and/or `max` columns.
4. Scroll down → **Commit directly to main branch** → **Commit changes to main**.
5. The Space restarts. Click **Logs** to confirm: `Data files updated successfully.`

**Option 2 - Locally**

1. Open `data/inputs_aggregated.csv` in Excel or a text editor.
2. Find the row with the correct `Implementation` and `Resource`.
3. Edit the `min`, `median`, and/or `max` columns.
4. Save the file, then start the app to verify:
   ```
   streamlit run app.py
   ```
5. Commit and push:
   ```
   git add -A -- ':!venv/'
   git commit -m "Update coefficients for [method name] and [resource name]"
   git push
   ```

### Case 2 - Add an existing resource to an existing method

**When to use:** A CDR method should now also use a resource that is already tracked for other methods (e.g., water is already listed for other methods, and you want to add it to Afforestation too). If the resource doesn't exist at all yet, use Case 4 first.

**Option 1 - On Hugging Face**

1. In the Space, click **Files** → open `data/inputs_aggregated.csv` → click the pencil icon.
2. Add a new row, making sure to include all the expected columns (see [Understanding inputs_aggregated.csv](#understanding-inputs_aggregatedcsv)).
3. Commit directly to main → Space restarts. Check Logs for `Data files updated successfully.`

**Option 2 - Locally**

1. Open `data/inputs_aggregated.csv` and add a new row (see [Understanding inputs_aggregated.csv](#understanding-inputs_aggregatedcsv)).
2. Save the file, then start the app to verify:
   ```
   streamlit run app.py
   ```
3. Commit and push:
   ```
   git add -A -- ':!venv/'
   git commit -m "Add [resource name] to [method name]"
   git push
   ```

### Case 3 - Add a new CDR method

**When to use:** A new carbon removal approach needs to be added to the tool.

**Option 1 - On Hugging Face**

1. In the Space, click **Files** → open `data/inputs_aggregated.csv` → click the pencil icon.
2. Add **one row per resource** the new method uses (see [Understanding inputs_aggregated.csv](#understanding-inputs_aggregatedcsv)). Use an existing `Method group` or create a new one.
3. Place the rows where you want the method to appear: display order follows row order in the CSV.
4. Commit directly to main → Space restarts. Check Logs for `Data files updated successfully.`

**Option 2 - Locally**

1. Open `data/inputs_aggregated.csv`.
2. Add one row per resource used by the new method. Fill in all columns (see [Understanding inputs_aggregated.csv](#understanding-inputs_aggregatedcsv)).
3. Save the file, then start the app to verify:
   ```
   streamlit run app.py
   ```
4. Commit and push:
   ```
   git add -A -- ':!venv/'
   git commit -m "Add new CDR method: [method name]"
   git push
   ```

---

### Case 4 - Add a new resource

**When to use:** A resource that doesn't exist yet needs to be used (e.g., a new type of mineral or energy).

**Option 1 - On Hugging Face**

1. Decide which **Resource group** the new resource belongs to. If it belongs to an existing group, use the exact same name. If it needs its own group, see Case 5.
2. In the Space, click **Files** → open `data/inputs_aggregated.csv` → pencil icon.
3. Add one row per method that uses this resource (see column descriptions in Case 2).
4. **Check the unit:** all resources in a group must share the same unit (e.g., all `Land` resources use `Mha/MtCO2`). If the unit differs, create a new group (Case 5).
5. Commit directly to main → Space restarts. Check Logs.

**Option 2 - Locally**

1. Decide which Resource group the new resource belongs to.
2. Open `data/inputs_aggregated.csv` and add one row per method that uses this resource.
3. Verify the unit is consistent with the rest of the group.
4. Save, then start the app to verify:
   ```
   streamlit run app.py
   ```
5. Commit and push:
   ```
   git add -A -- ':!venv/'
   git commit -m "Add new resource: [resource name]"
   git push
   ```

---

### Case 5 - Add a new resource group (new category)

**When to use:** Your new resource doesn't fit any existing category.

**Option 1 - On Hugging Face**

Same steps as Case 4, but write a **new group name** (never used before) in the `Resource group` column. The app will create a new section for it automatically. Place the rows where you want the group to appear: it appears at the position of its first row in the CSV.

**Option 2 - Locally**

Same steps as Case 4, but write a **new group name** in the `Resource group` column. The new group appears in the app at the position of its first row in the CSV.

---

### Case 6 - Change whether a resource is optional (secondary) for a method

**When to use:** You want to make a resource optional (user can enable/disable it) or make it always required.

**Option 1 - On Hugging Face**

1. In the Space, click **Files** → open `data/inputs_aggregated.csv` → pencil icon.
2. Find the row with the correct `Implementation` and `Resource`.
3. Change `Is secondary` to `TRUE` (optional) or `FALSE` (always active).
4. Commit directly to main → Space restarts. Check Logs.

**Option 2 - Locally**

1. Open `data/inputs_aggregated.csv`.
2. Find the row with the correct `Implementation` and `Resource`.
3. Change `Is secondary` to `TRUE` (optional) or `FALSE` (always active).
4. Save, then start the app to verify:
   ```
   streamlit run app.py
   ```
5. Commit and push:
   ```
   git add -A -- ':!venv/'
   git commit -m "Update secondary status for [resource] in [method]"
   git push
   ```

---

### Case 7 - Rename a method or resource

**When to use:** Correcting a spelling mistake or updating nomenclature.

> ⚠️ If users have saved scenario files (`.xlsx` or `.zip`) that reference the old name, those files will no longer load correctly after the rename.

**Option 1 - On Hugging Face**

Editing inline is not recommended for a rename because Find & Replace is not available in the browser editor. Instead:
1. Download `data/inputs_aggregated.csv` from the Files tab.
2. Open it in Excel. Use **Find & Replace** (Ctrl+H / Cmd+H) to replace the old name with the new name in the `Implementation` / `Resource` columns and, if the group name changed, in `Method group` / `Resource group`.
3. Also download `data/tooltips.xlsx`, update the name in the `Method` or `Resource` column of the relevant sheet, and save.
4. Upload both files via **Files** → `data/` folder → **+ Contribute** → **Upload Files** → Commit.
5. Space restarts → check Logs.

**Option 2 - Locally**

1. Open `data/inputs_aggregated.csv`. Use **Find & Replace** (Ctrl+H / Cmd+H) to replace the old name everywhere in the file.
2. Open `data/tooltips.xlsx` and update the name in the `Method` or `Resource` column of the relevant sheet. Save.
3. Start the app to verify:
   ```
   streamlit run app.py
   ```
4. Commit and push:
   ```
   git add -A -- ':!venv/'
   git commit -m "Rename [old name] to [new name]"
   git push
   ```

---

### Case 8 - Edit tooltip texts (hover descriptions)

**When to use:** You want to add, update, or remove the short description that appears when a user hovers over a resource or method name in the app.

> ⚠️ The name in column `Resource` or `Method` must match **exactly** the name used in `inputs_aggregated.csv` (same spelling, same capitalisation). If you rename a method or resource in the CSV, remember to update the name in `tooltips.xlsx` too.

**Option 1 - On Hugging Face**

The `.xlsx` file cannot be edited inline on Hugging Face. Instead:
1. In the Files tab, open the `data/` folder → download `tooltips.xlsx`.
2. Open it in Excel. Go to the relevant sheet (**Resources** or **Methods**).
3. Find the row with the name you want to edit. Fill in or change the `Tooltip` column. Leave it empty to remove a tooltip.
4. Save the file.
5. Back in the Files tab, go to `data/` → **+ Contribute** → **Upload Files**.
6. Upload your edited `tooltips.xlsx` → Commit directly to main.
7. The Space restarts. **No update script runs for tooltips** - the file is read directly by the app on each restart.

**Option 2 - Locally**

1. Open `data/tooltips.xlsx` in Excel.
2. Go to the relevant sheet (**Resources** or **Methods**).
3. Find the row with the name you want to edit. Fill in or change the `Tooltip` column. Leave it empty to remove a tooltip.
4. Save the file.
5. Restart the app (`Ctrl+C` then `streamlit run app.py`). **No need to run `update.py`.**
6. Commit and push:
   ```
   git add data/tooltips.xlsx
   git commit -m "Update tooltip texts"
   git push
   ```

---

### Case 10 - Change the display order of methods or resources

**When to use:** You want a method or resource group to appear earlier or later in the app (Resource inputs, Method constraints tabs...).

**Option 1 - On Hugging Face**

Moving rows is easier in Excel than in a browser editor:
1. Download `data/inputs_aggregated.csv` from the Files tab.
2. Open it in Excel. Move the relevant rows up or down - the app reads the CSV from top to bottom.
3. Upload the updated file via **Files** → `data/` → **+ Contribute** → **Upload Files** → Commit.
4. Space restarts → check Logs for `Data files updated successfully.`

**Option 2 - Locally**

1. Open `data/inputs_aggregated.csv`.
2. Move the relevant rows up or down.
3. Save, then start the app to verify:
   ```
   streamlit run app.py
   ```
4. Commit and push:
   ```
   git add -A -- ':!venv/'
   git commit -m "Reorder [methods/resources]"
   git push
   ```

---

### After any change - checklist

**Option 1 (Hugging Face direct):**
- [ ] Edited and committed `inputs_aggregated.csv` on Hugging Face
- [ ] Checked the Space logs: confirmed `Data files updated successfully.`
- [ ] Verified the change appears correctly in the app

**Option 2 (local):**
- [ ] Saved `inputs_aggregated.csv`
- [ ] Started the app locally (`streamlit run app.py`): confirmed `Data files updated successfully.` in the terminal
- [ ] Verified the change appears correctly in the app
- [ ] Committed all modified files (CSV, JSON files, templates) and pushed

---

## Library Maintenance Guide

This section explains how to keep the tool's software dependencies up to date, safely and without breaking anything.

---

### What is a library (dependency)?

The tool is built on top of external software packages ("libraries") that handle things like displaying the interface, reading Excel files, or running the optimization. These libraries are listed in `requirements.txt` with their exact version numbers (e.g., `streamlit==1.45.1`).

Libraries sometimes release updates - fixing bugs, patching security holes, or adding new features. The risk is that an update can also change how a library behaves and break the tool.

---

### The key libraries and what they do

These are the libraries that matter most for this tool:

| Library | Role | Risk if updated |
|---------|------|----------------|
| **streamlit** | Powers the entire web interface (tabs, sliders, buttons, charts) - currently 1.58.0 | 🔴 High - UI layout and widget behavior can change |
| **streamlit-option-menu** | The navigation sidebar menu | 🟡 Medium - must stay compatible with streamlit version |
| **pandas** | Reads and processes CSV and Excel files | 🟡 Medium - occasional API changes |
| **numpy** | Numerical operations (arrays, math) | 🟡 Medium - used by scipy and pandas |
| **scipy** | Runs the LP optimization (`linprog`) | 🟡 Medium - optimization results could change |
| **plotly** | All charts and interactive graphs | 🟡 Medium - chart API evolves |
| **openpyxl** | Reads and writes Excel files (.xlsx) | 🟢 Low - stable API |
| **pillow** | Image handling (indirect use) | 🟢 Low |
| **requests / urllib3 / certifi** | Network and security | 🟢 Low - update freely for security patches |

---

### Dependency chain (what depends on what)

Some libraries require specific versions of others to work. Updating one can force updates to others:

```
streamlit ──────────────► tornado (web server)
            ├───────────► altair (charts)
            ├───────────► pyarrow (data performance)
            ├───────────► narwhals (data compatibility)
            ├───────────► protobuf
            └───────────► cachetools, blinker, click, toml

pandas ─────────────────► numpy
        ├───────────────► pyarrow
        └───────────────► python-dateutil, pytz, tzdata

scipy ──────────────────► numpy

plotly ─────────────────► narwhals

openpyxl ──────────────► pillow
```

**Practical implication:** if you update `streamlit`, you may also need to update `tornado`, `altair`, `pyarrow`, etc. If you update `pandas`, you may need to update `numpy`. These cascading updates are handled automatically by pip (see below), but they can introduce unexpected changes.

---

### When to update

#### ✅ Update immediately
- **`pip-audit` flags a vulnerability** - run `pip-audit` regularly (see [Check for vulnerabilities](#checking-for-known-security-vulnerabilities)); any flagged library should be patched without delay.
- **Security vulnerabilities** in `requests`, `urllib3`, `certifi`, or `pillow` - these are reported publicly and should be patched fast even before `pip-audit` catches them.
- Check https://pypi.org/project/[library-name] or subscribe to GitHub security alerts.

#### ✅ Update when needed
- **Streamlit** - when a new feature in the app requires it, or when a critical bug in the current version is causing problems.
- **pandas / numpy / scipy** - when a bug in the current version is causing incorrect data processing or optimization results.
- **openpyxl** - when Excel import/export breaks with newer `.xlsx` files.

#### ❌ Do NOT update just because a newer version exists
- "Latest is always better" is not true for production tools. An update that works fine in isolation can break interactions between libraries.
- The tool has been tested with the versions in `requirements.txt`. Changing them without testing is risky.

---

### Checking for outdated libraries

To see which installed libraries have newer versions available:

```
pip list --outdated
```

This shows a list with current and latest versions. Use this to **identify** what could be updated but don't update everything at once.

---

### How to safely update a library

**Never update all libraries at once.** Update one at a time, test the tool after each one.

**Step 1 - Back up the current working state**

```
cp requirements.txt requirements.txt.backup
```

**Step 2 - Simulate first (dry run)**

Before applying anything, check whether the update would cause conflicts:
```
pip install --upgrade streamlit --dry-run
```
If pip reports conflicts, investigate before proceeding.

**Step 3 - Update the library**

```
pip install --upgrade streamlit
```

Or to a specific version:
```
pip install streamlit==1.58.0
```

**Step 4 - Verify no dependencies are broken**

```
pip check
```
This confirms that all installed packages are mutually compatible. If it reports any conflict, do not proceed to the next step - restore the backup instead (see Step 6).

**Step 5 - Freeze the new versions to requirements.txt**

```
pip freeze > requirements.txt
```
`pip freeze` lists every library currently installed in the environment with its exact version number, and `> requirements.txt` writes that list into the file. This ensures that anyone who installs from `requirements.txt` later, including Hugging Face when the Space rebuilds, gets exactly the same versions you just tested.

**Step 6 - Test the tool**

- Start the app: `streamlit run app.py`
- Go through every tab
- Import and export an Excel file
- Run an optimization
- Check that charts display correctly

**Step 7 - If something breaks**

Restore the backup:
```
pip install -r requirements.txt.backup
cp requirements.txt.backup requirements.txt
```

**Step 8 - If everything works**

Delete the backup file, it's no longer needed:
```
rm requirements.txt.backup
```
Then commit the updated `requirements.txt` and push.

---

### How to install all libraries on a new machine

```
python -m venv venv
source venv/bin/activate        # Mac/Linux
# or: venv\Scripts\activate     # Windows
pip install -r requirements.txt
```

---

### Checking for known security vulnerabilities

Install the security scanner:
```
pip install pip-audit
```

Run it:
```
pip-audit
```

This will flag any library with a known security issue and suggest a fix. These should be addressed promptly.

> **Recommended frequency:** run `pip-audit` at least once a year, even if the tool appears stable. Security vulnerabilities are discovered independently of whether the library has changed - a version that was safe last year may be flagged today.

---

### Using an AI assistant to handle updates

Library updates sometimes require code changes alongside the pip update - for example, a deprecated function that was renamed, or a new argument that became required. Doing this manually requires reading changelogs, finding every affected line, and testing the result.

An AI coding assistant such as **Claude Code** can handle this end-to-end:

- Run `pip-audit` or `pip list --outdated` and paste the output into Claude Code
- Describe the update you want to make (e.g., "update pandas and fix any deprecation warnings")
- Claude Code will update the library, search the codebase for affected code, apply the necessary fixes, and update `requirements.txt`

This is especially useful when:
- A deprecation warning points to a function that is used in multiple places across the codebase
- A major version bump changes how a library behaves (e.g., pandas 3.x changing its default string backend)
- You are not comfortable navigating the code yourself

> The AI handles the mechanical parts; you remain responsible for testing the result with `streamlit run app.py` before committing and pushing.

---

### Summary: update strategy

```
Security fix needed?  ──► Update immediately (requests, urllib3, certifi, pillow)
Something is broken?  ──► Update only the relevant library, test thoroughly
Nothing is broken?    ──► Do not update
```