# Agents

The agents package contains the LLM-based extraction components used in the HelpScout processing pipeline. Each agent is a self-contained class responsible for one well-defined task.

---

## Architecture

```
BaseAgent  (base_agent.py)
    │
    ├── SentimentAnalysisAgent  (sentiment_analysis_agent.py)
    │       Classifies overall sentiment polarity and emotions
    │       from a customer support conversation.
    │
    └── TopicExtractionAgent  (topic_extraction_agent.py)
            Assigns one or more topic tags and extracts
            billing-specific boolean flags.
```

All agents follow the same contract defined in `BaseAgent`:

| Method | Required | Description |
|--------|----------|-------------|
| `validate_input(input_data)` | Yes | Returns `True` if the input dict has the required fields |
| `process(input_data)` | Yes | Main entry point — validates, calls LLM, returns result dict |
| `log_processing(message, level)` | Inherited | Logs `[AgentName] message` at the given level |
| `handle_error(error, context)` | Inherited | Returns a standardised `{"success": False, "error": ...}` dict |

The workflow (`workflow/conversation_processor.py`) calls `agent.process(input_data)` for each node. Agents never call each other — they are orchestrated exclusively by the workflow.

---

## BaseAgent (`base_agent.py`)

Defines the interface every agent must implement. Contains no LLM logic.

### Key attributes set from config

```python
self.model        # LLM model name, e.g. "gpt-4o-mini"
self.temperature  # Sampling temperature (default: 0.2)
self.max_retries  # Reserved for retry logic in subclasses
```

These are read from the agent's block in `config_files/processing_config.json`:
```json
"agents": {
  "sentiment_analysis": { "model": "gpt-4o-mini", "temperature": 0.2, "max_retries": 3 }
}
```

### Return contract

Every `process()` implementation must return a dict with at minimum:
```python
{"success": True, ...}   # on success — include extracted fields
{"success": False, "error": "<reason>"}  # on failure
```

The workflow checks `success` to decide whether to mark a conversation as failed.

---

## SentimentAnalysisAgent (`sentiment_analysis_agent.py`)

Classifies the overall **sentiment polarity** and **emotions** expressed across a customer's conversation messages.

### Input

```python
agent.process({
    "conversation_text": "<formatted, truncated customer messages>"
})
```

The `conversation_text` is prepared by the workflow before calling the agent — it is numbered, pipe-delimited messages truncated to `max_conversation_chars`.

### Output (on success)

```python
{
    "success": True,
    "sentiment_polarity": "negative",        # one of the 5 polarity values
    "emotions": "frustration, disappointment", # comma-separated, or None (soft-fail)
    "sentiment_confidence": "high",
    "sentiment_notes": "Customer is frustrated by repeated login failures."
}
```

### Validation rules

| Field | Behaviour on invalid value |
|-------|---------------------------|
| `sentiment_polarity` | Hard fail — conversation is not stored |
| `emotions` | Soft fail — `None` is stored, conversation is still written |
| `confidence` | Silently corrected to `"medium"` |

### Where categories are defined

Polarity and emotion categories (their `value` and `description` strings) live in `config_files/processing_config.json` under `"sentiment_polarity"` and `"emotions"`. The system prompt is **built at init time from the config**, so updating the config is all you need to change what the LLM is instructed to classify.

### Modifying the sentiment prompt

The system prompt is assembled in `_build_system_prompt()`. To change the framing or add additional instructions, edit that method directly. The category lists are injected automatically from config — do not hardcode them in the prompt.

---

## TopicExtractionAgent (`topic_extraction_agent.py`)

Assigns one or more **topic tags** from the Musora HelpScout taxonomy, extracts three **billing/membership boolean flags**, and produces a brief **neutral summary** of the conversation.

### Input

```python
agent.process({
    "conversation_text": "<formatted, truncated customer messages>"
})
```

### Output (on success)

```python
{
    "success": True,
    "topics": "billing_and_subscription, account_and_access",  # comma-separated IDs
    "is_refund_request": True,    # customer explicitly asked for money back
    "is_cancellation": False,     # customer did NOT explicitly ask to cancel
    "is_membership": False,       # customer wants to join/rejoin and purchase membership
    "topic_confidence": "high",
    "topic_notes": "Customer was unexpectedly charged and is requesting a refund.",
    "summary": "The customer reports being charged after believing they had cancelled their subscription. They are requesting a full refund and confirmation that no further charges will occur."
}
```

### Validation rules

| Field | Behaviour on invalid value |
|-------|---------------------------|
| `topics` | Hard fail if no valid topic IDs remain after filtering |
| `is_refund_request` / `is_cancellation` / `is_membership` | Coerced to `bool`; defaults to `False` if missing |
| `confidence` | Silently corrected to `"medium"` |
| `summary` | Soft fail — `""` stored if missing; conversation still written |

### Where topics are defined

All topic definitions live in `config_files/topics.json`. The agent builds its system prompt directly from this file at init time — adding, removing, or rewriting a topic description requires only a config change.

### Billing and membership flags

`is_refund_request`, `is_cancellation`, and `is_membership` are extracted on every conversation regardless of which topics are assigned. They are defined in `topics.json` under `billing_and_subscription.flags` for documentation purposes, but the agent always asks the LLM to evaluate them independently.

### Summary

The `summary` field is a 2-3 sentence factual, third-person overview of the conversation — what the customer contacted support about, relevant context they provided, and their core request. It is designed to give a reader instant context without reading the full conversation, and can also be used as compact input when chaining LLM calls.

---

## How to Add a New Agent

Follow these steps to add a third extraction step (e.g. urgency scoring):

### Step 1 — Create the agent file

```python
# agents/urgency_agent.py
from agents.base_agent import BaseAgent
from langchain_openai import ChatOpenAI
from langchain.schema import HumanMessage, SystemMessage
import json, logging

logger = logging.getLogger(__name__)

class UrgencyAgent(BaseAgent):

    def __init__(self, config, api_key):
        super().__init__("UrgencyAgent", config)
        self.llm = ChatOpenAI(
            model=self.model,
            temperature=self.temperature,
            api_key=api_key,
            model_kwargs={"response_format": {"type": "json_object"}},
        )
        self._system_prompt = (
            "Classify the urgency of this customer support conversation.\n"
            'Return JSON: {"urgency": "high"|"medium"|"low", "urgency_notes": "<reason>"}'
        )

    def validate_input(self, input_data):
        return "conversation_text" in input_data and bool(input_data["conversation_text"])

    def process(self, input_data):
        if not self.validate_input(input_data):
            return {"success": False, "error": "Missing conversation_text"}
        try:
            response = self.llm.invoke([
                SystemMessage(content=self._system_prompt),
                HumanMessage(content=input_data["conversation_text"]),
            ])
            raw = json.loads(response.content)
            urgency = raw.get("urgency", "medium")
            if urgency not in {"high", "medium", "low"}:
                urgency = "medium"
            return {
                "success": True,
                "urgency": urgency,
                "urgency_notes": raw.get("urgency_notes", ""),
            }
        except Exception as e:
            return self.handle_error(e, "urgency_classification")
```

### Step 2 — Add config for the new agent

In `config_files/processing_config.json`:
```json
"agents": {
  "sentiment_analysis": { ... },
  "topic_extraction": { ... },
  "urgency": {
    "model": "gpt-4o-mini",
    "temperature": 0.1,
    "max_retries": 3
  }
}
```

### Step 3 — Add a node to the workflow

In `workflow/conversation_processor.py`:

```python
# 1. Import the new agent
from agents.urgency_agent import UrgencyAgent

# 2. Instantiate in __init__
self.urgency_agent = UrgencyAgent(config["agents"]["urgency"], api_key)

# 3. Add fields to ConversationState
urgency: str
urgency_notes: str

# 4. Add the node method
def _urgency_node(self, state):
    try:
        result = self.urgency_agent.process({"conversation_text": state["conversation_text"]})
        if result.get("success"):
            state["urgency"] = result.get("urgency")
            state["urgency_notes"] = result.get("urgency_notes", "")
        else:
            state["processing_errors"] = state.get("processing_errors", []) + [
                f"Urgency failed: {result.get('error')}"
            ]
            state["urgency"] = None
    except Exception as e:
        state["processing_errors"] = state.get("processing_errors", []) + [str(e)]
    return state

# 5. Wire into the graph in _build_workflow()
graph.add_node("urgency", self._urgency_node)
graph.add_edge("topic_extraction", "urgency")   # replaces the old edge to END
graph.add_edge("urgency", END)
```

### Step 4 — Add output columns

In `main.py`, add to the `column_map` dict:
```python
"urgency":       "URGENCY",
"urgency_notes": "URGENCY_NOTES",
```

In `sql/create_features_table.sql`, add:
```sql
URGENCY         VARCHAR(20),
URGENCY_NOTES   TEXT,
```

Run `ALTER TABLE` or recreate the table for the new columns to appear.

---

## How to Modify an Existing Agent

### Change the LLM model or temperature

Edit `config_files/processing_config.json` — no code change needed.

### Add or rename a sentiment category

In `config_files/processing_config.json`, update `sentiment_polarity.categories` or `emotions.categories`. The agent reads these at init and builds the prompt and validation set dynamically. The only code-level change is updating the output table column type/constraint if the new value is longer than the current `VARCHAR` size.

### Add or rename a topic

In `config_files/topics.json`, add or edit an entry in the `"topics"` array. The `TopicExtractionAgent` reads this file at init — the new topic appears in the prompt and validation automatically.

### Change the conversation truncation limit

In `config_files/processing_config.json`:
```json
"processing": {
  "max_conversation_chars": 3000
}
```

This is read by the workflow (`conversation_processor.py`) before formatting the conversation text — no agent code changes needed.

### Modify the system prompt framing

Each agent builds its prompt in a `_build_system_prompt()` method. Edit that method directly. Category lists are always injected from config — avoid hardcoding values that already live in the JSON.