process_helpscout/agents/README.md

Agents

The agents package contains the LLM-based extraction components used in the HelpScout processing pipeline. Each agent is a self-contained class responsible for one well-defined task.

---

Architecture

BaseAgent  (base_agent.py)
    │
    ├── SentimentAnalysisAgent  (sentiment_analysis_agent.py)
    │       Classifies overall sentiment polarity and emotions
    │       from a customer support conversation.
    │
    └── TopicExtractionAgent  (topic_extraction_agent.py)
            Assigns one or more topic tags and extracts
            billing-specific boolean flags.

All agents follow the same contract defined in BaseAgent:

Method	Required	Description
validate_input(input_data)	Yes	Returns True if the input dict has the required fields
process(input_data)	Yes	Main entry point — validates, calls LLM, returns result dict
log_processing(message, level)	Inherited	Logs [AgentName] message at the given level
handle_error(error, context)	Inherited	Returns a standardised {"success": False, "error": ...} dict

The workflow (workflow/conversation_processor.py) calls agent.process(input_data) for each node. Agents never call each other — they are orchestrated exclusively by the workflow.

---

BaseAgent (base_agent.py)

Defines the interface every agent must implement. Contains no LLM logic.

Key attributes set from config

self.model        # LLM model name, e.g. "gpt-4o-mini"
self.temperature  # Sampling temperature (default: 0.2)
self.max_retries  # Reserved for retry logic in subclasses

These are read from the agent's block in config_files/processing_config.json:

"agents": {
  "sentiment_analysis": { "model": "gpt-4o-mini", "temperature": 0.2, "max_retries": 3 }
}

Return contract

Every process() implementation must return a dict with at minimum:

{"success": True, ...}   # on success — include extracted fields
{"success": False, "error": "<reason>"}  # on failure

The workflow checks success to decide whether to mark a conversation as failed.

---

SentimentAnalysisAgent (sentiment_analysis_agent.py)

Classifies the overall sentiment polarity and emotions expressed across a customer's conversation messages.

Input

agent.process({
    "conversation_text": "<formatted, truncated customer messages>"
})

The conversation_text is prepared by the workflow before calling the agent — it is numbered, pipe-delimited messages truncated to max_conversation_chars.

Output (on success)

{
    "success": True,
    "sentiment_polarity": "negative",        # one of the 5 polarity values
    "emotions": "frustration, disappointment", # comma-separated, or None (soft-fail)
    "sentiment_confidence": "high",
    "sentiment_notes": "Customer is frustrated by repeated login failures."
}

Validation rules

Field	Behaviour on invalid value
sentiment_polarity	Hard fail — conversation is not stored
emotions	Soft fail — None is stored, conversation is still written
confidence	Silently corrected to "medium"

Where categories are defined

Polarity and emotion categories (their value and description strings) live in config_files/processing_config.json under "sentiment_polarity" and "emotions". The system prompt is built at init time from the config, so updating the config is all you need to change what the LLM is instructed to classify.

Modifying the sentiment prompt

The system prompt is assembled in _build_system_prompt(). To change the framing or add additional instructions, edit that method directly. The category lists are injected automatically from config — do not hardcode them in the prompt.

---

TopicExtractionAgent (topic_extraction_agent.py)

Assigns one or more topic tags from the Musora HelpScout taxonomy, extracts three billing/membership boolean flags, and produces a brief neutral summary of the conversation.

Input

agent.process({
    "conversation_text": "<formatted, truncated customer messages>"
})

Output (on success)

{
    "success": True,
    "topics": "billing_and_subscription, account_and_access",  # comma-separated IDs
    "is_refund_request": True,    # customer explicitly asked for money back
    "is_cancellation": False,     # customer did NOT explicitly ask to cancel
    "is_membership": False,       # customer wants to join/rejoin and purchase membership
    "topic_confidence": "high",
    "topic_notes": "Customer was unexpectedly charged and is requesting a refund.",
    "summary": "The customer reports being charged after believing they had cancelled their subscription. They are requesting a full refund and confirmation that no further charges will occur."
}

Validation rules

Field	Behaviour on invalid value
topics	Hard fail if no valid topic IDs remain after filtering
is_refund_request / is_cancellation / is_membership	Coerced to bool; defaults to False if missing
confidence	Silently corrected to "medium"
summary	Soft fail — "" stored if missing; conversation still written

Where topics are defined

All topic definitions live in config_files/topics.json. The agent builds its system prompt directly from this file at init time — adding, removing, or rewriting a topic description requires only a config change.

Billing and membership flags

is_refund_request, is_cancellation, and is_membership are extracted on every conversation regardless of which topics are assigned. They are defined in topics.json under billing_and_subscription.flags for documentation purposes, but the agent always asks the LLM to evaluate them independently.

Summary

The summary field is a 2-3 sentence factual, third-person overview of the conversation — what the customer contacted support about, relevant context they provided, and their core request. It is designed to give a reader instant context without reading the full conversation, and can also be used as compact input when chaining LLM calls.

---

How to Add a New Agent

Follow these steps to add a third extraction step (e.g. urgency scoring):

Step 1 — Create the agent file

# agents/urgency_agent.py
from agents.base_agent import BaseAgent
from langchain_openai import ChatOpenAI
from langchain.schema import HumanMessage, SystemMessage
import json, logging

logger = logging.getLogger(__name__)

class UrgencyAgent(BaseAgent):

    def __init__(self, config, api_key):
        super().__init__("UrgencyAgent", config)
        self.llm = ChatOpenAI(
            model=self.model,
            temperature=self.temperature,
            api_key=api_key,
            model_kwargs={"response_format": {"type": "json_object"}},
        )
        self._system_prompt = (
            "Classify the urgency of this customer support conversation.\n"
            'Return JSON: {"urgency": "high"|"medium"|"low", "urgency_notes": "<reason>"}'
        )

    def validate_input(self, input_data):
        return "conversation_text" in input_data and bool(input_data["conversation_text"])

    def process(self, input_data):
        if not self.validate_input(input_data):
            return {"success": False, "error": "Missing conversation_text"}
        try:
            response = self.llm.invoke([
                SystemMessage(content=self._system_prompt),
                HumanMessage(content=input_data["conversation_text"]),
            ])
            raw = json.loads(response.content)
            urgency = raw.get("urgency", "medium")
            if urgency not in {"high", "medium", "low"}:
                urgency = "medium"
            return {
                "success": True,
                "urgency": urgency,
                "urgency_notes": raw.get("urgency_notes", ""),
            }
        except Exception as e:
            return self.handle_error(e, "urgency_classification")

Step 2 — Add config for the new agent

In config_files/processing_config.json:

"agents": {
  "sentiment_analysis": { ... },
  "topic_extraction": { ... },
  "urgency": {
    "model": "gpt-4o-mini",
    "temperature": 0.1,
    "max_retries": 3
  }
}

Step 3 — Add a node to the workflow

In workflow/conversation_processor.py:

# 1. Import the new agent
from agents.urgency_agent import UrgencyAgent

# 2. Instantiate in __init__
self.urgency_agent = UrgencyAgent(config["agents"]["urgency"], api_key)

# 3. Add fields to ConversationState
urgency: str
urgency_notes: str

# 4. Add the node method
def _urgency_node(self, state):
    try:
        result = self.urgency_agent.process({"conversation_text": state["conversation_text"]})
        if result.get("success"):
            state["urgency"] = result.get("urgency")
            state["urgency_notes"] = result.get("urgency_notes", "")
        else:
            state["processing_errors"] = state.get("processing_errors", []) + [
                f"Urgency failed: {result.get('error')}"
            ]
            state["urgency"] = None
    except Exception as e:
        state["processing_errors"] = state.get("processing_errors", []) + [str(e)]
    return state

# 5. Wire into the graph in _build_workflow()
graph.add_node("urgency", self._urgency_node)
graph.add_edge("topic_extraction", "urgency")   # replaces the old edge to END
graph.add_edge("urgency", END)

Step 4 — Add output columns

In main.py, add to the column_map dict:

"urgency":       "URGENCY",
"urgency_notes": "URGENCY_NOTES",

In sql/create_features_table.sql, add:

URGENCY         VARCHAR(20),
URGENCY_NOTES   TEXT,

Run ALTER TABLE or recreate the table for the new columns to appear.

---

How to Modify an Existing Agent

Change the LLM model or temperature

Edit config_files/processing_config.json — no code change needed.

Add or rename a sentiment category

In config_files/processing_config.json, update sentiment_polarity.categories or emotions.categories. The agent reads these at init and builds the prompt and validation set dynamically. The only code-level change is updating the output table column type/constraint if the new value is longer than the current VARCHAR size.

Add or rename a topic

In config_files/topics.json, add or edit an entry in the "topics" array. The TopicExtractionAgent reads this file at init — the new topic appears in the prompt and validation automatically.

Change the conversation truncation limit

In config_files/processing_config.json:

"processing": {
  "max_conversation_chars": 3000
}

This is read by the workflow (conversation_processor.py) before formatting the conversation text — no agent code changes needed.

Modify the system prompt framing

Each agent builds its prompt in a _build_system_prompt() method. Edit that method directly. Category lists are always injected from config — avoid hardcoding values that already live in the JSON.