Score improvement changes

README.md

@@ -24,42 +24,53 @@ The core analysis logic is driven by a LangGraph StateGraph defined in `src/work
 
 ### Workflow Architecture
 ```text
-                 [ IntakeAgent ]
+                  [ IntakeAgent ]
                         |
                         v
-                   [ Router ]
-                  /          \
-      (If Audio) /            \ (If Text)
-                v              \
-    [ TranscriptionAgent ]      |
-                \              /
-                 \            /
-                  v          v
-              [ ModerationAgent ]
-                        |
-                        v
-            [ SummarizationAgent ]
-                        |
-                        v
-            [ QualityScoringAgent ]
-                        |
-                        v
-                      (END)
+               [ Router (file type) ]
+              /         |           \
+ (CSV invalid)/     (If Audio)     (If Text)
+          v          v               v
+        (END) [ TranscriptionAgent ] |
+                  |                  |
+                  v                  |
+            [ ModerationAgent ] <-----/
+                  |
+                  v
+          [ SummarizationAgent ]
+                  |
+                  v
+     [ PostSummarizeRouter (output) ]
+              |                |
+           (END)        [ QualityScoringAgent ]
+                               |
+                               v
+                             (END)
 ```
 
+Notes:
+- The workflow uses a LangGraph memory checkpointer (`MemorySaver`) and fallbacks on critical nodes (summarization/scoring) to avoid UI breakage on transient API/parse errors.
+- If CSV headers are invalid, the workflow terminates immediately and the UI shows a validation error.
+- After summarization, the workflow can short-circuit to `END` if the transcript is too short or the model output is missing/empty.
+
 ### Agent Classes
 
-1. **`IntakeAgent` (`src/agents/intake_agent.py`)**
+1. **`IntakeAgent` (`src/agents/IntakeAgent.py`)**
    - *Entry Point*.
-   - Reads the uploaded file, validates the file format, extracts basic metadata, and runs a first-pass profanity scrub (using an LLM).
+   - Reads the uploaded file, validates the file format and CSV schema, extracts basic metadata, and runs a first-pass clean-up (using an LLM).
+   - **CSV requirement:** headers must include `id` and `transcript` (case-insensitive).
 
-2. **`Router` (`src/agents/router.py`)**
+2. **`Router` (`src/agents/Router.py`)**
    - *Conditional Routing Node*.
-   - Determines the next step based on the file type.
+   - Determines the next step based on the file type and intake validation state.
    - **If Audio (`.mp3`, `.wav`)**: Routes to the `TranscriptionAgent`.
-   - **If Text (`.csv`)**: Routes directly to the `SummarizationAgent`.
+   - **If Text (`.csv`)**: Routes to the `ModerationAgent` then summarization/scoring.
+   - **If CSV invalid**: Routes to `END` (the UI displays `metadata.intake_error`).
 
-3. **`TranscriptionAgent` (`src/agents/transcription_agent.py`)**
+   **`PostSummarizeRouter` (`src/agents/Router.py`)**
+   - Routes based on model output quality (e.g., short transcript or missing summary can skip scoring).
+
+3. **`TranscriptionAgent` (`src/agents/TranscriptionAgent.py`)**
    - Utilizes `openai-whisper` to convert audio files into text.
    - Also scrubs the resulting transcript of profanity before passing it down the pipeline.
 
@@ -67,13 +78,13 @@ The core analysis logic is driven by a LangGraph StateGraph defined in `src/work
    - Receives text either directly from the `Router` (if text upload) or from the `TranscriptionAgent` (if audio upload).
    - Identifies any obscene words or profanity using an LLM and replaces them entirely with a `***` mask to safely prepare the text for downstream analysis.
 
-5. **`SummarizationAgent` (`src/agents/summarization_agent.py`)**
+5. **`SummarizationAgent` (`src/agents/SummarizationAgent.py`)**
    - Takes the redacted text from the `ModerationAgent`.
-   - Generates a concise summary and extracts key points using OpenAI (`gpt-3.5-turbo`).
+   - Generates a concise **summary**, **key points**, **tags**, and **highlights** using OpenAI (`gpt-4o`) with Pydantic-structured output.
 
-6. **`QualityScoringAgent` (`src/agents/quality_scoring_agent.py`)**
+6. **`QualityScoringAgent` (`src/agents/QualityScoringAgent.py`)**
    - Takes the clean text and evaluates it against a predefined rubric.
-   - Scores the transcript based on Tone, Professionalism, and Structured Resolution. Automatically applies a 3-point penalty to each score and logs a count of policy violations if the `ModerationAgent` detected and masked any profanity (`***`).
+   - Scores the transcript based on Tone, Professionalism, and Structured Resolution using Pydantic structured output (function calling when supported). Automatically applies a 3-point penalty to each score and logs a count of policy violations if the `ModerationAgent` detected and masked any profanity (`***`).
 
 ## Prerequisites
 
@@ -87,7 +98,7 @@ The core analysis logic is driven by a LangGraph StateGraph defined in `src/work
 
 1. Create a virtual environment and activate it (if you haven't already):
    ```bash
-   python -m venv .venv
+   python3 -m venv .venv
    source .venv/bin/activate
    ```
 2. Install the required dependencies:
@@ -104,9 +115,8 @@ The core analysis logic is driven by a LangGraph StateGraph defined in `src/work
 
 2. Start the Streamlit application:
    ```bash
-   streamlit run app.py
+   streamlit run src/streamlit_app.py
    ```
 
 3. Open your browser to the local URL provided by Streamlit (usually `http://localhost:8501`).
 4. Use the sidebar to upload a `.csv`, `.mp3`, or `.wav` file and watch the agents analyze your data!
-

requirements.txt

@@ -7,5 +7,6 @@ langchain
 langchain-openai
 openai
 openai-whisper
+pydantic
 python-dotenv
-ffmpeg-python
+ffmpeg-python

src/agents/CallState.py

@@ -1,4 +1,4 @@
-from typing import TypedDict, Optional, Dict, Any
+from typing import TypedDict, Optional, Dict, Any, List
 
 class CallState(TypedDict):
     file_path: str
@@ -7,5 +7,7 @@ class CallState(TypedDict):
     clean_content: Optional[str]
     metadata: Optional[Dict[str, Any]]
     summary: Optional[str]
-    key_points: Optional[str]
+    key_points: Optional[List[str]]
+    tags: Optional[List[str]]
+    highlights: Optional[List[str]]
     quality_scores: Optional[Dict[str, Any]]

src/agents/IntakeAgent.py

@@ -5,7 +5,7 @@ from src.agents.CallState import CallState
 
 class IntakeAgent:
     def __init__(self):
-        self.llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)
+        self.llm = ChatOpenAI(model="gpt-4o", temperature=0)
         
     def __call__(self, state: CallState) -> CallState:
         """Validates input formats, cleans up profanity, extracts meta."""
@@ -17,13 +17,30 @@ class IntakeAgent:
         if file_type == "csv":
             try:
                 df = pd.read_csv(file_path)
-                if "transcript" in df.columns:
-                    content = " ".join(df["transcript"].astype(str).tolist())
-                else:
-                    content = df.to_string()
-                state["content"] = content
+                normalized = {str(c).strip().lower(): c for c in df.columns}
+                required = {"id", "transcript"}
+                missing = sorted(required - set(normalized.keys()))
+                if missing:
+                    state["metadata"]["intake_error"] = (
+                        f"CSV is missing required headers: {', '.join(missing)}"
+                    )
+                    state["content"] = ""
+                    state["clean_content"] = ""
+                    return state
+
+                id_col = normalized["id"]
+                transcript_col = normalized["transcript"]
+                transcripts = df[transcript_col].fillna("").astype(str).tolist()
+                state["content"] = " ".join(transcripts).strip()
+                state["metadata"]["row_count"] = int(len(df))
+                state["metadata"]["ids_preview"] = (
+                    df[id_col].fillna("").astype(str).head(20).tolist()
+                )
             except Exception as e:
-                state["content"] = f"Error reading CSV: {e}"
+                state["metadata"]["intake_error"] = str(e)
+                state["content"] = ""
+                state["clean_content"] = ""
+                return state
 
         if state.get("content"):
             prompt = PromptTemplate.from_template(

src/agents/ModerationAgent.py

@@ -9,7 +9,7 @@ class ModerationAgent:
     """
 
     def __init__(self):
-        self.llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)
+        self.llm = ChatOpenAI(model="gpt-4o", temperature=0)
 
     def __call__(self, state: CallState) -> CallState:
         text = state.get("clean_content") or state.get("content") or ""

src/agents/QualityScoringAgent.py

@@ -1,11 +1,12 @@
-import json
 from langchain_openai import ChatOpenAI
 from langchain_core.prompts import PromptTemplate
 from src.agents.CallState import CallState
+from src.agents.schemas import QualityScores
+import inspect
 
 class QualityScoringAgent:
     def __init__(self):
-        self.llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)
+        self.llm = ChatOpenAI(model="gpt-4o", temperature=0)
         
     def __call__(self, state: CallState) -> CallState:
         """Evaluates tone, professionalism, and structured resolution with rubric."""
@@ -16,24 +17,57 @@ class QualityScoringAgent:
         prompt = PromptTemplate.from_template(
             "Evaluate the following call transcript for tone, professionalism, and structured resolution. "
             "Score each out of 10 based on a strict rubric.\n"
-            "Return as JSON string with keys 'tone', 'professionalism', 'structured_resolution', and 'notes'.\n\n"
+            "Return scores and brief notes.\n\n"
             "Transcript:\n{text}"
         )
-        
-        chain = prompt | self.llm
-        result_text = chain.invoke({"text": clean_text}).content
-        
-        try:
-            result_json = json.loads(result_text)
-            profanity_count = clean_text.count("***")
-            if profanity_count > 0:
-                result_json["profanity"] = profanity_count
-                for key in ["tone", "professionalism", "structured_resolution"]:
-                    if key in result_json and isinstance(result_json[key], (int, float)):
-                        result_json[key] = max(0, result_json[key] - 3)
-                        
-            state["quality_scores"] = result_json
-        except json.JSONDecodeError:
-            state["quality_scores"] = {"raw_evaluation": result_text}
+
+        structured_llm = self._structured_llm()
+        chain = prompt | structured_llm
+        result = chain.invoke({"text": clean_text})
+
+        if isinstance(result, QualityScores):
+            if hasattr(result, "model_dump"):
+                result_dict = result.model_dump()
+            else:
+                result_dict = result.dict()
+        else:
+            result_dict = dict(result or {})
+
+        profanity_count = clean_text.count("***")
+        if profanity_count > 0:
+            result_dict["profanity"] = profanity_count
+            for key in ["tone", "professionalism", "structured_resolution"]:
+                if key in result_dict and isinstance(result_dict[key], (int, float)):
+                    result_dict[key] = max(0, result_dict[key] - 3)
+
+        state["quality_scores"] = result_dict
             
         return state
+
+    def _structured_llm(self):
+        # Prefer OpenAI function calling enforcement when supported by the installed LangChain version.
+        sig = None
+        try:
+            sig = inspect.signature(self.llm.with_structured_output)
+        except Exception:
+            sig = None
+
+        if sig and "method" in sig.parameters:
+            try:
+                return self.llm.with_structured_output(
+                    QualityScores, method="function_calling"
+                )
+            except Exception:
+                pass
+
+        return self.llm.with_structured_output(QualityScores)
+
+    @staticmethod
+    def fallback(state: CallState) -> CallState:
+        state["quality_scores"] = {
+            "tone": None,
+            "professionalism": None,
+            "structured_resolution": None,
+            "notes": "Scoring failed (rate limit or parse error). Showing placeholders.",
+        }
+        return state

src/agents/Router.py

@@ -3,7 +3,22 @@ from src.agents.CallState import CallState
 class Router:
     def __call__(self, state: CallState) -> str:
         """Invokes above agents accordingly."""
+        if state.get("metadata", {}).get("intake_error"):
+            return "end"
         file_type = state["file_type"]
         if file_type in ["mp3", "wav"] and not state.get("content"):
             return "transcribe"
         return "summarize"
+
+
+class PostSummarizeRouter:
+    def __call__(self, state: CallState) -> str:
+        """Routes based on model output quality."""
+        if state.get("metadata", {}).get("intake_error"):
+            return "end"
+        text = state.get("clean_content") or state.get("content") or ""
+        if len(text.strip()) < 40:
+            return "end"
+        if not (state.get("summary") or "").strip():
+            return "end"
+        return "score"

src/agents/SummarizationAgent.py

@@ -1,11 +1,11 @@
-import json
 from langchain_openai import ChatOpenAI
 from langchain_core.prompts import PromptTemplate
 from src.agents.CallState import CallState
+from src.agents.schemas import CallSummary
 
 class SummarizationAgent:
     def __init__(self):
-        self.llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)
+        self.llm = ChatOpenAI(model="gpt-4o", temperature=0)
         
     def __call__(self, state: CallState) -> CallState:
         """Generates summaries and key points."""
@@ -15,19 +15,72 @@ class SummarizationAgent:
             
         prompt = PromptTemplate.from_template(
             "Summarize the following call transcript and extract key points.\n"
-            "Return as a JSON string with 'summary' and 'key_points' keys.\n\n"
+            "Return a concise summary, a short list of key points, 3-8 topic tags, and 2-6 highlights.\n\n"
             "Transcript:\n{text}"
         )
         
-        chain = prompt | self.llm
-        result_text = chain.invoke({"text": clean_text}).content
-        
-        try:
-            result_json = json.loads(result_text)
-            state["summary"] = result_json.get("summary", "")
-            state["key_points"] = result_json.get("key_points", "")
-        except json.JSONDecodeError:
-            state["summary"] = result_text
-            state["key_points"] = "Failed to parse JSON for key points."
+        structured_llm = self._structured_llm()
+        chain = prompt | structured_llm
+        result = chain.invoke({"text": clean_text})
+
+        if isinstance(result, CallSummary):
+            state["summary"] = result.summary
+            state["key_points"] = result.key_points
+            state["tags"] = result.tags
+            state["highlights"] = result.highlights
+        else:
+            state["summary"] = (result or {}).get("summary", "")
+            state["key_points"] = (result or {}).get("key_points", [])
+            state["tags"] = (result or {}).get("tags", [])
+            state["highlights"] = (result or {}).get("highlights", [])
             
         return state
+
+    def _structured_llm(self):
+        # Some LangChain versions support different output enforcement methods.
+        for kwargs in ({"method": "function_calling"}, {"method": "json_mode"}, {}):
+            try:
+                return self.llm.with_structured_output(CallSummary, **kwargs)
+            except TypeError:
+                continue
+        return self.llm.with_structured_output(CallSummary)
+
+    @staticmethod
+    def fallback(state: CallState) -> CallState:
+        text = state.get("clean_content") or state.get("content") or ""
+        if not text:
+            return state
+
+        # Best-effort retry using structured output with alternate enforcement methods.
+        llm = ChatOpenAI(model="gpt-4o", temperature=0)
+        prompt = PromptTemplate.from_template(
+            "Summarize the following call transcript.\n"
+            "Return: summary, key_points, tags, highlights.\n\n"
+            "Transcript:\n{text}"
+        )
+
+        for kwargs in ({"method": "function_calling"}, {"method": "json_mode"}, {}):
+            try:
+                structured_llm = llm.with_structured_output(CallSummary, **kwargs)
+                result = (prompt | structured_llm).invoke({"text": text})
+                if isinstance(result, CallSummary):
+                    state["summary"] = result.summary
+                    state["key_points"] = result.key_points
+                    state["tags"] = result.tags
+                    state["highlights"] = result.highlights
+                    return state
+                state["summary"] = (result or {}).get("summary", state.get("summary", ""))
+                state["key_points"] = (result or {}).get("key_points", state.get("key_points") or [])
+                state["tags"] = (result or {}).get("tags", state.get("tags") or [])
+                state["highlights"] = (result or {}).get("highlights", state.get("highlights") or [])
+                return state
+            except Exception:
+                continue
+
+        # Last resort heuristic fallback that avoids breaking the UI.
+        summary = (text.strip()[:800] + ("…" if len(text.strip()) > 800 else "")).strip()
+        state["summary"] = summary or state.get("summary", "")
+        state["key_points"] = state.get("key_points") or []
+        state["tags"] = state.get("tags") or []
+        state["highlights"] = state.get("highlights") or []
+        return state

src/agents/TranscriptionAgent.py

@@ -5,7 +5,7 @@ from src.agents.CallState import CallState
 
 class TranscriptionAgent:
     def __init__(self):
-        self.llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)
+        self.llm = ChatOpenAI(model="gpt-4o", temperature=0)
         
     def __call__(self, state: CallState) -> CallState:
         """Converts audio to text using whisper."""

src/streamlit_app.py

@@ -45,6 +45,32 @@ def apply_material_css():
             box-shadow: 0 4px 6px rgba(0,0,0,0.3);
         }
 
+        /* Expander (Transcript) - keep dark theme even on hover/focus */
+        div[data-testid="stExpander"] details,
+        div[data-testid="stExpander"] summary,
+        div[data-testid="stExpander"] summary:hover,
+        div[data-testid="stExpander"] summary:focus,
+        div[data-testid="stExpander"] summary:active,
+        div[data-testid="stExpander"] summary:focus-visible {
+            background-color: #1E1E1E !important;
+            color: #FFFFFF !important;
+        }
+
+        div[data-testid="stExpander"] details {
+            border: 1px solid #333333 !important;
+            border-radius: 8px !important;
+        }
+
+        /* Transcript text area */
+        div[data-testid="stExpander"] textarea,
+        div[data-testid="stExpander"] textarea:hover,
+        div[data-testid="stExpander"] textarea:focus,
+        div[data-testid="stExpander"] textarea:active {
+            background-color: #121212 !important;
+            color: #FFFFFF !important;
+            border-color: #333333 !important;
+        }
+
         </style>
     """, unsafe_allow_html=True)
 
@@ -52,6 +78,23 @@ def apply_material_css():
 def display_results(final_state):
     st.subheader("Workflow Results")
 
+    metadata = final_state.get("metadata", {})
+    if isinstance(metadata, dict) and metadata.get("intake_error"):
+        st.error(f"CSV validation failed: {metadata['intake_error']}")
+        st.info("This CSV was not accepted for processing. Please fix the headers and re-upload.")
+        st.markdown("### Metadata")
+        for k, v in metadata.items():
+            st.markdown(f"- **{k.replace('_', ' ').title()}**: {v}")
+        return
+
+    st.markdown("### Transcript")
+    transcript_text = final_state.get("clean_content") or final_state.get("content") or ""
+    if transcript_text:
+        with st.expander("View transcript", expanded=False):
+            st.text_area("Transcript", transcript_text, height=220)
+    else:
+        st.write("No transcript available.")
+
     col1, col2 = st.columns(2)
     with col1:
         st.markdown("### Summary")
@@ -65,6 +108,20 @@ def display_results(final_state):
         else:
             st.write(key_points)
 
+        st.markdown("### Tags / Highlights")
+        tags = final_state.get("tags") or []
+        highlights = final_state.get("highlights") or []
+        if tags:
+            st.write("**Tags:** " + ", ".join([str(t) for t in tags]))
+        else:
+            st.write("**Tags:** None")
+        if highlights:
+            st.write("**Highlights:**")
+            for h in highlights:
+                st.markdown(f"- {h}")
+        else:
+            st.write("**Highlights:** None")
+
     with col2:
         st.markdown("### Quality Scores")
         quality_scores = final_state.get("quality_scores", {})
@@ -98,7 +155,7 @@ def display_results(final_state):
                             plot_bgcolor='#1E1E1E',
                             font={'color': '#FFFFFF'}
                         )
-                        st.plotly_chart(fig, use_container_width=True)
+                        st.plotly_chart(fig, width="stretch")
                     except (ValueError, TypeError):
                         st.write(f"**{metric.replace('_', ' ').title()}**: {quality_scores[metric]}")
 
@@ -108,7 +165,6 @@ def display_results(final_state):
             st.write("No specific scores generated.", quality_scores)
 
         st.markdown("### Metadata")
-        metadata = final_state.get("metadata", {})
         if isinstance(metadata, dict) and metadata:
             for k, v in metadata.items():
                 st.markdown(f"- **{k.replace('_', ' ').title()}**: {v}")
@@ -137,7 +193,7 @@ def main():
     st.sidebar.header("Upload New File")
     uploaded_file = st.sidebar.file_uploader(
         "Upload a file",
-        type=["json", "mp3", "wav", "csv"],
+        type=["mp3", "wav", "csv"],
         on_change=set_upload_mode
     )
 
@@ -191,15 +247,21 @@ def main():
                     "file_type": file_extension
                 }
 
-                final_state = workflow.invoke(initial_state)
+                thread_id = Path(file_path).name
+                final_state = workflow.invoke(
+                    initial_state, config={"configurable": {"thread_id": thread_id}}
+                )
 
-                # Cache the results
-                with open(cached_json_path, 'w') as f:
-                    json.dump(final_state, f)
+                if final_state.get("metadata", {}).get("intake_error"):
+                    display_results(final_state)
+                else:
+                    # Cache the results
+                    with open(cached_json_path, 'w') as f:
+                        json.dump(final_state, f)
+                    st.rerun()
 
-                st.rerun()
-
-            display_results(final_state)
+            if not final_state.get("metadata", {}).get("intake_error"):
+                display_results(final_state)
 
     elif (
             st.session_state.view_mode == "dropdown" or st.session_state.view_mode == "none") and selected_file is not None:
@@ -235,11 +297,18 @@ def main():
                     "file_path": file_path,
                     "file_type": file_extension
                 }
-                final_state = workflow.invoke(initial_state)
-                with open(cached_json_path, 'w') as f:
-                    json.dump(final_state, f)
-                st.rerun()
-            display_results(final_state)
+                thread_id = Path(file_path).name
+                final_state = workflow.invoke(
+                    initial_state, config={"configurable": {"thread_id": thread_id}}
+                )
+                if final_state.get("metadata", {}).get("intake_error"):
+                    display_results(final_state)
+                else:
+                    with open(cached_json_path, 'w') as f:
+                        json.dump(final_state, f)
+                    st.rerun()
+            if not final_state.get("metadata", {}).get("intake_error"):
+                display_results(final_state)
 
     else:
         st.info("Please upload a file or select a previously processed file.")

src/workflow.py

@@ -1,4 +1,5 @@
 from langgraph.graph import StateGraph, END
+from langchain_core.runnables import RunnableLambda
 from src.agents import (
     CallState,
     IntakeAgent,
@@ -8,6 +9,7 @@ from src.agents import (
     ModerationAgent,
     Router
 )
+from src.agents.Router import PostSummarizeRouter
 
 def build_workflow():
     workflow = StateGraph(CallState)
@@ -19,13 +21,20 @@ def build_workflow():
     summarization_agent = SummarizationAgent()
     quality_scoring_agent = QualityScoringAgent()
     router = Router()
+    post_summarize_router = PostSummarizeRouter()
     
     # Add nodes
     workflow.add_node("intake", intake_agent)
     workflow.add_node("transcribe", transcription_agent)
     workflow.add_node("moderate", moderation_agent)
-    workflow.add_node("summarize", summarization_agent)
-    workflow.add_node("score", quality_scoring_agent)
+    summarize_node = RunnableLambda(summarization_agent).with_fallbacks(
+        [RunnableLambda(SummarizationAgent.fallback)]
+    )
+    score_node = RunnableLambda(quality_scoring_agent).with_fallbacks(
+        [RunnableLambda(QualityScoringAgent.fallback)]
+    )
+    workflow.add_node("summarize", summarize_node)
+    workflow.add_node("score", score_node)
     
     # Define entry point
     workflow.set_entry_point("intake")
@@ -36,14 +45,25 @@ def build_workflow():
         router,
         {
             "transcribe": "transcribe",
-            "summarize": "moderate"
+            "summarize": "moderate",
+            "end": END
         }
     )
     
     # Add standard edges
     workflow.add_edge("transcribe", "moderate")
     workflow.add_edge("moderate", "summarize")
-    workflow.add_edge("summarize", "score")
+    workflow.add_conditional_edges(
+        "summarize",
+        post_summarize_router,
+        {"score": "score", "end": END},
+    )
     workflow.add_edge("score", END)
-    
-    return workflow.compile()
+
+    # Compile with a MemorySaver checkpointer when available.
+    try:
+        from langgraph.checkpoint.memory import MemorySaver
+
+        return workflow.compile(checkpointer=MemorySaver())
+    except Exception:
+        return workflow.compile()