Add Scoring Rubric

README.md

@@ -16,7 +16,7 @@ args: ["--server.enableCORS", "false", "--server.enableXsrfProtection", "false"]
 ---
 # Call Center Data Analysis Agent
 
-This project implements a multi-agent workflow using **LangGraph** to process, transcribe, summarize, and score call center data. It comes with a **Streamlit** user interface to easily upload `.csv`, `.mp3`, or `.wav` files and view the resulting insights.
+This project implements a multi-agent workflow using **LangGraph** to process, transcribe, summarize, and score call center data. It comes with a **Streamlit** user interface to easily upload `.csv`, `.json`, `.mp3`, or `.wav` files and view the resulting insights.
 
 ## Workflow Flow & Agent Classes
 
@@ -57,8 +57,9 @@ Notes:
 
 1. **`IntakeAgent` (`src/agents/IntakeAgent.py`)**
    - *Entry Point*.
-   - Reads the uploaded file, validates the file format and CSV schema, extracts basic metadata, and runs a first-pass clean-up (using an LLM).
+   - Reads the uploaded file, validates the file format and schema, extracts basic metadata, and runs a first-pass clean-up (using an LLM).
    - **CSV requirement:** headers must include `id` and `transcript` (case-insensitive).
+   - **JSON supported shapes:** a list of `{id, transcript}` objects, a single `{id, transcript}` object, or a dict with `transcripts`/`calls` arrays containing `{id, transcript}` objects.
 
 2. **`Router` (`src/agents/Router.py`)**
    - *Conditional Routing Node*.
@@ -80,7 +81,7 @@ Notes:
 
 5. **`SummarizationAgent` (`src/agents/SummarizationAgent.py`)**
    - Takes the redacted text from the `ModerationAgent`.
-   - Generates a concise **summary**, **key points**, **tags**, and **highlights** using OpenAI (`gpt-4o`) with Pydantic-structured output.
+   - Generates a concise **summary**, **key points**, **action items**, **tags**, and **highlights** using OpenAI (`gpt-4o`) with Pydantic-structured output.
 
 6. **`QualityScoringAgent` (`src/agents/QualityScoringAgent.py`)**
    - Takes the clean text and evaluates it against a predefined rubric.

sample_data.json

@@ -0,0 +1,13 @@
+  [
+    {
+      "id": "call-001",
+      "transcript": "Agent: Thanks for calling Acme Support. How can I help today?\nCustomer: My internet has been dropping every evening.\nAgent: I’m sorry about that. Let’s run a quick modem reset and check outages in your area.
+  \nCustomer: Okay.\nAgent: I see intermittent signal loss. I’ll schedule a technician for tomorrow between 2–4pm and apply a service credit request.\nCustomer: Thank you.\nAgent: Before we wrap up, can you confirm the best phone
+  number for the tech to call?\nCustomer: 555-0101.\nAgent: Great — you’ll get a confirmation text shortly."
+    },
+    {
+      "id": "call-002",
+      "transcript": "Agent: Hi! I can help with billing.\nCustomer: I was charged twice for last month.\nAgent: I understand. I’m reviewing the account now.\nCustomer: I’m frustrated because this happened before.\nAgent: I’m
+  sorry. I see two identical charges; I’ll void the duplicate and email you a confirmation within 10 minutes.\nCustomer: Please do.\nAgent: Also, I’ll add a note to prevent a recurring duplicate payment."
+    }
+  ]

src/agents/CallState.py

@@ -8,6 +8,7 @@ class CallState(TypedDict):
     metadata: Optional[Dict[str, Any]]
     summary: Optional[str]
     key_points: Optional[List[str]]
+    action_items: Optional[List[str]]
     tags: Optional[List[str]]
     highlights: Optional[List[str]]
     quality_scores: Optional[Dict[str, Any]]

src/agents/IntakeAgent.py

@@ -1,4 +1,5 @@
 import pandas as pd
+import json
 from langchain_openai import ChatOpenAI
 from langchain_core.prompts import PromptTemplate
 from src.agents.CallState import CallState
@@ -42,6 +43,111 @@ class IntakeAgent:
                 state["clean_content"] = ""
                 return state
 
+        if file_type == "json":
+            try:
+                with open(file_path, "r", encoding="utf-8") as f:
+                    raw = f.read()
+
+                try:
+                    payload = json.loads(raw)
+                except json.JSONDecodeError:
+                    # Common failure mode: users paste multi-line transcripts with raw control characters
+                    # (e.g., literal newlines) inside JSON strings. JSON requires these to be escaped.
+                    def _sanitize_control_chars_in_strings(text: str) -> str:
+                        out: list[str] = []
+                        in_string = False
+                        escape = False
+                        for ch in text:
+                            if not in_string:
+                                out.append(ch)
+                                if ch == '"':
+                                    in_string = True
+                                continue
+
+                            # Inside string
+                            if escape:
+                                out.append(ch)
+                                escape = False
+                                continue
+                            if ch == "\\":
+                                out.append(ch)
+                                escape = True
+                                continue
+                            if ch == '"':
+                                out.append(ch)
+                                in_string = False
+                                continue
+
+                            code = ord(ch)
+                            if ch == "\n":
+                                out.append("\\n")
+                            elif ch == "\r":
+                                out.append("\\r")
+                            elif ch == "\t":
+                                out.append("\\t")
+                            elif code < 0x20:
+                                out.append(f"\\u{code:04x}")
+                            else:
+                                out.append(ch)
+                        return "".join(out)
+
+                    payload = json.loads(_sanitize_control_chars_in_strings(raw))
+
+                transcripts: list[str] = []
+                ids_preview: list[str] = []
+
+                def add_item(item: object) -> None:
+                    if not isinstance(item, dict):
+                        return
+                    transcript = item.get("transcript")
+                    if isinstance(transcript, str) and transcript.strip():
+                        transcripts.append(transcript.strip())
+                    item_id = item.get("id")
+                    if item_id is not None and len(ids_preview) < 20:
+                        ids_preview.append(str(item_id))
+
+                if isinstance(payload, list):
+                    for item in payload:
+                        add_item(item)
+                elif isinstance(payload, dict):
+                    if isinstance(payload.get("transcript"), str):
+                        add_item(payload)
+                    elif isinstance(payload.get("transcripts"), list):
+                        for item in payload["transcripts"]:
+                            add_item(item)
+                    elif isinstance(payload.get("calls"), list):
+                        for item in payload["calls"]:
+                            add_item(item)
+                    else:
+                        state["metadata"]["intake_error"] = (
+                            "JSON must be either a list of {id, transcript} objects, "
+                            "a single {id, transcript} object, or a dict with 'transcripts'/'calls' arrays."
+                        )
+                        state["content"] = ""
+                        state["clean_content"] = ""
+                        return state
+                else:
+                    state["metadata"]["intake_error"] = "Unsupported JSON root type."
+                    state["content"] = ""
+                    state["clean_content"] = ""
+                    return state
+
+                if not transcripts:
+                    state["metadata"]["intake_error"] = "JSON contained no non-empty 'transcript' fields."
+                    state["content"] = ""
+                    state["clean_content"] = ""
+                    return state
+
+                state["content"] = " ".join(transcripts).strip()
+                state["metadata"]["row_count"] = int(len(transcripts))
+                if ids_preview:
+                    state["metadata"]["ids_preview"] = ids_preview
+            except Exception as e:
+                state["metadata"]["intake_error"] = str(e)
+                state["content"] = ""
+                state["clean_content"] = ""
+                return state
+
         if state.get("content"):
             prompt = PromptTemplate.from_template(
                 "Clean the following text of any profanity and fix basic grammatical errors. Return only the clean text:\n{text}"

src/agents/QualityScoringAgent.py

@@ -5,6 +5,15 @@ from src.agents.schemas import QualityScores
 import inspect
 
 class QualityScoringAgent:
+    RUBRIC_VERSION = "v1"
+    RUBRIC_TEXT = (
+        "Scoring rubric (0–10 each):\n"
+        "- Tone: 0 hostile/arguing; 3 curt/tense; 5 neutral; 7 friendly/empathic; 10 consistently calm, respectful, and de-escalating.\n"
+        "- Professionalism: 0 rude/unprofessional; 3 unclear or dismissive; 5 acceptable; 7 clear, courteous, policy-aligned; 10 excellent clarity, appropriate boundaries, and ownership.\n"
+        "- Structured resolution: 0 no attempt; 3 vague/no next steps; 5 partial (some questions/steps); 7 clear diagnosis + next steps + confirmation; 10 fully structured (issue, actions, timelines, confirmation, and closure).\n"
+        "Notes must cite 1–3 specific behaviors from the transcript (no long quotes)."
+    )
+
     def __init__(self):
         self.llm = ChatOpenAI(model="gpt-4o", temperature=0)
         
@@ -15,15 +24,15 @@ class QualityScoringAgent:
             return state
             
         prompt = PromptTemplate.from_template(
-            "Evaluate the following call transcript for tone, professionalism, and structured resolution. "
-            "Score each out of 10 based on a strict rubric.\n"
+            "Evaluate the following call transcript for tone, professionalism, and structured resolution.\n\n"
+            "{rubric}\n\n"
             "Return scores and brief notes.\n\n"
             "Transcript:\n{text}"
         )
 
         structured_llm = self._structured_llm()
         chain = prompt | structured_llm
-        result = chain.invoke({"text": clean_text})
+        result = chain.invoke({"text": clean_text, "rubric": self.RUBRIC_TEXT})
 
         if isinstance(result, QualityScores):
             if hasattr(result, "model_dump"):
@@ -33,6 +42,9 @@ class QualityScoringAgent:
         else:
             result_dict = dict(result or {})
 
+        result_dict["rubric_version"] = self.RUBRIC_VERSION
+        result_dict["rubric"] = self.RUBRIC_TEXT
+
         profanity_count = clean_text.count("***")
         if profanity_count > 0:
             result_dict["profanity"] = profanity_count
@@ -40,6 +52,10 @@ class QualityScoringAgent:
                 if key in result_dict and isinstance(result_dict[key], (int, float)):
                     result_dict[key] = max(0, result_dict[key] - 3)
 
+        if state.get("metadata") is None or not isinstance(state.get("metadata"), dict):
+            state["metadata"] = {}
+        state["metadata"]["qa_rubric_version"] = self.RUBRIC_VERSION
+
         state["quality_scores"] = result_dict
             
         return state

src/agents/SummarizationAgent.py

@@ -15,7 +15,8 @@ class SummarizationAgent:
             
         prompt = PromptTemplate.from_template(
             "Summarize the following call transcript and extract key points.\n"
-            "Return a concise summary, a short list of key points, 3-8 topic tags, and 2-6 highlights.\n\n"
+            "Return a concise summary, a short list of key points, 2-8 action items, 3-8 topic tags, and 2-6 highlights.\n"
+            "Action items must be concrete follow-ups; include the owner (Agent/Customer) when you can.\n\n"
             "Transcript:\n{text}"
         )
         
@@ -26,11 +27,13 @@ class SummarizationAgent:
         if isinstance(result, CallSummary):
             state["summary"] = result.summary
             state["key_points"] = result.key_points
+            state["action_items"] = result.action_items
             state["tags"] = result.tags
             state["highlights"] = result.highlights
         else:
             state["summary"] = (result or {}).get("summary", "")
             state["key_points"] = (result or {}).get("key_points", [])
+            state["action_items"] = (result or {}).get("action_items", [])
             state["tags"] = (result or {}).get("tags", [])
             state["highlights"] = (result or {}).get("highlights", [])
             
@@ -55,7 +58,7 @@ class SummarizationAgent:
         llm = ChatOpenAI(model="gpt-4o", temperature=0)
         prompt = PromptTemplate.from_template(
             "Summarize the following call transcript.\n"
-            "Return: summary, key_points, tags, highlights.\n\n"
+            "Return: summary, key_points, action_items, tags, highlights.\n\n"
             "Transcript:\n{text}"
         )
 
@@ -66,11 +69,13 @@ class SummarizationAgent:
                 if isinstance(result, CallSummary):
                     state["summary"] = result.summary
                     state["key_points"] = result.key_points
+                    state["action_items"] = result.action_items
                     state["tags"] = result.tags
                     state["highlights"] = result.highlights
                     return state
                 state["summary"] = (result or {}).get("summary", state.get("summary", ""))
                 state["key_points"] = (result or {}).get("key_points", state.get("key_points") or [])
+                state["action_items"] = (result or {}).get("action_items", state.get("action_items") or [])
                 state["tags"] = (result or {}).get("tags", state.get("tags") or [])
                 state["highlights"] = (result or {}).get("highlights", state.get("highlights") or [])
                 return state
@@ -81,6 +86,7 @@ class SummarizationAgent:
         summary = (text.strip()[:800] + ("…" if len(text.strip()) > 800 else "")).strip()
         state["summary"] = summary or state.get("summary", "")
         state["key_points"] = state.get("key_points") or []
+        state["action_items"] = state.get("action_items") or []
         state["tags"] = state.get("tags") or []
         state["highlights"] = state.get("highlights") or []
         return state

src/agents/schemas.py

@@ -11,6 +11,10 @@ class CallSummary(BaseModel):
         default_factory=list,
         description="A short list of the most important takeaways from the call.",
     )
+    action_items: List[str] = Field(
+        default_factory=list,
+        description="Concrete follow-up actions with an owner when possible.",
+    )
     tags: List[str] = Field(
         default_factory=list,
         description="Short topic tags (e.g., billing, outage, refund).",

src/streamlit_app.py

@@ -108,6 +108,17 @@ def display_results(final_state):
         else:
             st.write(key_points)
 
+        st.markdown("### Action Items")
+        action_items = final_state.get("action_items", "No action items generated.")
+        if isinstance(action_items, list):
+            if action_items:
+                for item in action_items:
+                    st.markdown(f"- {item}")
+            else:
+                st.write("No action items generated.")
+        else:
+            st.write(action_items)
+
         st.markdown("### Tags / Highlights")
         tags = final_state.get("tags") or []
         highlights = final_state.get("highlights") or []
@@ -123,7 +134,7 @@ def display_results(final_state):
             st.write("**Highlights:** None")
 
     with col2:
-        st.markdown("### Quality Scores")
+        st.markdown("### Scoring Rubric")
         quality_scores = final_state.get("quality_scores", {})
         if isinstance(quality_scores, dict) and quality_scores:
             import plotly.graph_objects as go
@@ -161,8 +172,46 @@ def display_results(final_state):
 
             if "notes" in quality_scores:
                 st.write("**Notes:**", quality_scores["notes"])
+
+            if "rubric" in quality_scores and quality_scores["rubric"]:
+                with st.expander("View scoring rubric", expanded=False):
+                    rubric_rows = [
+                        {
+                            "Dimension": "Tone",
+                            "0": "Hostile/arguing",
+                            "3": "Curt/tense",
+                            "5": "Neutral",
+                            "7": "Friendly/empathic",
+                            "10": "Consistently calm, respectful, de-escalating",
+                        },
+                        {
+                            "Dimension": "Professionalism",
+                            "0": "Rude/unprofessional",
+                            "3": "Unclear or dismissive",
+                            "5": "Acceptable",
+                            "7": "Clear, courteous, policy-aligned",
+                            "10": "Excellent clarity, appropriate boundaries, ownership",
+                        },
+                        {
+                            "Dimension": "Structured resolution",
+                            "0": "No attempt",
+                            "3": "Vague / no next steps",
+                            "5": "Partial (some questions/steps)",
+                            "7": "Clear diagnosis + next steps + confirmation",
+                            "10": "Fully structured (issue, actions, timelines, confirmation, closure)",
+                        },
+                    ]
+
+                    st.dataframe(
+                        rubric_rows,
+                        hide_index=True,
+                        use_container_width=True,
+                    )
+                    st.caption(
+                        "Notes must cite 1–3 specific behaviors from the transcript (avoid long quotes)."
+                    )
         else:
-            st.write("No specific scores generated.", quality_scores)
+            st.write("No scoring rubric results generated.", quality_scores)
 
         st.markdown("### Metadata")
         if isinstance(metadata, dict) and metadata:
@@ -193,7 +242,7 @@ def main():
     st.sidebar.header("Upload New File")
     uploaded_file = st.sidebar.file_uploader(
         "Upload a file",
-        type=["mp3", "wav", "csv"],
+        type=["mp3", "wav", "csv", "json"],
         on_change=set_upload_mode
     )