TASKS.md

TASKS — Conversational Analytics Assistant (chat-service)

Step-by-step build checklist, derived from the implementation plan. Check items off as you go. Order matters — each milestone assumes the previous one works.

---

Milestone 0 — Repo & environment setup

• Create chat-service/ folder structure
• git init, set origin remote
• Create virtual environment (python -m venv venv)
• Activate venv
• Create .env.example with all vars from plan §10 (placeholders)
• Create local .env (real values, gitignored)
• Add .gitignore (venv, .env, __pycache__, *.pyc)
• Initial commit ("chore: project skeleton")

---

Milestone 1 — Service skeleton (boots + Groq round-trip)

• Add core deps to requirements.txt: fastapi, uvicorn[standard], pydantic-settings, groq, python-dotenv
• pip install -r requirements.txt
• app/config.py — Settings(BaseSettings) class reading env vars
• app/agent/llm.py — thin Groq client wrapper (single ask(prompt: str) -> str function)
• app/main.py:
  • GET /health → {"status": "ok"}
  • Temporary POST /chat → calls Groq wrapper directly, no LangGraph yet, just to prove the chain works
• Run locally: uvicorn app.main:app --reload --port 8002
• Test /health with curl/Postman
• Test /chat with curl/Postman — confirm real Groq response comes back
• Write Dockerfile (Python 3.12 base, matches existing services)
• Build + run container locally, re-test /health and /chat inside Docker
• Commit ("feat: service skeleton with Groq round-trip")

Done when: /health and /chat both work locally and in Docker.

---

Milestone 2 — First real tool + LangGraph agent (sql_query_tool)

• Add deps: langgraph, langchain-groq, sqlalchemy, psycopg[binary] (or asyncpg)
• Create read-only Postgres role (chat_ro_user) — DB side, not app code
• app/db/postgres.py — read-only connection pool using POSTGRES_READONLY_URL
• Confirm exact schema fields needed (resolve plan §12.1: ArticleFakeNews, ArticleTopic, etc. field names)
• app/tools/sql_tool.py:
  • Define metric enum: fake_news_count, articles_by_topic, sentiment_breakdown, propaganda_count, hate_speech_count, article_lookup, dialect_breakdown
  • Write one parametrized SQL query per metric
  • Return shape: {"rows": [...], "sql_used": "...", "source_refs": [...]}
• Write unit tests for each metric in tests/test_sql_tool.py (against a test/staging DB)
• app/agent/state.py — AgentState TypedDict (messages, sources, session_id, user_id)
• app/agent/prompts.py — system prompt (identity, tool-preference rule, citation rule, language-matching rule, no-fabrication rule)
• app/agent/graph.py — LangGraph ReAct-style agent wired to sql_query_tool only
• Wire /chat in main.py to call the LangGraph agent instead of the raw Groq wrapper
• app/schemas/request.py / app/schemas/response.py — formalize ChatRequest / ChatResponse models per plan §4
• Manually test: "How many fake news articles were detected last month?" → correct tool call + correct answer
• Commit ("feat: sql_query_tool + LangGraph agent")

Done when: agent correctly answers at least 3 different stat questions using real DB data, citing article IDs.

---

Milestone 3 — Django proxy endpoint

• In Django: add CHAT_SERVICE_URL and CHAT_SERVICE_INTERNAL_TOKEN to settings/env
• New view: POST /api/v1/chat/
  • Verify JWT (reuse existing auth)
  • Apply DRF throttling/rate limit (no need for now)
  • Forward {session_id, message, user_id} to chat service with internal token header
  • Return chat service's JSON response unchanged
• Confirm chat service rejects requests without the internal token
• Test end-to-end: frontend-style request → Django → chat service → Groq → back
• Commit on Django repo ("feat: chat proxy endpoint")

Done when: a JWT-authenticated request through Django reaches the chat service and gets a real answer.

---

Milestone 4 — graph_query_tool (Neo4j)

• Confirm current Neo4j label/relationship conventions used by kg_sync (resolve plan §12.2)
• Create read-only Neo4j role/user — N/A, Aura Free has no RBAC; enforcement moved to app-level (read-only transactions + no write Cypher in tool code)
• app/db/neo4j.py — driver wrapper using NEO4J_* vars, enforcing read-only via explicit read transactions
• app/tools/graph_tool.py:
  • Define query_type enum (7 total): entity_connections, entity_mentions, shared_entities_between_articles, most_connected_entities, article_verdict, claims_for_article, analysis_for_article
  • Write one parametrized Cypher template per query_type
  • Return shape: {"rows": [...], "source_refs": [...]}
• Add graph_query_tool to the LangGraph agent's tool list
• Unit tests in tests/test_graph_tool.py
• Manually test: "Who is connected to [entity]?" → correct Cypher template used, correct answer
• Commit ("feat: graph_query_tool")

Done when: agent correctly answers at least 2 relationship questions using real graph data.

---

Milestone 5 — Memory (multi-turn)

• Add dep: Redis client (redis / redis[hiredis])
• app/memory/redis_checkpointer.py — LangGraph checkpointer backed by Redis, key prefix chat:checkpoint:{session_id}
• Wire checkpointer into app/agent/graph.py compilation
• Set CHAT_SESSION_TTL_SECONDS TTL on session keys
• Implement history trimming: keep last CHAT_HISTORY_MAX_TURNS verbatim, summarize older turns beyond that
• Manual test: ask a question, then a follow-up ("and what about last month?") using the same session_id — confirm context carries over
• Manual test: new session_id → confirm no memory leaks across sessions
• Commit ("feat: Redis-backed multi-turn memory")

Done when: follow-up questions correctly resolve using prior turn context, and TTL/trimming work as expected.

---

Milestone 6 — Citations end-to-end

• app/tools/article_tool.py — get_article_detail(article_id) for resolving full title/snippet
• Confirm every tool (sql_query_tool, graph_query_tool) consistently returns source_refs
• Agent post-processing step: merge all source_refs collected during the turn into final sources[] in the response
• Update system prompt to explicitly require citing every ID it used
• Manual test: ask a question, verify sources[] in the JSON response matches what was actually used
• Commit ("feat: citation resolution")

Done when: every factual answer includes a non-empty, accurate sources[] array.

---

Milestone 7 — hybrid_search_tool

• Confirm reuse path for _embed (call existing Django/analysis embedding function - the function is in app/services/nlp_client.py)
• app/tools/hybrid_tool.py:
  • Step 1: embed query text
  • Step 2: pgvector cosine similarity search over search_vector
  • Step 3: for each top article, pull mentioned entities from Neo4j
  • Step 4: merge into single ranked result set with combined source_refs
• Add tool to agent's tool list
• Manual test: "Articles about the election, and who's mentioned" → correct combined results
• Commit ("feat: hybrid_search_tool")

Done when: at least one combined semantic+graph question is answered correctly with merged citations.

---

Milestone 8 — Frontend chat widget

• Generate/persist session_id client-side (e.g. localStorage, created on first load)
• Chat UI component: message list, input box, send button
• Call POST /api/v1/chat/ with {session_id, message}
• Render answer text
• Render sources[] as clickable links (article/entity)
• Loading state while waiting for response
• Basic error state (service down / timeout)
• Commit ("feat: chat widget")

Done when: a real user can open the widget, ask a question, get an answer with clickable citations, and ask a follow-up.

---

Milestone 9 — Tests, docs, hardening

• Unit tests for all tools (sql_tool, graph_tool, hybrid_tool) with mocked/test DB data
• tests/test_agent_e2e.py — at least 3 end-to-end scenarios (stat question, relationship question, follow-up question)
• chat-service/README.md — setup instructions, env vars, how to run locally + in Docker
• Verify .env.example is complete and matches plan §10
• Security checklist review (plan §8) — confirm every box is actually true in the running system:
  • Read-only DB roles confirmed (test that a write attempt fails)
  • No raw SQL/Cypher ever comes from the LLM — code review confirms only enum params reach tools
  • Internal token required and enforced
  • Rate limiting active on Django proxy
  • Groq key not logged anywhere
  • Tool calls logged (name + params, not raw rows)
• Add chat_service to docker-compose.yml (no published host port in prod config)
• Final walkthrough / demo run-through for defense
• Tag/commit ("chore: v1 complete")

Done when: everything above is checked and you can demo the full flow live without surprises.

---

Open items to resolve before/while coding (carry over from plan §12)

• Confirm exact Postgres field names per metric
• Confirm Neo4j label/relationship naming conventions
• Decide: authenticated-only sessions, or also support anonymous/demo sessions for defense
• Decide: citation links deep-link into real frontend routes, or raw IDs for now