README.md

---
title: Sherlock Project Assistant
emoji: 🔍
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: "4.44.0"
app_file: app.py
pinned: false
license: cc-by-nc-sa-4.0
---

<div align="center">
  <img src="assets/logo-transparent-bg.png" alt="Sherlock Logo" width="120">
  <h1>Sherlock Project Assistant</h1>
  <p><em>Your intelligent assistant for managing multiple projects through meeting summaries</em></p>

  [![License: CC BY-NC-SA 4.0](https://img.shields.io/badge/License-CC%20BY--NC--SA%204.0-lightgrey.svg)](https://creativecommons.org/licenses/by-nc-sa/4.0/)
  [![Python Version](https://img.shields.io/badge/python-3.10-blue.svg)](https://github.com/sebasmos/sherlock)
  [![Hugging Face Space](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Space-blue)](https://huggingface.co/spaces/sebasmos/sherlock-project-assistant)
</div>

---

An AI-powered project assistant that helps you manage and query your project meetings and documentation.

## Table of Contents

- [Demo](#demo)
- [Features](#features)
- [System Architecture](#system-architecture)
- [Tech Stack](#tech-stack)
- [Agentic Capabilities](#agentic-capabilities)
- [Quick Start](#quick-start)
- [Usage](#usage)
- [LLM Providers](#llm-providers)
- [Observability](#observability-langsmith)
- [Performance Features](#performance-features)
- [Agent Evaluation](#agent-evaluation)
- [Testing](#testing)
- [Project Structure](#project-structure)
- [Unsorted To-Dos](#unsorted-to-dos)

## Demo

🚀 [Try the live demo here](https://huggingface.co/spaces/sebasmos/sherlock-project-assistant)

## Features

- **RAG-powered Q&A** - Ask questions about your projects using ChromaDB vector search
- **LangGraph AI Agent** - Intelligent query routing for action items, blockers, and status
- **Multi-project support** - Manage and filter across multiple projects
- **Meeting structuring** - Upload raw notes and get AI-structured markdown
- **Action item tracking** - Track open/completed tasks with assignees and deadlines
- **Blocker & decision tracking** - Surface blockers and key decisions from meetings
- **Multiple LLM Providers** - Choose between HuggingFace (free) or Google Gemini (paid)
- **Meeting summary generation** - Get comprehensive summaries with key takeaways
- **Trend analysis** - Analyze patterns across meetings: recurring topics, blocker trends, progress
- **LangSmith Observability** - Trace LLM calls, monitor latency, token usage, and errors
- **Agent Evaluation** - Automated quality testing with keyword matching and latency metrics
- **Streaming Responses** - See LLM output in real-time as it generates (better UX)
- **Response Caching** - Faster repeated queries with 5-minute TTL cache (lower API costs)
- **Export Chat** - Download your conversation history as PDF

## System Architecture

![fig](assets/arch.png)

## Tech Stack

| Category | Technology | Purpose |
|----------|------------|---------|
| **Frontend** | Gradio 4.44 | Web UI framework |
| **LLM Framework** | LangChain | LLM orchestration |
| **Agent Framework** | LangGraph | State machine for agent routing |
| **Vector Store** | ChromaDB | Persistent vector storage |
| **Embeddings** | Sentence Transformers | Text embeddings (all-MiniLM-L6-v2) |
| **LLM (Free)** | HuggingFace Inference | Llama 3.2 3B Instruct |
| **LLM (Paid)** | Google Generative AI | Gemini 2.5 Flash Lite |
| **Data Models** | Pydantic | Data validation |
| **Testing** | Pytest | Unit and integration tests |
| **Observability** | LangSmith | LLM tracing and monitoring |

## Agentic Capabilities

| Capability | Description | Trigger Keywords |
|------------|-------------|------------------|
| **Query Analysis** | Understands user intent and extracts project context | All queries |
| **Context Retrieval** | Semantic search across meeting notes | All queries |
| **Action Item Extraction** | Surfaces open tasks with assignees and deadlines | "action item", "todo", "task", "what's next", "what should" |
| **Blocker Detection** | Identifies and lists current blockers | "blocker", "issue", "problem", "stuck" |
| **Decision Tracking** | Retrieves decisions made in meetings | "decision", "decided", "agreed" |
| **Project Filtering** | Scopes queries to specific projects | Mention project name in query |
| **Meeting Structuring** | Converts raw notes to formatted markdown | Upload tab |

## Quick Start

### Using uv (recommended)

```bash
# Clone the repository
git clone https://github.com/sebasmos/sherlock.git
cd sherlock

# Create venv and install dependencies
uv venv --python 3.10
source .venv/bin/activate  # On Windows: .venv\Scripts\activate
uv pip install -r requirements.txt

# Run the app
python app.py
```

### Using pip

```bash
# Clone the repository
git clone https://github.com/sebasmos/sherlock.git
cd sherlock

# Create virtual environment (Python 3.10)
python3.10 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Run the app
python app.py
```

The app will be available at http://localhost:7860

## Usage

1. Choose your LLM provider and enter your API token
2. Add your meeting notes to `data/your_project/meetings/*.md`
3. Start asking questions about your projects

### Meeting Notes Format

```markdown
# Meeting: Sprint Planning
Date: 2025-01-15
Participants: Alice, Bob

## Discussion
Key points discussed...

## Decisions
- Decision 1
- Decision 2

## Action Items
- [ ] Alice: Implement login by Jan 20
- [x] Bob: Review PR (completed)

## Blockers
- Waiting for API credentials
```

## LLM Providers

### HuggingFace (Free)

| Property | Value |
|----------|-------|
| **Model** | Llama 3.2 3B Instruct |
| **Cost** | Free (rate limited) |
| **Token URL** | [huggingface.co/settings/tokens](https://huggingface.co/settings/tokens) |
| **Setup** | 1. Create account → 2. New token → 3. Select "Read" permission |

### Google AI (Paid)

| Property | Value |
|----------|-------|
| **Model** | Gemini 2.5 Flash Lite |
| **Cost** | Pay-per-use |
| **API Key URL** | [aistudio.google.com/apikey](https://aistudio.google.com/apikey) |
| **Setup** | 1. Create project → 2. Enable API → 3. Create API key |

## Observability (LangSmith)

Enable LLM tracing and monitoring with [LangSmith](https://smith.langchain.com):

| Property | Value |
|----------|-------|
| **Dashboard** | [smith.langchain.com](https://smith.langchain.com) |
| **Cost** | Free tier available |
| **Features** | Trace LLM calls, latency, token usage, errors |

### Setup

1. Create account at [smith.langchain.com](https://smith.langchain.com)
2. Get your API key from Settings
3. Set environment variables:

```bash
export LANGCHAIN_API_KEY=your_langsmith_api_key
export LANGCHAIN_PROJECT=sherlock  # optional, defaults to "sherlock"
```

Or add to `.env` file:

```
LANGCHAIN_API_KEY=your_langsmith_api_key
LANGCHAIN_PROJECT=sherlock
```

Once configured, all LLM calls are automatically traced and visible in the LangSmith dashboard.

## Performance Features

### Streaming Responses

LLM responses are streamed token-by-token for a better user experience. You see the answer as it's being generated, reducing perceived latency.

### Response Caching

Repeated queries are cached for 5 minutes to reduce API costs and improve response times:

| Property | Value |
|----------|-------|
| **TTL** | 5 minutes |
| **Cache Key** | Query + Project + Provider |
| **Indicator** | "_⚡ Cached response_" shown for cached answers |

### Chat Export

Export your conversation history as a PDF file:

1. Click the **📥 Export** button in the chat interface
2. Download the `.pdf` file with all Q&A pairs
3. Includes project name, timestamp, and nicely formatted conversation

## Agent Evaluation

Automated evaluation measures agent quality across different query types.

| Metric | Value |
|--------|-------|
| **Test Cases** | 5 |
| **Pass Rate** | 100% |
| **Keyword Match** | 68% |
| **Avg Latency** | 5.2s |
| **Avg Response** | 1404 chars |

Run evaluation:
```bash
GOOGLE_API_KEY=your_key pytest tests/test_evaluation.py -v -s
```

## Testing

Run all tests:
```bash
HF_TOKEN=your_token GOOGLE_API_KEY=your_key pytest tests/ -v
```

| File | Description |
|------|-------------|
| `test_parsers.py` | Date & action item parsing |
| `test_rag.py` | RAG indexing & search |
| `test_app.py` | Upload & project management |
| `test_integration.py` | LLM provider tests |
| `test_evaluation.py` | Agent quality metrics |

## Project Structure

```
sherlock/
├── app.py                 # Main Gradio application
├── requirements.txt       # Python dependencies
├── README.md              # This file
├── assets/
│   ├── logo.png           # Project logo
│   ├── logo-transparent-bg.png
│   └── favicon/           # Favicon assets
├── src/
│   ├── __init__.py
│   ├── agent.py           # LangGraph AI agent
│   ├── rag.py             # ChromaDB RAG system
│   └── parsers.py         # Meeting note parsers
├── tests/
│   ├── conftest.py        # Pytest fixtures
│   ├── test_parsers.py    # Parser tests
│   ├── test_rag.py        # RAG tests
│   ├── test_app.py        # Upload meeting & project tests
│   ├── test_integration.py # LLM provider tests
│   └── test_evaluation.py # Agent quality evaluation
└── data/                  # Sample projects included
    ├── quantum_computing/
    │   └── meetings/      # Quantum Error Correction research
    └── covid_prediction/
        └── meetings/      # COVID-19 Variant Prediction ML project
```

## Sample Projects

The repository includes two realistic research project examples:

### Quantum Computing (Quantum Error Correction)
- **Team**: 5 researchers (physics, CS, mathematics)
- **Topics**: Surface codes, IBM Quantum hardware, decoder algorithms
- **Example queries**: "What's the decoder latency issue?", "What hardware access do we have?"

### COVID-19 Prediction (Variant Forecasting)
- **Team**: 5 researchers (epidemiology, ML, bioinformatics)
- **Topics**: ESM-2 model, GISAID data, CDC collaboration
- **Example queries**: "What's our model accuracy?", "What are the data quality issues?"

## Unsorted To-Dos

- [ ] Add support for more LLM providers (OpenAI, Anthropic, Ollama)
- [ ] Implement meeting calendar integration (Google Calendar, Outlook)
- [ ] Add user authentication for multi-user support