README.md

metadata

title: Sherlock Project Assistant
emoji: 🔍
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 4.44.0
app_file: app.py
pinned: false
license: cc-by-nc-sa-4.0

Sherlock Project Assistant

Your intelligent assistant for managing multiple projects through meeting summaries

---

An AI-powered project assistant that helps you manage and query your project meetings and documentation.

Table of Contents

• Demo
• Features
• System Architecture
• Tech Stack
• Agentic Capabilities
• Quick Start
• Usage
• LLM Providers
• Observability
• Performance Features
• Agent Evaluation
• Testing
• Project Structure
• Unsorted To-Dos

Demo

🚀 Try the live demo here

Features

• RAG-powered Q&A - Ask questions about your projects using ChromaDB vector search
• LangGraph AI Agent - Intelligent query routing for action items, blockers, and status
• Multi-project support - Manage and filter across multiple projects
• Meeting structuring - Upload raw notes and get AI-structured markdown
• Action item tracking - Track open/completed tasks with assignees and deadlines
• Blocker & decision tracking - Surface blockers and key decisions from meetings
• Multiple LLM Providers - Choose between HuggingFace (free) or Google Gemini (paid)
• Meeting summary generation - Get comprehensive summaries with key takeaways
• Trend analysis - Analyze patterns across meetings: recurring topics, blocker trends, progress
• LangSmith Observability - Trace LLM calls, monitor latency, token usage, and errors
• Agent Evaluation - Automated quality testing with keyword matching and latency metrics
• Streaming Responses - See LLM output in real-time as it generates (better UX)
• Response Caching - Faster repeated queries with 5-minute TTL cache (lower API costs)
• Export Chat - Download your conversation history as PDF

System Architecture

Tech Stack

Category	Technology	Purpose
Frontend	Gradio 4.44	Web UI framework
LLM Framework	LangChain	LLM orchestration
Agent Framework	LangGraph	State machine for agent routing
Vector Store	ChromaDB	Persistent vector storage
Embeddings	Sentence Transformers	Text embeddings (all-MiniLM-L6-v2)
LLM (Free)	HuggingFace Inference	Llama 3.2 3B Instruct
LLM (Paid)	Google Generative AI	Gemini 2.5 Flash Lite
Data Models	Pydantic	Data validation
Testing	Pytest	Unit and integration tests
Observability	LangSmith	LLM tracing and monitoring

Agentic Capabilities

Capability	Description	Trigger Keywords
Query Analysis	Understands user intent and extracts project context	All queries
Context Retrieval	Semantic search across meeting notes	All queries
Action Item Extraction	Surfaces open tasks with assignees and deadlines	"action item", "todo", "task", "what's next", "what should"
Blocker Detection	Identifies and lists current blockers	"blocker", "issue", "problem", "stuck"
Decision Tracking	Retrieves decisions made in meetings	"decision", "decided", "agreed"
Project Filtering	Scopes queries to specific projects	Mention project name in query
Meeting Structuring	Converts raw notes to formatted markdown	Upload tab

Quick Start

Using uv (recommended)

# Clone the repository
git clone https://github.com/sebasmos/sherlock.git
cd sherlock

# Create venv and install dependencies
uv venv --python 3.10
source .venv/bin/activate  # On Windows: .venv\Scripts\activate
uv pip install -r requirements.txt

# Run the app
python app.py

Using pip

# Clone the repository
git clone https://github.com/sebasmos/sherlock.git
cd sherlock

# Create virtual environment (Python 3.10)
python3.10 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Run the app
python app.py

The app will be available at http://localhost:7860

Usage

1. Choose your LLM provider and enter your API token
2. Add your meeting notes to data/your_project/meetings/*.md
3. Start asking questions about your projects

Meeting Notes Format

# Meeting: Sprint Planning
Date: 2025-01-15
Participants: Alice, Bob

## Discussion
Key points discussed...

## Decisions
- Decision 1
- Decision 2

## Action Items
- [ ] Alice: Implement login by Jan 20
- [x] Bob: Review PR (completed)

## Blockers
- Waiting for API credentials

LLM Providers

HuggingFace (Free)

Property	Value
Model	Llama 3.2 3B Instruct
Cost	Free (rate limited)
Token URL	huggingface.co/settings/tokens
Setup	1. Create account → 2. New token → 3. Select "Read" permission

Google AI (Paid)

Property	Value
Model	Gemini 2.5 Flash Lite
Cost	Pay-per-use
API Key URL	aistudio.google.com/apikey
Setup	1. Create project → 2. Enable API → 3. Create API key

Observability (LangSmith)

Enable LLM tracing and monitoring with LangSmith:

Property	Value
Dashboard	smith.langchain.com
Cost	Free tier available
Features	Trace LLM calls, latency, token usage, errors

Setup

1. Create account at smith.langchain.com
2. Get your API key from Settings
3. Set environment variables:

export LANGCHAIN_API_KEY=your_langsmith_api_key
export LANGCHAIN_PROJECT=sherlock  # optional, defaults to "sherlock"

Or add to .env file:

LANGCHAIN_API_KEY=your_langsmith_api_key
LANGCHAIN_PROJECT=sherlock

Once configured, all LLM calls are automatically traced and visible in the LangSmith dashboard.

Performance Features

Streaming Responses

LLM responses are streamed token-by-token for a better user experience. You see the answer as it's being generated, reducing perceived latency.

Response Caching

Repeated queries are cached for 5 minutes to reduce API costs and improve response times:

Property	Value
TTL	5 minutes
Cache Key	Query + Project + Provider
Indicator	"⚡ Cached response" shown for cached answers

Chat Export

Export your conversation history as a PDF file:

1. Click the 📥 Export button in the chat interface
2. Download the .pdf file with all Q&A pairs
3. Includes project name, timestamp, and nicely formatted conversation

Agent Evaluation

Automated evaluation measures agent quality across different query types.

Metric	Value
Test Cases	5
Pass Rate	100%
Keyword Match	68%
Avg Latency	5.2s
Avg Response	1404 chars

Run evaluation:

GOOGLE_API_KEY=your_key pytest tests/test_evaluation.py -v -s

Testing

Run all tests:

HF_TOKEN=your_token GOOGLE_API_KEY=your_key pytest tests/ -v

File	Description
test_parsers.py	Date & action item parsing
test_rag.py	RAG indexing & search
test_app.py	Upload & project management
test_integration.py	LLM provider tests
test_evaluation.py	Agent quality metrics

Project Structure

sherlock/
├── app.py                 # Main Gradio application
├── requirements.txt       # Python dependencies
├── README.md              # This file
├── assets/
│   ├── logo.png           # Project logo
│   ├── logo-transparent-bg.png
│   └── favicon/           # Favicon assets
├── src/
│   ├── __init__.py
│   ├── agent.py           # LangGraph AI agent
│   ├── rag.py             # ChromaDB RAG system
│   └── parsers.py         # Meeting note parsers
├── tests/
│   ├── conftest.py        # Pytest fixtures
│   ├── test_parsers.py    # Parser tests
│   ├── test_rag.py        # RAG tests
│   ├── test_app.py        # Upload meeting & project tests
│   ├── test_integration.py # LLM provider tests
│   └── test_evaluation.py # Agent quality evaluation
└── data/                  # Sample projects included
    ├── quantum_computing/
    │   └── meetings/      # Quantum Error Correction research
    └── covid_prediction/
        └── meetings/      # COVID-19 Variant Prediction ML project

Sample Projects

The repository includes two realistic research project examples:

Quantum Computing (Quantum Error Correction)

• Team: 5 researchers (physics, CS, mathematics)
• Topics: Surface codes, IBM Quantum hardware, decoder algorithms
• Example queries: "What's the decoder latency issue?", "What hardware access do we have?"

COVID-19 Prediction (Variant Forecasting)

• Team: 5 researchers (epidemiology, ML, bioinformatics)
• Topics: ESM-2 model, GISAID data, CDC collaboration
• Example queries: "What's our model accuracy?", "What are the data quality issues?"

Unsorted To-Dos

• Add support for more LLM providers (OpenAI, Anthropic, Ollama)
• Implement meeting calendar integration (Google Calendar, Outlook)
• Add user authentication for multi-user support