README 2.md

Agentic Defensor

An agentic RAG (Retrieval-Augmented Generation) system for legal defense analysis. This system goes beyond traditional RAG by employing multiple specialized agents to process queries, aggregate context, and generate comprehensive legal analyses.

Key Features

• Multi-Agent Architecture: Different specialized agents handle different parts of the RAG pipeline
• Query Analysis: Extracts key entities, refines ambiguous queries, and decomposes complex questions
• Context Aggregation: Summarizes and organizes retrieved document chunks for better context understanding
• Answer Generation: Produces structured, comprehensive answers with source references
• Agent Reasoning: Optional debug mode to see the agent's thinking process

Project Structure

.
├── data/                  # Data storage directory
├── embeddings/            # Embeddings and FAISS index storage
├── pdfs/                  # PDF documents
├── src/                   # Source code
│   ├── agents/            # Agent implementations
│   ├── api/               # FastAPI application
│   ├── data/              # Data processing utilities
│   ├── embeddings/        # Embedding generation
│   ├── models/            # Retrieval models
│   └── utils/             # Utility functions
├── .env                   # Environment variables
├── requirements.txt       # Python dependencies
└── README.md              # Project documentation

Setup

1. Clone the repository
2. Install the dependencies:

pip install -r requirements.txt

3. Make sure your environment variables are set up in the .env file:

OPENAI_API_KEY=your_openai_api_key

4. Create data directories for large files (these are git-ignored):

mkdir -p data embeddings pdfs

5. Add your data files:
   • Place your document chunks in data/doc_chunks.pkl
   • Place your embeddings in embeddings/embeddings.pkl
   • Place your FAISS index in embeddings/faiss_index.index

Quick Start

The easiest way to get started is to use the provided Makefile commands:

# Setup the environment
make setup

# Run API server
make run

# Run interactive mode
make run-interactive

# Run interactive mode with agent reasoning
make run-interactive-debug

# Run a specific query with agent
make run-agent QUERY="Your query here"

# Run a query with agent reasoning
make run-debug QUERY="Your query here"

Or use the start.sh script directly:

# API server
./start.sh api

# Interactive mode
./start.sh interactive

# Interactive mode with agent reasoning
./start.sh interactive-debug

# Run a query with standard agent
./start.sh cli "Your query here"

# Run a query with multi-agent system
./start.sh agent "Your query here"

# Run a query with agent reasoning
./start.sh agent-debug "Your query here"

Usage

All-in-One Script

The run.py script provides multiple ways to interact with the system:

# Global options
python run.py [--model MODEL_NAME] [--verbose] [--debug]

Interactive Mode (default)

Start an interactive session to ask multiple questions:

python run.py [interactive]

For an interactive session with the multi-agent system:

python run.py interactive --agent

With agent reasoning shown:

python run.py --debug interactive --agent

API Mode

Run the FastAPI server to handle queries over HTTP:

python run.py api [--port 8000]

CLI Mode

Run a single query from the command line:

python run.py cli "Your legal query here" [--top-k 100]

Agent Mode

Use the multi-agent system to process a query:

python run.py agent "Your legal query here" [--top-k 50]

With agent reasoning shown:

python run.py --debug agent "Your legal query here" [--top-k 50]

Save the result to a file:

python run.py agent "Your legal query here" --output result.json

Agent Debug Mode

The system includes a special debug mode that shows the agent's thinking process as it works through your query. This is helpful for:

• Understanding how the agent analyzes your query
• Seeing which documents it retrieves and why
• Following the context organization process
• Viewing the reasoning behind the final answer

To enable debug mode, use one of these methods:

# Using Makefile
make run-interactive-debug
make run-debug QUERY="Your query here"

# Using start.sh
./start.sh interactive-debug
./start.sh agent-debug "Your query here"
./start.sh agent "Your query here" --debug

# Using run.py directly
python run.py --debug interactive --agent
python run.py --debug agent "Your query here"

When debug mode is enabled, you'll see detailed reasoning steps marked with 🧠, showing:

1. Query analysis and understanding
2. Document retrieval decisions
3. Context organization strategies
4. Answer formulation process

This makes the system more transparent and helps you understand how it arrives at its conclusions.

Example Script

Run the example script to see the system in action:

python example.py

You can customize the query:

python example.py --query "Your legal query here" --top-k 100

Agentic Example

Run the agentic example script to see the multi-agent system in action:

python agentic_example.py --query "Your legal query here" --save

This uses multiple specialized agents to:

1. Analyze and structure the query
2. Retrieve relevant documents
3. Aggregate and organize the context
4. Generate a comprehensive answer

API Server

You can run the FastAPI server directly using:

python run_api.py

Or directly with uvicorn:

uvicorn src.api.app:app --reload

The API will be available at http://localhost:8000

API endpoints:

• GET /: Root endpoint
• POST /query: Query endpoint

API Usage Example

Query the API with curl:

curl -X POST "http://localhost:8000/query" \
     -H "Content-Type: application/json" \
     -d '{"query": "En qué tomo se encuentra Contrato Andrea Monsalve", "top_k": 100}'

Docker Deployment

For containerized deployment:

# Build and run the Docker container
make build-docker
make run-docker

# Or directly with docker-compose
docker-compose up -d

Large Files Handling

This project uses large data files for embeddings and document storage. These files are not included in the git repository and should be managed locally:

1. Data Storage: All data files are stored in git-ignored directories:

   • data/: For document chunks and other structured data
   • embeddings/: For vector embeddings and FAISS indices
   • pdfs/: For source PDF documents

2. Backup Strategy: Regularly backup these directories as they contain critical data:

   # Example backup command
   tar -czf agentic-defensor-data-backup.tar.gz data/ embeddings/ pdfs/
   

3. Data Transfer: When deploying to a new environment, transfer the data directories:

   # On production server
   mkdir -p data embeddings pdfs
   
   # Transfer from local machine (example)
   scp -r data/* user@server:/path/to/agentic-defensor/data/
   scp -r embeddings/* user@server:/path/to/agentic-defensor/embeddings/
   

4. Docker Volumes: When using Docker, the data directories are mounted as volumes to persist between container restarts.

Multi-Agent Architecture

The system uses several specialized agents working together:

1. Query Analyzer: Analyzes the user's query to extract key entities, refine ambiguous queries, and decompose complex questions
2. Retriever: Retrieves relevant document chunks based on the processed query
3. Context Aggregator: Summarizes and organizes retrieved chunks for coherent context
4. Answer Generator: Produces comprehensive, structured answers with source references
5. Agent Director: Coordinates the agents and manages the information flow

Installation as a Package

You can also install the project as a Python package:

pip install -e .

Then use the command-line tool:

agentic-defensor "Your legal query here"

Future Enhancements

• PDF document processing and chunking
• Multi-agent reasoning with competing hypotheses
• Enhanced context aggregation strategies
• Question refinement and decomposition
• Interactive chat interface
• Fine-tuning for legal domain

License

This project is licensed under the MIT License.

Cloud Deployment Options

In addition to local and Docker-based deployments, Agentic Defensor can be deployed to cloud platforms for wider accessibility.

Replicate Deployment

Agentic Defensor can be deployed as a Replicate model, making it accessible through their API:

1. Ensure you have a Replicate account

2. Install the Replicate CLI:

   pip install cog
   

3. Build and push your model:

   cog push
   

4. Once deployed, you can use it via their API:

   import replicate
   
   # Run a prediction
   output = replicate.run(
       "vichudo/agentic-defensor:latest",
       input={
           "query": "Your legal query here",
           "top_k": 50,
           "debug": True
       }
   )
   print(output)
   

Hugging Face Spaces Deployment

Deploy Agentic Defensor to Hugging Face Spaces for a user-friendly web interface:

1. Create a new Space on Hugging Face Spaces

2. Choose "Gradio" as the SDK

3. Upload all the required files, including:

   • Code files
   • Data files (embeddings, document chunks)
   • app_huggingface.py
   • requirements-huggingface.txt
   • space_config.json

4. The Space will automatically build and deploy your application

5. Your application will be available at https://huggingface.co/spaces/vichudo/agentic-defensor

Data Management for Cloud Deployments

When deploying to cloud platforms, you'll need to handle large data files appropriately:

1. Replicate:

   • Data files are included in the Docker image when building with Cog
   • For large datasets, consider hosting them separately and downloading at runtime

2. Hugging Face Spaces:

   • Upload data files directly to your Space (up to the storage limit)
   • For larger datasets, implement a download script that retrieves data from a storage service on startup

3. API Keys:

   • Set your OpenAI API key as a secret in your deployment platform
   • For Replicate, set it as a secret in your model settings
   • For Hugging Face Spaces, set it as a repository secret