SYSTEM_GUIDE.md

Nexus AI | Enterprise Audio Intelligence System Guide

Welcome to the Nexus AI system. This system acts as a conversational CRM intelligence engine that extracts structured insights from sales recordings or typed texts and scores leads for conversion probability.

---

1. System Architecture & Flow

The application processes text inputs or audio recordings through a multi-stage pipeline:

graph TD
    A[Audio Recording / Text Input] --> B[FastAPI Web Server]
    B --> C[Whisper Speech-to-Text]
    C --> D[Pyannote Speaker Diarization]
    D --> E[Local PII Redaction / Sanitization]
    E --> F[Llama 3 Extraction via Groq API]
    F --> G[Taxonomy Normalization]
    G --> H[XGBoost ML Classification Model]
    H --> I[CRM Lead Score & Follow-Up Alerts]

1. Transcription & Diarization: Audio files are transcribed using OpenAI Whisper. Speaker turn durations and customer/agent separations are calculated using Pyannote.audio.
2. Local PII Redaction: Sensitive Customer PII (Names, Phone Numbers, Emails) is detected and redacted locally before calling LLMs to maintain privacy constraints.
3. Llama 3 Information Extraction: Using Groq's high-speed API, Llama-3-70B extracts mentions of product features, brands, budget details, customer objections, and intent levels.
4. XGBoost Lead Scoring: Extracted signals are aligned into tabular features and fed into an XGBoost classifier which predicts whether the lead is hot (prob >= 0.7), warm (prob >= 0.4), or cold (prob < 0.4).
5. Follow-Up Engine: Actionable reminders and priority tasks are generated automatically from conversations and logged in a SQLite3 store.

---

2. Directory Structure

• src/ - Python core application codebase.
  • src/api/server.py - FastAPI entrypoint containing HTTP and WebSocket routes.
  • src/api/worker.py - Task worker handling background queue for audio files.
  • src/aspect_sentiment/ - Signal detection, NLP scoring rules, VADER sentiment, PII privacy filters, and model fusion.
• frontend/ - Next.js React Dashboard styled with Tailwind CSS and Framer Motion.
• data/ - Dataset processing directories. Contains raw transcript inputs, SQL databases, and SQLite metrics.
• models/ - Pickled artifacts of the trained XGBoost model (sales_conversion_model.pkl) and tabular schemas.
• audio/ - Sample WAV files.
• scripts/ - Shell/Batch files to easily start backend and frontend services.

---

3. Configuration & Startup

Ensure you copy .env.example to .env and .env.local inside the root directory and update them with your Groq and Hugging Face tokens:

LLAMA_API_KEY=your_groq_api_key
HUGGINGFACE_TOKEN=your_huggingface_token_if_using_pyannote

Starting the System

To start the servers:

1. Automated Startup (Windows): Double-click the START.bat script to run checks and launch backend & frontend servers automatically.
2. Manual Startup:
   • Backend:

     .venv\Scripts\python.exe -m uvicorn src.api.server:app --reload --port 8000
     

   • Frontend:

     cd frontend
     npm run dev
     

---

4. Diagnostics & Testing

We provide three layers of test verification to ensure everything runs perfectly:

1. System Sanity Check (test_system.py): Checks that the virtual environment imports all packages correctly, the spaCy NLP engine compiles, and pre-trained XGBoost pickle models are loaded.

   .venv\Scripts\python.exe test_system.py
   

2. API Endpoint Integration Test (test_audio_upload.py): Fires live API requests to /api/health, runs text processing, uploads a local test WAV file, and polls the job worker.

   .venv\Scripts\python.exe test_audio_upload.py
   

3. Browser Diagnostic Dashboard (audio-upload-debug.html): Open http://localhost:5173/audio-upload-debug.html in your browser once the frontend is running to trace status logs and debug connection failures.