README.md

---
title: "VocalSync Intelligence: Speech-to-Text"
emoji: 🎙️
colorFrom: blue
colorTo: green
sdk: gradio
sdk_version: 6.4.0
python_version: '3.10'
app_file: app.py
pinned: false
---


# 🎙️ VocalSync Intelligence: Deconstructing Speech-to-Text

**A curiosity-driven experiment in deconstructing the ASR-to-LLM pipeline.**

VocalSync Intelligence is a learning experiment designed to explore the bridge between raw audio waves and structured digital thoughts. Instead of treating AI as a "black box," this project deconstructs the process of capturing scattered brainstorming and streamlining it into detailed guidelines using local hardware constraints.

---

## ✨ Features

* **🎤 Live Transcription**: Real-time speech-to-text conversion from microphone input.
* **🤖 AI Meeting Analysis**: Integrated Meeting Manager logic using Llama-3.2-3B to generate action items and key insights from raw transcripts.
* **🌐 Web Interface**: A modern Gradio-based UI designed for seamless interaction with the ASR engine.
* **📹 Universal Video Support**: Ability to ingest and transcribe audio from YouTube, Vimeo, Teams, and 1000+ other platforms via URL.
* **🔄 Hybrid Modes**: Support for real-time streaming, after-speech accumulation, and direct file uploads.
* **⚡ Optimized Engine**: Leverages Faster Whisper with `int8` quantization for high-speed local CPU inference.
* **💾 Auto-Scribe**: Automatic persistence of all sessions to the `/outputs` directory with unique timestamps.
* **🔒 Privacy-First**: 100% local processing: no audio data or transcripts ever leave your machine.

---

## 🏗️ Technical Architecture

To balance semantic clarity with local CPU limitations, the project focuses on three technical pillars:

### 1. Signal Normalization
Using **PyAudio** to sample sound at 16kHz and normalizing 16-bit integers into `float32` decimals. This is the essential digital handshake between the microphone and the neural network.



### 2. Contextual Anchoring
Implementing a **Sliding Window** history. By feeding the last 200 characters of the transcript back into the `initial_prompt`, the system fixes phonetic hallucinations (e.g., ensuring "AI" isn't misheard as "Ali").

### 3. Inference Pipeline
* **ASR:** `faster-whisper` (Base model) using `int8` quantization for CPU efficiency.
* **LLM:** `Llama-3.2-3B-Instruct` acting as a "Meeting Manager" to align scattered thoughts into a streamlined roadmap.

---

## 📂 Project Structure

```plaintext
.
├── app.py                  # Main entry point (Gradio UI)
├── src/
│   ├── transcription/      # ASR Logic (Live, File, and Streaming engines)
│   ├── analysis/           # Llama-3.2-3B Integration
│   ├── handlers/           # Orchestration between audio and text processing
│   └── io/                 # Logic for persistent storage
├── outputs/                # Local storage for transcripts and AI analysis
└── requirements.txt        # Project dependencies
🚀 Getting Started
1. Prerequisites
Python 3.10+

FFmpeg: Essential for audio stream handling and URL processing.

Windows: choco install ffmpeg

Mac: brew install ffmpeg

Linux: sudo apt install ffmpeg

2. Installation
Clone the repository and set up a local environment:

Bash

git clone [https://github.com/mahnoor-khalid9/vocal-sync-speech-to-text.git](https://github.com/mahnoor-khalid9/vocal-sync-speech-to-text.git)
cd vocal-sync-speech-to-text
python -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate
pip install -r requirements.txt
3. Environment Setup
Create a .env file in the root directory:

Bash

HF_TOKEN=your_huggingface_token
4. Running the Experiment
Launch the interface to start the live thought-collection process:

Bash

python app.py
🎓 Findings & Learning Autopsy
The Warm-up Pulse: Solved the "Cold Start" lag where the model would miss the first few words by injecting a 1s silent np.zeros buffer at launch to initialize the engine.

VAD Gating: Implemented a Voice Activity Detection threshold of 0.5 to prevent the model from hallucinating text during silent periods or background noise.

Context > Model Size: Discovered that a "Base" model with a smart sliding-window prompt can often provide more coherent brainstorming flow than a "Large" model listening in a vacuum.

Note: This project is a learning exercise in seeing how data architecture: from signal normalization to metadata syncing: directly influences AI behavior.