data/README.md

Audio Transcription Pipeline

A modular audio transcription pipeline with speech recognition, audience response classification, speaker diarization, meeting summarization, and ASCII visualization.

Architecture

Audio Input (WAV/FLAC/MP3)
    │
    ▼
┌────────────────────┐
│  Transcriber       │  ← faster-whisper (base model, CPU int8)
│  Word-level timing │     Language detection, beam search
└────────┬───────────┘
         │
         ▼
┌────────────────────┐
│  Audience Class.   │  ← Rule-based + AST (MIT/ast-finetuned-audioset)
│  10 event classes  │     Applause, laughter, cheering, music, etc.
└────────┬───────────┘
         │
         ▼
┌────────────────────┐
│  Diarizer          │  ← pyannote/speaker-diarization-3.1 or mock
│  Speaker labeling  │     HF_TOKEN required for full diarization
└────────┬───────────┘
         │
         ▼
┌────────────────────┐
│  Summarizer        │  ← Qwen2.5-0.5B-Instruct GGUF
│  Structured JSON   │     Overview, decisions, action items
└────────┬───────────┘
         │
         ▼
┌────────────────────┐
│  GlyphCast ASCII   │  ← Spectrogram → ASCII art
│  7 charsets/3 modes│     Dark, light, hallow themes
└────────────────────┘

Quick Start

CLI

cd /app/audio_transcription_pipeline_1527

# Full transcription
python main.py transcribe data/jfk_speech.wav --output json

# With summary + ASCII visualization
python main.py transcribe data/jfk_speech.wav --summary --ascii

# All pipeline modules at once
python main.py all data/jfk_speech.wav --output json

# Audience response classification only
python main.py audience data/test_tone.wav --output json

# ASCII visualization only
python main.py ascii-viz data/jfk_speech.wav --mode dark

Web Demo (Runs as background service)

• Frontend (Gradio): Port 8080
• Backend API (FastAPI): Port 8081

# Start backend
python -m uvicorn api:app --host 0.0.0.0 --port 8081 --log-level warning

# Start frontend (in another terminal)
python app.py

API Endpoints

Endpoint	Method	Description
/health	GET	Health check
/transcribe	POST	Upload audio → transcription + analysis

Models Used

Model	Source	Size	Purpose
faster-whisper base	Systran/faster-whisper-base	~150 MB	Speech recognition
AST audioset	MIT/ast-finetuned-audioset-10-10-0.4593	~230 MB	Audience classification
Qwen2.5-0.5B-Instruct	Qwen/Qwen2.5-0.5B-Instruct-GGUF	~350 MB	Meeting summarization
Whisper-large-v3-turbo	openai/whisper-large-v3-turbo	~2 GB	Encoder for hybrid model
Qwen3-8B-Instruct	Qwen/Qwen3-8B-GGUF	~5 GB	LLM for hybrid model

Hybrid Speech-Transcription Model

See hybrid_model/ARCHITECTURE.md for details on the Encoder-Projector-LLM architecture.

Requirements

• Python 3.10+
• CPU with ~8GB RAM minimum
• Optional: HuggingFace token for pyannote diarization

License

MIT