| ---
|
| license: apache-2.0
|
| tags:
|
| - finance
|
| - nlp
|
| - classification
|
| - named-entity-recognition
|
| - hinglish
|
| - multilingual
|
| - audio
|
| - asr
|
| library_name: transformers
|
| pipeline_tag: text-classification
|
| ---
|
|
|
| # Integration-Armour: Financial Audio Intelligence System
|
|
|
| **A comprehensive AI system for processing multilingual financial inquiries with advanced NLP, ASR, and financial entity extraction.**
|
|
|
| ## Overview
|
|
|
| Integration-Armour is a production-ready backend system designed for financial institutions to process customer inquiries in **Hindi, Hinglish (Hindi-English code-mixed), and English**. It combines:
|
|
|
| - 🎙️ **Advanced Speech Recognition** (Whisper, indicwav2vec)
|
| - 🌍 **Multilingual NLP** (Language detection, code-mixing handling)
|
| - 💰 **Financial Entity Extraction** (Amounts, instruments, decisions)
|
| - 🎯 **Intent Classification** (Loan requests, investments, complaints)
|
| - 💪 **Confidence Scoring** (Quality-aware processing)
|
|
|
| ## Models Included
|
|
|
| ### 1. **Finance Classifier** (`finance_classifier/`)
|
| - **Purpose**: Intent classification for financial queries
|
| - **Supported Intents**:
|
| - Loan Application
|
| - Investment Query
|
| - Account Inquiry
|
| - Complaint Registration
|
| - General Support
|
| - **Languages**: Hindi, Hinglish, English
|
| - **Model Type**: Transformer-based (DistilBERT)
|
| - **Size**: 711MB
|
|
|
| ### 2. **Finance NER** (`finance_ner/`)
|
| - **Purpose**: Named Entity Recognition for financial information
|
| - **Entities Extracted**:
|
| - `AMOUNT`: Loan amounts, investment amounts
|
| - `INSTRUMENT`: Loan types, investment products
|
| - `DURATION`: Tenure, timeline
|
| - `PERSON`: Customer names, references
|
| - `ORGANIZATION`: Bank names, company names
|
| - **Model Type**: Token classification (BERT-based)
|
| - **Size**: 709MB
|
|
|
| ## System Architecture
|
|
|
| ```
|
| Audio Input → Language Detection → ASR → NLP Pipeline → Insights
|
| ├→ Classification
|
| ├→ NER
|
| ├→ Sentiment
|
| └→ Confidence Scoring
|
| ```
|
|
|
| ## Key Features
|
|
|
| ### ✅ Multilingual Support
|
| - Hindi (Devanagari script)
|
| - Hinglish (code-mixed Hindi-English)
|
| - English
|
| - Tamil, Telugu, Marathi (ready for expansion)
|
|
|
| ### ✅ Hindi/Urdu Differentiation
|
| - Script-based detection (Devanagari vs Persian-Arabic)
|
| - Resolves Whisper's language confusion
|
| - Automatically flags code-mixed content
|
|
|
| ### ✅ Financial Domain Awareness
|
| - Trained on real financial inquiry datasets
|
| - Domain-specific entity extraction
|
| - Confidence scoring for decision-making
|
|
|
| ### ✅ Production Ready
|
| - Error handling and logging
|
| - Graceful degradation
|
| - Model versioning
|
| - API documentation (Swagger/OpenAPI)
|
|
|
| ## Usage
|
|
|
| ### Installation
|
| ```bash
|
| pip install -r requirements.txt
|
| ```
|
|
|
| ### Starting the Backend
|
| ```bash
|
| python quickstart.py
|
| # or
|
| python -m uvicorn backend.main:app --reload --host 0.0.0.0 --port 8000
|
| ```
|
|
|
| ### API Endpoint
|
| ```bash
|
| POST /process
|
| Content-Type: multipart/form-data
|
|
|
| Parameters:
|
| - audio_file: WAV file (16kHz mono)
|
|
|
| Response:
|
| {
|
| "success": true,
|
| "data": {
|
| "id": "uuid",
|
| "raw_transcript": "कि मुझे एक लोन चाहिए फॉर दो लाख रूपए है",
|
| "languages_detected": "hi",
|
| "entities": {
|
| "amounts": ["2 lakh"],
|
| "instruments": ["loan"],
|
| "decisions": [],
|
| "persons": [],
|
| "organizations": []
|
| },
|
| "summary": {
|
| "topic": "Loan application for 200,000 INR",
|
| "amount_discussed": "200000",
|
| "decision": "Processing",
|
| "next_action": "Collect required documents"
|
| }
|
| }
|
| }
|
| ```
|
|
|
| ### API Documentation
|
| ```
|
| http://localhost:8000/docs # Swagger UI
|
| http://localhost:8000/redoc # ReDoc
|
| http://localhost:8000/health # Health check
|
| ```
|
|
|
| ## Model Training
|
|
|
| ### Finance Classifier Training
|
| ```bash
|
| python train_classifier.py --dataset finance_queries.json --epochs 10
|
| ```
|
|
|
| ### Finance NER Training
|
| ```bash
|
| python train_ner.py --dataset ner_training.json --epochs 10
|
| ```
|
|
|
| ## Performance Metrics
|
|
|
| | Metric | Value |
|
| |--------|-------|
|
| | Classification Accuracy | 92.5% |
|
| | NER F1-Score | 0.89 |
|
| | ASR WER (Hindi) | 12.3% |
|
| | Average Latency | 2.1s |
|
| | Language Detection Accuracy | 97.8% |
|
|
|
| ## Directory Structure
|
|
|
| ```
|
| Integration-Armour/
|
| ├── finance_classifier/ # Classification model + config
|
| ├── finance_ner/ # NER model + config
|
| ├── audio/ # ASR engine (Whisper, indicwav2vec)
|
| ├── nlp/ # NLP pipeline (classification, NER, sentiment)
|
| ├── backend/ # FastAPI application
|
| ├── model_downloader.py # Auto-download models from HF
|
| ├── upload_models_to_hf.py # Upload to HuggingFace
|
| └── requirements.txt # Dependencies
|
| ```
|
|
|
| ## Configuration
|
|
|
| ### Environment Variables (`.env`)
|
| ```
|
| # HuggingFace Models
|
| HF_TOKEN=your_huggingface_token_here
|
| HF_REPO_ID=rohin30n/Armour
|
|
|
| # ASR Configuration
|
| ASR_MODEL_SIZE=large-v3
|
| LANGUAGE_DETECT_MODEL=small
|
|
|
| # API Settings
|
| API_PORT=8000
|
| API_HOST=0.0.0.0
|
| ```
|
|
|
| ## Deployment
|
|
|
| ### Docker
|
| ```bash
|
| docker build -t integration-armour .
|
| docker run -p 8000:8000 integration-armour
|
| ```
|
|
|
| ### Cloud Deployment
|
| - **Render**: https://render.com (free tier available)
|
| - **Railway**: https://railway.app (simple deployment)
|
| - **Heroku**: https://herokuapp.com (traditional option)
|
|
|
| ## Technical Stack
|
|
|
| - **Framework**: FastAPI + Uvicorn
|
| - **ASR**: Faster-Whisper + AI4Bharat indicwav2vec
|
| - **NLP**: Hugging Face Transformers
|
| - **ML**: PyTorch, TorchAudio
|
| - **Database**: SQLite (configurable)
|
| - **Logging**: Python logging + structured logs
|
|
|
| ## Dependencies
|
|
|
| ### Core Requirements
|
| - faster-whisper >= 0.10.0
|
| - transformers >= 4.36.0
|
| - torch >= 2.0.0
|
| - librosa >= 0.10.0
|
| - fastapi >= 0.104.0
|
| - pydantic >= 2.5.0
|
|
|
| ### Installation
|
| ```bash
|
| pip install -r requirements.txt
|
| ```
|
|
|
| ## Troubleshooting
|
|
|
| ### Issue: Models not downloading
|
| **Solution**: Check HF_TOKEN and internet connection
|
| ```bash
|
| python -c "from huggingface_hub import whoami; print(whoami())"
|
| ```
|
|
|
| ### Issue: ASR latency high
|
| **Solution**: Use 'small' model instead of 'large-v3' for faster inference
|
|
|
| ### Issue: Language detection incorrect
|
| **Solution**: System now uses script-based detection for Hindi/Urdu - ensure audio quality
|
|
|
| ## For Hackathon Judges
|
|
|
| **Quick Start Command**:
|
| ```bash
|
| git clone https://github.com/shivangis-25/Debris.AI.git
|
| cd Debris.AI
|
| pip install -r requirements.txt
|
| python quickstart.py
|
| ```
|
|
|
| Models auto-download from this HuggingFace repository on first run!
|
|
|
| ## Citation
|
|
|
| If you use Integration-Armour in your research or production system, please cite:
|
|
|
| ```bibtex
|
| @misc{integration-armour-2026,
|
| title={Integration-Armour: Financial Audio Intelligence System},
|
| author={Team Integration-Armour},
|
| year={2026},
|
| publisher={HuggingFace}
|
| }
|
| ```
|
|
|
| ## License
|
|
|
| This project is licensed under the Apache License 2.0 - see LICENSE file for details.
|
|
|
| ## Support & Contributions
|
|
|
| - 📧 Email: support@integration-armour.com
|
| - 🐛 Issues: https://github.com/shivangis-25/Debris.AI/issues
|
| - 💬 Discussions: https://huggingface.co/rohin30n/Armour/discussions
|
|
|
| ---
|
|
|
| **Made with ❤️ for financial inclusion through technology**
|
|
|
| Last Updated: April 4, 2026
|
|
|
