| | --- |
| | license: mit |
| | title: Data Extractor Using Gemini |
| | sdk: docker |
| | emoji: 🏃 |
| | colorFrom: yellow |
| | colorTo: blue |
| | --- |
| | # 📊 Financial Data Extractor Using Gemini |
| |
|
| | A powerful AI-driven financial document analysis system that automatically extracts, organizes, and generates professional Excel reports from financial documents using Google's Gemini AI models. |
| |
|
| | ## 🚀 Features |
| |
|
| | ### Core Functionality |
| |
|
| | - **📄 Multi-format Document Support**: PDF, DOCX, TXT, and image files |
| | - **🔍 Intelligent Data Extraction**: AI-powered extraction of financial data points |
| | - **📊 Smart Data Organization**: Automatic categorization into 12+ financial categories |
| | - **💻 Excel Report Generation**: Professional multi-worksheet Excel reports with charts |
| | - **🎯 Real-time Processing**: Live streaming interface with progress tracking |
| |
|
| | ### Advanced Capabilities |
| |
|
| | - **🤖 Multi-Agent Workflow**: Specialized AI agents for extraction, arrangement, and code generation |
| | - **💾 Session Management**: Persistent storage with SQLite caching |
| | - **🔄 Auto-shutdown**: Intelligent resource management for cloud deployments |
| | - **📱 Modern UI**: Beautiful Gradio-based web interface |
| | - **🌐 Cross-platform**: Works on Windows, Mac, and Linux |
| | - **🐳 Docker Support**: Containerized deployment ready |
| |
|
| | ## 🏗️ Architecture |
| |
|
| | The system uses a sophisticated multi-agent workflow powered by the Agno framework: |
| |
|
| | ``` |
| | 📄 Document Upload |
| | ↓ |
| | 🔍 Data Extractor Agent |
| | ↓ (Structured Financial Data) |
| | 📊 Data Arranger Agent |
| | ↓ (Organized Categories) |
| | 💻 Code Generator Agent |
| | ↓ (Python Excel Code) |
| | 📊 Excel Report Output |
| | ``` |
| |
|
| | ### Agent Specialization |
| |
|
| | - **Data Extractor**: Extracts financial data points with confidence scoring |
| | - **Data Arranger**: Organizes data into 12+ professional categories |
| | - **Code Generator**: Creates Python code for Excel report generation |
| |
|
| | ## 📋 Requirements |
| |
|
| | ### System Requirements |
| |
|
| | - Python 3.8+ |
| | - Google API Key (for Gemini models) |
| | - 2GB+ RAM recommended |
| | - Cross-platform compatible |
| |
|
| | ### Dependencies |
| |
|
| | ``` |
| | agno>=1.7.4 |
| | gradio |
| | google-generativeai |
| | PyPDF2 |
| | Pillow |
| | python-dotenv |
| | pandas |
| | matplotlib |
| | openpyxl |
| | python-docx |
| | lxml |
| | markdown |
| | requests |
| | seaborn |
| | sqlalchemy |
| | websockets |
| | ``` |
| |
|
| | ## 🚀 Quick Start |
| |
|
| | ### 1. Clone the Repository |
| |
|
| | ```bash |
| | git clone <repository-url> |
| | cd Data_Extractor_Using_Gemini |
| | ``` |
| |
|
| | ### 2. Install Dependencies |
| |
|
| | ```bash |
| | pip install -r requirements.txt |
| | ``` |
| |
|
| | ### 3. Configure Environment |
| |
|
| | Create a `.env` file: |
| |
|
| | ```env |
| | GOOGLE_API_KEY=your_gemini_api_key_here |
| | ``` |
| |
|
| | ### 4. Run the Application |
| |
|
| | ```bash |
| | python app.py |
| | ``` |
| |
|
| | The application will be available at `http://localhost:7860` |
| |
|
| | ## 🐳 Docker Deployment |
| |
|
| | ### Build and Run |
| |
|
| | ```bash |
| | docker build -t financial-extractor . |
| | docker run -p 7860:7860 --env-file .env financial-extractor |
| | ``` |
| |
|
| | ### Environment Variables |
| |
|
| | - `GOOGLE_API_KEY`: Your Google Gemini API key |
| | - `INACTIVITY_TIMEOUT_MINUTES`: Auto-shutdown timeout (default: 30) |
| |
|
| | ## 📖 Usage Guide |
| |
|
| | ### 1. Upload Document |
| |
|
| | - Drag and drop or select your financial document |
| | - Supported formats: PDF, DOCX, TXT, PNG, JPG, JPEG |
| |
|
| | ### 2. Select Processing Mode |
| |
|
| | - **Quick Analysis**: Standard extraction and organization |
| | - **Custom Prompts**: Use predefined prompt templates for specific document types |
| |
|
| | ### 3. Monitor Progress |
| |
|
| | - Real-time streaming interface shows each processing step |
| | - Progress indicators for all workflow stages |
| | - Live terminal output for code execution |
| |
|
| | ### 4. Download Results |
| |
|
| | - Professional Excel report with multiple worksheets |
| | - Organized data categories with charts and formatting |
| | - All intermediate files available for download |
| |
|
| | ## 📊 Output Structure |
| |
|
| | The generated Excel reports include: |
| |
|
| | ### Worksheets |
| |
|
| | - **Summary**: Executive overview with key metrics |
| | - **Revenue**: Income and revenue streams |
| | - **Expenses**: Operating and non-operating expenses |
| | - **Assets**: Current and non-current assets |
| | - **Liabilities**: Short-term and long-term liabilities |
| | - **Equity**: Shareholder equity components |
| | - **Cash Flow**: Cash flow statements |
| | - **Ratios**: Financial ratio analysis |
| | - **Charts**: Visual representations of key data |
| | - **Raw Data**: Original extracted data points |
| |
|
| | ### Features |
| |
|
| | - Professional formatting with consistent styling |
| | - Interactive charts and visualizations |
| | - Dynamic period handling (auto-detects years/quarters) |
| | - Cross-referenced data validation |
| | - Print-ready layouts |
| |
|
| | ## 🔧 Configuration |
| |
|
| | ### Model Settings |
| |
|
| | Configure AI models in `config/settings.py`: |
| |
|
| | - Data Extractor Model |
| | - Data Arranger Model |
| | - Code Generator Model |
| | - Thinking budgets and retry settings |
| |
|
| | ### Prompt Customization |
| |
|
| | Customize agent instructions in `instructions/agents/`: |
| |
|
| | - `data_extractor.md`: Data extraction prompts |
| | - `data_arranger.md`: Data organization prompts |
| | - `code_generator.md`: Excel generation prompts |
| |
|
| | ### Workflow Configuration |
| |
|
| | Modify workflow behavior in `workflow/financial_workflow.py`: |
| |
|
| | - Agent configurations |
| | - Tool assignments |
| | - Output formats |
| |
|
| | ## 🛠️ Development |
| |
|
| | ### Project Structure |
| |
|
| | ``` |
| | ├── app.py # Main Gradio application |
| | ├── workflow/ # Core workflow implementation |
| | ├── instructions/ # Agent instruction templates |
| | ├── prompts/ # Prompt gallery configurations |
| | ├── config/ # Application settings |
| | ├── utils/ # Utility functions |
| | ├── static/ # Static assets |
| | ├── models/ # Data models |
| | └── terminal_stream.py # Real-time terminal streaming |
| | ``` |
| |
|
| | ### Key Components |
| |
|
| | - **WorkflowUI**: Main interface controller |
| | - **FinancialDocumentWorkflow**: Core processing pipeline |
| | - **AutoShutdownManager**: Resource management |
| | - **TerminalLogHandler**: Real-time logging |
| | - **PromptGallery**: Template management |
| |
|
| | ## 🔒 Security & Privacy |
| |
|
| | - **Local Processing**: All document processing happens locally |
| | - **No Data Storage**: Documents are processed and cleaned up automatically |
| | - **API Key Security**: Environment-based configuration |
| | - **Session Isolation**: Each session has isolated temporary directories |
| |
|
| | ## 🌐 Deployment Options |
| |
|
| | ### Local Development |
| |
|
| | ```bash |
| | python app.py |
| | ``` |
| |
|
| | ### Production (Gunicorn) |
| |
|
| | ```bash |
| | gunicorn -w 4 -b 0.0.0.0:7860 app:app |
| | ``` |
| |
|
| | ### Cloud Platforms |
| |
|
| | - **Hugging Face Spaces**: Ready for deployment |
| | - **Google Cloud Run**: Containerized deployment |
| | - **AWS/Azure**: Standard container deployment |
| |
|
| | ## 🤝 Contributing |
| |
|
| | 1. Fork the repository |
| | 2. Create a feature branch |
| | 3. Make your changes |
| | 4. Add tests if applicable |
| | 5. Submit a pull request |
| |
|
| | ## 📝 License |
| |
|
| | This project is licensed under the MIT License - see the LICENSE file for details. |
| |
|
| | ## 🆘 Support |
| |
|
| | ### Common Issues |
| |
|
| | - **API Key Errors**: Ensure your Google API key is valid and has Gemini access |
| | - **Memory Issues**: Increase system RAM or reduce document size |
| | - **Processing Timeouts**: Check network connectivity and API quotas |
