initial commit

Dockerfile

@@ -0,0 +1,13 @@
+FROM python:3.12-slim
+WORKDIR /app
+RUN apt-get update && apt-get install -y \
+    gcc \
+    g++ \
+    libffi-dev \
+    libssl-dev \
+    python3-dev \
+    && rm -rf /var/lib/apt/lists/*
+COPY app.py .
+RUN pip install --no-cache-dir requirements.txt
+EXPOSE 7860
+CMD ["python", "app.py"]

README.md

@@ -1,11 +1,130 @@
 ---
-title: Misty Climate Agent
-emoji: 🏃
-colorFrom: pink
-colorTo: indigo
+title: MistyClimate Agent
+emoji: 📈
+colorFrom: red
+colorTo: pink
 sdk: docker
 pinned: false
 short_description: This is a agent created using mistral models
+tags:
+  - agent-demo-track
+Usage: Mistral
 ---
+# MistyClimate Agent 📈
+
+This is an agent created using Mistral models, designed to process climate-related documents, analyze images, perform JSON data analysis, and convert text to speech. It provides a multi-agent system for document processing, image analysis, JSON analysis, and text-to-speech functionalities, all integrated into a user-friendly Gradio interface.
+
+## Video Demo
+Below is an embedded YouTube video demonstrating the Link2Doc MCP Server for the Hackathon:
+
+<div style="text-align: center; margin: 20px 0;">
+    <iframe width="560" height="400" src="" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
+</div>
+## Features
+
+- **Document Processing**: Extract structured data from climate-related PDFs using OCR capabilities.
+- **Image Analysis**: Analyze image-based documents (e.g., PNG, JPG, PDF) to extract text, charts, and tables.
+- **JSON Analysis**: Analyze JSON data to extract insights and patterns, with a focus on climate data.
+- **Text-to-Speech**: Convert text analysis into speech using the gTTS library.
+- **Gradio Interface**: A web-based UI to interact with all features seamlessly.
+
+## Setup
+
+This project is containerized using Docker and deployed on a Gradio Space. Follow the steps below to set up and run the project locally or on Hugging Face Spaces.
+
+### Prerequisites
+
+- Docker (if running locally)
+- A Mistral API key (obtain from [Mistral AI](https://mistral.ai/))
+- Python 3.10+ (if running locally without Docker)
+
+### Installation
+
+1. **Clone the Repository** (if running locally):
+   ```bash
+   git clone <repository-url>
+   cd <repository-directory>
+   ```
+
+2. **Install Dependencies**:
+   The project uses a `requirements.txt` file to manage dependencies. If running locally without Docker, install the dependencies using:
+   ```bash
+   pip install -r requirements.txt
+   ```
+
+3. **Set Up the Mistral API Key**:
+   - You will need a Mistral API key to use the Mistral models.
+   - In the Gradio interface, input your API key in the "Mistral API Key" field.
+
+4. **Run with Docker** (recommended for local testing):
+   - Build the Docker image:
+     ```bash
+     docker build -t mistyclimate-agent .
+     ```
+   - Run the Docker container:
+     ```bash
+     docker run -p 7860:7860 mistyclimate-agent
+     ```
+   - Access the Gradio interface at `http://localhost:7860`.
+
+5. **Deploy on Hugging Face Spaces**:
+   - This project is already configured for Hugging Face Spaces with the `sdk: docker` setting.
+   - Push your code to a Hugging Face Space repository.
+   - The Space will automatically build and deploy using the provided `Dockerfile`.
+
+## Usage
+
+1. **Access the Gradio Interface**:
+   - If running locally, open `http://localhost:7860` in your browser.
+   - If deployed on Hugging Face Spaces, visit the Space URL.
+
+2. **Enter Your Mistral API Key**:
+   - In the Gradio interface, provide your Mistral API key in the designated input field.
+
+3. **Interact with the Tabs**:
+   - **Document Processing**:
+     - Upload a PDF document (e.g., a climate report).
+     - Select the document type (e.g., `climate_report`).
+     - Click "Process Document" to extract structured data in JSON format.
+   - **Image Analysis**:
+     - Upload an image file (PNG, JPG, or PDF).
+     - Choose an analysis focus (e.g., `text_extraction`, `chart_analysis`).
+     - Click "Analyze Image" to get structured data from the image.
+   - **JSON Analysis & Speech**:
+     - Input JSON data (e.g., temperature or emissions data).
+     - Select an analysis type (e.g., `content`).
+     - Click "Run Analysis & Speech" to analyze the JSON and generate a speech output.
+   - **Text-to-Speech**:
+     - Enter text to convert to speech (e.g., "hello, and good luck for the hackathon").
+     - Click "Generate Speech" to produce and play an audio file.
+
+## File Structure
+
+- `agent.py`: Core logic for the multi-agent system, including document processing, image analysis, JSON analysis, and text-to-speech workflows.
+- `app.py`: Gradio interface setup and workflow orchestration.
+- `requirements.txt`: List of Python dependencies.
+- `Dockerfile`: Docker configuration for containerizing the app.
+- `README.md`: Project documentation (this file).
+
+## Notes
+
+- **File Paths**: In a Gradio Space, files like PDFs, images, and WAVs are handled dynamically via uploads. Output files (e.g., WAVs) are saved to `/tmp/` during runtime.
+- **Mistral API Key**: Ensure you have a valid Mistral API key to use the models. Without it, the workflows will fail.
+- **Docker Deployment**: The project is configured to run in a Docker container, making it compatible with Hugging Face Spaces.
+
+## Configuration Reference
+
+For more details on configuring Hugging Face Spaces, refer to the [Hugging Face Spaces Config Reference](https://huggingface.co/docs/hub/spaces-config-reference).
+
+## Tags
+
+- `agent-demo-track`
+
+## License
+
+This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details (if applicable).
+
+---
+
+Built with ❤️ by  Samudrala Dinesh Naveen Kumar.
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

agent.py

@@ -0,0 +1,645 @@
+import os
+import base64
+import json
+import requests
+from typing import Dict, Any, Optional, Union
+from pathlib import Path
+import asyncio
+from mistralai.extra.mcp.sse import MCPClientSSE, SSEServerParams
+from mistralai import Mistral
+from mistralai.models import UserMessage, AssistantMessage, ToolMessage
+from pydantic import BaseModel
+from IPython.display import Audio, display
+import platform
+import subprocess
+import urllib.parse
+from gtts import gTTS
+
+# Pydantic Models for structured outputs
+class AnalysisDescription(BaseModel):
+    document_type: str
+    key_findings: list[str]
+    summary: str
+    metadata: Dict[str, Any]
+    confidence_score: float
+
+MODEL = "mistral-medium-latest"
+
+def play_wav(url: str, save_path: str = "/tmp/audio.wav"):
+    """
+    Plays a WAV file from a URL or local file path.
+    Args:
+        url (str): URL or file path (e.g., file://path/to/file.wav).
+        save_path (str, optional): Path to save downloaded files. Defaults to "/tmp/audio.wav".
+    Returns:
+        str: Status message
+    """
+    try:
+        # Handle local file paths
+        if url.startswith("file://"):
+            file_path = urllib.parse.urlparse(url).path
+            if platform.system() == "Windows":
+                # On Windows, remove leading slash AND decode percent-encoding
+                file_path = urllib.parse.unquote(file_path.lstrip("/"))
+            else:
+                file_path = urllib.parse.unquote(file_path)
+            print(f"Playing local file: {file_path}")
+        else:
+            # Download from URL
+            print(f"Attempting to download WAV file from {url}...")
+            response = requests.get(url, timeout=10)
+            response.raise_for_status()
+            
+            with open(save_path, 'wb') as f:
+                f.write(response.content)
+            print(f"WAV file successfully downloaded and saved to {save_path}")
+            file_path = save_path
+        
+        print(f"Attempting to play {file_path}...")
+        try:
+            # Jupyter playback
+            display(Audio(filename=file_path))
+        except NameError:
+            # Non-Jupyter playback
+            if platform.system() == "Windows":
+                os.startfile(file_path)
+            elif platform.system() == "Darwin":  # macOS
+                subprocess.run(["open", file_path], check=True)
+            else:  # Linux
+                subprocess.run(["xdg-open", file_path], check=True)
+        
+        return "Audio played successfully"
+    
+    except Exception as e:
+        print(f"Error playing audio: {str(e)}")
+        return f"Error: {str(e)}"
+
+# Create DocAgent for OCR PDF processing
+def create_doc_agent(client: Mistral):
+    return client.beta.agents.create(
+        model=MODEL,
+        name="DocAgent",
+        description="Converts OCR PDFs to JSON using document processing capabilities",
+        instructions="Process documents by extracting text and structure, then convert to JSON format. Focus on climate-related documents and extract key data points.",
+        tools=[
+            {
+                "type": "function",
+                "function": {
+                    "name": "process_climate_document",
+                    "description": "Process climate documents from file path or URL and extract structured data",
+                    "parameters": {
+                        "type": "object",
+                        "properties": {
+                            "file_path": {
+                                "type": "string",
+                                "description": "Path to the document file"
+                            },
+                            "url": {
+                                "type": "string",
+                                "description": "URL to the document"
+                            },
+                            "document_type": {
+                                "type": "string",
+                                "description": "Type of climate document (report, analysis, data, etc.)"
+                            }
+                        }
+                    }
+                }
+            }
+        ]
+    )
+
+# Create ImageAgent for image PDF processing
+def create_image_agent(client: Mistral):
+    return client.beta.agents.create(
+        model=MODEL,
+        name="ImageAgent", 
+        description="Converts image PDFs to JSON using image analysis capabilities",
+        instructions="Analyze image-based documents, extract text and visual elements, then structure the data as JSON. Handle charts, graphs, and tabular data effectively.",
+        tools=[
+            {
+                "type": "function",
+                "function": {
+                    "name": "analyze_image",
+                    "description": "Analyze image documents and extract structured data",
+                    "parameters": {
+                        "type": "object",
+                        "properties": {
+                            "image_data": {
+                                "type": "string",
+                                "description": "Base64-encoded image data"
+                            },
+                            "image_format": {
+                                "type": "string",
+                                "description": "Image format (png, jpg, pdf, etc.)"
+                            },
+                            "analysis_focus": {
+                                "type": "string",
+                                "description": "Specific focus for analysis (text_extraction, chart_analysis, table_extraction)"
+                            }
+                        },
+                        "required": ["image_data", "image_format"]
+                    }
+                }
+            }
+        ]
+    )
+
+# Create Other Agents (similar changes for JsonAnalyzerAgent and SpeechAgent)
+def create_json_analyzer_agent(client: Mistral):
+    return client.beta.agents.create(
+        model=MODEL,
+        name="JsonAnalyzerAgent",
+        description="Analyzes JSON outputs from DocAgent or ImageAgent, producing detailed descriptions",
+        instructions="Analyze JSON data structures, identify patterns, extract insights, and provide comprehensive analysis. Output should be structured and detailed.",
+        tools=[
+            {
+                "type": "function", 
+                "function": {
+                    "name": "analyze_json_data",
+                    "description": "Process and analyze JSON data to extract insights and patterns",
+                    "parameters": {
+                        "type": "object",
+                        "properties": {
+                            "json_data": {
+                                "type": "object",
+                                "description": "JSON data to analyze"
+                            },
+                            "analysis_type": {
+                                "type": "string",
+                                "description": "Type of analysis to perform (statistical, content, structural)"
+                            }
+                        },
+                        "required": ["json_data"]
+                    }
+                }
+            }
+        ]
+    )
+
+def create_speech_agent(client: Mistral):
+    return client.beta.agents.create(
+        model=MODEL,
+        name="SpeechAgent",
+        description="Converts text analysis from JsonAnalyzerAgent into speech",
+        instructions="Convert text analysis into natural speech format. Optimize text for spoken delivery and handle technical content appropriately.",
+        tools=[
+            {
+                "type": "function",
+                "function": {
+                    "name": "text_to_speech",
+                    "description": "Convert text to speech audio",
+                    "parameters": {
+                        "type": "object",
+                        "properties": {
+                            "text": {
+                                "type": "string",
+                                "description": "Text to convert to speech"
+                            },
+                            "voice_settings": {
+                                "type": "object",
+                                "properties": {
+                                    "speed": {"type": "number", "default": 1.0},
+                                    "pitch": {"type": "number", "default": 1.0},
+                                    "voice_type": {"type": "string", "default": "neutral"}
+                                }
+                            }
+                        },
+                        "required": ["text"]
+                    }
+                }
+            }
+        ]
+    )
+
+# Helper functions for agent interactions
+def simulate_process_climate_document(file_path: Optional[str] = None, url: Optional[str] = None, document_type: str = "report") -> Dict[str, Any]:
+    """Simulate document processing function"""
+    return {
+        "document_id": "doc_001",
+        "source": file_path or url,
+        "type": document_type,
+        "extracted_text": "Climate change impacts are increasing globally...",
+        "key_data": {
+            "temperature_increase": "1.5°C",
+            "co2_levels": "420ppm",
+            "affected_regions": ["Arctic", "Coastal Areas", "Tropical Regions"]
+        },
+        "metadata": {
+            "pages": 45,
+            "extraction_confidence": 0.92,
+            "processing_time": "2.3s"
+        }
+    }
+
+def simulate_analyze_image(image_data: str, image_format: str, analysis_focus: str = "text_extraction") -> Dict[str, Any]:
+    """Simulate image analysis function"""
+    return {
+        "image_id": "img_001",
+        "format": image_format,
+        "analysis_type": analysis_focus,
+        "extracted_content": {
+            "text": "Global Temperature Anomalies 2020-2024",
+            "charts": ["line_chart_temperatures", "bar_chart_emissions"],
+            "tables": [{"headers": ["Year", "Temperature", "Anomaly"], "rows": 5}]
+        },
+        "visual_elements": {
+            "charts_detected": 2,
+            "tables_detected": 1,
+            "text_regions": 8
+        },
+        "confidence": 0.88
+    }
+
+def simulate_analyze_json_data(json_data: Dict[str, Any], analysis_type: str = "content") -> Dict[str, Any]:
+    """Simulate JSON analysis function"""
+    return {
+        "analysis_summary": "Comprehensive climate document analysis completed",
+        "key_insights": [
+            "Temperature data shows accelerating warming trend",
+            "Regional variations indicate uneven climate impacts",
+            "Emission data correlates with temperature increases"
+        ],
+        "data_quality": {
+            "completeness": 0.91,
+            "consistency": 0.87,
+            "reliability": 0.89
+        },
+        "recommendations": [
+            "Focus on high-impact regions for intervention",
+            "Monitor temperature trends quarterly",
+            "Implement emission reduction strategies"
+        ]
+    }
+
+def simulate_text_to_speech(text: str, voice_settings: Dict[str, Any] = None) -> str:
+    print(f"Converting to speech: {text[:100]}...")
+    save_path = "/tmp/generated_speech.wav"
+    tts = gTTS(text=text, lang="en")
+    tts.save(save_path)
+    return f"file://{os.path.abspath(save_path)}"
+
+async def process_document_workflow(client: Mistral, file_path: str, document_type: str = "climate_report"):
+    print("Starting document processing workflow...")
+    
+    try:
+        # Define the tool as a dictionary
+        doc_tool = [
+            {
+                "type": "function",
+                "function": {
+                    "name": "process_climate_document",
+                    "description": "Process climate documents from file path or URL and extract structured data",
+                    "parameters": {
+                        "type": "object",
+                        "properties": {
+                            "file_path": {"type": "string", "description": "Path to the document file"},
+                            "url": {"type": "string", "description": "URL to the document"},
+                            "document_type": {"type": "string", "description": "Type of climate document"}
+                        }
+                    }
+                }
+            }
+        ]
+        
+        messages = [
+            UserMessage(content=f"Process the climate document at {file_path} of type {document_type}")
+        ]
+        
+        response = await client.chat.complete_async(
+            model=MODEL,
+            messages=messages,
+            tools=doc_tool
+        )
+        
+        print("Document processing response:")
+        print(response.choices[0].message.content)
+        
+        if response.choices[0].message.tool_calls:
+            for tool_call in response.choices[0].message.tool_calls:
+                if tool_call.function.name == "process_climate_document":
+                    doc_result = simulate_process_climate_document(file_path=file_path, document_type=document_type)
+                    print("Document processing result:")
+                    print(json.dumps(doc_result, indent=2))
+        
+        return response
+        
+    except Exception as e:
+        print(f"Error in document workflow: {str(e)}")
+        return None
+
+async def process_image_workflow(client: Mistral, image_path: str, analysis_focus: str = "text_extraction"):
+    print("Starting image processing workflow...")
+    
+    try:
+        # Verify image file exists
+        if not os.path.exists(image_path):
+            raise FileNotFoundError(f"Image file not found: {image_path}")
+        
+        # Convert image to base64
+        with open(image_path, "rb") as image_file:
+            image_data = base64.b64encode(image_file.read()).decode("utf-8")
+        
+        # Define image analysis tool
+        image_tool = [
+            {
+                "type": "function",
+                "function": {
+                    "name": "analyze_image",
+                    "description": "Analyze image documents and extract structured data",
+                    "parameters": {
+                        "type": "object",
+                        "properties": {
+                            "image_data": {"type": "string", "description": "Base64-encoded image data"},
+                            "image_format": {"type": "string", "description": "Image format (png, jpg, pdf, etc.)"},
+                            "analysis_focus": {"type": "string", "description": "Specific focus for analysis"}
+                        },
+                        "required": ["image_data", "image_format"]
+                    }
+                }
+            }
+        ]
+        
+        messages = [
+            UserMessage(content=f"Analyze the image document at {image_path} with focus on {analysis_focus}")
+        ]
+        
+        response = await client.chat.complete_async(
+            model=MODEL,
+            messages=messages,
+            tools=image_tool
+        )
+        
+        print("Image processing response:")
+        print(response.choices[0].message.content)
+        
+        if response.choices[0].message.tool_calls:
+            for tool_call in response.choices[0].message.tool_calls:
+                if tool_call.function.name == "analyze_image":
+                    image_result = simulate_analyze_image(
+                        image_data=image_data,
+                        image_format="jpg",
+                        analysis_focus=analysis_focus
+                    )
+                    print("Image analysis result:")
+                    print(json.dumps(image_result, indent=2))
+        
+        return response
+        
+    except Exception as e:
+        print(f"Error in image workflow: {str(e)}")
+        return None
+
+async def complete_analysis_workflow(client: Mistral, input_data: Dict[str, Any], max_retries: int = 3, initial_delay: float = 5.0):
+    print("Starting complete analysis workflow...")
+
+    async def make_api_call(messages, tools, retry_count=0):
+        try:
+            response = await client.chat.complete_async(
+                model=MODEL,
+                messages=messages,
+                tools=tools
+            )
+            return response
+        except Exception as e:
+            if "429" in str(e) and retry_count < max_retries:
+                delay = initial_delay * (2 ** retry_count)
+                print(f"Rate limit hit, retrying in {delay} seconds... (Attempt {retry_count + 1}/{max_retries})")
+                await asyncio.sleep(delay)
+                return await make_api_call(messages, tools, retry_count + 1)
+            raise e
+
+    try:
+        # Define JSON analysis tool
+        json_analysis_tool = [
+            {
+                "type": "function",
+                "function": {
+                    "name": "analyze_json_data",
+                    "description": "Process and analyze JSON data to extract insights and patterns",
+                    "parameters": {
+                        "type": "object",
+                        "properties": {
+                            "json_data": {"type": "object", "description": "JSON data to analyze"},
+                            "analysis_type": {"type": "string", "description": "Type of analysis to perform"}
+                        },
+                        "required": ["json_data"]
+                    }
+                }
+            }
+        ]
+
+        # Step 1: Analyze JSON data
+        messages = [
+            UserMessage(content="Analyze the provided JSON data and create a comprehensive analysis")
+        ]
+
+        json_response = await make_api_call(messages, json_analysis_tool)
+
+        print("JSON Analysis response:")
+        print(json_response.choices[0].message.content)
+
+        # Simulate JSON analysis
+        if json_response.choices[0].message.tool_calls:
+            for tool_call in json_response.choices[0].message.tool_calls:
+                if tool_call.function.name == "analyze_json_data":
+                    analysis_result = simulate_analyze_json_data(json_data=input_data)
+                    print("Analysis result:")
+                    print(json.dumps(analysis_result, indent=2))
+
+        # Delay before next API call
+        await asyncio.sleep(2.0)
+
+        # Define speech tool
+        speech_tool = [
+            {
+                "type": "function",
+                "function": {
+                    "name": "text_to_speech",
+                    "description": "Convert text to speech audio",
+                    "parameters": {
+                        "type": "object",
+                        "properties": {
+                            "text": {"type": "string", "description": "Text to convert to speech"},
+                            "voice_settings": {
+                                "type": "object",
+                                "properties": {
+                                    "speed": {"type": "number", "default": 1.0},
+                                    "pitch": {"type": "number", "default": 1.0},
+                                    "voice_type": {"type": "string", "default": "neutral"}
+                                }
+                            }
+                        },
+                        "required": ["text"]
+                    }
+                }
+            }
+        ]
+
+        # Step 2: Convert analysis to speech
+        analysis_text = "Climate analysis reveals significant warming trends with regional variations requiring immediate attention."
+
+        speech_messages = [
+            UserMessage(content=f"Convert this analysis to speech: {analysis_text}")
+        ]
+
+        speech_response = await make_api_call(speech_messages, speech_tool)
+
+        print("Speech conversion response:")
+        print(speech_response.choices[0].message.content)
+
+        # Simulate TTS
+        if speech_response.choices[0].message.tool_calls:
+            for tool_call in speech_response.choices[0].message.tool_calls:
+                if tool_call.function.name == "text_to_speech":
+                    audio_url = simulate_text_to_speech(text=analysis_text)
+                    print(f"Generated audio URL: {audio_url}")
+
+                    # Play the audio
+                    play_result = play_wav(audio_url)
+                    print(f"Audio play result: {play_result}")
+
+        return json_response, speech_response
+
+    except Exception as e:
+        print(f"Error in complete analysis workflow: {str(e)}")
+        return None, None
+
+async def tts_with_mcp(client: Mistral, text: str = "hello, and good luck for the hackathon"):
+    try:
+        # Define TTS tool
+        tts_tool = [
+            {
+                "type": "function",
+                "function": {
+                    "name": "text_to_speech",
+                    "description": "Convert text to speech audio",
+                    "parameters": {
+                        "type": "object",
+                        "properties": {
+                            "text": {"type": "string", "description": "Text to convert to speech"},
+                            "voice_settings": {
+                                "type": "object",
+                                "properties": {
+                                    "speed": {"type": "number", "default": 1.0},
+                                    "pitch": {"type": "number", "default": 1.0},
+                                    "voice_type": {"type": "string", "default": "neutral"}
+                                }
+                            }
+                        },
+                        "required": ["text"]
+                    }
+                }
+            }
+        ]
+        
+        print("Running TTS workflow...")
+        messages = [
+            UserMessage(content=f"Say '{text}' out loud!")
+        ]
+        
+        response = await client.chat.complete_async(
+            model=MODEL,
+            messages=messages,
+            tools=tts_tool
+        )
+        
+        print("TTS Agent response:")
+        print(response.choices[0].message.content)
+        
+        if response.choices[0].message.tool_calls:
+            for tool_call in response.choices[0].message.tool_calls:
+                if tool_call.function.name == "text_to_speech":
+                    audio_url = simulate_text_to_speech(text=text)
+                    print(f"Generated audio URL: {audio_url}")
+                    play_result = play_wav(audio_url)
+                    print(f"Audio play result: {play_result}")
+        
+        return response
+        
+    except Exception as e:
+        print(f"Error in TTS workflow: {str(e)}")
+        return None
+
+async def main(client: Mistral):
+    print("Running TTS workflow...")
+
+    try:
+        # Generate speech with gTTS
+        text = "hello, and good luck for the hackathon"
+        save_path = "/tmp/output.wav"
+        tts = gTTS(text=text, lang="en")
+        tts.save(save_path)
+        print(f"Audio saved to {save_path}")
+
+        # Play the audio
+        play_result = play_wav(f"file://{os.path.abspath(save_path)}")
+        print(f"Audio play result: {play_result}")
+
+        # Optional: Run SpeechAgent to simulate conversational interaction
+        run_result = await tts_with_mcp(client, text)
+
+        if run_result:
+            print("All run entries:")
+            for entry in run_result.choices[0].message.content.splitlines():
+                print(entry)
+
+        return run_result
+
+    except Exception as e:
+        print(f"Error in TTS workflow: {str(e)}")
+        return None
+
+async def main_workflow(client: Mistral):
+    print("Mistral Multi-Agent Document Processing System Initialized")
+    doc_agent = create_doc_agent(client)
+    image_agent = create_image_agent(client)
+    json_analyzer_agent = create_json_analyzer_agent(client)
+    speech_agent = create_speech_agent(client)
+    
+    print("Available agents:")
+    print(f"- DocAgent ID: {doc_agent.id}")
+    print(f"- ImageAgent ID: {image_agent.id}")
+    print(f"- JsonAnalyzerAgent ID: {json_analyzer_agent.id}")
+    print(f"- SpeechAgent ID: {speech_agent.id}")
+    print("-" * 50)
+
+    # Skip hardcoded file processing since Gradio handles file uploads
+    print("Skipping hardcoded document and image processing workflows in main_workflow.")
+    print("Use the Gradio interface to upload and process files.")
+    print("-" * 50)
+
+    # Complete analysis workflow
+    print("3. Running complete analysis workflow...")
+    sample_data = {
+        "temperature_data": [20.1, 20.5, 21.2, 21.8],
+        "emissions": [400, 410, 415, 420],
+        "regions": ["Global", "Arctic", "Tropical"]
+    }
+    analysis_response, speech_response = await complete_analysis_workflow(client, sample_data)
+    print("-" * 50)
+    
+    if analysis_response:
+        print("Analysis Response:")
+        print(analysis_response.choices[0].message.content)
+    else:
+        print("No analysis response received")
+    
+    if speech_response:
+        print("Speech Response:")
+        print(speech_response.choices[0].message.content)
+    else:
+        print("No speech response received")
+    
+    print("All workflows completed!")
+
+async def full_run(client: Mistral):
+    await main_workflow(client)
+    print("\n" + "="*50)
+    print("Running TTS workflow...")
+    await main(client)
+
+if __name__ == "__main__":
+    # This block is for testing purposes; actual client will be passed from app.py
+    client = Mistral(api_key="YOUR_API_KEY")
+    asyncio.run(full_run(client))

app.py

@@ -0,0 +1,239 @@
+import gradio as gr
+import asyncio
+import json
+import os
+import base64
+from agent import (create_doc_agent, create_image_agent, create_json_analyzer_agent, 
+                 create_speech_agent, process_document_workflow, process_image_workflow, 
+                 complete_analysis_workflow, tts_with_mcp, simulate_process_climate_document,
+                 simulate_analyze_image, simulate_analyze_json_data, simulate_text_to_speech, play_wav)
+from mistralai import Mistral
+from typing import Dict, Any
+# Function to initialize Mistral client and agents
+
+custom_css = """
+body {
+    background: #121212;
+    color: #ffffff;
+}
+.gradio-container {
+    background-color: #1e1e1e;
+    border-radius: 12px;
+    box-shadow: 0 4px 12px rgba(0,0,0,0.4);
+}
+h1, h2 {
+    color: #80cbc4;
+}
+.gr-button {
+    background-color: #26a69a;
+    color: white;
+}
+.gr-button:hover {
+    background-color: #00897b;
+}
+input, textarea, select {
+    background-color: #2c2c2c !important;
+    color: #ffffff;
+    border: 1px solid #4db6ac;
+}
+.gr-file label {
+    background-color: #26a69a;
+    color: white;
+}
+.gr-audio {
+    border-radius: 12px;
+    box-shadow: 0 0 8px #4db6ac;
+}
+"""
+
+
+def initialize_client_and_agents(api_key: str):
+    try:
+        client = Mistral(api_key=api_key)
+        doc_agent = create_doc_agent(client)
+        image_agent = create_image_agent(client)
+        json_analyzer_agent = create_json_analyzer_agent(client)
+        speech_agent = create_speech_agent(client)
+        return client, {
+            "doc_agent_id": doc_agent.id,
+            "image_agent_id": image_agent.id,
+            "json_analyzer_agent_id": json_analyzer_agent.id,
+            "speech_agent_id": speech_agent.id
+        }
+    except Exception as e:
+        return None, f"Error initializing client: {str(e)}"
+
+# Function to handle document processing workflow
+async def run_document_workflow(api_key: str, file, document_type):
+    if not api_key:
+        return "Error: Please provide a valid API key."
+    if file is None:
+        return "Error: Please upload a document file."
+    file_path = file.name
+    client, agents_or_error = initialize_client_and_agents(api_key)
+    if client is None:
+        return agents_or_error
+    try:
+        response = await process_document_workflow(client, file_path, document_type)
+        if response and response.choices and response.choices[0].message.tool_calls:
+            for tool_call in response.choices[0].message.tool_calls:
+                if tool_call.function.name == "process_climate_document":
+                    result = simulate_process_climate_document(file_path=file_path, document_type=document_type)
+                    return json.dumps(result, indent=2)
+        return response.choices[0].message.content if response and response.choices else "No response received."
+    except Exception as e:
+        return f"Error: {str(e)}"
+
+# Function to handle image processing workflow
+async def run_image_workflow(api_key: str, image_file, analysis_focus):
+    if not api_key:
+        return "Error: Please provide a valid API key."
+    if image_file is None:
+        return "Error: Please upload an image file."
+    image_path = image_file.name
+    client, agents_or_error = initialize_client_and_agents(api_key)
+    if client is None:
+        return agents_or_error
+    try:
+        response = await process_image_workflow(client, image_path, analysis_focus)
+        if response and response.choices and response.choices[0].message.tool_calls:
+            for tool_call in response.choices[0].message.tool_calls:
+                if tool_call.function.name == "analyze_image":
+                    with open(image_path, "rb") as f:
+                        image_data = base64.b64encode(f.read()).decode("utf-8")
+                    result = simulate_analyze_image(image_data, image_format="jpg", analysis_focus=analysis_focus)
+                    return json.dumps(result, indent=2)
+        return response.choices[0].message.content if response and response.choices else "No response received."
+    except Exception as e:
+        return f"Error: {str(e)}"
+
+# Function to handle JSON analysis and speech workflow
+async def run_analysis_and_speech_workflow(api_key: str, json_input, analysis_type):
+    if not api_key:
+        return "Error: Please provide a valid API key.", None
+    try:
+        json_data = json.loads(json_input)
+        client, agents_or_error = initialize_client_and_agents(api_key)
+        if client is None:
+            return agents_or_error, None
+        json_response, speech_response = await complete_analysis_workflow(client, json_data, max_retries=3)
+        
+        output = []
+        if json_response and json_response.choices:
+            output.append("JSON Analysis Response:")
+            output.append(json_response.choices[0].message.content)
+            for tool_call in json_response.choices[0].message.tool_calls or []:
+                if tool_call.function.name == "analyze_json_data":
+                    analysis_result = simulate_analyze_json_data(json_data, analysis_type)
+                    output.append("Analysis Result:")
+                    output.append(json.dumps(analysis_result, indent=2))
+        
+        if speech_response and speech_response.choices:
+            output.append("\nSpeech Response:")
+            output.append(speech_response.choices[0].message.content)
+            for tool_call in speech_response.choices[0].message.tool_calls or []:
+                if tool_call.function.name == "text_to_speech":
+                    analysis_text = "Climate analysis reveals significant warming trends with regional variations requiring immediate attention."
+                    audio_url = simulate_text_to_speech(analysis_text)
+                    output.append(f"Generated Audio URL: {audio_url}")
+                    play_result = play_wav(audio_url)
+                    output.append(f"Audio Play Result: {play_result}")
+                    if "file://" in audio_url:
+                        audio_path = audio_url.replace("file://", "")
+                        if os.path.exists(audio_path):
+                            return "\n".join(output), audio_path
+                        else:
+                            output.append("Error: Audio file not found.")
+        
+        return "\n".join(output), None
+    except Exception as e:
+        return f"Error: {str(e)}", None
+
+# Function to handle TTS workflow
+async def run_tts_workflow(api_key: str, text_input):
+    if not api_key:
+        return "Error: Please provide a valid API key.", None
+    client, agents_or_error = initialize_client_and_agents(api_key)
+    if client is None:
+        return agents_or_error, None
+    try:
+        response = await tts_with_mcp(client, text_input)
+        output = []
+        if response and response.choices:
+            output.append("TTS Agent Response:")
+            output.append(response.choices[0].message.content)
+            for tool_call in response.choices[0].message.tool_calls or []:
+                if tool_call.function.name == "text_to_speech":
+                    audio_url = simulate_text_to_speech(text=text_input)
+                    output.append(f"Generated Audio URL: {audio_url}")
+                    play_result = play_wav(audio_url)
+                    output.append(f"Audio Play Result: {play_result}")
+                    if "file://" in audio_url:
+                        audio_path = audio_url.replace("file://", "")
+                        if os.path.exists(audio_path):
+                            return "\n".join(output), audio_path
+                        else:
+                            output.append("Error: Audio file not found.")
+        return "\n".join(output), None
+    except Exception as e:
+        return f"Error: {str(e)}", None
+
+# Gradio interface
+with gr.Blocks(css=custom_css) as demo:
+
+    gr.Markdown("# MistyClimate Multi-Agent System")
+    gr.Markdown("## Mistral Multi-Agent Processing System")
+    gr.Markdown("Enter your Mistral API key and interact with document processing, image analysis, JSON analysis, and text-to-speech functionalities.")
+    
+    api_key_input = gr.Textbox(label="Mistral API Key", type="password", placeholder="Enter your Mistral API key here")
+    
+    with gr.Tab("Document Processing"):
+        doc_file = gr.File(label="Upload Document (PDF)")
+        doc_type = gr.Dropdown(choices=["climate_report", "analysis", "data"], label="Document Type", value="climate_report")
+        doc_button = gr.Button("Process Document")
+        doc_output = gr.Textbox(label="Document Processing Output", lines=10)
+        doc_button.click(
+            fn=run_document_workflow,
+            inputs=[api_key_input, doc_file, doc_type],
+            outputs=doc_output
+        )
+
+    with gr.Tab("Image Analysis"):
+        img_file = gr.File(label="Upload Image (PNG/JPG/PDF)")
+        analysis_focus = gr.Dropdown(choices=["text_extraction", "chart_analysis", "table_extraction"], 
+                                  label="Analysis Focus", value="text_extraction")
+        img_button = gr.Button("Analyze Image")
+        img_output = gr.Textbox(label="Image Analysis Output", lines=10)
+        img_button.click(
+            fn=run_image_workflow,
+            inputs=[api_key_input, img_file, analysis_focus],
+            outputs=img_output
+        )
+
+    with gr.Tab("JSON Analysis & Speech"):
+        json_input = gr.Textbox(label="JSON Data Input", lines=5, 
+                              placeholder='{"temperature_data": [20.1, 20.5, 21.2, 21.8], "emissions": [400, 410, 415, 420], "regions": ["Global", "Arctic", "Tropical"]}')
+        analysis_type = gr.Dropdown(choices=["statistical", "content", "structural"], 
+                                  label="Analysis Type", value="content")
+        analysis_button = gr.Button("Run Analysis & Speech")
+        analysis_output = gr.Textbox(label="Analysis and Speech Output", lines=10)
+        audio_output = gr.Audio(label="Generated Audio")
+        analysis_button.click(
+            fn=run_analysis_and_speech_workflow,
+            inputs=[api_key_input, json_input, analysis_type],
+            outputs=[analysis_output, audio_output]
+        )
+
+    with gr.Tab("Text-to-Speech"):
+        tts_input = gr.Textbox(label="Text Input", value="hello, and good luck for the hackathon")
+        tts_button = gr.Button("Generate Speech")
+        tts_output = gr.Textbox(label="TTS Output", lines=5)
+        tts_audio = gr.Audio(label="Generated Audio")
+        tts_button.click(
+            fn=run_tts_workflow,
+            inputs=[api_key_input, tts_input],
+            outputs=[tts_output, tts_audio]
+        )
+
+if __name__ == "__main__":
+    demo.launch()

requirements.txt

@@ -0,0 +1,9 @@
+mistralai 
+requests
+pydantic 
+IPython 
+gtts 
+gradio 
+asyncio 
+json
+mcp