refactor: Remove Sweetviz and use YData Profiling as primary EDA tool

- Remove Sweetviz from requirements.txt due to NumPy 2.x incompatibility
- Remove generate_sweetviz_report and generate_combined_eda_report functions
- Update orchestrator to use only generate_ydata_profiling_report
- Remove Sweetviz imports from tools __init__.py
- Update tools_registry.py to remove Sweetviz tool definitions
- Update README.md to remove all Sweetviz references
- Uninstall sweetviz package

YData Profiling provides:
- Full NumPy 2.x compatibility
- More comprehensive analysis than Sweetviz
- Better maintained with regular updates
- Automated insights and data quality warnings

README.md

@@ -39,7 +39,7 @@ The frontend is built with React 19 and TypeScript 5.8, featuring a modern glass
 
 - **Landing Page**: Geometric hero section with animated background paths, key capabilities showcase, problem-solution presentation, process flow visualization, and technology stack display
 - **Chat Interface**: Real-time message streaming, file upload support for CSV and Parquet formats, markdown rendering for formatted responses with code syntax highlighting, loading states with animated indicators, and error handling with user-friendly messages
-- **Report Viewer**: In-application modal viewer for HTML reports generated by YData Profiling, Sweetviz, and custom dashboard tools. Full-screen modal with professional styling, iframe embedding for report content, and download capabilities
+- **Report Viewer**: In-application modal viewer for HTML reports generated by YData Profiling and custom dashboard tools. Full-screen modal with professional styling, iframe embedding for report content, and download capabilities
 - **Session Management**: Maintains conversation history across browser sessions, allows users to review previous analyses, and provides context for follow-up questions
 
 ### Complete Machine Learning Pipeline
@@ -127,7 +127,7 @@ Type your request in natural language in the chat input box. The agent understan
 
 **Step 5: Review Results**
 
-The agent will execute the requested workflow and display results in the chat interface. For analyses that generate HTML reports (such as YData Profiling or Sweetviz), a "View Report" button will appear. Click this button to open the report in a full-screen modal viewer.
+The agent will execute the requested workflow and display results in the chat interface. For analyses that generate HTML reports (such as YData Profiling), a "View Report" button will appear. Click this button to open the report in a full-screen modal viewer.
 
 ### Example Queries and Use Cases
 
@@ -220,21 +220,489 @@ The startup script will:
 - **Matplotlib 3.8+**: Fundamental plotting library for Python offering publication-quality static visualizations
 - **Pydantic 2.5+**: Data validation library using Python type annotations for request/response models
 
-### Data Processing and Storage
+###Docker Deployment
 
-- **Polars**: Primary dataframe library for all data manipulation operations
-- **Pandas 2.1+**: Secondary support for compatibility with legacy tools and libraries
-- **SQLite**: Embedded database for caching query results and session management
-- **Python-dotenv**: Environment variable management from .env files
+The application includes a multi-stage Dockerfile for optimized containerized deployment.
 
-### Development and Deployment
+### Building the Docker Image
 
-- **Docker**: Containerization platform with multi-stage builds for optimized image size and consistent deployment
-- **Uvicorn**: Lightning-fast ASGI server for running FastAPI applications
-- **Git**: Version control system for code management and collaboration
-**4. Install frontend dependencies**
+Build the Docker image with the following command:
+
+```bash
+docker build -t ds-agent:latest .
+```
+
+The multi-stage build process:
+1. **Stage 1 (Builder)**: Installs Node.js dependencies and builds the React frontend
+2. **Stage 2 (Runtime)**: Sets up Python environment, installs backend dependencies, and copies built frontend
+3. Result: Optimized image size by excluding development dependencies and build tools
+
+### Running the Container
+
+Run the containerized application:
+
+```bash
+docker run -d \
+  -p 8080:8080 \
+  --env-file .env \
+  --name ds-agent-container \
+  ds-agent:latest
+```
+
+Parameters explained:
+- `-d`: Run container in detached mode (background)
+- `-p 8080:8080`: Map container port 8080 to host port 8080
+- `--env-file .env`: Load environment variables from .env file
+- `--name ds-agent-container`: Assign a name to the container for easy management
+
+### Docker Compose (Recommended)
+
+For easier management, create a `docker-compose.yml` file:
+
+```yaml
+version: '3.8'
+
+services:
+  ds-agent:
+    build: .
+    container_name: ds-agent
+    ports:
+      - "8080:8080"
+    env_file:
+      - .env
+    volumes:
+   Environment Configuration
+
+The application uses environment variables for configuration management. Create a `.env` file in the project root directory with the following variables:
+
+### Required Configuration
+
+```bash
+# LLM Provider Selection
+LLM_PROVIDER=gemini
+# Options: gemini (currently supported)
+
+# Google Gemini API Key (REQUIRED)
+GOOGLE_API_KEY=your_api_key_here
+# Obtain from: https://ai.google.dev/
+# Free tier limits: 10 RPM, 20 RPD
+
+# Gemini Model Selection
+GEMINI_MODEL=gemini-2.5-flash
+# Options:
+#   - gemini-2.5-flash (recommended, balanced performance)
+#   - gemini-1.5-pro (higher capability, lower rate limits)
+#   - gemini-1.5-flash (faster, lower cost)
+```
+
+### Optional Configuration
+Advanced Features
+
+### Intelligent Intent Detection and Classification
+
+The orchestration system employs sophisticated intent detection to automatically classify user requests and route them to appropriate workflow pipelines. The classification system analyzes incoming natural language queries using keyword matching, pattern recognition, and contextual understanding.
+
+**Intent Categories:**
+
+1. **Full ML Pipeline Intent**: Triggered by keywords such as "train", "model", "predict", "machine learning", "regression", "classification". Executes complete workflow including data profiling, cleaning, feature engineering, model training, hyperparameter tuning, and evaluation.
+
+2. **Exploratory Analysis Intent**: Activated by keywords like "explore", "profile", "report", "analysis", "overview", "insights", "understand". Performs comprehensive data profiling with statistical summaries, distribution analysis, correlation matrices, and automated insights generation.
+
+3. **Data Cleaning Intent**: Detected via keywords such as "clean", "missing", "outliers", "duplicates", "impute", "handle". Focuses on data quality improvement operations without proceeding to modeling.
+
+4. **Visualization Intent**: Identified through keywords like "plot", "visualize", "chart", "graph", "heatmap", "distribution". Generates requested visualizations without performing modeling or extensive preprocessing.
+
+5. **Feature Engineering Intent**: Recognized by keywords such as "feature", "engineer", "create features", "transform", "encode". Applies feature transformation and creation operations.
+
+6. **Multi-Intent Workflows**: The system can detect and handle requests combining multiple intents, executing them in a logical sequence.
+
+The intent classification system uses confidence scoring to handle ambiguous requests and can ask clarifying questions when intent is unclear.
+
+### Context-Aware Session Memory
+
+The agent implements persistent session memory that maintains conversation context across multiple turns. This enables natural multi-turn dialogues where subsequent requests can reference previous operations without requiring full context repetition.
+
+**Session Memory Capabilities:**
+
+- **Workflow History**: Stores complete history of executed tools, parameters, and results for the current session
+- **File State Tracking**: Maintains references to uploaded files, intermediate processed datasets, and generated outputs
+- **Model Persistence**: Remembers trained models and their performance metrics for comparison and further tuning
+- **Error Context**: Stores information about encountered errors to avoid repeating failed operations
+- **User Preferences**: Learns from user choices (e.g., preferred visualization types, imputation strategies)
+
+**Example Multi-Turn Conversation:**
+
+```Complete Workflow Example
+
+This section demonstrates a complete end-to-end workflow for a real-world dataset, showing the agent's autonomous decision-making and execution capabilities.
+
+### Dataset: Earthquake Magnitude Prediction
+
+**Input Dataset:** `earthquake_data.csv`
+- Rows: 175,947 earthquake records
+- Columns: 22 features including latitude, longitude, depth, time, location, and magnitude
+- Target Variable: Earthquake magnitude (continuous regression task)
+- Data Quality: 11.67% missing values, presence of outliers, mixed data types
+
+**User Prompt:**
+```
+"Train a model to predict earthquake magnitude with the highest possible accuracy"
+```
+
+### Automated Workflow Execution
+
+**Phase 1: Data Profiling and Assessment** (Step 1)
+- Tool: `generate_ydata_profile`
+- Action: Comprehensive statistical analysis of all 22 features
+- Findings:
+  - Total records: 175,947
+  - Missing values detected in 8 columns
+  - Outliers present in depth, latitude, longitude
+  - High cardinality in location column (15,000+ unique values)
+  - Strong correlation between depth and magnitude (r=0.62)
+- Output: YData Profiling HTML report saved to `outputs/earthquake_profile.html`
+- Time: 18.3 seconds
+API Reference
+
+The FastAPI backend exposes several endpoints for programmatic interaction.
+
+### Endpoints
+
+**POST /chat**
+- Description: Send a message to the agent with optional file upload
+- Content-Type: multipart/form-data
+- Parameters:
+  - message (string, required): User's natural language request
+  - file (file, optional): Dataset file (CSV or Parquet)
+- Response: JSON with agent's response message and workflow history
+- Example:
 ```bash
-cd FRRONTEEEND
+curl -X POST http://localhost:8080/chat \
+  -F "message=Generate a data profile report" \
+  -F "file=@dataset.csv"
+```
+
+**POST /run**
+- Description: Execute a complete analysis workflow
+- Content-Type: application/json
+- Parameters:
+  - query (string, required): Analysis request
+  - use_cache (boolean, optional): Enable caching (default: true)
+- Response: JSON with analysis results and generated artifacts
+- Example:
+```json
+{
+  "query": "Train a regression model to predict sales",
+  "use_cache": true
+}
+```
+
+**GET /outputs/{file_path}**
+- Description: Retrieve generated reports and artifacts
+- Parameters:
+  - file_path (string, required): Path to output file
+- Response: File content (HTML, PNG, CSV, etc.)
+- Example:
+```bash
+curl http://localhost:8080/outputs/ydata_profile.html
+```
+
+**GET /api/health**
+- Description: Health check endpoint
+- Response: JSON with status information
+- Example response:
+```json
+{
+  "status": "healthy",
+  "version": "1.0.0",
+  "timestamp": "2025-12-27T10:30:00Z"
+}
+```
+
+### Interactive API Documentation
+
+FastAPI automatically generates interactive API documentation:
+- Swagger UI: http://localhost:8080/docs
+- ReDoc: http://localhost:8080/redoc
+
+## Contributing
+
+Contributions to improve the AI-Powered Data Science Agent are welcome. Please follow these guidelines:
+
+### Development Setup
+
+1. Fork the repository and clone your fork
+2. Create a new branch for your feature: `git checkout -b feature/your-feature-name`
+3. Install development dependencies: `pip install -r requirements-dev.txt`
+4. Make your changes with appropriate tests
+5. Ensure all tests pass: `pytest tests/`
+6. Format code with black: `black src/`
+7. Lint code with flake8: `flake8 src/`
+8. Commit with descriptive messages
+9. Push to your fork and submit a pull request
+
+### Code Style
+
+- Follow PEP 8 guidelines for Python code
+- Use type hints for function parameters and return values
+- Write docstrings for all functions and classes
+- Keep functions focused and under 50 lines when possible
+- Use meaningful variable names
+
+### Testing
+
+- Write unit tests for new features
+- Ensure existing tests pass before submitting PR
+- Aim for >80% code coverage
+
+## License
+
+This project is licensed under the MIT License. See the LICENSE file for complete terms.
+
+Copyright (c) 2025 Pulastya B
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+## Acknowledgments
+
+This project builds upon several excellent open-source technologies and frameworks:
+
+- **Google Gemini 2.5 Flash**: Advanced language model with function calling capabilities enabling intelligent agent orchestration
+- **FastAPI**: Modern, high-performance web framework for building APIs with Python, providing automatic documentation and validation
+- **React**: JavaScript library for building user interfaces, enabling component-based architecture and efficient rendering
+- **Polars**: High-performance DataFrame library written in Rust, offering significant speed improvements over traditional data processing libraries
+- **Scikit-learn**: Machine learning library providing simple and efficient tools for data analysis and modeling
+- **XGBoost, LightGBM, CatBoost**: Gradient boosting frameworks offering state-of-the-art performance for structured data
+- **Optuna**: Hyperparameter optimization framework with efficient search algorithms
+- **YData Profiling**: Automated exploratory data analysis tool generating comprehensive reports
+- **Plotly**: Interactive visualization library for creating publication-quality graphs
+- **TypeScript**: Typed superset of JavaScript enhancing code quality and developer experience
+- **Tailwind CSS**: Utility-first CSS framework for rapid UI development
+- **Vite**: Next-generation frontend build tool with instant server start
+
+Special thanks to the open-source community for creating and maintaining these exceptional tools.
+
+## Contact and Support
+
+**Developer:** Pulastya B
+
+**GitHub Profile:** [@Pulastya-B](https://github.com/Pulastya-B)
+
+**Project Repository:** [DevSprint-Data-Science-Agent](https://github.com/Pulastya-B/DevSprint-Data-Science-Agent)
+
+**Issues and Bug Reports:** Please use the GitHub Issues page to report bugs or request features
+
+**Documentation:** Additional documentation and tutorials available in the repository wiki
+
+**Project Status:** Active development - Built for DevSprint Hackathon
+
+For questions, suggestions, or collaboration opportunities, please open an issue on GitHub or contact through the repository.
+
+---
+
+**Last Updated:** December 27, 2025
+
+**Version:** 1.0.0
+Step 6 - Temporal Feature Extraction:
+- Tool: `extract_time_features`
+- Input column: 'timestamp'
+- Features created:
+  - year, month, day_of_week, hour
+  - Cyclical encodings: hour_sin, hour_cos, month_sin, month_cos
+- Justification: Earthquakes may have temporal patterns
+- New columns: 8 time-based features
+
+Step 7 - Categorical Encoding:
+- Tool: `encode_categorical_features`
+- Method: Target encoding for 'location' (high cardinality), one-hot encoding for 'type'
+- Result: All categorical variables converted to numeric
+- New columns: 3 (reduced from high-cardinality location)
+
+Step 8 - Statistical Features:
+- Tool: `create_statistical_features`
+- Features created:
+  - Distance from nearest plate boundary (calculated from lat/lon)
+  - Depth-to-magnitude ratio
+  - Regional earthquake frequency (rolling count)
+- New columns: 3 domain-specific features
+
+Final feature count: 28 engineered features
+
+**Phase 5: Model Training and Selection** (Step 9)
+- Tool: `train_baseline_models`
+- Algorithms trained in parallel:
+
+1. Ridge Regression: R² = 0.534, RMSE = 0.312
+2. Lasso Regression: R² = 0.541, RMSE = 0.309
+3. ElasticNet: R² = 0.538, RMSE = 0.311
+4. Random Forest: R² = 0.698, RMSE = 0.251
+5. XGBoost: R² = 0.716, RMSE = 0.243 (BEST)
+6. LightGBM: R² = 0.709, RMSE = 0.247
+7. CatBoost: R² = 0.712, RMSE = 0.245
+
+- Best model selected: XGBoost
+- Validation split: 80/20 stratified split
+- Time: 124.7 seconds
+
+**Phase 6: Hyperparameter Optimization** (Step 10)
+- Tool: `optimize_hyperparameters_optuna`
+- Framework: Optuna with Tree-structured Parzen Estimator (TPE)
+- Search space:
+  - max_depth: [3, 10]
+  - learning_rate: [0.001, 0.3] (log scale)
+  - n_estimators: [100, 1000]
+  - min_child_weight: [1, 10]
+  - subsample: [0.6, 1.0]
+  - colsample_bytree: [0.6, 1.0]
+- Trials: 50 iterations
+- Best parameters found:
+  - max_depth: 7
+  - learning_rate: 0.0847
+  - n_estimators: 673
+  - min_child_weight: 3
+  - subsample: 0.8234
+  - colsample_bytree: 0.9123
+- Optimized performance: R² = 0.743, RMSE = 0.231
+- Improvement: +3.8% R² over baseline
+- Time: 312.4 seconds
+
+**Phase 7: Model Validation** (Step 11)
+- Tool: `cross_validate_model`
+- Method: 5-fold stratified cross-validation
+- Results:
+  - Fold 1: R² = 0.741, RMSE = 0.232
+  - Fold 2: R² = 0.745, RMSE = 0.230
+  - Fold 3: R² = 0.738, RMSE = 0.234
+  - Fold 4: R² = 0.747, RMSE = 0.229
+  - Fold 5: R² = 0.742, RMSE = 0.232
+- Mean performance: R² = 0.743 ± 0.003, RMSE = 0.231 ± 0.002
+- Interpretation: Low variance across folds indicates robust, generalizable model
+- Time: 267.8 seconds
+
+**Phase 8: Visualization and Reporting** (Steps 12-13)
+
+Step 12 - Feature Importance Analysis:
+- Tool: `plot_feature_importance`
+- Top 10 features by importance:
+  1. depth (0.284)
+  2. distance_to_plate_boundary (0.167)
+  3. latitude (0.142)
+  4. longitude (0.138)
+  5. regional_frequency (0.095)
+  6. depth_magnitude_ratio (0.067)
+  7. hour_sin (0.034)
+  8. month (0.028)
+  9. location_encoded (0.024)
+  10. year (0.021)
+- Output: Interactive Plotly bar chart saved to `outputs/feature_importance.html`
+
+Step 13 - Comprehensive Dashboard:
+- Tool: `create_plotly_dashboard`
+- Visualizations included:
+  - Correlation heatmap (28x28 features)
+  - Actual vs Predicted scatter plot
+  - Residual distribution plot
+  - Feature importance ranking
+  - Temporal patterns in predictions
+- Output: Multi-panel interactive dashboard saved to `outputs/model_dashboard.html`
+
+### Final Results Summary
+
+**Model Performance:**
+- Algorithm: XGBoost with optimized hyperparameters
+- Training R²: 0.743
+- Cross-validated R²: 0.743 ± 0.003
+- RMSE: 0.231 (on magnitude scale 0-10)
+- MAE: 0.176
+- Explanation: Model explains 74.3% of variance in earthquake magnitudes
+
+**Artifacts Generated:**
+- Trained model file: `outputs/xgboost_model_optimized.pkl`
+- YData profiling report: `outputs/earthquake_profile.html`
+- Feature importance plot: `outputs/feature_importance.html`
+- Interactive dashboard: `outputs/model_dashboard.html`
+- Cleaned dataset: `data/earthquake_data_cleaned.parquet`
+- Feature engineered dataset: `data/earthquake_data_featured.parquet`
+
+**Total Execution Time:** 12 minutes 43 seconds
+
+**Key Insights:**
+1. Depth is the strongest predictor of earthquake magnitude (28.4% importance)
+2. Spatial features (distance to plate boundaries, lat/lon) are highly informative
+3. Temporal patterns show cyclical variations in earthquake characteristics
+4. Model performance is consistent across cross-validation folds (low variance)
+5. The optimized XGBoost model provides reliable magnitude predictions suitable for deployment
+
+### Robust Error Recovery System
+
+The agent implements a comprehensive error recovery system designed to handle failures gracefully and guide users toward successful task completion.
+
+**Error Recovery Mechanisms:**
+
+1. **Automatic Retry with Correction**: When a tool execution fails due to incorrect parameters, the agent analyzes the error message, adjusts parameters based on the error type, and automatically retries the operation with corrected inputs.
+
+2. **File Existence Validation**: Before executing tools that require specific file inputs, the system validates file existence and accessibility, providing clear guidance when files are missing.
+
+3. **Column Name Validation**: Validates that requested column names exist in the dataset before performing operations, suggesting similar column names when exact matches aren't found.
+
+4. **Dependency Tracking**: Ensures tools are executed in proper sequence, checking that prerequisite operations (e.g., data cleaning before training) have been completed.
+
+5. **Loop Detection**: Monitors tool execution patterns to detect and prevent infinite retry loops. If the same operation fails multiple times with the same error, the agent stops retrying and requests user intervention.
+
+6. **Recovery Guidance**: When errors cannot be automatically resolved, the system provides detailed guidance including:
+   - Clear explanation of what went wrong
+   - The last successful file state that can be used to continue
+   - Suggested alternative approaches
+   - Specific parameter corrections needed
+
+7. **Graceful Degradation**: If a requested operation cannot be completed, the agent attempts to provide partial results or alternative analysis that may still be valuable.
+
+**Example Error Recovery Flow:**
+
+```
+Request: "Train a model to predict 'Price' column"
+
+Error Detected: Column 'Price' not found in dataset
+Recovery Action: Search for similar columns → Find 'price', 'PRICE', 'SalePrice'
+Agent Response: "Column 'Price' not found. Did you mean 'SalePrice'? I found these similar columns: ['SalePrice', 'price_usd']. Please specify which column to use."
+
+User: "Yes, use SalePrice"
+Agent: [Continues with corrected column name]
+```
+
+### Interactive Report Viewing
+
+The web interface includes an integrated report viewer that displays comprehensive HTML reports generated during analysis without requiring users to download files or switch to external tools.
+
+**Report Viewer Features:**
+
+- **In-Application Display**: Reports open in a full-screen modal overlay within the chat interface
+- **Multiple Report Types**: Supports YData Profiling reports and custom HTML dashboards
+- **Professional Styling**: Modal features glassmorphism design, smooth animations, and responsive layout
+- **Interactive Navigation**: Users can zoom, scroll, and interact with report elements directly in the viewer
+- **Download Option**: Reports can be downloaded as standalone HTML files for sharing or archival
+- **Automatic Detection**: System automatically detects when tools generate HTML reports and creates "View Report" buttons in the chat interface
+
+**Supported Report Types:**
+
+1. **YData Profiling Reports**: Comprehensive automated EDA with variable statistics, distributions, correlations, missing value analysis, and alerts for data quality issues
+
+2. **Custom Dashboards**: User-created Plotly dashboards with multiple interactive visualizations
+
+The report extraction system uses multiple strategies to locate report files, including checking tool return values, parsing workflow history, and using regex pattern matching on agent responses.
+- Use different API keys for development and production
+- Rotate API keys periodically
+- Set restrictive file permissions on `.env` (chmod 600 on Linux/macOS)inux/macOS:**
+```bash
+chmod +x build-and-deploy.sh
+./build-and-deploy.sh
+```
+
+These scripts handle building the image, stopping any existing containers, and starting a new container with proper configuration.FRRONTEEEND
 npm install
 npm run build
 cd ..

requirements.txt

@@ -31,8 +31,7 @@ seaborn>=0.13.1
 plotly>=5.18.0  # Interactive visualizations
 
 # EDA Report Generation
-sweetviz>=2.3.1  # Beautiful fast EDA reports
-ydata-profiling>=4.17.0  # Updated for Python 3.13 compatibility
+ydata-profiling>=4.17.0  # Comprehensive automated EDA reports with Python 3.13 compatibility
 
 # User Interface
 # gradio>=5.49.1  # Replaced with React frontend

src/orchestrator.py

@@ -104,10 +104,8 @@ from tools import (
     generate_interactive_box_plots,
     generate_interactive_time_series,
     generate_plotly_dashboard,
-    # EDA Report Generation (3) - NEW PHASE 2
-    generate_sweetviz_report,
+    # EDA Report Generation (1) - NEW PHASE 2
     generate_ydata_profiling_report,
-    generate_combined_eda_report,
     # Code Interpreter (2) - NEW PHASE 2 - TRUE AI AGENT CAPABILITY
     execute_python_code,
     execute_code_from_file,
@@ -332,10 +330,8 @@ class DataScienceCopilot:
             "generate_interactive_box_plots": generate_interactive_box_plots,
             "generate_interactive_time_series": generate_interactive_time_series,
             "generate_plotly_dashboard": generate_plotly_dashboard,
-            # EDA Report Generation (3) - NEW PHASE 2
-            "generate_sweetviz_report": generate_sweetviz_report,
+            # EDA Report Generation (1) - NEW PHASE 2
             "generate_ydata_profiling_report": generate_ydata_profiling_report,
-            "generate_combined_eda_report": generate_combined_eda_report,
             # Code Interpreter (2) - NEW PHASE 2 - TRUE AI AGENT CAPABILITY
             "execute_python_code": execute_python_code,
             "execute_code_from_file": execute_code_from_file,
@@ -668,7 +664,7 @@ Use specialized tools FIRST. Only use execute_python_code for:
 - NEW Automation: auto_ml_pipeline (zero-config full pipeline), auto_feature_selection
 - NEW Visualization: generate_all_plots, generate_data_quality_plots, generate_eda_plots, generate_model_performance_plots, generate_feature_importance_plot
 - NEW Interactive Plotly Visualizations: generate_interactive_scatter, generate_interactive_histogram, generate_interactive_correlation_heatmap, generate_interactive_box_plots, generate_interactive_time_series, generate_plotly_dashboard (interactive web-based plots with zoom/pan/hover)
-- NEW EDA Report Generation: generate_sweetviz_report (beautiful fast reports), generate_ydata_profiling_report (comprehensive detailed analysis), generate_combined_eda_report (both in one call)
+- NEW EDA Report Generation: generate_ydata_profiling_report (comprehensive detailed analysis with full statistics, distributions, correlations, and data quality insights)
 - NEW Enhanced Feature Engineering: create_ratio_features, create_statistical_features, create_log_features, create_binned_features
 
 **RULES:**

src/tools/__init__.py

@@ -141,11 +141,9 @@ from .plotly_visualizations import (
     generate_plotly_dashboard
 )
 
-# EDA Report Generation (3) - NEW PHASE 2
+# EDA Report Generation (1) - NEW PHASE 2
 from .eda_reports import (
-    generate_sweetviz_report,
-    generate_ydata_profiling_report,
-    generate_combined_eda_report
+    generate_ydata_profiling_report
 )
 
 # Code Interpreter (2) - NEW PHASE 2 - CRITICAL for True AI Agent
@@ -279,10 +277,8 @@ __all__ = [
     "generate_interactive_time_series",
     "generate_plotly_dashboard",
     
-    # EDA Report Generation (3) - NEW PHASE 2
-    "generate_sweetviz_report",
+    # EDA Report Generation (1) - NEW PHASE 2
     "generate_ydata_profiling_report",
-    "generate_combined_eda_report",
     
     # Code Interpreter (2) - NEW PHASE 2 - CRITICAL for True AI Agent
     "execute_python_code",

src/tools/eda_reports.py

@@ -1,6 +1,6 @@
 """
 EDA Report Generation Tools
-Generates comprehensive HTML reports using Sweetviz and ydata-profiling.
+Generates comprehensive HTML reports using ydata-profiling.
 """
 
 import os
@@ -9,128 +9,6 @@ from typing import Dict, Any, Optional
 import polars as pl
 
 
-def generate_sweetviz_report(
-    file_path: str,
-    output_path: str = "./outputs/reports/sweetviz_report.html",
-    target_column: Optional[str] = None,
-    compare_file_path: Optional[str] = None
-) -> Dict[str, Any]:
-    """
-    Generate a beautiful HTML report using Sweetviz.
-    
-    Sweetviz creates stunning visualizations for EDA with:
-    - Target analysis (associations with target variable)
-    - Feature distributions and statistics
-    - Correlations and relationships
-    - Missing value analysis
-    - Comparison between datasets (train vs test)
-    
-    Args:
-        file_path: Path to the dataset CSV file
-        output_path: Where to save the HTML report
-        target_column: Optional target variable for analysis
-        compare_file_path: Optional second dataset to compare against
-        
-    Returns:
-        Dict with success status, report path, and summary
-    """
-    try:
-        import warnings
-        import pandas as pd
-        
-        # Suppress NumPy deprecation warnings that Sweetviz triggers
-        warnings.filterwarnings('ignore', category=DeprecationWarning)
-        
-        import sweetviz as sv
-        
-        # Read dataset (Sweetviz requires pandas)
-        if file_path.endswith('.csv'):
-            df = pd.read_csv(file_path)
-        elif file_path.endswith('.parquet'):
-            df = pd.read_parquet(file_path)
-        else:
-            raise ValueError(f"Unsupported file format: {file_path}")
-        
-        # Create output directory if needed
-        os.makedirs(os.path.dirname(output_path) or "./outputs/reports", exist_ok=True)
-        
-        # Generate report based on configuration
-        if compare_file_path:
-            # Comparison report (e.g., train vs test)
-            if compare_file_path.endswith('.csv'):
-                df_compare = pd.read_csv(compare_file_path)
-            elif compare_file_path.endswith('.parquet'):
-                df_compare = pd.read_parquet(compare_file_path)
-            else:
-                raise ValueError(f"Unsupported compare file format: {compare_file_path}")
-            
-            report = sv.compare([df, "Dataset 1"], [df_compare, "Dataset 2"], target_column)
-        elif target_column:
-            # Analysis with target variable
-            if target_column not in df.columns:
-                available = list(df.columns)
-                return {
-                    "success": False,
-                    "error": f"Column '{target_column}' not found. Available columns: {', '.join(available)}",
-                    "suggestion": f"Did you mean one of: {', '.join(available[:5])}?"
-                }
-            report = sv.analyze([df, "Dataset"], target_feat=target_column)
-        else:
-            # Basic analysis without target
-            report = sv.analyze(df)
-        
-        # Generate HTML report
-        report.show_html(filepath=output_path, open_browser=False, layout='vertical', scale=1.0)
-        
-        # Get summary statistics
-        num_features = len(df.columns)
-        num_rows = len(df)
-        num_numeric = df.select_dtypes(include=['number']).shape[1]
-        num_categorical = df.select_dtypes(include=['object', 'category']).shape[1]
-        missing_pct = (df.isnull().sum().sum() / (num_rows * num_features)) * 100
-        
-        return {
-            "success": True,
-            "report_path": output_path,
-            "message": f"✅ Sweetviz report generated successfully at: {output_path}",
-            "summary": {
-                "features": num_features,
-                "rows": num_rows,
-                "numeric_features": num_numeric,
-                "categorical_features": num_categorical,
-                "missing_percentage": round(missing_pct, 2),
-                "target_column": target_column,
-                "has_comparison": compare_file_path is not None
-            }
-        }
-        
-    except ImportError:
-        return {
-            "success": False,
-            "error": "Sweetviz not installed. Install with: pip install sweetviz",
-            "error_type": "MissingDependency",
-            "workaround": "Use generate_ydata_profiling_report as an alternative for comprehensive EDA reports."
-        }
-    except AttributeError as e:
-        if "VisibleDeprecationWarning" in str(e) or "numpy" in str(e).lower():
-            return {
-                "success": False,
-                "error": "Sweetviz is incompatible with NumPy 2.x. NumPy version downgrade required.",
-                "error_type": "DependencyConflict",
-                "solution": "Downgrade NumPy to 1.x: py -m pip install 'numpy<2.0'",
-                "workaround": "Use generate_ydata_profiling_report instead - it's fully compatible with NumPy 2.x and provides more comprehensive analysis.",
-                "alternative_report_path": output_path.replace("sweetviz", "ydata_profile")
-            }
-        raise
-    except Exception as e:
-        return {
-            "success": False,
-            "error": f"Failed to generate Sweetviz report: {str(e)}",
-            "error_type": type(e).__name__,
-            "workaround": "Try generate_ydata_profiling_report for a comprehensive EDA report instead."
-        }
-
-
 def generate_ydata_profiling_report(
     file_path: str,
     output_path: str = "./outputs/reports/ydata_profile.html",
@@ -250,90 +128,3 @@ def generate_ydata_profiling_report(
             "error": f"Failed to generate ydata-profiling report: {str(e)}",
             "error_type": type(e).__name__
         }
-
-
-def generate_combined_eda_report(
-    file_path: str,
-    output_dir: str = "./outputs/reports",
-    target_column: Optional[str] = None,
-    minimal: bool = False
-) -> Dict[str, Any]:
-    """
-    Generate both Sweetviz and ydata-profiling reports in one call.
-    
-    This convenience function creates comprehensive EDA reports using both tools,
-    giving you the best of both worlds:
-    - Sweetviz: Beautiful, fast, focused visualizations
-    - ydata-profiling: Comprehensive, detailed analysis
-    
-    Args:
-        file_path: Path to the dataset CSV file
-        output_dir: Directory to save both reports
-        target_column: Optional target variable for Sweetviz analysis
-        minimal: If True, uses minimal mode for ydata-profiling
-        
-    Returns:
-        Dict with success status and paths to both reports
-    """
-    try:
-        # Create output directory
-        os.makedirs(output_dir, exist_ok=True)
-        
-        # Generate Sweetviz report
-        sweetviz_path = os.path.join(output_dir, "sweetviz_report.html")
-        sweetviz_result = generate_sweetviz_report(
-            file_path=file_path,
-            output_path=sweetviz_path,
-            target_column=target_column
-        )
-        
-        # Generate ydata-profiling report
-        ydata_path = os.path.join(output_dir, "ydata_profile.html")
-        ydata_result = generate_ydata_profiling_report(
-            file_path=file_path,
-            output_path=ydata_path,
-            minimal=minimal
-        )
-        
-        # Check if both succeeded
-        both_success = sweetviz_result["success"] and ydata_result["success"]
-        
-        if both_success:
-            return {
-                "success": True,
-                "message": f"✅ Generated both EDA reports successfully in: {output_dir}",
-                "reports": {
-                    "sweetviz": {
-                        "path": sweetviz_path,
-                        "summary": sweetviz_result.get("summary", {})
-                    },
-                    "ydata_profiling": {
-                        "path": ydata_path,
-                        "statistics": ydata_result.get("statistics", {})
-                    }
-                },
-                "recommendation": "Open both reports in your browser to get comprehensive insights!"
-            }
-        else:
-            # At least one failed
-            errors = []
-            if not sweetviz_result["success"]:
-                errors.append(f"Sweetviz: {sweetviz_result['error']}")
-            if not ydata_result["success"]:
-                errors.append(f"ydata-profiling: {ydata_result['error']}")
-            
-            return {
-                "success": False,
-                "error": " | ".join(errors),
-                "partial_results": {
-                    "sweetviz": sweetviz_result,
-                    "ydata_profiling": ydata_result
-                }
-            }
-            
-    except Exception as e:
-        return {
-            "success": False,
-            "error": f"Failed to generate combined reports: {str(e)}",
-            "error_type": type(e).__name__
-        }

src/tools/tools_registry.py

@@ -1431,29 +1431,12 @@ TOOLS = [
             }
         }
     },
-    # EDA Report Generation (3) - NEW PHASE 2
-    {
-        "type": "function",
-        "function": {
-            "name": "generate_sweetviz_report",
-            "description": "Generate beautiful HTML EDA report using Sweetviz. Creates stunning visualizations with target analysis, feature distributions, correlations, missing values. Fast and visually appealing. Supports dataset comparison (train vs test).",
-            "parameters": {
-                "type": "object",
-                "properties": {
-                    "file_path": {"type": "string", "description": "Path to the dataset CSV/Parquet file"},
-                    "output_path": {"type": "string", "description": "Where to save HTML report (default: ./outputs/reports/sweetviz_report.html)"},
-                    "target_column": {"type": "string", "description": "Optional target variable for association analysis"},
-                    "compare_file_path": {"type": "string", "description": "Optional second dataset to compare (e.g., train vs test)"}
-                },
-                "required": ["file_path"]
-            }
-        }
-    },
+    # EDA Report Generation (1) - NEW PHASE 2
     {
         "type": "function",
         "function": {
             "name": "generate_ydata_profiling_report",
-            "description": "Generate comprehensive HTML report using ydata-profiling (formerly pandas-profiling). Provides extensive analysis: overview, variable statistics, interactions, correlations (Pearson, Spearman, Cramér's V), missing values matrix, duplicate analysis, and more. Most detailed profiling tool.",
+            "description": "Generate comprehensive HTML report using ydata-profiling (formerly pandas-profiling). Provides extensive analysis: overview, variable statistics, interactions, correlations (Pearson, Spearman, Cramér's V), missing values matrix, duplicate analysis, and more. Most detailed and comprehensive profiling tool with automated insights and data quality warnings.",
             "parameters": {
                 "type": "object",
                 "properties": {
@@ -1466,23 +1449,6 @@ TOOLS = [
             }
         }
     },
-    {
-        "type": "function",
-        "function": {
-            "name": "generate_combined_eda_report",
-            "description": "Generate BOTH Sweetviz and ydata-profiling reports in one call. Best of both worlds: Sweetviz for beautiful fast visualizations + ydata-profiling for comprehensive detailed analysis. Recommended for complete EDA.",
-            "parameters": {
-                "type": "object",
-                "properties": {
-                    "file_path": {"type": "string", "description": "Path to the dataset CSV/Parquet file"},
-                    "output_dir": {"type": "string", "description": "Directory to save both reports (default: ./outputs/reports)"},
-                    "target_column": {"type": "string", "description": "Optional target variable for Sweetviz analysis"},
-                    "minimal": {"type": "boolean", "description": "If true, uses minimal mode for ydata-profiling (default: false)"}
-                },
-                "required": ["file_path"]
-            }
-        }
-    },
     # ========================================
     # CODE INTERPRETER - THE GAME CHANGER 🚀
     # ========================================