README.md

---
title: Intelligent_PID
emoji: 🔍
colorFrom: blue
colorTo: red
sdk: gradio
sdk_version: 5.3.0
app_file: gradioChatApp.py
pinned: false
license: mit
---

# P&ID Processing with AI-Powered Graph Construction

## Overview

This project processes P&ID (Piping and Instrumentation Diagram) images using multiple AI models for symbol detection, text recognition, and line detection. It constructs a graph representation of the diagram and provides an interactive interface for querying the diagram's contents.

## Process Flow

```mermaid
graph TD
    subgraph "Document Input"
        A[Upload Document] --> B[Validate File]
        B -->|PDF/Image| C[Document Processor]
        B -->|Invalid| ERR[Error Message]
        C -->|PDF| D1[Extract Pages]
        C -->|Image| D2[Direct Process]
    end

    subgraph "Image Preprocessing"
        D1 --> E[Optimize Image]
        D2 --> E
        E -->|CLAHE Enhancement| E1[Contrast Enhancement]
        E1 -->|Denoising| E2[Clean Image]
        E2 -->|Binarization| E3[Binary Image]
        E3 -->|Resize| E4[Normalized Image]
    end

    subgraph "Line Detection Pipeline"
        E4 --> L1[Load DeepLSD Model]
        L1 --> L2[Scale Image 0.1x]
        L2 --> L3[Grayscale Conversion]
        L3 --> L4[Model Inference]
        L4 --> L5[Scale Coordinates]
        L5 --> L6[Draw Lines]
    end

    subgraph "Detection Pipeline"
        E4 --> F[Symbol Detection]
        E4 --> G[Text Detection]
        
        F --> S1[Load YOLO Models]
        G --> T1[Load OCR Models]
        
        S1 --> S2[Detect Symbols]
        T1 --> T2[Detect Text]
        
        S2 --> S3[Process Symbols]
        T2 --> T3[Process Text]
        
        L6 --> L7[Process Lines]
    end

    subgraph "Data Integration"
        S3 --> I[Data Aggregation]
        T3 --> I
        L7 --> I
        I --> J[Create Edges]
        J --> K[Build Graph Network]
        K --> L[Generate Knowledge Graph]
    end

    subgraph "User Interface"
        L --> M[Interactive Visualization]
        M --> N[Chat Interface]
        N --> O[Query Processing]
        O --> P[Response Generation]
        P --> N
    end

    style A fill:#f9f,stroke:#333,stroke-width:2px
    style F fill:#fbb,stroke:#333,stroke-width:2px
    style G fill:#bfb,stroke:#333,stroke-width:2px
    style H fill:#bbf,stroke:#333,stroke-width:2px
    style I fill:#fbf,stroke:#333,stroke-width:2px
    style N fill:#bbf,stroke:#333,stroke-width:2px
    
    %% Add style for model nodes
    style SM1 fill:#ffe6e6,stroke:#333,stroke-width:2px
    style SM2 fill:#ffe6e6,stroke:#333,stroke-width:2px
    style LM1 fill:#e6e6ff,stroke:#333,stroke-width:2px
    style DC1 fill:#e6ffe6,stroke:#333,stroke-width:2px
    style DC2 fill:#e6ffe6,stroke:#333,stroke-width:2px
```

## Architecture

![Project Architecture](./assets/P&ID_to_Graph.drawio.png)

## Features

- **Multi-modal AI Processing**: 
  - Combined OCR approach using Tesseract, EasyOCR, and DocTR
  - Symbol detection with optimized thresholds
  - Intelligent line and connection detection
- **Document Processing**:
  - Support for PDF, PNG, JPG, JPEG formats
  - Automatic page extraction from PDFs
  - Image optimization pipeline
- **Text Detection Types**:
  - Equipment Tags
  - Line Numbers
  - Instrument Tags
  - Valve Numbers
  - Pipe Sizes
  - Flow Directions
  - Service Descriptions
  - Process Instruments
  - Nozzles
  - Pipe Connectors
- **Data Integration**:
  - Automatic edge detection
  - Relationship mapping
  - Confidence scoring
  - Detailed detection statistics
- **User Interface**:
  - Interactive visualization tabs
  - Real-time processing feedback
  - AI-powered chat interface
  - Knowledge graph exploration

The entire process is visualized through an interactive Gradio-based UI, allowing users to upload a P&ID image, follow the detection steps, and view both the results and insights in real time.

## Key Files

- **gradioChatApp.py**: The main Gradio app script that handles the frontend and orchestrates the overall flow.
- **symbol_detection.py**: Module for detecting symbols using YOLO models.
- **text_detection_combined.py**: Unified module for text detection using multiple OCR engines (Tesseract, EasyOCR, DocTR).
- **line_detection_ai.py**: Module for detecting lines and connections using AI.
- **data_aggregation.py**: Aggregates detected elements into a structured format.
- **graph_construction.py**: Constructs the graph network from aggregated data.
- **graph_processor.py**: Handles graph visualization and processing.
- **pdf_processor.py**: Handles PDF document processing and page extraction.

## Setup and Installation

1. Clone the repository:
```bash
git clone https://github.com/IntuigenceAI/intui-PnID-POC.git
cd intui-PnID-POC
```

2. Install dependencies using uv:
```bash
# Install uv if you haven't already
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create and activate virtual environment
uv venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install dependencies
uv pip install -r requirements.txt
```

3. Download required models:
```bash
python download_model.py  # Downloads DeepLSD model for line detection
```

4. Run the application:
```bash
python gradioChatApp.py
```

## Models

### Line Detection Model
- **DeepLSD Model**: 
  - File: deeplsd_md.tar
  - Purpose: Line segment detection in P&ID diagrams
  - Input Resolution: Variable (scaled to 0.1x for performance)
  - Processing: Grayscale conversion and binary thresholding

### Text Detection Models
- **Combined OCR Approach**:
  - Tesseract OCR
  - EasyOCR
  - DocTR
  - Purpose: Text recognition and classification

### Graph Processing
- **NetworkX-based**:
  - Purpose: Graph construction and analysis
  - Features: Node linking, edge creation, path analysis

## Updating the Environment

To update the environment, use the following:

```bash
conda env update --file environment.yml --prune
```

This command will update the environment according to changes made in the `environment.yml`.

### Step 6: Deactivate the environment

When you're done, deactivate the environment by:

```bash
conda deactivate
```

2. Upload a P&ID image through the interface.
3. Follow the sequential steps of symbol, text, and line detection.
4. View the generated graph and AI agent's reasoning in the real-time chat box.
5. Save and export the results if satisfactory.

## Folder Structure

```
├── assets/
│   └── AiAgent.png
│   └── llm.png
├── gradioApp.py
├── symbol_detection.py
├── text_detection_combined.py
├── line_detection_ai.py
├── data_aggregation.py
├── graph_construction.py
├── graph_processor.py
├── pdf_processor.py
├── pnid_agent.py
├── requirements.txt
├── results/
├── models/
│   └── symbol_detection_model.pth
```

## /models Folder

- **models/symbol_detection_model.pth**: This folder contains the pre-trained model for symbol detection in P&ID diagrams. This model is crucial for detecting key symbols such as valves, instruments, and pipes in the diagram. Make sure to download the model and place it in the `/models` directory before running the app.

## Future Work

- **Advanced Symbol Recognition**: Improve symbol detection by integrating more sophisticated recognition models.
- **Graph Enhancement**: Introduce more complex graph structures and logic for representing the relationships between the diagram's elements.
- **Data Export**: Allow export in additional formats such as DEXPI-compliant XML or JSON.


# Docker Information

We'll cover the basic docker operations here.

## Building

There is a dockerfile for each different project (they have slightly different requiremnts). 

###  `gradioChatApp.py`

Run this one as follows:

```
> docker build -t exp-pnid-to-graph_chat-w-graph:0.0.4 -f Dockerfile-chatApp .
> docker tag exp-pnid-to-graph_chat-w-graph:0.0.4 intaicr.azurecr.io/intai/exp-pnid-to-graph_chat-w-graph:0.0.4
```

## Deploying to ACR

###  `gradioChatApp.py`

```
> az login
> az acr login --name intaicr
> docker push intaicr.azurecr.io/intai/exp-pnid-to-graph_chat-w-graph:0.0.4
```

## Models

### Symbol Detection Models
- **Intui_SDM_41.pt**: Primary model for equipment and large symbol detection
  - Classes: Equipment, Vessels, Heat Exchangers
  - Input Resolution: 1280x1280
  - Confidence Threshold: 0.3-0.7 (adaptive)

- **Intui_SDM_20.pt**: Secondary model for instrument and small symbol detection
  - Classes: Instruments, Valves, Indicators
  - Input Resolution: 1280x1280
  - Confidence Threshold: 0.3-0.7 (adaptive)

### Line Detection Model
- **intui_LDM_01.pt**: Specialized model for line and connection detection
  - Classes: Solid Lines, Dashed Lines
  - Input Resolution: 1280x1280
  - Confidence Threshold: 0.5

### Text Detection Models
- **Tesseract**: v5.3.0
  - Configuration: 
    - OEM Mode: 3 (Default)
    - PSM Mode: 11 (Sparse text)
    - Custom Whitelist: A-Z, 0-9, special characters

- **EasyOCR**: v1.7.1
  - Configuration:
    - Language: English
    - Paragraph Mode: False
    - Height Threshold: 2.0
    - Width Threshold: 2.0
    - Contrast Threshold: 0.2

- **DocTR**: v0.6.0
  - Models:
    - fast_base-688a8b34.pt
    - crnn_vgg16_bn-9762b0b0.pt

# P&ID Line Detection

A deep learning-based pipeline for detecting lines in P&ID diagrams using DeepLSD.

## Architecture
```mermaid
graph TD
    A[Input Image] --> B[Line Detection]
    B --> C[DeepLSD Model]
    C --> D[Post-processing]
    D --> E[Output JSON/Image]
    
    subgraph Line Detection Pipeline
    B --> F[Image Preprocessing]
    F --> G[Scale Image 0.1x]
    G --> H[Grayscale Conversion]
    H --> C
    C --> I[Scale Coordinates]
    I --> J[Draw Lines]
    J --> E
    end
```

## Setup

### Prerequisites
- Python 3.12+
- uv (for dependency management)
- Git
- CUDA-capable GPU (optional)

### Installation

1. Clone the repository:
```bash
git clone https://github.com/IntuigenceAI/intui-PnID-POC.git
cd intui-PnID-POC
```

2. Install dependencies using uv:
```bash
# Install uv if you haven't already
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create and activate virtual environment
uv venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install dependencies
uv pip install -r requirements.txt
```

3. Download DeepLSD model:
```bash
python download_model.py
```

## Usage

1. Run the line detection:
```bash
python line_detection_ai.py
```

The script will:
- Load the DeepLSD model
- Process input images at 0.1x scale for performance
- Generate line detections
- Save results as JSON and annotated images

## Configuration

Key parameters in `line_detection_ai.py`:
- `scale_factor`: Image scaling (default: 0.1)
- `device`: CPU/GPU selection
- `mask_json_paths`: Paths to text/symbol detection results

## Input/Output

### Input
- Original P&ID images
- Optional text/symbol detection JSON files for masking

### Output
- Annotated images with detected lines
- JSON files containing line coordinates and metadata

## Project Structure

```
├── line_detection_ai.py    # Main line detection script
├── detectors.py           # Line detector implementation
├── download_model.py      # Model download utility
├── models/               # Directory for model files
│   └── deeplsd_md.tar   # DeepLSD model weights
├── results/              # Output directory
└── requirements.txt      # Project dependencies
```

## Dependencies

Key dependencies:
- torch
- opencv-python
- numpy
- DeepLSD

See `requirements.txt` for the complete list.

## Contributing

1. Fork the repository
2. Create your feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add some amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## Acknowledgments

- [DeepLSD](https://github.com/cvg/DeepLSD) for the line detection model
- Original P&ID processing pipeline by IntuigenceAI

---
title: PnID Diagram Analyzer
emoji: 🔍
colorFrom: blue
colorTo: red
sdk: gradio
sdk_version: 4.19.2
app_file: gradioChatApp.py
pinned: false
---

# PnID Diagram Analyzer

This app analyzes PnID diagrams using AI to detect and interpret various elements.

## Features
- Line detection
- Symbol recognition
- Text detection
- Graph construction