INTEGRATION_GUIDELINES.md

# Lung Cancer Classification API - Flutter Integration Guidelines

Complete guide for integrating the Grad-CAM backend API into your Flutter application.

---

## Table of Contents
1. [API Overview](#api-overview)
2. [Base URL & Configuration](#base-url--configuration)
3. [Endpoints](#endpoints)
4. [Request/Response Formats](#requestresponse-formats)
5. [Image Handling](#image-handling)
6. [Error Handling](#error-handling)
7. [Code Examples](#code-examples)
8. [UI Integration](#ui-integration)
9. [Best Practices](#best-practices)
10. [Troubleshooting](#troubleshooting)

---

## API Overview

This API provides:
- **YOLO Tumor Detection**: Detects lung tumors in CT scan images
- **DenseNet121 Classification**: Classifies detected tumors into 4 cancer types
- **Grad-CAM Visualization**: Explains model predictions with attention heatmaps
- **CT Scan Validation**: Validates input is a medical image (not a color photo)

### Supported Cancer Classes
```
0. Adenocarcinoma (Class A)
1. Small Cell (Class B)
2. Large Cell (Class E)
3. Squamous Cell (Class G)
```

---

## Base URL & Configuration

### Development
```
Base URL: http://localhost:5001
```

### Production (Railway Deployment)
```
Base URL: https://your-railway-app.up.railway.app
```

### Environment Configuration
```dart
class ApiConfig {
  static const String baseUrl = 'http://localhost:5001'; // Change for production
  static const int timeout = 120; // seconds (image processing takes time)
  static const String apiVersion = 'v1.0.0';
}
```

---

## Endpoints

### 1. Health Check
**GET** `/health`

Check if API is running and models are loaded.

**Response:**
```json
{
  "status": "healthy",
  "model_loaded": true
}
```

---

### 2. Validate CT Scan
**POST** `/validate-ct`

Validate if uploaded image is a CT scan (grayscale check).

**Request:**
```
Content-Type: multipart/form-data
Body: 
  - file: <image_file>
```

**Response (Valid CT):**
```json
{
  "is_ct_scan": true,
  "color_score": 3.52,
  "message": "Valid CT scan - grayscale image detected"
}
```

**Response (Invalid - Color Image):**
```json
{
  "is_ct_scan": false,
  "color_score": 45.2,
  "message": "NOT a CT scan - color image detected (color_score=45.2)"
}
```

**Color Score Interpretation:**
- `< 6.0`: Valid CT scan (grayscale)
- `>= 6.0`: Not a CT scan (color image detected)

---

### 3. Analyze (YOLO + Classification + Grad-CAM)
**POST** `/analyze`

Complete pipeline: detect tumors → classify → generate heatmaps.

**Request:**
```
Content-Type: multipart/form-data
Body:
  - file: <image_file>
```

**Response (With Tumors):**
```json
{
  "success": true,
  "tumors_detected": 1,
  "detection_image": "base64_encoded_image_with_boxes",
  "heatmap_image": "base64_encoded_heatmap",
  "detections": [
    {
      "tumor_id": 1,
      "bbox": [161, 319, 225, 390],
      "bbox_with_padding": [149, 305, 237, 404],
      "prediction": "Small Cell (Class B)",
      "confidence": 44.62,
      "all_confidences": {
        "Adenocarcinoma (Class A)": 11.07,
        "Small Cell (Class B)": 44.62,
        "Large Cell (Class E)": 25.52,
        "Squamous Cell (Class G)": 18.79
      },
      "crop_image": "base64_encoded_crop",
      "heatmap_image": "base64_encoded_heatmap_of_crop"
    }
  ]
}
```

**Response (No Tumors):**
```json
{
  "success": true,
  "tumors_detected": 0,
  "detections": [],
  "detection_image": "base64_encoded_original_image",
  "message": "No tumors detected in this image"
}
```

**Response (Error):**
```json
{
  "error": "Could not load image"
}
```

---

## Request/Response Formats

### Multipart Form Data (Image Upload)

```dart
// Example using http package
var request = http.MultipartRequest('POST', Uri.parse('$baseUrl/analyze'));
request.files.add(
  http.MultipartFile.fromBytes(
    'file',
    imageBytes,
    filename: 'scan.jpg',
  ),
);
```

### Response Image Decoding

All images in responses are **base64-encoded JPEG**.

```dart
// Decode base64 to Image widget
Image.memory(
  base64Decode(response['detection_image']),
  fit: BoxFit.contain,
)
```

---

## Image Handling

### Supported Formats
- JPEG (.jpg, .jpeg)
- PNG (.png)
- Recommended: **JPEG** (faster processing)

### Image Size Recommendations
- **Minimum**: 256x256 pixels
- **Optimal**: 512x512 pixels
- **Maximum**: 2048x2048 pixels (will be resized internally)

### Image Acquisition

#### From Camera
```dart
import 'package:image_picker/image_picker.dart';

final picker = ImagePicker();
final pickedFile = await picker.pickImage(source: ImageSource.camera);

if (pickedFile != null) {
  final bytes = await pickedFile.readAsBytes();
  // Use bytes for upload
}
```

#### From Gallery
```dart
final pickedFile = await picker.pickImage(source: ImageSource.gallery);
if (pickedFile != null) {
  final bytes = await pickedFile.readAsBytes();
  // Use bytes for upload
}
```

#### From File
```dart
import 'dart:io';

File imageFile = File('/path/to/image.jpg');
final bytes = await imageFile.readAsBytes();
// Use bytes for upload
```

### Base64 Image Display

```dart
import 'dart:convert';

// Decode and display
Widget displayBase64Image(String base64String) {
  return Image.memory(
    base64Decode(base64String),
    fit: BoxFit.contain,
  );
}

// Or with error handling
Widget safeDisplayImage(String base64String) {
  try {
    return Image.memory(
      base64Decode(base64String),
      fit: BoxFit.contain,
      errorBuilder: (context, error, stackTrace) {
        return Center(child: Text('Failed to load image'));
      },
    );
  } catch (e) {
    return Center(child: Text('Invalid image data'));
  }
}
```

---

## Error Handling

### HTTP Status Codes

| Status | Meaning | Action |
|--------|---------|--------|
| 200 | Success | Process response |
| 400 | Bad Request | Check file format/size |
| 500 | Server Error | Retry or check logs |

### Common Errors

#### "No file uploaded"
```json
{"error": "No file uploaded"}
```
**Cause**: File not attached to request  
**Fix**: Ensure `file` field is included in multipart request

#### "Could not load image"
```json
{"error": "Could not load image"}
```
**Cause**: Corrupted or unsupported image format  
**Fix**: Use JPEG/PNG, verify file integrity

#### "YOLO model not loaded"
```json
{"error": "YOLO model not loaded"}
```
**Cause**: Backend not initialized properly  
**Fix**: Restart backend, check `models/best.pt` exists

#### Timeout
**Cause**: Image processing takes >120 seconds  
**Fix**: Reduce image size, check server performance

---

## Code Examples

### Complete Service Class

```dart
import 'package:http/http.dart' as http;
import 'dart:convert';

class LungCancerAnalysisService {
  final String baseUrl = 'http://localhost:5001';
  final int timeout = Duration(seconds: 120).inSeconds;

  // Health check
  Future<bool> healthCheck() async {
    try {
      final response = await http
          .get(Uri.parse('$baseUrl/health'))
          .timeout(Duration(seconds: 10));
      
      return response.statusCode == 200;
    } catch (e) {
      print('Health check failed: $e');
      return false;
    }
  }

  // Validate CT scan
  Future<ValidateCTResponse> validateCT(List<int> imageBytes) async {
    try {
      var request = http.MultipartRequest('POST', 
        Uri.parse('$baseUrl/validate-ct'));
      
      request.files.add(
        http.MultipartFile.fromBytes('file', imageBytes, filename: 'scan.jpg'),
      );
      
      request.headers['Accept'] = 'application/json';
      
      final streamedResponse = await request.send()
          .timeout(Duration(seconds: this.timeout));
      
      final response = await http.Response.fromStream(streamedResponse);
      
      if (response.statusCode == 200) {
        final json = jsonDecode(response.body);
        return ValidateCTResponse.fromJson(json);
      } else {
        throw Exception('Validation failed: ${response.body}');
      }
    } catch (e) {
      throw Exception('CT validation error: $e');
    }
  }

  // Full analysis with YOLO + Classification
  Future<AnalysisResponse> analyze(List<int> imageBytes) async {
    try {
      var request = http.MultipartRequest('POST', 
        Uri.parse('$baseUrl/analyze'));
      
      request.files.add(
        http.MultipartFile.fromBytes('file', imageBytes, filename: 'scan.jpg'),
      );
      
      request.headers['Accept'] = 'application/json';
      
      final streamedResponse = await request.send()
          .timeout(Duration(seconds: this.timeout));
      
      final response = await http.Response.fromStream(streamedResponse);
      
      if (response.statusCode == 200) {
        final json = jsonDecode(response.body);
        return AnalysisResponse.fromJson(json);
      } else {
        throw Exception('Analysis failed: ${response.body}');
      }
    } catch (e) {
      throw Exception('Analysis error: $e');
    }
  }
}

// Response models
class ValidateCTResponse {
  final bool isCTScan;
  final double colorScore;
  final String message;

  ValidateCTResponse({
    required this.isCTScan,
    required this.colorScore,
    required this.message,
  });

  factory ValidateCTResponse.fromJson(Map<String, dynamic> json) {
    return ValidateCTResponse(
      isCTScan: json['is_ct_scan'] ?? false,
      colorScore: (json['color_score'] ?? 0).toDouble(),
      message: json['message'] ?? '',
    );
  }
}

class AnalysisResponse {
  final bool success;
  final int tumorsDetected;
  final List<TumorDetection> detections;
  final String detectionImage; // base64
  final String heatmapImage;   // base64
  final String? message;

  AnalysisResponse({
    required this.success,
    required this.tumorsDetected,
    required this.detections,
    required this.detectionImage,
    required this.heatmapImage,
    this.message,
  });

  factory AnalysisResponse.fromJson(Map<String, dynamic> json) {
    return AnalysisResponse(
      success: json['success'] ?? false,
      tumorsDetected: json['tumors_detected'] ?? 0,
      detections: (json['detections'] as List)
          .map((d) => TumorDetection.fromJson(d))
          .toList(),
      detectionImage: json['detection_image'] ?? '',
      heatmapImage: json['heatmap_image'] ?? '',
      message: json['message'],
    );
  }
}

class TumorDetection {
  final int tumorId;
  final List<int> bbox;
  final List<int> bboxWithPadding;
  final String prediction;
  final double confidence;
  final Map<String, double> allConfidences;
  final String cropImage;     // base64
  final String heatmapImage;  // base64

  TumorDetection({
    required this.tumorId,
    required this.bbox,
    required this.bboxWithPadding,
    required this.prediction,
    required this.confidence,
    required this.allConfidences,
    required this.cropImage,
    required this.heatmapImage,
  });

  factory TumorDetection.fromJson(Map<String, dynamic> json) {
    return TumorDetection(
      tumorId: json['tumor_id'] ?? 0,
      bbox: List<int>.from(json['bbox'] ?? []),
      bboxWithPadding: List<int>.from(json['bbox_with_padding'] ?? []),
      prediction: json['prediction'] ?? 'Unknown',
      confidence: (json['confidence'] ?? 0).toDouble(),
      allConfidences: Map<String, double>.from(
        (json['all_confidences'] as Map).map(
          (k, v) => MapEntry(k, (v as num).toDouble()),
        ),
      ),
      cropImage: json['crop_image'] ?? '',
      heatmapImage: json['heatmap_image'] ?? '',
    );
  }
}
```

### Usage in Widget

```dart
class CancerAnalysisScreen extends StatefulWidget {
  @override
  _CancerAnalysisScreenState createState() => _CancerAnalysisScreenState();
}

class _CancerAnalysisScreenState extends State<CancerAnalysisScreen> {
  final service = LungCancerAnalysisService();
  
  bool isLoading = false;
  AnalysisResponse? result;
  String? errorMessage;

  void analyzeImage(List<int> imageBytes) async {
    setState(() {
      isLoading = true;
      errorMessage = null;
    });

    try {
      // Step 1: Validate CT scan
      final validation = await service.validateCT(imageBytes);
      
      if (!validation.isCTScan) {
        setState(() {
          errorMessage = 'Not a CT scan (score: ${validation.colorScore})';
          isLoading = false;
        });
        return;
      }

      // Step 2: Full analysis
      final analysis = await service.analyze(imageBytes);
      
      setState(() {
        result = analysis;
        isLoading = false;
      });

    } catch (e) {
      setState(() {
        errorMessage = 'Error: $e';
        isLoading = false;
      });
    }
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(title: Text('Lung Cancer Analysis')),
      body: SingleChildScrollView(
        child: Padding(
          padding: EdgeInsets.all(16),
          child: Column(
            children: [
              if (isLoading)
                Center(
                  child: Column(
                    children: [
                      CircularProgressIndicator(),
                      SizedBox(height: 16),
                      Text('Analyzing image...'),
                    ],
                  ),
                ),
              if (errorMessage != null)
                Container(
                  padding: EdgeInsets.all(12),
                  decoration: BoxDecoration(
                    color: Colors.red.shade100,
                    borderRadius: BorderRadius.circular(8),
                  ),
                  child: Text(
                    errorMessage!,
                    style: TextStyle(color: Colors.red.shade900),
                  ),
                ),
              if (result != null)
                Column(
                  crossAxisAlignment: CrossAxisAlignment.start,
                  children: [
                    Card(
                      child: Padding(
                        padding: EdgeInsets.all(12),
                        child: Column(
                          crossAxisAlignment: CrossAxisAlignment.start,
                          children: [
                            Text(
                              'Tumors Detected: ${result!.tumorsDetected}',
                              style: Theme.of(context).textTheme.titleLarge,
                            ),
                            SizedBox(height: 16),
                            // Detection image
                            Text('Detection Image:'),
                            Image.memory(
                              base64Decode(result!.detectionImage),
                              fit: BoxFit.contain,
                            ),
                            SizedBox(height: 16),
                            // Heatmap image
                            Text('Heatmap (Highest Confidence):'),
                            Image.memory(
                              base64Decode(result!.heatmapImage),
                              fit: BoxFit.contain,
                            ),
                          ],
                        ),
                      ),
                    ),
                    SizedBox(height: 16),
                    // Individual tumor details
                    ...result!.detections.map((tumor) {
                      return Card(
                        child: Padding(
                          padding: EdgeInsets.all(12),
                          child: Column(
                            crossAxisAlignment: CrossAxisAlignment.start,
                            children: [
                              Text(
                                'Tumor #${tumor.tumorId}',
                                style: Theme.of(context).textTheme.titleMedium,
                              ),
                              Text('Classification: ${tumor.prediction}'),
                              Text('Confidence: ${tumor.confidence.toStringAsFixed(2)}%'),
                              SizedBox(height: 12),
                              Text('All Confidences:'),
                              ...tumor.allConfidences.entries.map((e) {
                                return Text(
                                  '  ${e.key}: ${e.value.toStringAsFixed(2)}%',
                                );
                              }).toList(),
                              SizedBox(height: 12),
                              Text('Bounding Box: ${tumor.bbox}'),
                            ],
                          ),
                        ),
                      );
                    }).toList(),
                  ],
                ),
            ],
          ),
        ),
      ),
    );
  }
}
```

---

## UI Integration

### Recommended UI Flow

```
┌─────────────────────┐
│  Home Screen        │
│  - Upload Image     │
│  - Take Photo       │
│  - Gallery          │
└──────────┬──────────┘
           │
           ▼
┌─────────────────────┐
│ Validation Screen   │
│ - Show Color Score  │
│ - CT Scan Check     │
│ - Retry/Continue    │
└──────────┬──────────┘
           │
           ▼
┌─────────────────────┐
│ Loading Screen      │
│ - Progress Indicator│
│ - "Analyzing..."    │
│ - Timeout Handling  │
└──────────┬──────────┘
           │
           ▼
┌──────────────────────┐
│ Results Screen       │
│ ┌──────────────────┐ │
│ │ Detection Image  │ │
│ ├──────────────────┤ │
│ │ Heatmap Image    │ │
│ ├──────────────────┤ │
│ │ Tumor Details    │ │
│ │ - Confidence     │ │
│ │ - Classification │ │
│ │ - Per-class %    │ │
│ └──────────────────┘ │
└──────────────────────┘
```

### Confidence Visualization

```dart
Widget buildConfidenceBar(String className, double confidence) {
  final color = confidence > 0.4
      ? Colors.red
      : confidence > 0.2
          ? Colors.orange
          : Colors.green;

  return Column(
    crossAxisAlignment: CrossAxisAlignment.start,
    children: [
      Text(className),
      LinearProgressIndicator(
        value: confidence / 100,
        minHeight: 8,
        backgroundColor: Colors.grey.shade300,
        valueColor: AlwaysStoppedAnimation(color),
      ),
      Text('${confidence.toStringAsFixed(2)}%'),
    ],
  );
}
```

### Image Display with Zoom

```dart
import 'package:photo_view/photo_view.dart';

Widget zoomableImage(String base64String) {
  return PhotoView(
    imageProvider: MemoryImage(base64Decode(base64String)),
    minScale: PhotoViewComputedScale.contained * 0.8,
    maxScale: PhotoViewComputedScale.covered * 2,
  );
}
```

---

## Best Practices

### 1. Handle Timeouts Gracefully
```dart
Future<AnalysisResponse> analyzeWithTimeout(List<int> imageBytes) async {
  try {
    return await service.analyze(imageBytes).timeout(
      Duration(seconds: 120),
      onTimeout: () => throw TimeoutException('Analysis took too long'),
    );
  } on TimeoutException {
    // Show user-friendly message
    // Suggest reducing image size
  }
}
```

### 2. Validate Before Upload
```dart
bool validateImage(List<int> imageBytes) {
  // Check file size (limit to 10MB)
  const maxSizeBytes = 10 * 1024 * 1024; // 10 MB
  if (imageBytes.length > maxSizeBytes) {
    return false;
  }
  
  // Check JPEG/PNG signature
  if (imageBytes[0] == 0xFF && imageBytes[1] == 0xD8) {
    return true; // JPEG
  }
  if (imageBytes[0] == 0x89 && imageBytes[1] == 0x50) {
    return true; // PNG
  }
  
  return false;
}
```

### 3. Cache Results
```dart
class AnalysisCache {
  static final Map<String, AnalysisResponse> _cache = {};

  static String hashImage(List<int> imageBytes) {
    return sha256.convert(imageBytes).toString();
  }

  static void store(List<int> imageBytes, AnalysisResponse response) {
    _cache[hashImage(imageBytes)] = response;
  }

  static AnalysisResponse? retrieve(List<int> imageBytes) {
    return _cache[hashImage(imageBytes)];
  }
}
```

### 4. Permission Handling
```dart
import 'package:permission_handler/permission_handler.dart';

Future<bool> requestCameraPermission() async {
  final status = await Permission.camera.request();
  return status.isGranted;
}

Future<bool> requestGalleryPermission() async {
  final status = await Permission.photos.request();
  return status.isGranted;
}
```

### 5. Network State Awareness
```dart
import 'package:connectivity_plus/connectivity_plus.dart';

bool isNetworkAvailable() {
  final connectivity = Connectivity();
  return connectivity.checkConnectivity().then((result) {
    return result != ConnectivityResult.none;
  });
}

// Show offline message if needed
```

---

## Troubleshooting

### Issue: "Connection refused"
**Cause**: Backend not running  
**Solution**: 
```bash
cd /path/to/backend
source .venv/bin/activate
python main.py
```

### Issue: "Image too large"
**Cause**: File size > 10MB  
**Solution**: Compress before upload
```dart
import 'package:flutter_image_compress/flutter_image_compress.dart';

List<int> compressImage(List<int> imageBytes) {
  return XFile.fromData(imageBytes)
      .compress(quality: 85)
      .then((file) => file.readAsBytes());
}
```

### Issue: "CORS Error"
**Cause**: Frontend URLs not whitelisted  
**Solution**: Backend already has CORS enabled via Flask-CORS

### Issue: "Timeout after 120s"
**Cause**: Large image or slow server  
**Solution**: 
- Reduce image resolution
- Check server CPU/memory
- Increase timeout threshold

### Issue: "Invalid base64 image"
**Cause**: Corrupted response data  
**Solution**: 
- Log full response: `print(response.body)`
- Check server error logs
- Retry with different image

### Issue: Model not loaded
**Error**: `"YOLO model not loaded"`  
**Cause**: `models/best.pt` missing  
**Solution**: Verify model files exist in backend

---

## Environment-Specific Configuration

### Development
```dart
const String API_BASE = 'http://localhost:5001';
const bool DEBUG = true;
const int TIMEOUT = 120;
```

### Staging
```dart
const String API_BASE = 'https://staging-api.railway.app';
const bool DEBUG = true;
const int TIMEOUT = 120;
```

### Production
```dart
const String API_BASE = 'https://your-production-api.railway.app';
const bool DEBUG = false;
const int TIMEOUT = 120;
```

### Switch at Runtime
```dart
String getBaseUrl() {
  if (kDebugMode) {
    return 'http://localhost:5001';
  }
  return 'https://your-production-api.railway.app';
}
```

---

## API Testing

### cURL Examples

**Validation:**
```bash
curl -X POST http://localhost:5001/validate-ct \
  -F "file=@path/to/scan.jpg"
```

**Analysis:**
```bash
curl -X POST http://localhost:5001/analyze \
  -F "file=@path/to/scan.jpg"
```

**Health Check:**
```bash
curl http://localhost:5001/health
```

### Postman Setup
1. Create POST request to `http://localhost:5001/analyze`
2. Go to "Body" tab → Select "form-data"
3. Add key "file" (type: File)
4. Select image file
5. Send & view response

---

## Dependencies for Flutter

Add to `pubspec.yaml`:
```yaml
dependencies:
  flutter:
    sdk: flutter
  http: ^1.1.0
  image_picker: ^1.0.0
  photo_view: ^0.14.0
  flutter_image_compress: ^2.1.0
  permission_handler: ^11.4.0
  connectivity_plus: ^5.0.0
  crypto: ^3.0.0
```

---

## Performance Metrics

| Operation | Time | Notes |
|-----------|------|-------|
| Image Upload | 1-5s | Depends on image size & network |
| YOLO Detection | 5-15s | GPU faster if available |
| DenseNet Classification | 2-5s | Per tumor detected |
| Grad-CAM Heatmap | 2-3s | Per tumor |
| **Total** | **10-30s** | End-to-end (single tumor) |

---

## Support & Contact

For issues or questions:
1. Check [Troubleshooting](#troubleshooting) section
2. Review API logs: `python main.py` console output
3. Test with cURL first: ensure backend works
4. Check Flutter app logs: `flutter logs`
5. Verify network connectivity & firewall rules

---

**API Version**: 1.0.0  
**Last Updated**: March 8, 2026  
**Compatibility**: Flutter 3.0+ , Dart 3.0+