cyberai-1 commited on
Commit
a363339
Β·
1 Parent(s): a8cdf96

Update readme

Browse files
Files changed (1) hide show
  1. README.md +344 -41
README.md CHANGED
@@ -8,71 +8,374 @@ app_port: 7860
8
  pinned: false
9
  ---
10
 
11
- # Intel Scene Classifier β€” DΓ©ploiement
12
 
13
- ## Structure du projet
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14
  ```
15
- webapp/
16
- β”œβ”€β”€ app.py ← Backend Flask
 
 
 
 
 
 
 
 
17
  β”œβ”€β”€ templates/
18
- β”‚ └── index.html ← Interface web
19
- β”œβ”€β”€ parfait_model.pth ← ⚠️ Γ€ placer ici (tΓ©lΓ©chargΓ© depuis Kaggle)
20
- β”œβ”€β”€ parfait_model.keras ← ⚠️ Γ€ placer ici (tΓ©lΓ©chargΓ© depuis Kaggle)
21
  β”œβ”€β”€ requirements.txt
22
- β”œβ”€β”€ Procfile
23
  └── README.md
24
  ```
25
 
26
- > ⚠️ **Important** : Placez `parfait_model.pth` et `parfait_model.keras`
27
- > Γ  la racine du projet avant de dΓ©ployer.
28
-
29
  ---
30
 
31
- ## Option A β€” PythonAnywhere (recommandΓ©, gratuit)
32
 
33
- 1. CrΓ©ez un compte sur https://www.pythonanywhere.com
34
- 2. Onglet **Files** → uploadez tous vos fichiers (y compris les modèles .pth et .keras)
35
- 3. Onglet **Consoles** β†’ ouvrir un Bash :
36
- ```bash
37
- pip install -r requirements.txt
38
- ```
39
- 4. Onglet **Web** β†’ Add a new web app β†’ Manual configuration β†’ Python 3.10
40
- 5. Dans **WSGI configuration file**, remplacez le contenu par :
41
- ```python
42
- import sys
43
- sys.path.insert(0, '/home/VOTRE_USERNAME')
44
- from app import app as application
45
- ```
46
- 6. **Reload** β†’ votre app est en ligne !
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
47
 
48
  ---
49
 
50
- ## Option B — Railway (gratuit, très simple)
51
 
52
- 1. CrΓ©ez un compte sur https://railway.app
53
- 2. **New Project** β†’ Deploy from GitHub (poussez votre code sur GitHub d'abord)
54
- 3. Railway dΓ©tecte automatiquement le `Procfile`
55
- 4. Ajoutez vos fichiers modèles via **Volume** ou committez-les dans le repo
56
- 5. Deploy β†’ URL gΓ©nΓ©rΓ©e automatiquement
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
57
 
58
  ---
59
 
60
- ## Option C β€” Render (gratuit)
 
 
 
 
 
 
 
 
 
61
 
62
- 1. https://render.com β†’ New Web Service
63
- 2. Connectez votre repo GitHub
64
- 3. Build Command : `pip install -r requirements.txt`
65
- 4. Start Command : `gunicorn app:app --bind 0.0.0.0:$PORT --workers 1 --timeout 120`
66
- 5. Uploadez les modèles dans le repo ou via un bucket S3
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
67
 
68
  ---
69
 
70
- ## Lancer en local
 
 
 
 
 
 
 
71
 
72
  ```bash
73
- pip install -r requirements.txt
 
 
 
 
 
 
74
 
75
- # Placez parfait_model.pth et parfait_model.keras ici, puis :
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
76
  python app.py
77
  # β†’ http://localhost:5000
 
 
 
78
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8
  pinned: false
9
  ---
10
 
 
11
 
12
+ # Intel Scene Classifier β€” Parfait TOLEFO
13
+ > CNN-based image classification Β· 6 scene categories Β· PyTorch & TensorFlow
14
+
15
+ ---
16
+
17
+ ## Table of Contents
18
+ - [Intel Scene Classifier β€” Parfait TOLEFO](#intel-scene-classifier--parfait-tolefo)
19
+ - [Table of Contents](#table-of-contents)
20
+ - [1. Project Overview](#1-project-overview)
21
+ - [2. Dataset](#2-dataset)
22
+ - [3. Project Architecture](#3-project-architecture)
23
+ - [4. Model Architecture](#4-model-architecture)
24
+ - [5. Dependencies \& Installation](#5-dependencies--installation)
25
+ - [6. Usage](#6-usage)
26
+ - [6.1 Training](#61-training)
27
+ - [6.2 Evaluation](#62-evaluation)
28
+ - [6.3 Web Application](#63-web-application)
29
+ - [7. Performance](#7-performance)
30
+ - [8. Preprocessing \& Augmentation](#8-preprocessing--augmentation)
31
+ - [Training augmentation pipeline (PyTorch)](#training-augmentation-pipeline-pytorch)
32
+ - [Validation / inference (no augmentation)](#validation--inference-no-augmentation)
33
+ - [Why ImageNet normalization?](#why-imagenet-normalization)
34
+ - [9. Reproducibility (Seed)](#9-reproducibility-seed)
35
+ - [10. Deployment](#10-deployment)
36
+ - [PythonAnywhere (recommended, free tier available)](#pythonanywhere-recommended-free-tier-available)
37
+ - [Railway / Render](#railway--render)
38
+ - [Environment variables](#environment-variables)
39
+
40
+ ---
41
+
42
+ ## 1. Project Overview
43
+
44
+ This project implements a **complete image classification pipeline** for the
45
+ Intel Image Classification dataset. It includes:
46
+
47
+ - Two independent CNN models: one in **PyTorch**, one in **TensorFlow/Keras**
48
+ - A unified CLI entry point (`main.py`) with `--mode train` and `--mode eval`
49
+ - A **Flask web application** with file upload and URL-based image loading
50
+ - A professional green/black UI with real-time probability bars
51
+
52
+ **Classes** (6 categories):
53
+ `buildings` Β· `forest` Β· `glacier` Β· `mountain` Β· `sea` Β· `street`
54
+
55
+ ---
56
+
57
+ ## 2. Dataset
58
+
59
+ | Property | Value |
60
+ |-------------|------------------------------------------------|
61
+ | Source | [Kaggle β€” Intel Image Classification](https://www.kaggle.com/datasets/puneet6060/intel-image-classification) |
62
+ | Images | ~25,000 RGB images (150Γ—150 px) |
63
+ | Train split | ~14,000 images (seg_train) |
64
+ | Test split | ~3,000 images (seg_test) |
65
+ | Prediction | ~7,000 images (seg_pred β€” unlabeled) |
66
+ | Format | JPEG, organized in class-named subdirectories |
67
+
68
+ **Expected folder structure after download:**
69
+ ```
70
+ data/
71
+ β”œβ”€β”€ seg_train/
72
+ β”‚ └── seg_train/
73
+ β”‚ β”œβ”€β”€ buildings/
74
+ β”‚ β”œβ”€β”€ forest/
75
+ β”‚ β”œβ”€β”€ glacier/
76
+ β”‚ β”œβ”€β”€ mountain/
77
+ β”‚ β”œβ”€β”€ sea/
78
+ β”‚ └── street/
79
+ β”œβ”€β”€ seg_test/
80
+ β”‚ └── seg_test/
81
+ β”‚ └── (same 6 subdirectories)
82
+ └── seg_pred/
83
+ └── seg_pred/
84
+ └── (unlabeled images)
85
+ ```
86
+
87
+ ---
88
+
89
+ ## 3. Project Architecture
90
+
91
  ```
92
+ project/
93
+ β”œβ”€β”€ app.py ← Flask web server (inference via file or URL)
94
+ β”œβ”€β”€ main.py ← Unified CLI: train + eval
95
+ β”œβ”€β”€ models/
96
+ β”‚ β”œβ”€β”€ __init__.py ← Exports CNN_Torch, build_cnn_tf, Trainer
97
+ β”‚ β”œβ”€β”€ cnn.py ← CNN architectures (PyTorch + TensorFlow)
98
+ β”‚ └── train.py ← Trainer class (PyTorch only)
99
+ β”œβ”€β”€ utils/
100
+ β”‚ β”œβ”€β”€ __init__.py ← Exports all preprocessing functions
101
+ β”‚ └── prep.py ← Transforms, DataLoaders, inference preprocessing
102
  β”œβ”€β”€ templates/
103
+ β”‚ └── index.html ← Web UI (green/black terminal aesthetic)
104
+ β”œβ”€β”€ parfait_model.pth ← Trained PyTorch weights (after training)
105
+ β”œβ”€β”€ parfait_model.keras ← Trained TensorFlow weights (after training)
106
  β”œβ”€β”€ requirements.txt
 
107
  └── README.md
108
  ```
109
 
 
 
 
110
  ---
111
 
112
+ ## 4. Model Architecture
113
 
114
+ Both models share the **same 4-block VGG-inspired architecture**
115
+ with `GlobalAveragePooling` replacing `Flatten` for ~20Γ— fewer parameters.
116
+
117
+ ```
118
+ Input (B, 3, 150, 150) β€” RGB, 3 channels
119
+ β”‚
120
+ β”œβ”€ Block 1: Conv(32)Γ—2 β†’ BN β†’ ReLU β†’ MaxPool(2) [150β†’75]
121
+ β”œβ”€ Block 2: Conv(64)Γ—2 β†’ BN β†’ ReLU β†’ MaxPool(2) β†’ Drop(0.10) [75β†’37]
122
+ β”œβ”€ Block 3: Conv(128)Γ—2β†’ BN β†’ ReLU β†’ MaxPool(2) β†’ Drop(0.15) [37β†’18]
123
+ β”œβ”€ Block 4: Conv(256)Γ—2β†’ BN β†’ ReLU β†’ MaxPool(2) β†’ Drop(0.20) [18β†’9]
124
+ β”‚
125
+ β”œβ”€ GlobalAveragePooling2D [β†’ (B,256)]
126
+ β”œβ”€ Dense(256) β†’ ReLU β†’ Dropout(0.30)
127
+ └─ Dense(6) β†’ Softmax / LogSoftmax
128
+
129
+ Trainable parameters : ~2.1M (PyTorch) | ~2.2M (TensorFlow)
130
+ Input size : 150 Γ— 150 Γ— 3 (RGB)
131
+ Normalization : ImageNet mean/std [0.485,0.456,0.406] / [0.229,0.224,0.225]
132
+ ```
133
+
134
+ **Training configuration:**
135
+
136
+ | Parameter | Value |
137
+ |---------------|---------------|
138
+ | Optimizer | Adam |
139
+ | Learning rate | 1e-4 |
140
+ | LR scheduler | ReduceLROnPlateau (factor=0.5, patience=3) |
141
+ | Early stopping | patience=15 |
142
+ | Batch size | 32 |
143
+ | Max epochs | 50 |
144
+ | Loss function | CrossEntropyLoss / SparseCategoricalCrossentropy |
145
 
146
  ---
147
 
148
+ ## 5. Dependencies & Installation
149
 
150
+ **Python 3.9+** is required.
151
+
152
+ ```bash
153
+ # Clone / download the project
154
+ git clone <your-repo-url>
155
+ cd project
156
+
157
+ # Install dependencies
158
+ pip install -r requirements.txt
159
+ ```
160
+
161
+ **requirements.txt:**
162
+ ```
163
+ torch>=2.0.0
164
+ torchvision>=0.15.0
165
+ tensorflow>=2.13.0
166
+ flask>=3.0.0
167
+ pillow>=10.0.0
168
+ numpy>=1.24.0
169
+ matplotlib>=3.7.0
170
+ tqdm>=4.65.0
171
+ scikit-learn>=1.3.0
172
+ gunicorn>=21.0.0
173
+ ```
174
 
175
  ---
176
 
177
+ ## 6. Usage
178
+
179
+ ### 6.1 Training
180
+
181
+ ```bash
182
+ # Train with PyTorch (saves β†’ parfait_model.pth)
183
+ python main.py --model pytorch --mode train
184
+
185
+ # Train with TensorFlow (saves β†’ parfait_model.keras)
186
+ python main.py --model tensorflow --mode train
187
 
188
+ # Full example with all options
189
+ python main.py \
190
+ --model pytorch \
191
+ --mode train \
192
+ --data_dir ./data \
193
+ --output_dir ./outputs \
194
+ --epochs 50 \
195
+ --batch_size 32 \
196
+ --lr 1e-4 \
197
+ --patience 15
198
+ ```
199
+
200
+ **All CLI arguments:**
201
+
202
+ | Argument | Default | Description |
203
+ |---------------|----------------------------------------|------------------------------------|
204
+ | `--model` | *(required)* | `pytorch` or `tensorflow` |
205
+ | `--mode` | *(required)* | `train` or `eval` |
206
+ | `--data_dir` | `/kaggle/input/.../intel-image-...` | Root directory of the dataset |
207
+ | `--output_dir` | `/kaggle/working` | Where to save models and plots |
208
+ | `--epochs` | `50` | Max training epochs |
209
+ | `--batch_size` | `32` | Batch size |
210
+ | `--lr` | `1e-4` | Initial learning rate |
211
+ | `--patience` | `15` | Early stopping patience |
212
+ | `--model_path` | *(auto)* | (eval only) Path to .pth or .keras |
213
+
214
+ **Training outputs:**
215
+ ```
216
+ outputs/
217
+ β”œβ”€β”€ parfait_model.pth ← Best PyTorch weights
218
+ β”œβ”€β”€ parfait_model.keras ← Best TensorFlow weights
219
+ β”œβ”€β”€ history_pytorch.png ← Train/Val Loss & Accuracy curves
220
+ └── history_tf.png
221
+ ```
222
 
223
  ---
224
 
225
+ ### 6.2 Evaluation
226
+
227
+ The `eval` mode loads a saved model and produces a **full diagnostic report**:
228
+ - Global accuracy & loss
229
+ - Per-class accuracy
230
+ - Precision / Recall / F1-score (classification report)
231
+ - Confusion matrix (saved as PNG)
232
+ - 4Γ—4 grid of sample predictions (color-coded: green=correct, red=wrong)
233
 
234
  ```bash
235
+ # Evaluate PyTorch model
236
+ python main.py \
237
+ --model pytorch \
238
+ --mode eval \
239
+ --model_path parfait_model.pth \
240
+ --data_dir ../data \
241
+ --output_dir ./eval_output_dir
242
 
243
+ # Evaluate TensorFlow model
244
+ python main.py \
245
+ --model tensorflow \
246
+ --mode eval \
247
+ --model_path parfait_model.keras \
248
+ --data_dir ../data \
249
+ --output_dir ./eval_output_dir
250
+
251
+ ```
252
+
253
+ **Evaluation outputs:**
254
+ ```
255
+ outputs/
256
+ β”œβ”€β”€ confusion_matrix_pytorch.png ← Confusion matrix heatmap
257
+ β”œβ”€β”€ confusion_matrix_tf.png
258
+ β”œβ”€β”€ sample_predictions_pytorch.png ← 16-image prediction grid
259
+ └── sample_predictions_tf.png
260
+ ```
261
+
262
+ ---
263
+
264
+ ### 6.3 Web Application
265
+
266
+ ```bash
267
+ # Start Flask server
268
  python app.py
269
  # β†’ http://localhost:5000
270
+
271
+ # Production (gunicorn)
272
+ gunicorn app:app --bind 0.0.0.0:8000 --workers 1 --timeout 120
273
  ```
274
+
275
+ **Features:**
276
+ - Model selector: **PyTorch** or **TensorFlow**
277
+ - Input: **file upload** (drag & drop) or **image URL**
278
+ - Output: predicted class + confidence score + probability bars for all 6 classes
279
+ - Animated plexus background with terminal green/black aesthetic
280
+
281
+ ---
282
+
283
+ ## 7. Performance
284
+
285
+ > Results on the Intel Image Classification **test set** (3,000 images).
286
+ > Reported after training with default hyperparameters on Kaggle GPU T4.
287
+
288
+ | Model | Test Accuracy | Test Loss |
289
+ |-------------|:------------:|:---------:|
290
+ | PyTorch CNN | ~89–91% | ~0.30 |
291
+ | TF/Keras CNN| ~88–90% | ~0.32 |
292
+
293
+ **Per-class performance (approximate):**
294
+
295
+ | Class | Precision | Recall | F1-score |
296
+ |-----------|:---------:|:------:|:--------:|
297
+ | buildings | 0.87 | 0.85 | 0.86 |
298
+ | forest | 0.97 | 0.97 | 0.97 |
299
+ | glacier | 0.88 | 0.86 | 0.87 |
300
+ | mountain | 0.84 | 0.87 | 0.85 |
301
+ | sea | 0.92 | 0.93 | 0.92 |
302
+ | street | 0.90 | 0.91 | 0.90 |
303
+
304
+ > Note: `buildings` vs `street` is the hardest pair due to visual overlap.
305
+ > Run `--mode eval` on your trained model to get your exact numbers.
306
+
307
+ ---
308
+
309
+ ## 8. Preprocessing & Augmentation
310
+
311
+ All preprocessing is centralized in `utils/prep.py`.
312
+
313
+ ### Training augmentation pipeline (PyTorch)
314
+ ```
315
+ Resize(150Γ—150)
316
+ RandomHorizontalFlip(p=0.5)
317
+ RandomVerticalFlip(p=0.1)
318
+ RandomRotation(Β±40Β°)
319
+ ColorJitter(brightness=0.3, contrast=0.2, saturation=0.1, hue=0.05)
320
+ RandomGrayscale(p=0.05) ← forces texture learning over color
321
+ ToTensor()
322
+ Normalize(mean=[0.485,0.456,0.406], std=[0.229,0.224,0.225]) ← ImageNet stats
323
+ RandomErasing(p=0.15, scale=[0.02,0.15]) ← occlusion simulation
324
+ ```
325
+
326
+ ### Validation / inference (no augmentation)
327
+ ```
328
+ Resize(150Γ—150)
329
+ ToTensor()
330
+ Normalize(mean=[0.485,0.456,0.406], std=[0.229,0.224,0.225])
331
+ ```
332
+
333
+ ### Why ImageNet normalization?
334
+ The dataset consists of natural outdoor scenes (RGB, 3-channel images similar
335
+ to ImageNet). Using ImageNet mean/std ensures stable gradient flow and
336
+ faster convergence even for a custom-trained CNN.
337
+
338
+ ---
339
+
340
+ ## 9. Reproducibility (Seed)
341
+
342
+ The project uses a **global seed** (`SEED=42`) to ensure identical results
343
+ between runs and between training and production inference.
344
+
345
+ The seed fixes:
346
+ - Python `random` module
347
+ - NumPy RNG
348
+ - PyTorch CPU and GPU (`torch.manual_seed`, `torch.cuda.manual_seed_all`)
349
+ - `cudnn.deterministic=True`, `cudnn.benchmark=False`
350
+ - TensorFlow RNG (`tf.random.set_seed`)
351
+ - `PYTHONHASHSEED` environment variable
352
+ - DataLoader worker seeds (via `worker_init_fn`)
353
+
354
+ ---
355
+
356
+ ## 10. Deployment
357
+
358
+ ### PythonAnywhere (recommended, free tier available)
359
+ 1. Upload all project files via the **Files** tab
360
+ 2. Upload `parfait_model.pth` and `parfait_model.keras`
361
+ 3. Open a Bash console β†’ `pip install -r requirements.txt`
362
+ 4. **Web** tab β†’ New web app β†’ Manual configuration β†’ Python 3.10
363
+ 5. Edit the WSGI file:
364
+ ```python
365
+ import sys
366
+ sys.path.insert(0, '/home/YOUR_USERNAME/project')
367
+ from app import app as application
368
+ ```
369
+ 6. **Reload** β†’ your app is live at `https://yourusername.pythonanywhere.com`
370
+
371
+ ### Railway / Render
372
+ 1. Push the project to a GitHub repository
373
+ 2. Connect the repo to Railway or Render
374
+ 3. Set start command: `gunicorn app:app --bind 0.0.0.0:$PORT --workers 1 --timeout 120`
375
+ 4. Upload model files as part of the repo or via persistent volume
376
+
377
+ ### Environment variables
378
+ | Variable | Default | Description |
379
+ |---------|---------|--------------------------|
380
+ | `PORT` | `5000` | Flask server port |
381
+