Upload 9 files

.gitignore

@@ -0,0 +1,112 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+env/
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+.hypothesis/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# pyenv
+.python-version
+
+# celery beat schedule file
+celerybeat-schedule
+
+# SageMath parsed files
+*.sage.py
+
+# dotenv
+.env
+
+# virtualenv
+.venv
+venv/
+ENV/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+
+# input data, saved log, checkpoints
+data/
+input/
+saved/
+datasets/
+
+# editor, os cache directory
+.vscode/
+.idea/
+__MACOSX/

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 Technische Universität Kaiserslautern (TUK) & National University of Sciences and Technology (NUST)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -1,3 +1,149 @@
----
-license: mit
----
+# NUST Wheat Rust Disease (NWRD): Semantic Segmentation using Suppervised Deep Learning
+Semantic segmentation of wheat yellow/stripe rust disease images to segment out rust and non-rust pixels using supervised deep learning.
+
+This repo contains the source code for the study presented in the work:
+`The NWRD Dataset: An Open-Source Annotated Segmentation Dataset of Diseased Wheat Crop`.
+
+## Dataset
+The NWRD dataset is a real-world segmentation dataset of wheat rust diseased and healthy leaf images specifically constructed for semantic segmentation of wheat rust disease.
+The NWRD dataset consists of 100 images in total at this moment.
+
+Sample images from The NWRD dataset; annotated images showing rust disease along with their binary masks:
+![Sample images from The NWRD dataset; annotated images showing rust disease along with their binary masks](https://github.com/saadulkh/nwrd/assets/38633812/c8677336-82a0-4637-a3f8-61f5cedbad37)
+
+
+Dataset is available at: https://dll.seecs.nust.edu.pk/downloads/
+
+### Directory Structure
+The NWRD dataset images are available in `.jpg` format and the annotated binary masks are available in `.png` format. Below is the directory structure of this dataset:
+```
+NWRD
+├── test
+│   ├── images
+│   └── masks
+└── train
+    ├── images
+    └── masks
+```
+
+### Data Splits
+Here are the data splits of the NWRD dataset:
+|   Split   |   Percentage  |
+|-----------|-----------|
+|   Train + Valid   |   90      |
+|   Test    |   10      |
+|   Total   |   100     |
+
+The experimentation with 22 images was done with the following set of images:
+|   Split   |   Images  |
+|-----------|-----------|
+|   Train + Valid   | 2, 7, 14, 30, 64, 83, 84, 90, 94, 95, 100, 118, 124, 125, 132, 133, 136, 137, 138, 146 |
+|   Test    | 67, 123 |
+
+## Requirements
+* Python
+* Pytorch
+* Torchvision
+* Numpy
+* Tqdm
+* Tensorboard
+* Scikit-learn
+* Pandas
+
+## Usage
+Different aspects of the training can be tunned from the `config.json` file.
+Use `python train.py -c config.json` to run code.
+
+### Config file format
+Config files are in `.json` format:
+```javascript
+{
+    "name": "WRS",                          // training session name
+    "n_gpu": 1,                             // number of GPUs to use for training.
+
+    "arch": {
+        "type": "UNet",                     // name of model architecture to train
+        "args": {
+            "n_channels": 3,
+            "n_classes": 2
+        }                                   // pass arguments to the model
+    },
+    "data_loader": {
+        "type": "PatchedDataLoader",        // selecting data loader
+        "args":{
+            "data_dir": "data/",            // dataset path
+            "patch_size": 128,              // patch size
+            "batch_size": 64,               // batch size
+            "patch_stride": 32,             // patch overlapping stride
+            "target_dist": 0.01,            // least percentage of rust pixels in a patch
+            "shuffle": true,                // shuffle training data before
+            "validation_split": 0.1,        // size of validation dataset. float(portion) or int(number of samples)
+            "num_workers": 2                // number of cpu processes to be used for data loading
+        }
+    },
+    "optimizer": {
+        "type": "RMSprop",
+        "args":{
+            "lr": 1e-6,                     // learning rate
+            "weight_decay": 0
+        }
+    },
+    "loss": "focal_loss",                   // loss function
+    "metrics": [                            // list of metrics to evaluate
+        "precision",
+        "recall",
+        "f1_score"
+    ],
+    "lr_scheduler": {
+        "type": "ExponentialLR",            // learning rate scheduler
+        "args": {
+            "gamma": 0.998
+        }
+    },
+    "trainer": {
+        "epochs": 500,                      // number of training epochs
+        "adaptive_step": 5,                 // update dataset after every adaptive_step epochs
+
+        "save_dir": "saved/",
+        "save_period": 1,                   // save checkpoints every save_period epochs
+        "verbosity": 2,                     // 0: quiet, 1: per epoch, 2: full
+        
+        "monitor": "min val_loss",          // mode and metric for model performance monitoring. set 'off' to disable.
+        "early_stop": 50,                   // early stop
+
+        "tensorboard": true                 // enable tensorboard visualization
+    }
+}
+```
+
+### Using config files
+Modify the configurations in `.json` config files, then run:
+
+  ```
+  python train.py --config config.json
+  ```
+
+### Resuming from checkpoints
+You can resume from a previously saved checkpoint by:
+
+  ```
+  python train.py --resume path/to/checkpoint
+  ```
+
+### Using Multiple GPU
+You can enable multi-GPU training by setting `n_gpu` argument of the config file to larger number.
+If configured to use smaller number of gpu than available, first n devices will be used by default.
+Specify indices of available GPUs by cuda environmental variable.
+  ```
+  python train.py --device 2,3 -c config.json
+  ```
+  This is equivalent to
+  ```
+  CUDA_VISIBLE_DEVICES=2,3 python train.py -c config.py
+  ```
+## License
+This project is licensed under the MIT License. See  [LICENSE](LICENSE) for more details
+
+## Acknowledgements
+* This research project was funded by German Academic Exchange Service (DAAD).
+* This project follows the template provided by [victoresque](https://github.com/victoresque/pytorch-template)

config.json

@@ -0,0 +1,57 @@
+{
+    "name": "WRS_Adaptive_Patching",
+    "n_gpu": 1,
+
+    "arch": {
+        "type": "UNet",
+        "args": {
+            "n_channels": 3,
+            "n_classes": 2
+        }
+    },
+    "data_loader": {
+        "type": "PatchedDataLoader",
+        "args":{
+            "data_dir": "/scratch/sukhan/wrs/datads-cv/",
+            "patch_size": 128,
+            "batch_size": 64,
+            "patch_stride": 32,
+            "target_dist": 0.01,
+            "shuffle": true,
+            "validation_split": 0.1,
+            "num_workers": 8
+        }
+    },
+    "optimizer": {
+        "type": "RMSprop",
+        "args":{
+            "lr": 1e-6,
+            "weight_decay": 0
+        }
+    },
+    "loss": "focal_loss",
+    "metrics": [
+        "precision",
+        "recall",
+        "f1_score"
+    ],
+    "lr_scheduler": {
+        "type": "ExponentialLR",
+        "args": {
+            "gamma": 0.998
+        }
+    },
+    "trainer": {
+        "epochs": 500,
+        "adaptive_step": 5,
+
+        "save_dir": "/scratch/sukhan/wrs/saved/",
+        "save_period": 1,
+        "verbosity": 2,
+        
+        "monitor": "min val_loss",
+        "early_stop": 50,
+
+        "tensorboard": true
+    }
+}

inference.py

@@ -0,0 +1,116 @@
+import argparse
+import collections
+import glob
+import shutil
+import sys
+from datetime import datetime
+from pathlib import Path
+import PIL.Image as Image
+import torch
+from torchvision.transforms import transforms
+from tqdm import tqdm
+import data_loader.data_loaders as module_data
+import model.model as module_arch
+from parse_config import ConfigParser
+
+
+def main(config):
+    logger = config.get_logger('infernece')
+
+    # setup data_loader instances
+    data_loader = getattr(module_data, config['data_loader']['type'])(
+        config['data_loader']['args']['data_dir'],
+        patch_size=config['data_loader']['args']['patch_size'],
+        batch_size=512,
+        shuffle=False,
+        validation_split=0.0,
+        training=False,
+        num_workers=2
+    )
+
+    # build model architecture
+    model = config.init_obj('arch', module_arch)
+    logger.info(model)
+
+    # prepare model for testing
+    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+    logger.info('Loading checkpoint: {} ...'.format(config.resume))
+    checkpoint = torch.load(config.resume, map_location=device)
+    state_dict = checkpoint['state_dict']
+    if config['n_gpu'] > 1:
+        model = torch.nn.DataParallel(model)
+    model.load_state_dict(state_dict)
+    model = model.to(device)
+
+    model.eval()
+    with torch.no_grad():
+        for i, (data, target) in enumerate(tqdm(data_loader)):
+            data, target = data.to(device), target.to(device)
+            output = model(data)
+
+            batch_size = data_loader.batch_size
+            patch_idx = torch.arange(
+                batch_size * i, batch_size * i + data.shape[0])
+            pred = torch.argmax(output, dim=1)
+            data_loader.dataset.patches.store_data(
+                patch_idx, [pred.unsqueeze(1)])
+
+    preds = [(data_loader.dataset.patches.combine(idx, data_idx=0).cpu(),
+              data_loader.dataset.data[idx])
+             for idx in range(len(data_loader.dataset.data))]
+    trsfm = transforms.ToPILImage()
+
+    out_dir = list(config.save_dir.parts)
+    out_dir[-3] = 'output'
+    out_dir = Path(*out_dir)
+    out_dir.mkdir(parents=True, exist_ok=True)
+    for pred, path in preds:
+        filename = Path(path).stem + '.png'
+        pred = trsfm(pred.float())
+        pred.save(out_dir / filename)
+
+
+if __name__ == '__main__':
+    args = argparse.ArgumentParser(description='PyTorch Template')
+    args.add_argument('-c', '--config', default=None, type=str,
+                      help='config file path (default: None)')
+    args.add_argument('-r', '--resume', default=None, type=str,
+                      help='path to latest checkpoint (default: None)')
+    args.add_argument('-d', '--device', default=None, type=str,
+                      help='indices of GPUs to enable (default: all)')
+    args.add_argument('--data', default=None, type=str,
+                      help='path to data (default: None)')
+
+    run_id = datetime.now().strftime(r'%m%d_%H%M%S')
+    dst_data = Path('.data/', run_id)
+
+    data_dir = dst_data / 'test' / 'images'
+    masks_dir = dst_data / 'test' / 'masks'
+
+    data_dir.mkdir(parents=True, exist_ok=True)
+    masks_dir.mkdir(parents=True, exist_ok=True)
+
+    if '--data' in sys.argv:
+        src_data = Path(sys.argv[sys.argv.index('--data') + 1])
+        if src_data.is_file():
+            shutil.copy(src_data, data_dir)
+            mask = Image.new('1', Image.open(src_data).size)
+            mask.save(masks_dir / (src_data.stem + '.png'), 'PNG')
+        else:
+            for filename in glob.glob('*.jpg', root_dir=src_data):
+                file_path = src_data / filename
+                if file_path.is_file():
+                    shutil.copy(file_path, data_dir)
+                    mask = Image.new('1', Image.open(file_path).size)
+                    mask.save(masks_dir / (file_path.stem + '.png'), 'PNG')
+        sys.argv += ['--data_dir', str(dst_data)]
+
+    # custom cli options to modify configuration from default values given in json file.
+    CustomArgs = collections.namedtuple('CustomArgs', 'flags type target')
+    options = [
+        CustomArgs(['--data_dir'], type=str,
+                   target='data_loader;args;data_dir')
+    ]
+    config = ConfigParser.from_args(args, options)
+    main(config)
+    shutil.rmtree(dst_data)

parse_config.py

@@ -0,0 +1,163 @@
+import logging
+import os
+from datetime import datetime
+from functools import partial, reduce
+from operator import getitem
+from pathlib import Path
+from logger import setup_logging
+from utils import read_json, write_json
+
+
+class ConfigParser:
+    def __init__(self, config, resume=None, modification=None, run_id=None):
+        """
+        class to parse configuration json file. Handles hyperparameters for training, initializations of modules, checkpoint saving
+        and logging module.
+        :param config: Dict containing configurations, hyperparameters for training. contents of `config.json` file for example.
+        :param resume: String, path to the checkpoint being loaded.
+        :param modification: Dict keychain:value, specifying position values to be replaced from config dict.
+        :param run_id: Unique Identifier for training processes. Used to save checkpoints and training log. Timestamp is being used as default
+        """
+        # load config file and apply modification
+        self._config = _update_config(config, modification)
+        self.resume = resume
+
+        # set save_dir where trained model and log will be saved.
+        save_dir = Path(self.config['trainer']['save_dir'])
+
+        exper_name = self.config['name']
+        if run_id is None:  # use timestamp as default run-id
+            run_id = datetime.now().strftime(r'%m%d_%H%M%S')
+        self._save_dir = save_dir / 'models' / exper_name / run_id
+        self._log_dir = save_dir / 'log' / exper_name / run_id
+
+        # make directory for saving checkpoints and log.
+        exist_ok = run_id == ''
+        self.save_dir.mkdir(parents=True, exist_ok=exist_ok)
+        self.log_dir.mkdir(parents=True, exist_ok=exist_ok)
+
+        # save updated config file to the checkpoint dir
+        write_json(self.config, self.save_dir / 'config.json')
+
+        # configure logging module
+        setup_logging(self.log_dir)
+        self.log_levels = {
+            0: logging.WARNING,
+            1: logging.INFO,
+            2: logging.DEBUG
+        }
+
+    @classmethod
+    def from_args(cls, args, options=''):
+        """
+        Initialize this class from some cli arguments. Used in train, test.
+        """
+        for opt in options:
+            args.add_argument(*opt.flags, default=None, type=opt.type)
+        if not isinstance(args, tuple):
+            args = args.parse_args()
+
+        if args.device is not None:
+            os.environ["CUDA_VISIBLE_DEVICES"] = args.device
+        if args.resume is not None:
+            resume = Path(args.resume)
+            cfg_fname = resume.parent / 'config.json'
+        else:
+            msg_no_cfg = "Configuration file need to be specified. Add '-c config.json', for example."
+            assert args.config is not None, msg_no_cfg
+            resume = None
+            cfg_fname = Path(args.config)
+
+        config = read_json(cfg_fname)
+        if args.config and resume:
+            # update new config for fine-tuning
+            config.update(read_json(args.config))
+
+        # parse custom cli options into dictionary
+        modification = {opt.target: getattr(args, _get_opt_name(opt.flags)) for opt in options}
+        return cls(config, resume, modification)
+
+    def init_obj(self, name, module, *args, **kwargs):
+        """
+        Finds a function handle with the name given as 'type' in config, and returns the
+        instance initialized with corresponding arguments given.
+
+        `object = config.init_obj('name', module, a, b=1)`
+        is equivalent to
+        `object = module.name(a, b=1)`
+        """
+        module_name = self[name]['type']
+        module_args = dict(self[name]['args'])
+        assert all([k not in module_args for k in kwargs]), 'Overwriting kwargs given in config file is not allowed'
+        module_args.update(kwargs)
+        return getattr(module, module_name)(*args, **module_args)
+
+    def init_ftn(self, name, module, *args, **kwargs):
+        """
+        Finds a function handle with the name given as 'type' in config, and returns the
+        function with given arguments fixed with functools.partial.
+
+        `function = config.init_ftn('name', module, a, b=1)`
+        is equivalent to
+        `function = lambda *args, **kwargs: module.name(a, *args, b=1, **kwargs)`.
+        """
+        module_name = self[name]['type']
+        module_args = dict(self[name]['args'])
+        assert all([k not in module_args for k in kwargs]), 'Overwriting kwargs given in config file is not allowed'
+        module_args.update(kwargs)
+        return partial(getattr(module, module_name), *args, **module_args)
+
+    def __getitem__(self, name):
+        """Access items like ordinary dict."""
+        return self.config[name]
+
+    def get_logger(self, name, verbosity=2):
+        msg_verbosity = 'verbosity option {} is invalid. Valid options are {}.'.format(
+            verbosity, self.log_levels.keys())
+        assert verbosity in self.log_levels, msg_verbosity
+        logger = logging.getLogger(name)
+        logger.setLevel(self.log_levels[verbosity])
+        return logger
+
+    # setting read-only attributes
+    @property
+    def config(self):
+        return self._config
+
+    @property
+    def save_dir(self):
+        return self._save_dir
+
+    @property
+    def log_dir(self):
+        return self._log_dir
+
+# helper functions to update config dict with custom cli options
+
+
+def _update_config(config, modification):
+    if modification is None:
+        return config
+
+    for k, v in modification.items():
+        if v is not None:
+            _set_by_path(config, k, v)
+    return config
+
+
+def _get_opt_name(flags):
+    for flg in flags:
+        if flg.startswith('--'):
+            return flg.replace('--', '')
+    return flags[0].replace('--', '')
+
+
+def _set_by_path(tree, keys, value):
+    """Set a value in a nested object in tree by sequence of keys."""
+    keys = keys.split(';')
+    _get_by_path(tree, keys[:-1])[keys[-1]] = value
+
+
+def _get_by_path(tree, keys):
+    """Access a nested object in tree by sequence of keys."""
+    return reduce(getitem, keys, tree)

requirements.txt

@@ -0,0 +1,7 @@
+torch>=1.12
+torchvision
+numpy
+tqdm
+tensorboard>=2.9.1
+scikit-learn
+pandas

test.py

@@ -0,0 +1,77 @@
+import argparse
+import torch
+from tqdm import tqdm
+import data_loader.data_loaders as module_data
+import model.loss as module_loss
+import model.metric as module_metric
+import model.model as module_arch
+from parse_config import ConfigParser
+
+
+def main(config):
+    logger = config.get_logger('test')
+
+    # setup data_loader instances
+    data_loader = getattr(module_data, config['data_loader']['type'])(
+        config['data_loader']['args']['data_dir'],
+        patch_size=config['data_loader']['args']['patch_size'],
+        batch_size=512,
+        shuffle=False,
+        validation_split=0.0,
+        training=False,
+        num_workers=2
+    )
+
+    # build model architecture
+    model = config.init_obj('arch', module_arch)
+    logger.info(model)
+
+    # prepare model for testing
+    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+    logger.info('Loading checkpoint: {} ...'.format(config.resume))
+    checkpoint = torch.load(config.resume, map_location=device)
+    state_dict = checkpoint['state_dict']
+    if config['n_gpu'] > 1:
+        model = torch.nn.DataParallel(model)
+    model.load_state_dict(state_dict)
+    model = model.to(device)
+
+    # get function handles of loss and metrics
+    loss_fn = getattr(module_loss, config['loss'])
+    metric_fns = [getattr(module_metric, met) for met in config['metrics']]
+
+    total_loss = 0.0
+    total_metrics = torch.zeros(len(metric_fns))
+
+    model.eval()
+    with torch.no_grad():
+        for i, (data, target) in enumerate(tqdm(data_loader)):
+            data, target = data.to(device), target.to(device)
+            output = model(data)
+
+            # computing loss, metrics on test set
+            loss = loss_fn(output, target)
+            batch_size = data.shape[0]
+            total_loss += loss.item() * batch_size
+            for i, metric in enumerate(metric_fns):
+                total_metrics[i] += metric(output, target) * batch_size
+
+    n_samples = len(data_loader.sampler)
+    log = {'loss': total_loss / n_samples}
+    log.update({
+        met.__name__: total_metrics[i].item() / n_samples for i, met in enumerate(metric_fns)
+    })
+    logger.info(log)
+
+
+if __name__ == '__main__':
+    args = argparse.ArgumentParser(description='PyTorch Template')
+    args.add_argument('-c', '--config', default=None, type=str,
+                      help='config file path (default: None)')
+    args.add_argument('-r', '--resume', default=None, type=str,
+                      help='path to latest checkpoint (default: None)')
+    args.add_argument('-d', '--device', default=None, type=str,
+                      help='indices of GPUs to enable (default: all)')
+
+    config = ConfigParser.from_args(args)
+    main(config)

train.py

@@ -0,0 +1,85 @@
+import argparse
+import collections
+import numpy as np
+import torch
+import data_loader.data_loaders as module_data
+import model.loss as module_loss
+import model.metric as module_metric
+import model.model as module_arch
+from parse_config import ConfigParser
+from trainer import Trainer
+from utils import prepare_device
+
+
+# fix random seeds for reproducibility
+SEED = 123
+torch.manual_seed(SEED)
+torch.backends.cudnn.deterministic = True
+torch.backends.cudnn.benchmark = False
+np.random.seed(SEED)
+
+
+def main(config):
+    logger = config.get_logger('train')
+
+    # setup data_loader instances
+    data_loader = config.init_obj('data_loader', module_data)
+    valid_data_loader = data_loader.split_validation()
+
+    # setup data_loader for inference
+    data_loader.inference = getattr(module_data, config['data_loader']['type'])(
+        config['data_loader']['args']['data_dir'],
+        patch_size=config['data_loader']['args']['patch_size'],
+        batch_size=config['data_loader']['args']['batch_size'],
+        shuffle=False,
+        validation_split=0.0,
+        training=True,
+        num_workers=config['data_loader']['args']['num_workers']
+    )
+
+    # build model architecture, then print to console
+    model = config.init_obj('arch', module_arch)
+    logger.info(model)
+
+    # prepare for (multi-device) GPU training
+    device, device_ids = prepare_device(config['n_gpu'])
+    model = model.to(device)
+    if len(device_ids) > 1:
+        model = torch.nn.DataParallel(model, device_ids=device_ids)
+
+    # get function handles of loss and metrics
+    criterion = getattr(module_loss, config['loss'])
+    metrics = [getattr(module_metric, met) for met in config['metrics']]
+
+    # build optimizer, learning rate scheduler. delete every lines containing lr_scheduler for disabling scheduler
+    trainable_params = filter(lambda p: p.requires_grad, model.parameters())
+    optimizer = config.init_obj('optimizer', torch.optim, trainable_params)
+    lr_scheduler = config.init_obj('lr_scheduler', torch.optim.lr_scheduler, optimizer)
+
+    trainer = Trainer(model, criterion, metrics, optimizer,
+                      config=config,
+                      device=device,
+                      data_loader=data_loader,
+                      valid_data_loader=valid_data_loader,
+                      lr_scheduler=lr_scheduler)
+
+    trainer.train()
+
+
+if __name__ == '__main__':
+    args = argparse.ArgumentParser(description='PyTorch Template')
+    args.add_argument('-c', '--config', default=None, type=str,
+                      help='config file path (default: None)')
+    args.add_argument('-r', '--resume', default=None, type=str,
+                      help='path to latest checkpoint (default: None)')
+    args.add_argument('-d', '--device', default=None, type=str,
+                      help='indices of GPUs to enable (default: all)')
+
+    # custom cli options to modify configuration from default values given in json file.
+    CustomArgs = collections.namedtuple('CustomArgs', 'flags type target')
+    options = [
+        CustomArgs(['--lr', '--learning_rate'], type=float, target='optimizer;args;lr'),
+        CustomArgs(['--bs', '--batch_size'], type=int, target='data_loader;args;batch_size')
+    ]
+    config = ConfigParser.from_args(args, options)
+    main(config)