runtraining.sh

This script runs CDeep3M training to generated what is known as a trained model. In the case of CDeep3M we are actually training 3 separate models which will be described below.

This script is actually a wrapper that invokes CreateTrainJob.m and run_all_train.sh

Usage:

runtraining.sh 
usage: runtraining.sh [-h] [--1fmonly] [--numiterations NUMITERATIONS]
                      augtrainimages trainoutdir

              Version: 0.13.0

              Trains CDeep3M model using caffe with training data
              passed into script. 
    
positional arguments:
  augtrainimages       Augmented training data from PreprocessTrainingData.m
  trainoutdir          Desired output directory

optional arguments:
  -h, --help           show this help message and exit
  --1fmonly            Only train 1fm model
  --numiterations      Number of training iterations to run (default 2000)

This script will create a new directory, denoted as trainoutdir in usage above, which will be structured as follows:

Tree view of directory showing only base files and directories

├── 1fm
│   ├── log
│   ├── trainedmodel
├── 3fm
│   ├── log
│   ├── trainedmodel
├── 5fm
│   ├── log
│   ├── trainedmodel
├── caffe_train.sh
├── readme.txt
├── run_all_train.sh
└── train_file.txt

1fm, 3fm, 5fm

These directories contain the trained models and each one has an identical structure as seen here with actual files:

├── #fm
│   ├── deploy.prototxt
│   ├── label_class_selection.prototxt
│   ├── log
│   │   ├── caffe.bin.INFO
│   │   ├── caffe.bin.ip-XXX.ubuntu.log.INFO.XXXX
│   │   └── out.log
│   ├── solver.prototxt
│   ├── trainedmodel
│   │   ├── 1fm_classifer_iter_###.caffemodel
│   │   └── 1fm_classifer_iter_###.solverstate
│   ├── train_file.txt
│   ├── train_val.prototxt
│   └── valid_file.txt

The actual trained model resides under #fm/trainedmodel in the .caffemodel file. The other file .solverstate is needed to resume training, but not needed for prediction. The ### in the .caffemodel and .solverstate denote the iteration of the model file. As caffe trains multiple .caffemodel files will be output so in this #fm/trainedmodel can exist multiple files at different iterations of completion.