L08.3: Leveraging a Pretrained Model

In this video I continue our discussion of convolutional neural networks by looking at you you can leverage pretrained models to improve image processing tasks like image classification. The Keras API contains many saved pretrained convolutional model architectures. In this example I use the VGG16 architecture that was trained originally on 1.4 million input images (ImageNet).

Using a pretrained model to do the feature extraction works for you because the series of convolutional layers learns a hierarchical representation of useful features in images, from low level features like edge detection and texture detection, up through higher level concepts. Often these concepts and be reused in your image processing task.

In this video I show two basic methods to use a pretrained model for feature extraction on your own image classification task. One method is to simply run all data through the model first and save the output results form the feature extraction. This new dataset is then used as the training and test data for the model you want. This is computationally much less expensive than the next method, but limits your use of data augmentation. In the second method you can create a new model, where you tack on your inputs and data augmentation to the convolutional base inputs, freeze the convolutional base weights, and add your own dense layers on the output to be trained. I also discuss fine tuning the second approach, where sometimes you can get further improvements in performance by unfreezing some of the highest level convolutional base layer representations and let them learn on your own input data.

Resources:

Textbook: Chollet (2022). "Deep Learning with Python (2ed)". Manning.
https://www.amazon.com/dp/1617296864/...

CSci 560 Class Repository: https://github.com/csci560-nndl/nndl
Contains video slides and iPython notebooks for this course.



00:00 Introduction
01:34 Feature extraction with a pretrained model
06:35 Method 1: Fast feature extraction without data augmentation
13:47 Method 2: Feature extraction together with data augmentation
18:23 Fine tuning a pretrained model
24:22 Summary