convolutional neural network fine-tuning Keras transfer learning vgg

Transfer Learning with Keras and Deep Learning

On this tutorial, you’ll discover ways to perform switch learning with Keras, Deep Learning, and Python on your own customized datasets.

Think about this:

You’re just hired by Yelp to work of their pc imaginative and prescient division.

Yelp has just launched a brand new function on its web site that permits reviewers to take photographs of their meals/dishes and then affiliate them with specific gadgets on a restaurant’s menu.

It’s a neat function…

…but they’re getting lots of unwanted image spam.

Certain nefarious customers aren’t taking photographs of their dishes…as an alternative, they’re taking photographs of… (properly, you’ll be able to in all probability guess).

Your process?

Work out how you can create an automated pc imaginative and prescient software that can distinguish between “food” and “not food”, thereby permitting Yelp to continue with their new function launch and present worth to their customers.

So, how are you going to construct such an software?

The reply lies in transfer studying by way of deep learning.

At this time marks the start of a brand new set of tutorials on transfer studying utilizing Keras. Transfer learning is the method of:

  1. Taking a network pre-trained on a dataset
  2. And using it to acknowledge image/object classes it was not educated on

Primarily, we will make the most of the strong, discriminative filters discovered by state-of-the-art networks on challenging datasets (similar to ImageNet or COCO), and then apply these networks to recognize objects the mannequin was never educated on.

Normally, there are two forms of switch studying within the context of deep learning:

  1. Transfer studying by way of function extraction
  2. Transfer studying by way of fine-tuning

When performing function extraction, we treat the pre-trained network as an arbitrary function extractor, allowing the input image to propagate forward, stopping at pre-specified layer, and taking the outputs of that layer as our options.

Positive-tuning, then again, requires that we replace the model structure itself by removing the previous fully-connected layer heads, providing new, freshly initialized ones, and then training the brand new FC layers to foretell our input courses.

We’ll be masking each methods in this collection right here on the PyImageSearch blog, however at the moment we are going to concentrate on function extraction.

To discover ways to perform transfer studying by way of function extraction with Keras, just maintain studying!

Transfer studying with Keras and Deep Learning

Word: Most of the switch learning ideas I’ll be overlaying in this collection tutorials also seem in my ebook, Deep Learning for Pc Imaginative and prescient with Python. Inside the e-book, I’m going into rather more element (and embrace more of my ideas, ideas, and greatest practices). If you need more element on transfer studying after going via this information, undoubtedly take a look at my ebook.

Within the first a part of this tutorial, we’ll evaluate two methods of switch studying: function extraction and fine-tuning.

I’ll then provide an in depth dialogue of the right way to carry out transfer learning by way of function extraction (the first focus of this tutorial).

From there, we’ll assessment Food-5k dataset, a dataset containing 5,000 pictures falling into two courses: “food” and “not-food”.

We’ll utilize switch studying by way of function extraction to recognize both of these courses on this tutorial.

As soon as we have now a very good deal with on the dataset, we’ll start coding.

We’ll have quite a lot of Python information to evaluate, every undertaking a selected step, together with:

  1. Creating a configuration file.
  2. Building our dataset (i.e., putting the pictures within the correct directory construction).
  3. Extracting options from our input photographs utilizing Keras and pre-trained CNNs.
  4. Coaching a Logistic Regression model on prime of the extracted options.

Elements of the code we’ll be reviewing right here immediately may even be utilized in the remainder of the switch learning collection — should you intend on following along with the tutorials, take the time now to ensure you perceive the code.

Two forms of switch learning: function extraction and fine-tuning

Figure 1: By way of “transfer learning”, we will make the most of a pre-existing model comparable to one educated to classify canine vs. cats. Using that pre-trained mannequin we will break open the CNN and then apply “transfer learning” to another, utterly totally different dataset (comparable to bears). We’ll discover ways to apply switch studying with Keras and deep studying in the rest of this blog submit.

Word: The following section has been tailored from my e-book, Deep Learning for Pc Vision with Python. For the complete set of chapters on switch learning, please seek advice from the textual content.

Think about a standard machine studying state of affairs where we’re given two classification challenges.

Within the first problem, our objective is to coach a Convolutional Neural Community to recognize canine vs. cats in a picture.

Then, in the second challenge, we’re tasked with recognizing three separate species of bears: grizzly bears, polar bears, and big pandas.

Utilizing normal practices in machine learning/deep studying, we might deal with these challenges as two separate issues:

  • First, we might collect a adequate labeled dataset of canine and cats, followed by training a model on the dataset
  • We might then repeat the method a second time, only this time, gathering pictures of our bear breeds, and then coaching a model on prime of the labeled dataset.

Transfer learning proposes a unique paradigm — what if we might utilize an present pre-trained classifier as a starting point for a brand new classification, object detection, or instance segmentation process?

Utilizing transfer studying in the context of the proposed challenges above, we might:

  • First practice a Convolutional Neural Network to recognize canine versus cats
  • Then, use the identical CNN educated on the dog and cat knowledge and use it to differentiate between the bear courses, despite the fact that no bear knowledge was combined with the canine and cat knowledge in the course of the initial training

Does this sound too good to be true?

It’s truly not.

Deep neural networks educated on large-scale datasets akin to ImageNet and COCO have confirmed to be wonderful at the activity of switch studying.

These networks study a set of wealthy, discriminative options able to recognizing 100s to 1,000s of object courses — it only is sensible that these filters may be reused for tasks aside from what the CNN was initially educated on.

Generally, there are two kinds of transfer learning when applied to deep studying for pc imaginative and prescient:

  1. Treating networks as arbitrary function extractors.
  2. Removing the fully-connected layers of an present network, putting a new set of FC layers on prime of the CNN, and then fine-tuning these weights (and optionally previous layers) to acknowledge the brand new object courses.

On this weblog submit, we’ll focus primarily on the first technique of transfer studying, treating networks as function extractors.

We’ll talk about fine-tuning networks later on this collection on transfer learning with deep learning.

Transfer studying by way of function extraction

Figure 2: Left: The unique VGG16 network architecture that outputs chances for every of the 1,000 ImageNet class labels. Right: Eradicating the FC layers from VGG16 and as an alternative of returning the final POOL layer. This output will serve as our extracted options.

Observe: The following part has been adapted from my ebook, Deep Learning for Pc Vision with Python. For the complete set of chapters on function extraction, please confer with the text.

Sometimes, you’ll deal with a Convolutional Neural Network as an end-to-end picture classifier:

  1. We input a picture to the network.
  2. The picture forward propagates by means of the network.
  3. We get hold of our last classification chances at the end of the network.

Nevertheless, there isn’t a “rule” that claims we must permit the image to ahead propagate via all the community.

As an alternative, we will:

  1. Cease propagation at an arbitrary, but pre-specified layer (similar to an activation or pooling layer).
  2. Extract the values from the required layer.
  3. Deal with the values as a function vector.

For example, let’s think about the VGG16 community by Simonyan and Zisserman in Determine 2 (left) on the prime of this part.

Alongside with the layers within the community, I have additionally included the enter and output quantity shapes for each layer.

When treating networks a function extractor, we primarily “chop off” the network at our pre-specified layer (sometimes previous to the fully-connected layers, nevertheless it actually is determined by your specific dataset).

If we have been to cease propagation before the fully-connected layers in VGG16, the last layer in the network would develop into the max-pooling layer (Figure 2, proper), which may have an output shape of seven x 7 x 512. Flattening, this volume into a function vector we might acquire an inventory of 7 x 7 x 512 = 25,088 values — this record of numbers serves as our function vector used to quantify the input image.

We will then repeat the method for our whole dataset of pictures.

Given a complete of N photographs in our community, our dataset would now be represented as an inventory of N vectors, each of 25,088-dim.

As soon as we now have our function vectors, we will practice off-the-shelf machine learning fashions resembling Linear SVM, Logistic Regression, Choice Timber, or Random Forests on prime of these features to obtain a classifier that can acknowledge new courses of pictures.

That stated, the 2 commonest machine learning models you’ll see for transfer studying by way of function extraction are:

  1. Logistic Regression
  2. Linear SVM

Why these two models?

First, have in mind our function extractor is a CNN.

CNN’s are non-linear fashions able to studying non-linear options — we are assuming that the options discovered by the CNN are already strong and discriminative.

The second, and perhaps arguably more necessary purpose, is that our function vectors are typically very giant and have excessive dimensionality.

We, subsequently, need a quick mannequin that can be educated on prime of the options — linear models are typically very quick to train.

For example, our dataset of 5,000 pictures, each represented by a function vector of 25,088-dim, could be educated in a number of seconds using a Logistic Regression model.

To wrap up this part, I would like you to needless to say the CNN itself is just not able to recognizing these new courses.

As an alternative, we’re utilizing the CNN as an intermediary function extractor.

The downstream machine studying classifier will care for learning the underlying patterns of the features extracted from the CNN.

The Meals-5K dataset

Figure three: We’ll apply switch studying to the Meals-5K dataset utilizing Python, Keras, and Deep Learning.

The dataset we’ll be using here immediately is the Meals-5K dataset, curated by the Multimedia Sign Processing Group (MSPG) of the Swiss Federal Institute of Know-how.

The dataset, because the identify suggests, consists of 5,000 photographs, belonging to 2 courses:

  1. Meals
  2. Non-food

Our objective of is to coach a classifier such that we will distinguish between these two courses.

MSPG has offered us with pre-split coaching, validation, and testing splits. We’ll be using these splits both on this guide on transfer learning by way of extraction in addition to the remainder of our tutorials on function extraction.

Downloading the Meals-5K dataset

Go forward and seize the zip related with at the moment’s “Downloads”.

When you’ve download the source code, change directory into
transfer-learning-keras :

In my expertise, I’ve found that downloading the Food-11 dataset is unreliable.

Subsequently I’m presenting two choices to obtain the dataset:

Choice 1: Use wget in your terminal

The wget software comes on Ubuntu and different Linux distros. On macOS, you have to set up it:

To obtain the Meals-5K dataset, let’s use
wget  in our terminal:

Word: No less than on macOS, I’ve found that if the
wget  command fails as soon as, just run it once more and then the download will begin.

Choice 2: Use FileZilla

FileZilla is a GUI software for FTP and SCP connections. You could obtain it in your OS right here.

Once you’ve put in and launched the appliance, enter the credentials:

You possibly can then join and obtain the file into the appropriate destination.

Determine 4: Downloading the Meals-5K dataset with FileZilla.

The username and password mixture have been obtained from the official Food-5K dataset web site. If the username/password combination stops working for you, examine to see if the dataset curators modified the login credentials.

As soon as downloaded, we will go ahead and unzip the dataset:

Challenge structure

Now that we’ve at present’s zip and the dataset, let’s inspect all the challenge listing.

First, navigate again as much as the venture’s root:

Then, use the
tree  command with arguments as shown:

As you possibly can see, the
Meals-5K/  incorporates
analysis/ ,
training/ , and
validation/  sub-directories. Every sub-directory accommodates 1,000
.jpg  picture information.

dataset/  listing, while empty now, will soon include the Food-5K pictures in a extra organized type (to be discussed within the part, “Building our dataset for feature extraction”).

Upon efficiently executing in the present day’s Python scripts, the 
output/  listing will home our extracted options (saved in three separate
.csv  information) in addition to our label encoder and mannequin (each of which are in
.cpickle  format).

Our Python scripts embrace:

  • pyimagesearch/ : Our customized configuration file will help us handle our dataset, class names, and paths. It’s written in Python immediately in order that we will use
    os.path  to build OS-specific formatted file paths instantly in the script.

  • : Using the configuration, this script will  create an organized dataset on disk, making it straightforward to extract options from.

  • : The transfer learning magic begins right here. This Python script will use a pre-trained CNN to extract raw features, storing the leads to a
    .csv  file. The label encoder
    .cpickle  file may also be output by way of this script.

  • : Our coaching script will practice a Logistic Regression mannequin on prime of the beforehand computed features. We’ll evaluate and save the resulting mannequin as a
    .cpickle .

The  and scripts shall be re-used in the rest of the collection on switch learning so ensure you pay shut consideration to them!

Our configuration file

Let’s get began by reviewing our configuration file.

Open up within the
pyimagesearch submodule and insert the following code:

We start with a single import. We’ll use the
os  module (Line 2) in this config to concatenate paths properly.

ORIG_INPUT_DATASET  is the path to the unique input dataset (i.e., where you downloaded and unarchived the Food-5K dataset).

The subsequent path,
BASE_PATH , might be where our dataset is organized (the results of executing .

Word: The listing structure is just not especially helpful for this specific submit, but it is going to be later in the collection as soon as we get to fine-tuning. Again, I contemplate organizing datasets on this method a “best practice” for reasons you’ll see in this collection.

Let’s specify more dataset configs in addition to our class labels and batch measurement:

The path to output coaching, evaluation, and validation directories is specified on Strains 13-15.

CLASSES  are specified in listing type on Line 18. As previously mentioned, we’ll be working with
“food”  and
“non_food”  pictures.

When extracting features, we’ll break our knowledge into bite-sized chunks referred to as batches. The
BATCH_SIZE  is specified on Line 21.

Finally, we will construct the rest of our paths:

Our label encoder path is concatenated on Line 25 the place the results of joining the paths is
output/le.cpickle  on Linux/Mac or
outputle.cpickle  on Windows.

The extracted features will reside in a CSV file within the path laid out in

Lastly, we assemble the path to our exported model file in

Building our dataset for function extraction

Earlier than we will extract options from our set of input photographs, let’s take the time to arrange our photographs on disk.

I favor to have my dataset on disk organized in the format of:


Maintaining this listing construction:

  • Not only retains our dataset organized on disk…
  • …but in addition allows us to utilize Keras’
    flow_from_directory perform once we get to fine-tuning later on this collection of tutorials.

Because the Meals-5K dataset additionally supplies pre-supplied knowledge splits, our ultimate listing construction may have the shape:


Let’s go ahead and construct our dataset + directory construction now.

Open up the file and insert the following code:

Our packages are imported on Strains 2-5. We’ll use our
config  (Line 2) all through this script to recall our settings. The other three imports —
paths ,
shutil , and
os  — will permit us to traverse directories, create folders, and copy information.

On Line eight we start looping over our training, testing, and validation splits.

Strains 11 and 12 create an inventory of all
imagePaths  within the cut up.

From there we’ll go ahead and loop over the
imagePaths :

For every
imagePath  in the cut up, we proceed to:

  • Extract the class
    label  from the filename (Strains 17 and 18).
  • Construct the trail to the output directory based mostly on the
    cut up , and
    label  (Line 21).
  • Create
    dirPath  (if mandatory) by way of Strains 24 and 25.
  • Copy the picture into the destination path (Strains 28 and 29).

Now that  has been coded, use the “Downloads” section of the tutorial to download an archive of the source code.

You possibly can then execute  using the next command:

Right here you’ll be able to see that our script executed efficiently.

To verify your listing construction on disk, use the
ls  command:

Contained in the dataset directory, we’ve our training, analysis, and validation splits.

And inside every of those directories, we have now class labels directories:

Extracting options from our dataset using Keras and pre-trained CNNs

Let’s move on to the precise function extraction element of transfer studying.

All code used for function extraction utilizing a pre-trained CNN will stay inside — open up that file and insert the following code:

On Strains 2-12, all of the packages needed for extracting features are imported. Most notably this consists of
VGG16 .

VGG16 is the convolutional neural community (CNN) we are using for transfer learning (Line 3).

On Line 16, we load the
model  whereas specifying two parameters:

  • weights=”imagenet” : Pre-trained ImageNet weights are loaded for switch learning.

  • include_top=False : We don’t embrace the fully-connected head with the softmax classifier. In other words, we chop off the top of the network.

With weights dialed in and by loading our model with out the top, we at the moment are prepared for transfer learning. We’ll use the output values of the community immediately, storing the outcomes as function vectors.

Finally, our label encoder is initialized on Line 17.

Let’s loop over our knowledge splits:

Looping over each
cut up  (training, testing, and validation) begins on Line 20.

First, we grab all
imagePaths  for the
cut up  (Strains 23 and 24).

Paths are randomly shuffled by way of Line 28, and from there, our class
labels  are extracted from the paths themselves (Line 29).

If vital, our label encoder is instantiated and fitted (Strains 32-34), making certain we will convert the string class labels to integers.

Next, we construct the path to output our CSV information (Strains 37-39). We could have three CSV information — one for each knowledge cut up. Every CSV could have N variety of rows — one for every of the pictures within the knowledge cut up.

The subsequent step is to loop over our
imagePaths  in
BATCH_SIZE  chunks:

To create our batches of
imagePaths , we use Python’s
vary  perform. The perform accepts three parameters:
start ,
cease , and
step . You’ll be able to learn more about
vary  on this detailed rationalization.

Our batches will step by means of the whole listing of
imagePaths . The
step  is our batch measurement (
32  until you modify it in the configuration settings).

On Strains 48 and 49 the current batch of image paths and labels are extracted using array slicing. Our
batchImages  record is then initialized on Line 50.

Let’s go forward and populate our
batchImages  now:

Looping over
batchPaths  (Line 53), we’ll load each
image , preprocess it, and gather it into 
batchImages .

picture  itself is loaded on Line 56.

Preprocessing consists of:

  • Resizing to 224×224 pixels by way of the
    target_size  parameter on Line 56.
  • Changing to array format (Line 57).
  • Adding a batch dimension (Line 62).
  • Mean subtraction (Line 63).

If these preprocessing steps seem overseas, please check with Deep Learning for Pc Vision with Python.

Finally, the
picture  is added to the batch by way of Line 66.

Now we’ll move the batch of pictures via our network to extract features:

Our batch of photographs is shipped via the network by way of Strains 71 and 72.

Understand that we have now removed the fully-connected layer head of the community. As an alternative, the ahead propagation stops on the max-pooling layer. We’ll treat the output of the max-pooling layer as an inventory of
features , also referred to as a “feature vector”.

The output dimension of the max-pooling layer is (batch_size, 7 x 7 x 512). We will thus 
reshape  the
options  right into a NumPy array of form
(batch_size, 7 * 7 * 512), treating the output of the CNN as a function vector.

Let’s wrap up this script:

Sustaining our batch efficiency, the
features  and associated class labels are written to our CSV file (Strains 76-80).

Contained in the CSV file, the class
label  is the first area in each row (enabling us to simply extract it from the row during training). The function
vec  follows.

Every CSV file shall be closed by way of Line 83. Recall that upon completion we may have one CSV file per knowledge cut up.

Lastly, we will dump the label encoder to disk (Strains 86-88).

Let’s go ahead and extract options from our dataset using the VGG16 community pre-trained on ImageNet.

Use the “Downloads” part of this tutorial to download the supply code, and from there, execute the next command:

On an NVIDIA Okay80 GPU it took 2m55s to extract options from the 5,000 photographs in the Food-5K dataset.

You should use a CPU as an alternative, but it is going to take fairly a bit longer.

Implementing our coaching script

The final step for switch studying by way of function extraction is to implement a Python script that may take our extracted features from the CNN and then practice a Logistic Regression mannequin on prime of the options.

Once more, remember that our CNN did not predict anything! As an alternative, the CNN was handled as an arbitrary function extractor.

We inputted a picture to the community, it was ahead propagated, and then we extracted the layer outputs from the max-pooling layer — these outputs serve as our function vectors.

To see how we will practice a model on these function vectors, open up the file and let’s get to work:

On Strains 2-7 we import our required packages. Notably, we’ll use
LogisticRegression  as our machine studying classifier. Fewer imports are required for our coaching script as in comparison with extracting options. This is partly as a result of the coaching script itself is actually easier.

Let’s outline a perform named
load_data_split  on Line 9. This perform is liable for loading all knowledge and labels given the trail of a knowledge cut up CSV file (the
splitPath  parameter).

Inside the perform, we start by initializing, our
knowledge  and
labels  lists (Strains 11 and 12).

From there we open the CSV and loop over all rows beginning on Line 15. In the loop, we:

  • Load all comma separated values from the
    row  into an inventory (Line 17).
  • Grab the category
    label  by way of Line 18 (it is the first value in the record).
  • Extract all
    options  from the row (Line 19). These are all values within the listing except the category label. The result’s our function vector.
  • From there, we append the function vector and
    label  to the
    knowledge  and
    labels  lists respectively (Strains 22 and 23).

Lastly, the
knowledge  and
labels  are returned to the calling perform (Line 30).

With the
load_data_spit  perform able to go, let’s put it to work by loading our knowledge:

Strains 33-41 load our coaching and testing function knowledge from disk. We’re using our perform from the earlier code block to handle the loading process.

Line 44 masses our label encoder.

With our knowledge in memory, we’re now ready to train our machine learning classifier:

Strains 48 and 49 are chargeable for initializing and training our Logistic Regression
mannequin .

Word: To study extra about Logistic Regression and different machine learning algorithms intimately, remember to confer with PyImageSearch Gurus, my flagship pc imaginative and prescient course and group.

Strains 53 and 54 facilitate evaluating the
mannequin  on the testing set and printing classification statistics within the terminal.

Finally, the
model  is output in Python’s pickle format (Strains 58-60).

That’s a wrap for our training script! As you’ve discovered, writing code for training a Logistic Regression model on prime of function knowledge could be very simple. Within the next part, we’ll run the coaching script.

In case you are wondering how we might deal with a lot function knowledge that it might’t fit into memory abruptly, stay tuned for subsequent week’s tutorial.

Notice: This tutorial is long enough as is, so I haven’t coated how you can tune the hyperparameters to the Logistic Regression mannequin, something I undoubtedly advocate doing to ensure to acquire the very best accuracy attainable. In the event you’re thinking about studying extra about switch learning, and find out how to tune hyperparameters, throughout function extraction, be sure you seek advice from Deep Learning for Pc Vision with Python where I cowl the methods in additional element.

Training a model on the extracted options

At this level, we are ready to carry out the final step on transfer learning by way of function extraction with Keras.

Let’s briefly evaluate what we have now finished thus far:

  1. Downloaded the Food-5K dataset (5,000 photographs belonging to two courses, “food” and “non-food”, respectively).
  2. Restructured the unique directory structure of the dataset in a format more suitable for switch learning (particularly, fine-tuning which we’ll be masking later on this collection).
  3. Extracted features from the pictures utilizing VGG16 pre-trained on ImageNet.

And now, we’re going to train a Logistic Regression model on prime of these extracted options.

Once more, take into account that VGG16 was not educated to recognize the “food” versus “non-food” courses. As an alternative, it was educated to acknowledge 1,000 ImageNet courses.

But, by leveraging:

  1. Function extraction with VGG16
  2. And applying a Logistic Regression classifier on prime of these extracted features

…we will recognize the brand new courses, regardless that VGG16 was by no means educated to recognize them!

Go ahead and use the “Downloads” section of this tutorial to obtain the source code to this guide.

From there, open up a terminal and execute the following command:

Training on my machine took only 27 seconds, and as you’ll be able to see from our output, we’re acquiring 98-99% accuracy on the testing set!

When ought to I exploit switch learning and function extraction?

Transfer studying by way of function extraction is usually one of the best methods to obtain a baseline accuracy in your personal tasks.

Each time I am confronted with a brand new deep learning venture, I typically throw function extraction with Keras at it simply to see what happens:

  • In some instances, the accuracy is adequate.
  • In others, it requires me to tune the hyperparameters to my Logistic Regression model or attempt one other pre-trained CNN.
  • And in different conditions, I have to discover fine-tuning and even training from scratch with a customized CNN architecture.

Regardless, in the perfect case switch learning by way of function extraction provides me good accuracy and the undertaking could be accomplished.

And within the worst case I’ll achieve a baseline to beat with my future experiments.

What’s next — where do I study more about switch learning and function extraction?

In this tutorial, you discovered find out how to perform switch studying by way of function extraction and then practice a mannequin on prime of the extracted features.

But I know as soon as this submit is revealed I’m going to obtain emails and questions within the feedback relating to:

  • “How do I classify images outside my training/testing set?”
  • “How do I load an image from disk, extract features from it using a CNN, and then classify it using the Logistic Regression model?”
  • “How do I correctly preprocess my input image before classification?”

Right now’s tutorial is lengthy sufficient as it’s. I can’t embrace these sections of Deep Learning for Pc Vision with Python inside this submit.

For those who’d wish to study extra about transfer learning, together with:

  1. More details on the concept of switch learning
  2. The way to carry out function extraction
  3. The right way to fine-tune networks
  4. Easy methods to classify pictures outdoors your training/testing set using both function extraction and fine-tuning

…you then’ll undoubtedly need to confer with Deep Learning for Pc Imaginative and prescient with Python.

In addition to chapters on transfer learning, you’ll also find:

  • Super sensible walkthroughs that present options to precise, real-world picture classification, object detection, and instance segmentation issues.
  • Arms-on tutorials (with numerous code) that not solely present you the algorithms behind deep learning for pc imaginative and prescient but their implementations as nicely.
  • A no-nonsense educating fashion that is guaranteed that will help you grasp deep learning for picture understanding and visible recognition.

To study extra concerning the e-book, and grab the table of contents + free pattern chapters, just click here!


At the moment marked the start of our collection on switch studying with Keras and Deep Learning.

The 2 main types of function extraction by way of deep learning are:

  1. Function extraction
  2. Superb-tuning

The main target of as we speak’s tutorial was on function extraction, the process of treating a pre-trained community as an arbitrary function extractor.

The steps to perform switch studying by way of function extraction embrace:

  1. Starting with a pre-trained community (sometimes on a dataset comparable to ImageNet or COCO; giant enough for the mannequin to study discriminative filters).
  2. Permitting an input image to ahead propagate to an arbitrary (pre-specified) layer.
  3. Taking the output of that layer and treating it as a function vector.
  4. Coaching a “standard” machine studying model on the dataset of extracted features.

The good thing about performing switch learning by way of function extraction is that we do not want to coach (or re-train) our neural network.

As an alternative, the community serves as a black field function extractor.

These extracted options, which are assumed to be non-linear in nature (since they have been extracted from a CNN), are then handed into a linear mannequin for classification.

In the event you’re keen on studying extra about switch learning, function extraction, and fine-tuning, remember to discuss with my guide, Deep Learning for Pc Vision with Python the place I cover the topic in more element.

I hope you loved right now’s submit! Keep tuned for next week once we talk about the best way to work with function extraction when our dataset is just too giant too match into memory.

To obtain the supply code to this publish (and be notified when future tutorials are revealed here on PyImageSearch), simply enter your e mail handle in the type under!