We will cover the following tasks in 1 hour and 7 minutes:
Introduction and Downloading VGG19 Model
Let’s launch our notebook instances that we created in the previous project. Let’s create a new notebook on this instance which we will use for our training process. Then we will download a VGG19 model from Keras along with its weights pre-trained on the imagenet dataset. We will not use any top layer from the VGG19 model and will add our own top layer to get 128 dimensional vectors for our face embedding.
Creating Embedding Model
We have already downloaded the VGG19 weights and architecture that we will base our embedding model on. We will use the Sequential class from Keras to construct our embedding model. The first layer of this model is going to be the previously downloaded VGG19 model. Of course the VGG19 model does not include a top layer in our case. Then we will
Flatten the VGG19 output, add a
Dense layer with
128 nodes and sigmoid activation function. We are also using a
kernel_initializer. At this point, we don’t need to compile our embedding model. We will use this as part of another computational graph later and set the compile settings of that graph.
Let’s first import all the indices of triplets that got saved in our previous example. In triplet loss, we are taking a look at three embedding at a time: anchor, positive and negative. Then, we calculate the euclidean distance between the anchor and positive, and between anchor and negative. The distance between the anchor and positive should be much lower than the distance between the anchor and negative. We will also need to do online mining for the hardest negative example for the model: hardest in the sense that they are the hardest for the model to distinguish from the positive examples given an anchor. Based on this, let’s implement our custom loss function!
In this task, we will see how to create a custom data generator that can be used while model training. When we have custom datasets which may be too large to load in memory in entirety, we will need to use data generators like this example to read only a small batch of data at a time and train the model on that batch and then read the next batch and so on.
We will randomly select the triplet examples while creating a new batch of training data. The dataset was already pre-processed before we saved it to disk in the last project so we will save some time in training because of that. We don’t need to worry about the labels as we won’t require those for the triplet loss function.
Face Verifier Network
In this task, we will create a new model using Keras’ Model class which we are calling our Face Verifier Network. This is essentially our Siamese Network which will take three inputs for the three image arrays from the triplet generator and will give us a concatenated output for the three embedding vectors. Then, those three embedding vectors will be used to calculate triplet loss as we will set the loss function for this Face Verifier as the previously defined triplet loss function.
Note that the embedding model being used to calculate embedding vectors is being shared in this network. That is, the weights are shared. We will also compile the Face Verifier with the
Adam optimizer with
Training with Triplet Loss
As we do this training, we will use an Early Stopping callback so that we don’t do any training that is not required and the training process stops when the network seems to stop making any significant improvement. Because of the negative mining concept, the batch size should be higher than what we would normally use. In fact, the Facenet paper uses almost 1800 examples in one batch. We are creating a much smaller version of that implementation and also happen to have lesser memory, so we will use only 16 steps per epoch with a batch size of about 100 examples.
You will see a general downward trend for your loss values. We will plot the loss values on a graph to take a look at the trend. We will save our model as a Keras
Is there a meaningful difference between the euclidean distances between the anchor and positive examples and between anchor and negative examples. In this task, we will randomly select a few examples and get the embedding vectors as predicted by our embedding model.
We will then define a simple function to calculate euclidean distances between two vectors. Then, we will find the distances between the anchor and positive examples, and between the anchor and negative examples. When these two distances are plotted on a graph, we will notice how the distance values between the anchor and positive examples are much lower, in general, compared to the distance values between the anchor and negative examples.
About the Host (Amit Yadav)
I am a machine learning engineer with focus in computer vision and sequence modelling for automated signal processing using deep learning techniques. My previous experiences include leading chatbot development for a large corporation.