We will cover the following tasks in 52 minutes:
In this task, we will launch our Notebook instance where we created and trained our model in the previous project. This instance has a GPU attached to it. We will also need to create a bucket on Google Storage. We will save our trained model here before we can run inference on it. Let’s create a new notebook where we will write code for deployment and inference during this project. We will also use
Create a SavedModel
In this task, we will load the model that we trained in the previous project. We will note the name of input of this model. This is because the input name will be used as a key in the object that we will use later as input for this model. We will also need to convert our model to what is called a
SavedModel format. We will use the
save_keras_model helper function to achieve this.
Saved Model to Cloud Storage
We will now transfer our model, which is now in the saved_model folder, to google cloud storage. We will transfer the entire folder because while running inference, GCP needs not only the
.pb file but also the assets and variables folders created along with it when the model is exported to this format. We had already created a bucket in the first task to store this model folder.
Prepare Input for Inference
In order to prepare the inputs for inference, we will create a
create_input function. In our example, we just want to use the anchor, positive and negative arrays that we had already created in the first project in this series. We will use our model input’s name as key when creating a json object that we will need for inference. Note that we will need to create multi-dimensional arrays to a list format for the json object to work. Then, we save the json object on disk. We will use this object in one of the next tasks.
Deploy the Model
Now, we will deploy our model! The model is already saved in the right format on Google Storage in a bucket. In order to use Google AI Platform model deployment, we need the models to be deployed in a Google Storage bucket. We will create a new version for our Google AI Platform model and set that version to the location of our model folder on Google Storage. Once the deployment is complete, go back to the Notebook instance and use the
create_input function to create some json objects that we can use as inputs when we run inference on the deployed model.
We will first run an inference from the terminal using
gcloud ml-engine. Once this works, and a list of numbers is returned back, we know that everything is setup correctly and now we can start working on creating a method that will make a similar call as what we just did but it will be able to use the information that is returned (the feature vectors) and compare the euclidean distances between the various embedding vectors.
Let’s define a function called
predict_json which will take a json object file path as argument. We need a service object in order to make a query using the
googleapiclient. We will need to create a json object instance by reading the json object we created on disk before. Once we make a request to the deployed model via the Google Cloud AI Platform, a response will be sent back. This response has the predicted embedding vector in the key
Check Euclidean Distances
Of course, for our application, we will need to calculate euclidean distances between any two given embedding vectors. If this distance, between two predicted embedding vectors, is less than a certain threshold, then we know that the two images are of the same person’s face. To implement this, we will create a new function called
check_distances. We will pass an index for a triplet to this function and the function will make calls to the model to get embedding vectors for anchor, positive and negative examples. Then, the function will calculate euclidean distances between the predicted embedding vectors for anchor and positive and between anchor and negative.
The distance between predicted embedding vectors for anchor and positive should be generally much higher compared to the distance between the predicted embedding vectors for anchor and negative. We will check that out for a few triplets!
About the Host (Amit Yadav)
I am a machine learning engineer with focus in computer vision and sequence modelling for automated signal processing using deep learning techniques. My previous experiences include leading chatbot development for a large corporation.