Deploy Models with TensorFlow Serving and Flask

In this course, you will learn how to create a model and load previously saved weights to it. Also, you will learn to use this pre-trained model in a Flask web application. By the end of the course, you will have a working Flask application running in your browser which will be able to make fairly accurate predictions on images to predict if an image is of a glass or a table.

Join for Free
Deploy Models with TensorFlow Serving and Flask

Duration (mins)


5.0 / 5


Task List

We will cover the following tasks in 54 minutes:


We will understand the Rhyme interface and our learning environment. You will get a virtual machine, you will need Jupyter Notebook and TensorFlow for this course and both of these are already installed on your virtual machine. Jupyter Notebooks are very popular with Data Science and Machine Learning Engineers as one can write code in cells and use other cells for documentation.

Getting Started with the Flask App

The machine is already setup with everything we will need for this course. As a starting point, we can use our virtual environment which has the packages already installed. Type venv\scripts\activate and this will launch the virtual environment. Now, if you type flask run, the app will launch. Now we can try out the app. We will test it out with a few images.

Index Template

We are trying to create a super simple hello world web app with Flask. This will have just one route for the home page which will display just a simple “hello world” message for now. In the next few chapters, we will iteratively keep working on this file.

TensorFlow Serving

In order to load our model, we will need to import Keras and TensorFlow. The way to load a pre-trained model is to first create an instance of a model with the exact same architecture as what you used when you trained the model. So, we trained this model in one of our previous courses on computer vision. The course was called transfer learning and our model was trained on a glass vs table data. The ResNet50 model trained on ImageNet is pretty big so it will take some time to load so just be patient.

Getting Predictions

In the previous chapter, we creted a model and loaded weights from an earlier training. This means that our model is now ready to make predictions. Except the problem is that we need to pre-process incoming images before we can actually get the model to make predictions on them. This is because the model expects tensors of certain shape so we need to convert our incoming images accordingly. We will need a couple of methods from the image preprocessing library of Keras. Let’s define a method called prepare_images which will return processed data suitable for our model given a list of images

Connecting the Model Server to the App

We will write a method which will return a string based on our model’s prediction given an image. Our model was trained on the Glass vs Table dataset and predicts whether an image is of a glass or a table. We will be sure to use TensorFlow’s default graph.

Displaying the Results in the App

In this chapter, we will finally finish writing our show_main method which will render a bit of html to let users upload images. Then, we will use the make_prediction method written in the previous chapter to make a prediction an uploaded image. Once everything is working, we will go to Chrome and go to the localhost. There, we have the upload form. We will upload an image from our tests folder and take a look at the prediction.

Watch Preview

Preview the instructions that you will follow along in a hands-on session in your browser.


Very useful course

Nadezhda Petrova
Nadezhda Petrova
Amit Yadav

About the Host (Amit Yadav)

I am a machine learning engineer with focus in computer vision and sequence modelling for automated signal processing using deep learning techniques. My previous experiences include leading chatbot development for a large corporation.

Frequently Asked Questions

In Rhyme, all projects are completely hands-on. You don't just passively watch someone else. You use the software directly while following the host's (Amit Yadav) instructions. Using the software is the only way to achieve mastery. With the "Live Guide" option, you can ask for help and get immediate response.
Nothing! Just join through your web browser. Your host (Amit Yadav) has already installed all required software and configured all data.
Absolutely! Your host (Amit Yadav) has provided this session completely free of cost!
You can go to, sign up for free, and follow this visual guide How to use Rhyme to create your own projects. If you have custom needs or company-specific environment, please email us at
Absolutely. We offer Rhyme for workgroups as well larger departments and companies. Universities, academies, and bootcamps can also buy Rhyme for their settings. You can select projects and trainings that are mission critical for you and, as well, author your own that reflect your own needs and tech environments. Please email us at
Rhyme strives to ensure that visual instructions are helpful for reading impairments. The Rhyme interface has features like resolution and zoom that will be helpful for visual impairments. And, we are currently developing a close-caption functionality to help with hearing impairments. Most of the accessibility options of the cloud desktop's operating system or the specific application can also be used in Rhyme. If you have questions related to accessibility, please email us at
We started with windows and linux cloud desktops because they have the most flexibility in teaching any software (desktop or web). However, web applications like Salesforce can run directly through a virtual browser. And, others like Jupyter and RStudio can run on containers and be accessed by virtual browsers. We are currently working on such features where such web applications won't need to run through cloud desktops. But, the rest of the Rhyme learning, authoring, and monitoring interfaces will remain the same.
Please email us at and we'll respond to you within one business day.

Ready to join this 54 minutes session for free?

More Projects by Amit Yadav

Your First Python Program
Your First Python Program
1 hour and 26 minutes
Linear Regression with Python
Linear Regression with Python
1 hour and 4 minutes