NA / 5
We will cover the following tasks in 1 hour and 13 minutes:
TensorFlow is an open source machine learning library. It is one of the most popular and widely used machine learning library at the moment. However, working with TensorFlow may seem a bit challenging at first because you’d need to understand a lot of underlying ideas on how computations with Neural Networks work. While it’s great to know those details, getting started with Neural Networks directly with writing computational graphs can be a bit intimidating.
This is where Keras comes in. Keras is a high-level API which can use TensorFlow as its backend and provide the users with a simple to use interface. Keras does the heavy lifting behind the scenes, leaving developers to focus on just the high level details. Some developers may or may not want to use Keras in a production setting but for testing out ideas quickly, it’s an amazing tool.
Loading the Data
In order to understand our problem better, we will first import the data that we’d be working with and take a closer look at it. We are going to use the popular MNIST dataset which has lots of images of hand-written digits along with their labels.
So, we have 60000 examples for the training set and 10000 examples for the test set. You will notice that each input x is of the shape (28, 28). What this means is that, for each example, there are 28 rows and 28 columns. Fortunately, we can simply print out these examples using the Pyplot module from Matplotlib.
One Hot Encoding
We will change the way this label is represented from a class name or number to a list of all possible classes with all the classes set to 0 except the one which this example belongs to - which will be set to 1.
So, now it’s as if our Neural Network will predict which switch is ON out of all the 10 switches instead of trying to predict an actual numeric value. This makes it a classification problem. If we were to try and predict an actual number, like 5 or 7 etc., it would be a regression problem instead whereas we are trying to classify the image examples in our case.
In a given network example, we have two hidden layers. The first layer with all the X features is called the input layer and the output y is called the output layer. In this example above, the output has only one “node”. The hidden layer can have a lot of nodes or a very few nodes depending on how complex the problem may be. Here, the both hidden layer have 2 nodes each. Each node is an output of a linear function which takes inputs from the nodes of the preceding layer. All the Ws and all the bs associated with all of these linear functions will have to be “learned” by our algorithm as it attempts to optimise those values in order to best fit the given data. In the hand-written digit classification problem, we will have 128 nodes for two hidden layers and of course we already know that the input is a 784 dimensional vector.
Preprocessing the Examples
We will create a Neural Network which will take 784 dimensional vectors as inputs (28 rows * 28 columns) and will output a 10 dimensional vector (For the 10 classes). We have already converted the outputs to 10 dimensional, one-hot encoded vectors. Now, let’s convert the input to the required format as well. We will use numpy to easily unroll the examples from (28, 28) arrays to (784, 1) vectors.
Pixel values, in this dataset, range from 0 to 255. While that’s fine if we want to display our images, for our neural network to learn the weights and biases for different layers, computations will be simply much more effective and fast if we normalised these values. In one of the future projects, we will take a look at how this normalisation affects the speed of learning.
Creating a Model
Creating a Neural Network model with the help of Keras is really simple. We simply use a Sequential class defined in Keras, and add some layers to it. As discussed before, we will use two hidden layers with 128 nodes each and one output layer with 10 nodes for the 10 classes. All the layers are going to be Dense layers. This means, like our examples above, all the nodes of a layer would be connected to all the nodes of the preceding layer i.e. densely connected.
We are instantiating a Sequential model. We pass on a list of layers that we want in our model, in the order that we want them. So, we have two hidden layers with 128 nodes each and one output layer with 10 nodes. We set the input shape on the first hidden layer to correspond to the shape of a single example from our reshaped training and test sets - we know each example is a 784 dimensional vector for the 784 pixels of the images.
Training the Model
Let’s train the model now. We will use our training set which has been normalised and reshaped! Also, we are going to train the model for 5 epochs. Think of epoch like an iteration of all the examples going through the model. So, by setting the epochs to 5, we will go through all the training examples 5 times.
We get a training set accuracy of over 98%. While this is probably not as good as a human level performance, it still seems quite good for a machine. But, in order to ensure that this is not a simple “memorization” by the machine, we should evaluate the performance on the test set. This is easy to do, we simply use the evaluate method on our model.
Let’s get our model’s predictions on the test dataset. Each prediction is a list of probability scores as we expected from our softmax output. What we are interested in, is the index of the highest probability score in each prediction. We can use numpy’s argmax function to do this.
We have a total of 10000 predictions. We probably can’t go through all the 10000 predictions for now, but we can take a look at the first few. Let’s plot the first few test set images along with their predicted and actual labels and see how our trained model actually performed.
About the Host (Amit Yadav)
I have been writing code since 1993, when I was 11, and my first passion project started with a database management software that I wrote for a local hospital. More recently, I wrote an award winning education Chatbot for a multi-billion-revenue company. I solved a recurrent problem for my client where they wanted to make basic cyber safety and privacy education accessible for their users. This bot enabled my client to reach out to their customers with personalised and real-time education. In the last one year, I’ve continued my interest in this field by constantly learning and growing in Machine Learning, NLP and Deep Learning. I'm very excited to share my variety of experience and learnings with you with the help of Rhyme.com.