# TensorFlow (Intermediate): Eager Execution & Automatic Differentiation

Welcome to Eager Execution! TensorFlow’s eager execution facilitates an imperative programming environment that allows the programmer to evaluate operations immediately, instead of first creating computational graphs to run later. In this course, we will not only get familiar with eager execution but will also look at how automatic differentiation works in TensorFlow.

Duration (mins)

Learners

#### 5.0 / 5

Rating

We will cover the following tasks in 53 minutes:

### Introduction

Normally, when you use TensorFlow to create and train machine learning models, you need to build computational graphs first required for your model training first and then run those graphs later to actually perform the computations. However, this approach is not very easy or intuitive to use. In production setting, this may not be a big problem but if you’re just researching and experimenting with your potential models, then this traditional approach can slow things down. This is where eager execution comes in.

### Eager Execution

TensorFlow’s eager execution facilitates an imperative programming environment that allows the programmer to evaluate operations immediately, instead of first creating computational graphs to run later. Let’s see how to enable this mode and also check if we are working with eager execution.

### Tensors

Tensors are multi-dimensional arrays that hold information on operations and data types that those operations are performed on. A tensor object has a data type and a shape of data that it expects. TensorFlow offers a comprehensive library of tensor operations like addition, matrix multiplication and so on. These operations automatically convert native Python types into tensors. Let’s look at a few examples in this chapter.

### NumPy Compatibility

Because both the data types and operations are inherent properties of tensors, it is not possible to mutate the tensors. While NumPy arrays are mutable, they can be easily converted to tensors and vice versa. Of course, when we convert a tensor to a NumPy array, we are only converting the resulting value of the tensor. By default, NumPy arrays are automatically converted to tensors and tensors are automatically converted to NumPy arrays. Tensors can also be explicitly converted to NumPy arrays by calling the `numpy()` method. Let’s look at some of these examples in this chapter.

### Device Placement

TensorFlow operations can be backed by either GPU or CPU. TensorFlow can automatically decide if GPU or CPU is used for the operations. Tensors produced by these operations are typically backed by the memory of the device the operation was executed on.

### Dataset from Tensors

We can use TensorFlow’s Dataset API to build pipelines to feed data to our models. We can use this to create data pipelines to feed our model’s training and evaluation loops. In eager execution mode, we don’t need to construct a TensorFlow iterator, but instead we can simply use Python’s iterator over the Dataset objects. There are also a bunch of methods to create dataset objects. You can create the dataset objects from tensors or by reading text or CSV files.

### Automatic Differentiation

Automatic differentiation is a way to programmatically compute derivative of a function. This process involves applying chain rule repeatedly to simpler operations. TensorFlow records all operations executed inside a Gradient Tape. In order to compute gradients associated with each recorded operation, gradients of all operations leading up to required values are computed using automatic differentiation.

### Control Flow

Python’s control flow is handled naturally by the gradient tapes. Let’s take a look at it in this chapter.

With Gradient Tapes, higher order gradients are actually quite easy to compute. If gradients are computed within a gradient tape context, those computations can be stored in a gradient tape as well. This way, higher order gradients can be computed. Let’s take a look!

## Watch Preview

Preview the instructions that you will follow along in a hands-on session in your browser.

I am a machine learning engineer with focus in computer vision and sequence modelling for automated signal processing using deep learning techniques. My previous experiences include leading chatbot development for a large corporation.

##### How is this different from YouTube, PluralSight, Udemy, etc.?
In Rhyme, all projects are completely hands-on. You don't just passively watch someone else. You use the software directly while following the host's (Amit Yadav) instructions. Using the software is the only way to achieve mastery. With the "Live Guide" option, you can ask for help and get immediate response.
##### Can I buy Rhyme sessions for my company or learning institution?
Absolutely. We offer Rhyme for workgroups as well larger departments and companies. Universities, academies, and bootcamps can also buy Rhyme for their settings. You can select projects and trainings that are mission critical for you and, as well, author your own that reflect your own needs and tech environments. Please email us at help@rhyme.com
##### What kind of accessibility options does Rhyme provide?
Rhyme strives to ensure that visual instructions are helpful for reading impairments. The Rhyme interface has features like resolution and zoom that will be helpful for visual impairments. And, we are currently developing a close-caption functionality to help with hearing impairments. Most of the accessibility options of the cloud desktop's operating system or the specific application can also be used in Rhyme. If you have questions related to accessibility, please email us at accessibility@rhyme.com
##### Why don't you just use containers or virtual browsers?
We started with windows and linux cloud desktops because they have the most flexibility in teaching any software (desktop or web). However, web applications like Salesforce can run directly through a virtual browser. And, others like Jupyter and RStudio can run on containers and be accessed by virtual browsers. We are currently working on such features where such web applications won't need to run through cloud desktops. But, the rest of the Rhyme learning, authoring, and monitoring interfaces will remain the same.
##### I have a different question
Please email us at help@rhyme.com and we'll respond to you within one business day.

## More Projects by Amit Yadav

45 minutes
56 minutes
27 minutes
###### Amazon Sagemaker: Image Classification
1 hour and 8 minutes
55 minutes
###### Amazon Sagemaker: Semantic Segmentation
1 hour and 2 minutes
60 minutes
###### Amazon Sagemaker: Custom Training with TensorFlow
1 hour and 8 minutes
###### Amazon Sagemaker: Object Detection
1 hour and 12 minutes
38 minutes
###### Deep Learning Fundamentals - Logistic Regression
1 hour and 12 minutes
49 minutes
###### Custom Prediction Routines with Google AI Platform
1 hour and 8 minutes
###### Linear Regression with Python
1 hour and 4 minutes
48 minutes
###### Classification with Transfer Learning in Keras
48 minutes
1 hour and 26 minutes
48 minutes
54 minutes
43 minutes
54 minutes
58 minutes
###### TensorFlow (Advanced): Neural Style Transfer
1 hour and 5 minutes
###### Create a Neural Network Visualizer Web App with Python
1 hour and 5 minutes
###### Create Your First Chatbot with Rasa and Python
1 hour and 20 minutes
49 minutes
46 minutes
###### Create Your First Game with Python
1 hour and 4 minutes
###### TensorFlow (Beginner) - Basic Image Classification
1 hour and 10 minutes
40 minutes