Deep Learning- Basics
What is Deep Learning, Future, Application, and Component of DL
- What is Deep Learning
- Why Deep learning is becoming popular now?
- Future of Deep Learning.
- Applications of Deep Learning.
- Components of Deep Learning and how it works?
If you are interested in learning the basics of Machine learning then please read here.
Machine Learning- Quick and Easy Part-1
Machine Learning Quick and Easy Part-2
What is Deep Learning
Deep Learning is a subset of machine learning in AI. It utilizes a hierarchical level of artificial neural networks to carry out the process of machine learning. The artificial neural networks are built like the Human Brain.
Deep learning is a type of machine learning that mimics the neuron of the neural networks present in the human brain.
Deep Learning is the most fascinating branch of machine learning not only the fascinating also the most powerful.
How you ever thought how human brain works ??
The human brain is one of the most important tools in this world for Learning, understanding and applying them. And If computers can work in the same way the brain works that are the core idea of Deep Learning.
Neurons are the base/processing unit of brain, so how can we create it in computers ??
we create an artificial structure like our brain
Like our brain has input unit eyes, nose and ears then neurons do some processing and output comes. “Below I have explained a little about Neuron”
In the same way, we have input neurons some hidden layers and output for the result.
Then why it is Deep Learning ?? there is nothing deep here ?
We call it deep learning because we take this to further steps by expanding it to more dimensions by creating multiple hidden layers.
Like brain we have a hidden process which we aren’t aware of in the same way we create multiple hidden layers for better processing. that is why we call it as DEEP LEARNING.
Why Deep Learning is becoming popular now ?? why not before
Deep Learning creates ANN and CNN which were introduced in the 1960s or 1970s. so why Deep learning is picking up now ??
ANN and CNN were there in the 1970s but we didn’t have enough computer resources or processing capabilities. Data was there but where to store ??
For example
Back then in the 1960s for storing 10 MB of data you had to pay $3000 and now in 2019 256 GB costs us around $12.
So we can say we have exponential progress in terms of technology not linear. Now storage doesn’t cost you anything it’s free to use like google cloud, Dropbox.
Right now Scientists are looking to store data in DNA, but it costs a lot for processing. Isn’t it sounds similar to the storage scenario which had in the 1960s?
Same goes for CPU or processing capacity back then and now. So we didn’t have enough CPU, Storage resources to process data that was the main reason deep learning and AI were not that popular.
Future of Deep Learning
Those days are not far enough when everyone will have their own personal assistant robots and they will help human beings for whatever needs they have, like cooking your food, cleaning your rooms and other things.
As of now Voice assistants like Siri, Ok Google, Alexa they are in the market and in coming few years we will be able to see Siri, Alexa, and many more live Robots.
Applications of Deep Learning
Components of Deep Learning and how it works ??
All explanation which I have done here is too basic, just to understand how things work in ANN and CNN and basic definition of all terms associated with Deep Learning. However, I am writing another Technical Article for Deep Learning.
- The Neurons
- The Activation Functions
- How does Neural Network work /Learn?
- Gradient Descent
- Stochastic Gradient Descent
- Back-propagation
The Neurons
The basic building block of Artificial Neural Network ANN and also the basic unit of our Brain. Deep Learning’s main idea is to learn how the human brain works.
This is the picture of Neuron from 1960, A scientist saw this in a microscope and drew an image at the same time.
Neurons are interconnected to each other by some veins. it’s enough of a real brain, let’s see how we create it in computers.
The brain is connected to other neurons or input signals. In the same way, real Artificial Neurons are connected to input values or another neuron. Like Linear regression, we had an independent variable and dependent variable so we also have the same variables here.
Standardization and Normalization of independent variables are required before processing.
Output value could be anything.
- Categorical
- Yes / No
- Continuous Price
Weights play an important role in the learning of neural networks. we adjust the weights by gradient descent and back-propagation. weights decide which input variable is important or not.
What happens inside Neurons ??
Step 1
All input values get added along with the weights.
Step 2
We apply Activation function to it.
The Activation Function
There are many different types of activation function, we will talk about a few popular Activation functions.
1. Threshold function
- Very simple YES / NO kind of function, If value of X>= 0 then 1 else 0.
2. Sigmoid Function
- 1/1-e(power-x) where X is the weighted sum of inputs.
- Sigmoid Function is mainly used in Logistic Regression. This function is useful in final or output layer.
3.Rectifier
- The rectified linear activation function is a simple calculation that returns the value provided as input directly, or the value 0.0 if the input is 0.0 or less.
We apply rectifier function in hidden layers and sigmoid function in output neuron.
How does Neural Network works / Learn?
Let suppose we want to predict house price then we have input values and the output value is a price that is continuous.
Input values could be Distance, size, Bedrooms(Input variables)
Input — — — ->Hidden Layers— —->Output
The hidden layer has many neurons and each neuron is a combination of different inputs we apply to the activation functions and get the output.
Learning takes place, either we hard code everything by telling all parameters or to leave it to the Neural network .
Example- If there is a case to Identify Cat/Dog then by writing all small parameters of Cat and Dog or to provide Cat/Dog pictures as a test dataset and pass them through ANN then when we provide next image it will be able to tell whether this picture is of Cat or Dog.
Here input values are passing to hidden layer then we are getting the output Y cap. Y is the actual output. Cost function is calculated then we adjust the weights in this way Neural network learns in their own.
We feed the values, it passes through the neuron and function, we get the Y cap, compare it with actual value Y, Cost function generated, we adjust the weights.
Gradient Descent
Gradient descent is an optimization algorithm used to find the values of parameters (coefficients) of a function (f) that minimizes a cost function (cost).
Gradient descent helps us to minimize the cost function which eventually helps us to improve our network.
We take a random point of Cost function and find its slopes and follow this till we get to the minimum point. Gradient descent is best used when the parameters cannot be calculated analytically (e.g. using linear algebra) and must be searched for by an optimization algorithm
Stochastic Gradient Descent
Gradient descent can be slow to run on very large datasets.
Because one iteration of the gradient descent algorithm requires a prediction for each instance in the training dataset, it can take a long time when you have many millions of instances.
In situations when you have large amounts of data, you can use a variation of gradient descent called stochastic gradient descent.
In this variation, the gradient descent procedure described above is run but the update to the coefficients is performed for each training instance, rather than at the end of the batch of instances.
Backpropagation
For a neural network to learn, there has to be an element of feedback involved. Feedback helps us to improve and we all use it, Don’t we?
When you do something for the first time you take note of how you did it, To improve it, you remembered what you’d done wrong before, modify your movements to get a better result.
Neural networks learn things in exactly the same way, typically by a feedback process called backpropagation.
There are two terms Forward propagation and Back-propagation, In Forward propagation information is entered into the input layer and propagated forward to get output Ycap, then we compare Ycap and Y and find out the error.
Y cap is predicted output & Y is real output.
The process of propagating errors in the backward direction and adjusting weights in the network simultaneously is called Back-propagation.
Training the ANN with Stochastic Gradient Descent
We need to follow the below steps in order to train our ANN.
- Initialize the weights to small numbers close to 0.
- Input your Dataset in the input layer.
- Forward propagation: from left to right, propagate until we get the predicted Y.
- Compare the predicted result to the Actual Result.
- Back-Propagation: from right to left, error is back-propagated.update the weights, learning rate decides by how much we update the weights.
- Repeat the above steps.
If you are interested in learning the basics of Machine learning then please read here.
Machine Learning- Quick and Easy Part-1
Machine Learning Quick and Easy Part-2
=====================================
