# Simple Linear Regression from Scratch!

Using the already created libraries like `sklearn` or `tensorflow` or `pytorch` or any other is very easy, & just after learning using these libraries, giving yourself a title of Machine Learning Engineer or Deep Learning Engineer is a complete disaster/blunder. In reality, what makes you capable of giving this title to yourself is the amount of knowledge in that field, & here, knowledge doesn’t mean the ability to use the already created libraries, here, knowledge means the internal working of the algorithms, ability to create them from scratch. Once a person is able to identify the actual working of the algorithms, then that person can solve any custom problems that can appear to be unsolvable at the first sight, or there is no already built solution for that problem.

## Pre-requisites for the best understanding of this blog:

1. Basic Understanding of Machine Learning & its types (no in-depth intuition is required).
2. Basic understanding of Gradient Descent as a whole, you should be aware of the weight & bias update process (no in-depth intuition is required). If you want to understand Gradient Descent as a whole in-depth, refer to my blog mentioned below:

# Linear Regression using Stochastic Gradient Descent

1. The maximum number of iterations is reached.
2. Error calculated is less than the set threshold (A threshold is set which means that if we are getting the error in the model prediction below/less than this number i.e., threshold, then we are satisfied. We do this because every time the error between the prediction & actual values can’t be 0).

# Linear Regression using Batch Gradient Descent

1. The maximum number of iterations is reached.
2. Error calculated is less than the set threshold (A threshold is set which means that if we are getting the error in the model prediction below/less than this number i.e., threshold, then we are satisfied. We do this because every time the error between the prediction & actual values can’t be 0).

# Linear Regression using Mini Batch Gradient Descent

1. The maximum number of iterations is reached.
2. Error calculated is less than the set threshold (A threshold is set which means that if we are getting the error in the model prediction below/less than this number i.e., threshold, then we are satisfied. We do this because every time the error between the prediction & actual values can’t be 0).

# Comparison of all the Gradient Descent Approaches with an example

`# Importing Standard Scalerfrom sklearn.preprocessing import StandardScaler# Generating the sample datax = np.arange(50, 150)# Transforming the data using Standard Scalerx = StandardScaler().fit_transform(x)# Generating the data for linear equation as y = 3x + 9y = 3 * x + 9# Converting features & target values into 2-D arrays, so that they can be passed into the above functions to calculate the weights & biases according to a particular Gradient Descent Algorithm.x = x.reshape(-1, 1)y = y.reshape(-1, 1)`

Each vertical line represents end of iterations required to calculate weights & biases for a particular Gradient Descent approach. Each algorithm loss/error showing line & line showing iteration ending is having the same colour.

# Conclusion

--

--

## More from Harshit Dawar

Big Data Enthusiast, have a demonstrated history of delivering large and complex projects. Interested in working in the field of AI and Data Science.

Love podcasts or audiobooks? Learn on the go with our new app.

## Harshit Dawar

Big Data Enthusiast, have a demonstrated history of delivering large and complex projects. Interested in working in the field of AI and Data Science.