Gradient Descent for a Logistic Regression

Python, NumPy

About

In this project I implemented a Gradient Descent algorithm for a Logistic Regression from Scratch using only the NumPy library in Python. This is essentially a binary classifier which utilizes a sigmoid activation function. This was tested on a dataset containing over 400 data samples.








How does it work?

The Function Call
1

1.) Standardize all Features


2.) Each weight (9 weights + bias) is initialized randomly and we calculate sigmoid(x) for each sample


3.) Calculate the error between the actual value and the predicted value (when inputted into sigmoid function), this is repeated for each sample and averaged.


4.) Use the Update rule to update each weight

Download free bootstrap 4 landing page, free boootstrap 4 templates, Download free bootstrap 4.1 landing page, free boootstrap 4.1.1 templates, meyawo Landing page
2

After each Iteration of the gradient descent algorithm we calculate the error using this cost function which is essentially the actual value subtracted from the predicted value

Download free bootstrap 4 landing page, free boootstrap 4 templates, Download free bootstrap 4.1 landing page, free boootstrap 4.1.1 templates, meyawo Landing page
3

To Update the weights, we need to apply the update rule along with its learning rate as seen below.

We need to take a partial derivative of the error function with respect to each individual weight (theta).

Download free bootstrap 4 landing page, free boootstrap 4 templates, Download free bootstrap 4.1 landing page, free boootstrap 4.1.1 templates, meyawo Landing page
Output

The Learning curves after running this algorithm can be seen below. For each Iteration and learning rate there are two plots. The first plot in each table element represents the learning curve for batch gradient descent, the second plot represents the learning curve for mini-batch gradient descent.


Iterations Learning Rate = 0.001 Learning Rate = 0.0001
500 Download free bootstrap 4 landing page, free boootstrap 4 templates, Download free bootstrap 4.1 landing page, free boootstrap 4.1.1 templates, meyawo Landing page Download free bootstrap 4 landing page, free boootstrap 4 templates, Download free bootstrap 4.1 landing page, free boootstrap 4.1.1 templates, meyawo Landing page Download free bootstrap 4 landing page, free boootstrap 4 templates, Download free bootstrap 4.1 landing page, free boootstrap 4.1.1 templates, meyawo Landing page Download free bootstrap 4 landing page, free boootstrap 4 templates, Download free bootstrap 4.1 landing page, free boootstrap 4.1.1 templates, meyawo Landing page
1000 Download free bootstrap 4 landing page, free boootstrap 4 templates, Download free bootstrap 4.1 landing page, free boootstrap 4.1.1 templates, meyawo Landing page Download free bootstrap 4 landing page, free boootstrap 4 templates, Download free bootstrap 4.1 landing page, free boootstrap 4.1.1 templates, meyawo Landing page Download free bootstrap 4 landing page, free boootstrap 4 templates, Download free bootstrap 4.1 landing page, free boootstrap 4.1.1 templates, meyawo Landing page Download free bootstrap 4 landing page, free boootstrap 4 templates, Download free bootstrap 4.1 landing page, free boootstrap 4.1.1 templates, meyawo Landing page
10000 Download free bootstrap 4 landing page, free boootstrap 4 templates, Download free bootstrap 4.1 landing page, free boootstrap 4.1.1 templates, meyawo Landing page Download free bootstrap 4 landing page, free boootstrap 4 templates, Download free bootstrap 4.1 landing page, free boootstrap 4.1.1 templates, meyawo Landing page Download free bootstrap 4 landing page, free boootstrap 4 templates, Download free bootstrap 4.1 landing page, free boootstrap 4.1.1 templates, meyawo Landing page Download free bootstrap 4 landing page, free boootstrap 4 templates, Download free bootstrap 4.1 landing page, free boootstrap 4.1.1 templates, meyawo Landing page

How can you communicate?

Contact Me

Copyright © DevCRUD