Python, NumPy
In this project I implemented a Gradient Descent algorithm for a Logistic Regression from Scratch using only the NumPy library in Python. This is essentially a binary classifier which utilizes a sigmoid activation function. This was tested on a dataset containing over 400 data samples.
How does it work?
1.) Standardize all Features
2.) Each weight (9 weights + bias) is initialized randomly and we calculate sigmoid(x) for each sample
3.) Calculate the error between the actual value and the predicted value (when inputted into sigmoid function), this is repeated for each sample and averaged.
4.) Use the Update rule to update each weight
After each Iteration of the gradient descent algorithm we calculate the error using this cost function which is essentially the actual value subtracted from the predicted value
To Update the weights, we need to apply the update rule along with its learning rate as seen below.
We need to take a partial derivative of the error function with respect to each individual weight (theta).
The Learning curves after running this algorithm can be seen below. For each Iteration and learning rate there are two plots. The first plot in each table element represents the learning curve for batch gradient descent, the second plot represents the learning curve for mini-batch gradient descent.
Iterations | Learning Rate = 0.001 | Learning Rate = 0.0001 |
---|---|---|
500 | ![]() ![]() |
![]() ![]() |
1000 | ![]() ![]() |
![]() ![]() |
10000 | ![]() ![]() |
![]() ![]() |
How can you communicate?