Sigmoid Function in Logistic Regression is an Advanced Regression Technique that can solve various classification problems. Being a classification model, it is termed “Regression” because the fundamental techniques are similar to Linear Regression.
Binary classification problems like a tumour is Malignant or not, an Email is spam or not and Multiclass Classification problems like classifying a random fruit as Apple, Mango or Orange can be solved using Logistic Regression.
In the Linear Regression problems we can simply predict a non discrete outcome (∈R) to an example by fitting the best line to the data given there’s a hypothesis (h) that maps X to Y.
In logistic regression where the outcome always belongs to a set Y, consisting discrete values (say 0 or 1 only)
A few concepts you should know before diving deeper
Hypothesis:
Hypothesis is an instance or model that maps inputs to outputs and can be evaluated and used to make predictions.
For example, for Univariate Linear Regression Hypothesis can be given as hθ(x) = θ0 + θ1(x) where θ0 and θ1 are the features.
If we generalize this hypothesis, we get
And on vectorizing the above equation we get : 
We then find out the maximum cost [say J(θ)] our model has to pay each time we run it. In our attempt to minimize this cost function J(θ), we use algorithms such as Gradient Descent and other complex algorithms such as Conjugate Gradient, BFGS, L-BFGS.
Gradient Descent
It is a function used to minimize the cost function [J(θ)]
A simple gradient Descent algorithm for Univariate Linear Regression can be given as:
Repeat until global minima is achieved
{
θj := θj – α (partial derivative of J(θj) ) for j=0,1
Simultaneously update for j=0 and 1
}
What if we use Linear Regression for classification problems?
Let’s say we have a binary classification problem, the outcomes belong to the set {0,1}
On using Linear Regression, we may get
hθ(x) > 1 or hθ(x) <0 , which seems absurd if we are solving a binary classification problem.
So we have to restrict our hypothesis to a function that maps X to Y such that Y lies between 0 and 1. We achieve this restriction using Sigmoid Function.
Let’s know what Sigmoid Function is and what is its nature.
Sigmoid Function
A sigmoid Function can be mathematically written as 
If we plot this function in on a Cartesian plane, we get the following graph:
The sigmoid curve is asymptotic at 1 and 0. It can restrict the outcome value to lie in between 0 and 1 no matter what is the value of X.
We see that even though x ranges from -∞ to ∞ the outcome remains in the range 0-1 with the value 0.5 when x=0.
Let’s see how we can use this function to tune Linear Regression which is an algorithm used for prediction in Logistic Regression that can be used for classification problems and how we can define the decision boundary.
We want our hypothesis to follow the condition 0≤hθ(x)≤1. As discussed above, the vectorized general form of Linear Regression Hypothesis is 
and if we apply sigmoid function to it:
Where 
So our hypothesis becomes 
We prefer sigmoid Function over other functions of similar nature because the loss function is smaller in the case of Sigmoid function.
Defining a decision boundary in Sigmoid Function
Now that we have applied the sigmoid function to the Linear Regression hypothesis and have obtained a hypothesis for Logistic Regression . Let’s see how the classification will be possible.
We can predict,
y=1 if hθ(x)≥0.5
y=0 if hθ(x)<0.5
Conclusion
In conclusion, we can say that first we had a Linear Regression Function θTx that could give any continuous value as output but when Sigmoid Function is applied to it [g{θTx)] our outcome will be restricted to any value in between 0 and 1.
Written By: RITIKA SAINI
Reviewed By: Viswanadh
If you are Interested In Machine Learning You Can Check Machine Learning Internship Program
Also Check Other Technical And Non Technical Internship Programs
