K- Nearest-Neighbors(KNN) Algorithm For Machine Learning

Introduction:

 Knn is a  machine learning algorithm which can be used for both classification and regression predictive problem. Most probably it is used in classification problem.Knn Algorithm has another name Lazy Algorithm and  it   comes under supervised learning.

Why  Knn algorithm is called as Lazy Algorithm?:

Most of the Algorithm like Bayesian ,Classification,Logistic  Regression ,SVM etc are  generalize over the training set before receiving the test data.They create a model based on the training data before receiving  the test data and then do the prediction /classification.So these Algorithms are called Eager Algorithm .But in case of Knn it never creates  a  model.Knn  Algorithm  try to keep whole dataset and give you some prediction.

Let’s understand how does Knn works by taking an interesting Example:

From this above diagram we can easily understand how Knn algorithm works:

Diagram1: In diagram no 1 we are taking two class problem  one class is class A and  second one is class B. Then we are initializing a  new data point to classify it’s  class. we will apply Knn Algorithm to classify it’s class.

Diagram 2: Second diagram representing the first steps towards Knn Algorithm.

Step1 : This Algorithm Will calculate all the distances from the new data point to all the data point present in two classes.Second question comes to our mind  how  it calculates the distance? Euclidean ,Manhattan or Hamming distance calculation method. These are the most commonly used methods for distance calculation.It can use any one method for distance calculation.

Step2 : After calculating all distances this algorithm will check which data  point is nearest to this new data point.Then it looks for the nearest data point that belongs to which class?If it belongs to class A then this new datapoint will also belongs to class A  or if  it belongs  to class B then this newpoint will also belongs to class B.

In Knn we have to give the k value

             K = How many nearest point you are checking in

 K=1: It means this Algorithm  have  to check for one nearest data point from this new datapoint.

 K=2: It means this Algorithm have to check for two nearest data point from this new data point. After that it will decide which class this new point will  belong  to.

If two nearest data points belong to the same class then the new data will belong to that class.

If two nearest data points belong to different classes then what will it do?

In this case this Algorithm will check which nearest data point is more near to this new data point.The new data point will belong to that data point class

Diagram 3: In Diagram no 3 it represents the class of new data point when k=3

K=3: It means this algorithm have to check for three nearest point from this new data point.Then it will check for the majority of class.After that it will decide which class this new point will belong to.

Majority of class:

This  Algorithm will check the majority of the class of the three nearest points after that it will assign the new datapoint into the same class.

So here we can easily understand that K value is playing a vital role.All the decision has been taken by KNN Algorithm based on  K value.

HOW TO TAKE OPTIMAL K VALUE  IN KNN?

1 There is no hard and fast rule to choose an optimal K value.we might have to perform Knn with different K      values.   

2  Whenever we perform KNN Algorithm on any dataset In this case  check for training  accuracy and testing accuracy and look where they are matching that will be the optimal k value.   

APPLICATION OF KNN IN REAL LIFE:

Without knowing the application of  Knn algorithm learning of Knn is waste.In different domain we can apply knn  like FInance,Medicine,Agriculture etc.

Examples:1 Loan management

                  2 Credit Rating

                  3  Identify the risk factor for cancer based on clinical and demographic variables.

CONCLUSION:

Knn Algorithm is a very powerful implementation of Machine learning.we can build our Knn Machine learning  for predictions using R,PYTHON.R and PYTHON both are having very interesting packages to perform machine learning  and statistical work.