MATLAB is a powerful tool for a lot of scientific tasks. It also has a Deep Learning toolbox that enables it to perform Deep Learning. We can use the concept of Transfer Learning to perform such tasks
What is Transfer Learning? | MATLAB
Transfer Learning refers to the transfer of information from a pre-trained network to a new network. Stevo Bozinovski and Ante Fulgosi, in 1976 came up with transfer learning in neural networks training. They gave a mathematical model and geometrical representation for transfer learning. Transfer learning is a method in machine learning especially deep learning which concentrates on storing the knowledge and using it while solving another similar problem.
Transfer Learning is related to problems like learning multiple tasks & concept drift and is not exclusively a field of study for deep learning. However, it is popular in deep learning given the vast resources required to train the deep learning models such as CNNs which have demanding datasets on which the models are trained.
There are two popular approaches listed below.
1) Develop Model Approach
2) Pre-trained model Approach
In 1981, transfer learning was first used to train images on a Neural Network (CNN). This method of training makes the convolutional neural network fast, where the random initialization is done. Some of the examples of pre-trained CNN models are AlexNet, VGG16, VGG19, ResNet, GoogleNet, etc. which are used for transfer learning by the researchers. Transfer learning is significantly used to reduce the training time or we can say that it provides faster learning ability.
The key problem of this learning process is that it finds the similarity between the old problem and the new problem. Here we are going to use the pre-trained model approach. We are going to apply Transfer Learning on ‘AlexNet’.
How to implement in MATLAB?
First, we need to use Transfer Learning to retrain the network, and Next, we need to check its inference accuracy and keep retraining until it is high enough.
1. Loading and Retraining the Network.
This step includes the collection of data, loading them into the pre-trained network, and then replacing the final few layers (fully connected layers), according to our convenience.
2. Gathering Labeled Data
In Transfer Learning, networks are trained with millions of images, so it doesn’t require a very big dataset to be retrained. The major purpose of retraining is to give the network an idea about the images it needs to classify (detect). For example, think we need to detect a black cap, but ‘AlexNet’ has labeled named ‘cowboy hat’ which is most correlated to the cap.
So, the network would assume it as a cowboy hat, but at the same time, there is a probability that it classifies it as a ‘crash helmet’ which is again much similar to a cap. To avoid this problem and improve the accuracy of networks, we retrain the model with images similar to the detection dataset. To gather such data, we use the Pi-Cam.
We used MATLAB to control the camera and collected 100 different images of each type of object and then chose 15 from each and labeled them into folders, each folder name referring to a label of the network. The dataset can be increased but we figured out this data-set is enough to retrain.
3. Loading the network and replacing final layers
We have loaded the network into MATLAB. We divided the labeled data in the ratio of 7:3 into training and validation. Validation data is used to check the inference accuracy which we would be discussing after a few sections. The Deep Learning Toolbox provides some functions to load and manipulate the Network. ‘net = alexnet’ is used to load the network into MATLAB and then we can review the architecture of that network using the ‘analyzeNetwork’ function.
This gives a visual representation of the layers of the network and thus making it easy to identify the layers that need to be replaced and any other structural changes if needed. We found that to make it a classifier for five objects we need to replace the final 1000 dimensional ‘softmax’ layer into a 5-dimensional ‘softmax’.
As we mentioned earlier, to change the last few layers of the Fully Connected network. This can be complete in two ways. One of them is using the available (GUI) named “Deep Network Designer’.
Image: Replacing the final layers.
On this GUI you will have a representation of AlexNet. We used this to delete the existing ‘fullyConnectedLayer’ and replaced it with a new one with custom hyperparameters and an output size of 5. Then we replaced the output ‘classificationLayer’ as well. We can also use custom code to perform this task, but this is more informative and error-free.
4. Retraining the Network
The input layer of “AlexNetx” takes 227×227 images and the images we took were of different sizes. We resized all the training images using the data augmenter available in the toolbox for this purpose. This augmenter is also use to create new data to prevent overfitting. The general practice involves always using an augmenter with at least two to three different processed versions of images.
‘imageDataAugmenter( )’ is a constructor that allowed us to create such augmented images. Then we used “trainingOptions( )’ to set various parameters and hyper-parameters required for a network to learn. This included ‘MiniBatchSize’, “InitialLearnRate’, ‘MaxEpochs’, etc. This function also takes the validation data to check the accuracy.
Image: Training accuracy on MATLAB
5. Checking the inference accuracy and retraining the model
After the network is trained, a validations dataset is use to check the accuracy. It means passing an image through the network on which it didn’t train and checking how good the network is to classify the image into the right category. As soon as training ends it pops up with a status report which has a detailed graph of accuracy vs epochs.
It also shows a graph for the loss which gives us more information about the network. Once the model has been retrain, accuracy is check. Generally, an accuracy of more than 95% is desire but it can be somewhat up to 90% as well depending on certain conditions. Accuracy can be bad if the hyperparameters are not set properly. A high learning rate can lead to a problem of infinite gradient descent. Having epochs less than the required can affect accuracy.
Both accuracy and loss become saturated after a definite number of epochs. Finding the sweet spot is completely trial and error. Fewer data brings the need for retraining after introducing more transformation parameters into the augmenter.
Image: Input and Inference on MATLAB
Also, the network architecture can be alter, reducing the number of neurons with the help of dropout regularization will help more. This process needs to repeat several times until the network is fit for deployment.
Conclusion
This is just one application of Deep Learning. You can make any network you want with MATLAB. The best thing about MATLAB is you can create networks without even knowing the code. Having a GUI lets you apply deep learning very easily.
Written By: Soumya Ranjan Acharya
Reviewed By: Rushikesh Lavate
If you are Interested In Machine Learning You Can Check Machine Learning Internship Program
Also Check Other Technical And Non Technical Internship Programs
