Confusion Matrix is simple method that is used to calculate the accuracy of an AI model, especially when dealing with an image based AI model.
The Confusion Matrix table is as follows,
The positive/Negative labels refer to the Predicted outcome of an experiment, while the True/False refers to the Actual outcome (if the Actual outcome is True then its positive, if it is not the Actual outcome is Negative). As an example, if the AI predicts that an Insect as FAW, but if the insect is actually from an other category, then it would fall into False Positive because the actual outcome was false (Negative) but the prediction was positive. Similary,
True Positive (TP) – If the Prediction is Positive and the Actual outcome is also True/Positive ( Predicted as FAW and it is a FAW in Actuality).
False Positive (FP) – If the Prediction is Positive but if the Actual outcome is False/Negative ( Predicted as FAW but in Actuality the insect is of other category).
False Negative (FN) – If the Prediction is Negative but if the Actual outcome is True/Positive (Predicted as other category but in Actuality the insect is FAW).
True Negative (TN) – If the Prediction is Negative and the Acutal outcome is also False/Negative (Predicted as other catergory and it is an insect belonging to the other category in Actuality as well).
This can be confusing at first ( they are called confusion matrix, after all) but will be clear to you with the example.
Calculating the Accuracy
The Accuracy of the of the model is calculated by using the below formula,
Accuracy = (TP+TN) / (TP+TN+FP+FN)
Let’s assume that we used an AI model to predict if insects in a sample population of 200 belongs to FAW or other Category and that there are 100 FAW and 100 insects belonging to the other category in the sample. The first step would be to create the Confusion Matrix table and it would be as below,
As shown above the Positive label would be FAW and the Negative would be the Other category.
Lets say after testing the model it predicted 75 FAW correctly as FAW, predicted 25 FAW incorrectly as other, predicted 90 other category insects correctly as other and predicted 10 other category insects incorrectly as FAW. Now lets sort out which of these values are TP, TN, FP and FN.
TP (predicted as FAW and is FAW in actuality) = 75
FN (predicted as other but is FAW in actuality) = 25
FP (predicted as FAW but is other in actuality) = 10
TN (predicted as other and is other in acutality) = 90
Now lets add these values to our Confusion Matrix and the tabel would look as below,
With the confusion matrix filled, now we can calculate the accuracy of the model by using the formula that we mentioned ealier in the article,
Accuracy = (TP+TN) / (TP+TN+FP+FN) = ( 75+90) / ( 75+90+10+25) = 165/ 200 = 0.825 or 82.5%
Hence, the accuracy of the AI model that predicts FAW vs Other is 82.5%.
We hope that this article gave you an undestanding as well as the confidence to calcuale an AI models accuracy. If you have any questions with regards to this, please contact kasun via email on firstname.lastname@example.org.