top of page

Top 4 Classification Metrics You Must Know

Updated: Oct 27, 2022

What are the business problems that we are trying to solve and how accurately can the model solve our problem?

The business problem that we are trying to solve is to predict the probability of a customer making a purchase on our website. We would like to be able to accurately predict whether a customer will make a purchase or not so that we can target them with ads accordingly. This will help us to make more money and to better target our ads. The model can accurately predict whether a customer will make a purchase or not with about 80% accuracy. So how do those 80% numbers come out? This is where evaluation metrics come into play.

What is a model without metric evaluation?

A model without metric evaluation is a model that has not been evaluated using any metrics. This means that the model's performance is unknown and it is not possible to say how well it will work on new data.

Why do we need this metric evaluation? Metric evaluation is important because it allows us to see how well a model is performing and to identify any areas where it may be lacking. Without metric evaluation, it would be difficult to improve the model or to know if it is working as well as it could be.

There are 4 types of evaluation metrics that can be used to see our model performance.

1. Accuracy

2. Precision

3. Recall

4. F1 Score

What is accuracy ?

Accuracy is the proportion of correct predictions out of all predictions.

When to use ?

Accuracy is important measures to consider when assessing the performance of a classifier.

Example :

You have a fraud detection system trained on imbalanced data. In normal transaction, the model is right 98% of the time. For the fraud class “Positive”, you are only right 50% of the time.

As the data is very sparse, do we still want accuracy as a metric of our model performance?

No. In this case, accuracy would not be a good metric of our model performance because the model could simply predict the majority class every time and still have a high accuracy.

What is precision in evaluation metrics?

Precision in evaluation metrics is a measure of how accurate a model is in predicting positive outcomes.

When to use?

Precision metrics are typically used to evaluate the performance of a classifier. The most common precision metric is the percentage of correctly classified instances, which is simply the accuracy of the classifier. Other precision metrics include the precision and recall of the classifier, which are typically used together to get a more complete picture of the classifier's performance.

Examples :

If most fraud detection is a positive result, but actually the detection is not a fraud. In this instance, precision helps when the costs of false positives are high.

What is Recall in evaluation metrics?

Recall is a metric that measures the ability of a model to correctly identify positive samples.

When to Use?

Recall in evaluation metrics typically used in classification tasks, precision and recall are used as a pair.

Examples :

If the model predicts the fraud a negative result, but the detection is actually a fraud. These events need recall metrics in assisting when the costs of false negatives are high.

What is F1 Score ?

F1 is the harmonic mean of precision and recall. A model with a high F1 score is said to be a balanced model.

When to use ?

As the F1 score is the overall measure of a model’s accuracy that combines precision and recall, let's take a look at the binary problem of 1 and 0. F1 scores are perfect when it’s 1 and a total failure when it is 0.