- \n
- A Quick Review of Classification<\/a><\/li>\n
- Defining Positive and Negative<\/a><\/li>\n
- Real World Examples of Positive and Negative Classes<\/a><\/li>\n
- How Positive and Negative Classes Relate to Classification Metrics<\/a><\/li>\n<\/ul>\n
That said, let’s dive in.<\/p>\n
We’ll start with a review of classification and build from there.<\/p>\n
<\/a><\/p>\n
A Quick Review of Classification<\/h2>\n
In machine learning, classification stands as one of the most fundamental and commonly performed tasks. <\/p>\n
At its core, classification involves categorizing data into predefined classes or categories, making it important in several applications, from email filtering to medical diagnostics. <\/p>\n
The essence of classification lies in its ability to evaluate incoming data examples and assign the correct label that defines what the example represents. <\/p>\n
Central to the concept of classification in machine learning is the understanding of positive and negative classes. This is especially true for binary classification problems (although we often need the concepts of “positive” and “negative” classes for multiclass classification as well, which I will briefly touch on later). <\/p>\n
Binary classification \u2013 the simplest form of classification \u2013 involves categorizing incoming data examples into two distinct groups, or classes. <\/p>\n
We commonly call these classes the “positive” class and the “negative” class.<\/p>\n
<\/a><\/p>\n
Defining ‘Positive’ and ‘Negative’<\/h2>\n
So, let’s quickly define what we mean by positive and negative.<\/p>\n
In the context of classification, “positive<\/strong>” refers to the class or outcome that the model is primarily interested in detecting or predicting. It represents the presence or occurrence of the specific characteristic or condition we’re trying to identify.<\/p>\n
In contrast, “negative<\/strong>” denotes the absence or non-occurrence of the characteristic that we’re tying to identify. Essentially, the “negative” class is largely complementary to the positive class. The negative class is important because it helps the model learn what features are unassociated with the condition of interest.<\/p>\n
![\"An](\"https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/12\/positive-and-negative_simple-example.png\")
<\/p>\n<\/a><\/p>\n
A couple of real-world examples of positive and negative classes<\/h3>\n
Let’s quickly run through a couple of real-world examples.<\/p>\n
Spam Detection<\/h4>\n
Let’s say that we’re performing a spam detection task, where we’re trying to distinguish between “spam” (i.e., junk email) and “non-spam” (AKA, “ham”, which is email you want to receive).<\/p>\n
In such a task, positive and negative would be:<\/p>\n
- \n
- positive<\/strong>: a spam email<\/li>\n
- negative<\/strong>: a non-spam email (AKA, ham)<\/li>\n<\/ul>\n
Cancer Detection<\/h4>\n
Let’s try another one.<\/p>\n
Let’s say that we’re trying to detect the presence of cancer in a diagnostic test.<\/p>\n
In such a task, the positive and negative classes would refer to:<\/p>\n
- \n
- positive<\/strong>: presence of cancer<\/li>\n
- negative<\/strong>: absence of cancer<\/li>\n<\/ul>\n
Fraud detection<\/h4>\n
Let’s do one more.<\/p>\n
Let’s assume that we’re working at a bank and we’re trying to detect fraudulent transactions.<\/p>\n
In such a task, positive and negative would typically be:<\/p>\n
- \n
- positive<\/strong>: presence of fraud<\/li>\n
- negative<\/strong>: absence of fraud<\/li>\n<\/ul>\n
The positive class typically signifies the presence of the quality under investigation<\/h4>\n
Notice that in all of the examples we just looked at, the positive class signifies the presence<\/em> of some condition or quality.<\/p>\n
To be clear: there may be tasks or situations where this is not the case, but most of the time<\/em> the positive class encodes the presence of something.<\/p>\n
More specifically, it signifies the presence of something that we’re trying to detect or identify.<\/p>\n
Alternative names for positive\/negative<\/h3>\n
We often use the terms positive and negative in the context of classification as terms that generalize across multiple tasks.<\/p>\n
So, as seen above, for spam classification, we can generally use the term “positive” to specify a spam email and “negative” to specify a non-spam email.<\/p>\n
Having said that, given a specific task, we may use the less general labels instead of the general positive\/negative labels.<\/p>\n
So in spam<\/a> classification, we might use “spam”\/”non-spam”.<\/p>\n
Or in fraud detection, we might use “fraud”\/”no-fraud”.<\/p>\n
In these cases, you need to realize that these task-specific class labels actually map to “positive” and “negative.”<\/p>\n
So whenever you’re doing binary classification, if you see task-specific class labels, you need to think about and be mindful of which class is actually “positive” and which is “negative” for the purposes of the task.<\/p>\n
Having said that, there are situations where it’s less clear which class should be considered positive and negative. There are circumstances where it’s somewhat arbitrary or much less clear which class to treat as positive and which to treat as negative.<\/p>\n
<\/a><\/p>\n
Positive and Negative are Important for Classification Metrics<\/h2>\n
Ok. <\/p>\n
Positive and negative. <\/p>\n
You might be thinking “it makes sense, so what?”<\/p>\n
As it turns out, almost all classification evaluation metrics depend on this framing of classes as positive and negative.<\/p>\n
How Positive\/Negative relate to Confusion Matrixes<\/h3>\n
For example, in a confusion matrix, we categorize all predictions according to the value of the actual class (positive or negative), and also the predicted class (positive or negative).<\/p>\n
And by categorizing examples this way, we get the 2×2 confusion matrix:<\/p>\n
![\"A](\"https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/09\/sklearn-confusion-matrix_visual-example.png\")
<\/p>\nMoreover, as seen above, by categorizing all examples according to the positive\/negative values of the actual class and predicted class, we get 4 different types of predictions:<\/p>\n
- \n
- True Positive<\/a><\/strong>: Predict
positive<\/code> when the actual input is positive<\/li>\n- True Negative<\/a><\/strong>: Predict
negative<\/code> when the actual input is negative<\/li>\n- False Positive<\/a><\/strong>: Predict
positive<\/code> when the actual input is negative<\/li>\n- False Negative<\/a><\/strong>: Predict
negative<\/code> when the actual input is positive<\/li>\n<\/ul>\nAnd these 4 different classification prediction types \u2013 True Positive, True Negative, False Positive, and False Negative \u2013 are themselves important for other metrics.<\/p>\n
Positive and Negative Classes in Precision, Recall and F1 Score<\/h3>\n
So the terms
positive<\/code> andnegative<\/code> are in turn important for classificaiton metrics, because almost all classification metrics are built on the quantities of True Positives, True Negatives, False Positives, and False Negatives.<\/p>\nPrecision<\/h4>\n
For example, precision is the proportion of positive predictions that were correct<\/a>.<\/p>\n
(1) <\/span> <\/span>
![\"\begin{equation*}](\"https:\/\/www.sharpsightlabs.com\/wp-content\/ql-cache\/quicklatex.com-044caa3289c7a9b705bb7a0698ce0340_l3.png\" "\\"Rendered")
<\/p>\nAs seen above, we calculate it with True Positives, and False Negatives (which obviously require us to understand the distinction between positive and negative classes).<\/p>\n
Recall<\/h4>\n
Recall is the proportion of actual positive examples that we correctly classify as positive<\/a>.<\/p>\n
(2) <\/span> <\/span>
![\"\begin{equation*}](\"https:\/\/www.sharpsightlabs.com\/wp-content\/ql-cache\/quicklatex.com-e69223257e8b997e82533f3237335bbb_l3.png\" "\\"Rendered")
<\/p>\nAgain: this requires us to understand positives and negatives.<\/p>\n
F1 Score<\/h4>\n
Finally, we have F1 score, which balances between precision and recall<\/a>.<\/p>\n
(3) <\/span> <\/span>
![\"\begin{equation*}](\"https:\/\/www.sharpsightlabs.com\/wp-content\/ql-cache\/quicklatex.com-f365623515304b8829652957cc60ed59_l3.png\" "\\"Rendered")
<\/p>\nAnd we see that to calculate F1, we need True Positives, False Positives, and False Negatives.<\/p>\n
There are other classification metrics as well, but these are three of the most important.<\/p>\n
And as you’ve seen, to calculate them, you need to have the number of True Positives, False Positives, and False Negatives. In turn, these require the positive\/negative class distinction.<\/p>\n
How this Fits with Multi-Class Classification<\/h2>\n
Finally, let’s briefly link this to multi class classification.<\/p>\n
How do positives and negatives work for classification tasks with more than 2 classes?<\/p>\n
Well, it’s complicated.<\/p>\n
There are actually a few ways to do multi-class classification.<\/p>\n
But, as it turns out, they still use these “positive” and “negative” classes.<\/p>\n
One vs the rest<\/h3>\n
In one vs the rest, we treat one of the classes as “positive” and we group together all of the other classes as “negative”.<\/p>\n
We literally analyze one class as “positive” vs the rest.<\/p>\n
This allows us to use the positive\/negative classes, which in turn allows us to use the classification metrics that I described above.<\/p>\n
One vs One<\/h3>\n
Additionally, we can perform one vs one.<\/p>\n
In one vs one, we analyze one class vs another class, one pair at a time.<\/p>\n
So if we have 3 classes \u2013 A, B and C \u2013 we can analyze A vs B. A vs C. B vs C.<\/p>\n
This turns a multi class situation into a set of binary classification tasks. In turn, that enables us to characterize one of each pair as positive and the other as negative, which then allows us to use classification metrics like precision, recall, etc.<\/p>\n
Wrapping Up: Why Positive and Negative Classes are Important for Machine Learning<\/h2>\n
The concepts of positive and negative classes are critical for understanding classification. Positive and negative are needed directly binary classification, but as noted above, the concepts can be extended as well to multiclass situations.<\/p>\n
I’m a firm believer in the principle of “master the fundamentals” and then build your knowledge up from there.<\/p>\n
These concepts of positive and negative are like machine learning bedrock. You need to know what they mean, and in turn, how they relate to classification metrics, etc.<\/p>\n
Having said that, there’s probably some detail and nuance that I’ve left out, so you should sign up for our email newsletter to get all of our future ML articles.<\/p>\n
Leave Your Questions and Comments Below<\/h3>\n
Having said all of that, do you have other questions about positive and negative in machine learning?<\/p>\n
Are you still confused about something?<\/p>\n
I want to hear from you.<\/p>\n
Leave your questions and comments in the comments section at the bottom of the page.<\/p>\n
Sign up for our email list<\/h3>\n
If you want to learn more about machine learning and AI, then sign up for our email list.<\/p>\n
Every week, we publish free tutorials about a variety of topics in machine learning, data science, and AI, including:<\/p>\n
- \n
- Scikit Learn<\/li>\n
- Numpy<\/li>\n
- Pandas<\/li>\n
- Machine Learning<\/li>\n
- Deep Learning<\/li>\n
- … and more<\/li>\n<\/ul>\n
If you sign up for our email list, then we’ll deliver those free tutorials to you, direct to your inbox.<\/p>\n","protected":false},"excerpt":{"rendered":"
In this blog post, I’m going to quickly explain positive and negative classes in machine learning classification. I’ll explain what the positive and negative classes are, how they relate to classification metric, some examples of positive and negative in real-world machine learning, and more. If you need something specific, just click on one of these … Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":7826,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","om_disable_all_campaigns":false,"footnotes":""},"categories":[86,19],"tags":[],"publishpress_future_action":{"enabled":false,"date":"2024-05-15 04:17:38","action":"change-status","newStatus":"draft","terms":[],"taxonomy":"category"},"_links":{"self":[{"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/posts\/7395"}],"collection":[{"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/comments?post=7395"}],"version-history":[{"count":1,"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/posts\/7395\/revisions"}],"predecessor-version":[{"id":7821,"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/posts\/7395\/revisions\/7821"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/media\/7826"}],"wp:attachment":[{"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/media?parent=7395"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/categories?post=7395"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/tags?post=7395"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
- True Negative<\/a><\/strong>: Predict
- negative<\/strong>: absence of fraud<\/li>\n<\/ul>\n
- negative<\/strong>: absence of cancer<\/li>\n<\/ul>\n
- negative<\/strong>: a non-spam email (AKA, ham)<\/li>\n<\/ul>\n
- Defining Positive and Negative<\/a><\/li>\n