{"id":7398,"date":"2023-12-04T20:02:26","date_gmt":"2023-12-05T02:02:26","guid":{"rendered":"https:\/\/www.sharpsightlabs.com\/?p=7398"},"modified":"2023-12-07T17:46:57","modified_gmt":"2023-12-07T23:46:57","slug":"f1-score-explained","status":"publish","type":"post","link":"https:\/\/www.sharpsightlabs.com\/blog\/f1-score-explained\/","title":{"rendered":"F1 Score, Explained"},"content":{"rendered":"<p>If you want to master modern machine learning and AI, one of the major sub-areas that you need to master is <em>classification<\/em>.  <\/p>\n<p>Classification is one of the most important types of task in machine learning and AI.<\/p>\n<p>But mastering classification, in part, means mastering how to <em>evaluate<\/em> classification systems.<\/p>\n<p>Which in turn, means understanding the wide range of classification evaluation metrics.<\/p>\n<p>That brings us to F1 score.<\/p>\n<p>F1 score is one of the most important classification evaluation metrics and you need to know it well.<\/p>\n<p>So in this post, I&#8217;m going to tell you all of the essentials that you need to know about F1 score.<\/p>\n<p>I&#8217;ll explain what F1 is, the pros and cons of F1, how to improve it, and more.<\/p>\n<p>If you need something specific, just click on any of these links and the link will take you to the appropriate point in the post.<\/p>\n<p><strong>Table of Contents:<\/strong><\/p>\n<ul>\n<li><a href=\"#classification-review\">A Quick Review of Classification<\/a><\/li>\n<li><a href=\"#f1-basics\">F1 Score Basics<\/a><\/li>\n<li><a href=\"#pros-and-cons\">Pros and Cons of F1 Score<\/a><\/li>\n<li><a href=\"#how-to-improve-f1\">How to Improve F1 Score<\/a><\/li>\n<li><a href=\"#alternatives\">Alternatives to F1 Score<\/a><\/li>\n<\/ul>\n<p>Having said that, F1 score is sometimes a little challenging to understand, so it would probably be helpful if you read the whole post.<\/p>\n<p>Ok.  Let&#8217;s get to it.<\/p>\n<p><a id = \"classification-review\"><\/a><\/p>\n<h2>A Quick Review of Classification<\/h2>\n<p>First, before we directly discuss F1 score, we should really review what classification is and how it works.  <\/p>\n<p>Everything you need to know about F1 hinges on a solid understanding of classification and different types of classification predictions (both correct and incorrect predictions).<\/p>\n<p>So here, we&#8217;ll quickly review classification and then move on to F1 score (and obviously, if you&#8217;re 100% positive that you know all of this cold, then you can just skip ahead).<\/p>\n<h3>Machine Learning Overview<\/h3>\n<p>Quickly, let&#8217;s discuss what machine learning is, since that frames our discussion of classification.<\/p>\n<p>Machine learning is the discipline of building computer systems that improve their performance when we expose them to data.<\/p>\n<p>So <a href=\"https:\/\/www.sharpsightlabs.com\/blog\/introduction-to-machine-learning\/\" target=\"_blank\" rel=\"noopener\">machine learning systems<\/a> are different from traditional computer programs.<\/p>\n<p>Whereas in a traditional computer system, a human programmer explicitly programs all of the computer&#8217;s steps, in a machine learning system, we set up an algorithm that&#8217;s capable of &#8220;learning&#8221; as we expose it to data examples.<\/p>\n<p>ML systems can learn to do a variety of tasks, but one of the most common tasks is <em>classification<\/em>.<\/p>\n<h4>What is Classification<\/h4>\n<p>Classification systems (also known as &#8220;classifiers&#8221;) learn to categorize (AKA, classify) input examples by predicting a categorical label.<\/p>\n<p>In classification, the potential labels come from a pre-defined list.  For example:<\/p>\n<ul>\n<li><code>spam<\/code> or <code>not spam<\/code><\/li>\n<li><code>cat<\/code> or <code>not cat<\/code><\/li>\n<li><code>fraud<\/code> or <code>not fraud<\/code><\/li>\n<li><code>positive<\/code> or <code>negative<\/code><\/li>\n<\/ul>\n<p>In classification, we already know the possible labels, and we just want the system to predict the correct label every time we give it an example.<\/p>\n<p>To better understand this, let&#8217;s look at an example.<\/p>\n<h5>A quick example of classification<\/h5>\n<p>The example that I like to use to explain how classification works is something that I like to call &#8220;The Cat Detector.&#8221;<\/p>\n<p>The Cat Detector is simple.<\/p>\n<p>It detects cats.<\/p>\n<p>More specifically, you feed the Cat Detector images and it outputs only one of two possible outputs: <\/p>\n<ul>\n<li><code>positive<\/code> (which indicates that it thinks the image is a cat), or\n<li><code>negative<\/code> (which indicates that it things the image is not a cat).\n<\/ul>\n<p>It&#8217;s a very simple binary classifier.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/cat-detector_positive-negative-outputs.png\" alt=\"An image that shows how our hypothetical cat detector system inputs images and outputs labels of &quot;positive&quot; or &quot;negative&quot; for cat or not cat.\" width=\"732\" class=\"aligncenter size-full wp-image-7749\" srcset=\"https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/cat-detector_positive-negative-outputs.png 1464w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/cat-detector_positive-negative-outputs-600x320.png 600w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/cat-detector_positive-negative-outputs-1024x546.png 1024w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/cat-detector_positive-negative-outputs-768x409.png 768w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/cat-detector_positive-negative-outputs-1080x575.png 1080w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/cat-detector_positive-negative-outputs-610x325.png 610w\" sizes=\"(max-width: 1464px) 100vw, 1464px\" \/><\/p>\n<p>Input an image, output a prediction of cat or not a cat (i.e., <a href=\"https:\/\/www.sharpsightlabs.com\/blog\/positive-and-negative-classes\/\" target=\"_blank\" rel=\"noopener\"><code>positive<\/code> or <code>negative<\/code><\/a>)<\/p>\n<p>Having said that, in spite of the seeming simplicity, it is a little more complicated than it appears.<\/p>\n<h5>Classification Mistakes<\/h5>\n<p>The main thing that makes classification systems more complicated is that essentially <em>all classifiers make mistakes<\/em> (unless the task is trivially easy).<\/p>\n<p>Said another way, classification predictions are sometimes incorrect.<\/p>\n<p>To show this, let&#8217;s go back to our Cat Detector system.<\/p>\n<p>In a perfect world, you&#8217;d feed the Cat Detector a picture of a cat, and it would always output <code>positive<\/code>.<\/p>\n<p>And you&#8217;d feed the Cat Detector a picture of a non-cat (like a dog), and it would always output <code>negative<\/code>.<\/p>\n<p>But that&#8217;s the idealized, perfect behavior.<\/p>\n<p>And that perfect behavior never happens when we&#8217;re working on difficult, real world problems.<\/p>\n<p>In the real world, with hard problems, our classifier will make mistakes.<\/p>\n<p>For example, you might show the Cat Classifier an image of a cat, but it outputs <code>negative<\/code>, indicating that it predicts not-a-cat.<\/p>\n<p>Or, you might show the Cat Classifier an image of a non-cat (such as a dog), and it outputs <code>positive<\/code>, indicating that it (incorrectly) predicts that the image is a cat.<\/p>\n<p>So in the real world, our binary classifier actually has 4 different prediction types &#8230; 2 &#8220;correct&#8221; prediction types, and 2 &#8220;incorrect&#8221; prediction types, as shown here:<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/cat-detector_potivive-negative_correct-and-incorrect-predictions.png\" alt=\"An image of different types of correct and incorrect predictions for our Cat Detector system.\" width=\"730\" class=\"aligncenter size-full wp-image-7750\" srcset=\"https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/cat-detector_potivive-negative_correct-and-incorrect-predictions.png 1460w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/cat-detector_potivive-negative_correct-and-incorrect-predictions-600x455.png 600w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/cat-detector_potivive-negative_correct-and-incorrect-predictions-1024x777.png 1024w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/cat-detector_potivive-negative_correct-and-incorrect-predictions-768x583.png 768w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/cat-detector_potivive-negative_correct-and-incorrect-predictions-1080x820.png 1080w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/cat-detector_potivive-negative_correct-and-incorrect-predictions-610x463.png 610w\" sizes=\"(max-width: 1460px) 100vw, 1460px\" \/><\/p>\n<p>At this point, you might be asking yourself &#8220;So what? Classifiers make mistakes.  How does this relate to F1 Score?&#8221;<\/p>\n<p>Good question.<\/p>\n<p>We&#8217;re getting there &#8230;<\/p>\n<p>We just need to talk a little more specifically about these correct and incorrect prediction types.<\/p>\n<p>Because they&#8217;re at the core of classifier evaluation, and therefore, at the core of F1.<\/p>\n<p>Just bear with me a little bit more, and we&#8217;ll get to F1.<\/p>\n<p><a id = \"types-of-predictions\"><\/a><\/p>\n<h5>Correct and Incorrect Prediction Types<\/h5>\n<p>The correct and incorrect prediction types that we just saw with the Cat Detector can be generalized for all binary classifiers.<\/p>\n<p>Remember that for the Cat Detector, there are 4 types of predictions:<\/p>\n<ul>\n<li>Correctly predict <code>positive<\/code> when the input is actually a cat<\/li>\n<li>Correctly predict <code>negative<\/code> when the input is not a cat<\/li>\n<li>Incorrectly predict <code>positive<\/code> when the input is <em>not<\/em> a cat<\/li>\n<li>Incorrectly predict <code>negative<\/code> when the input is actually a cat<\/li>\n<\/ul>\n<p>But if we generalize the possible input classes for any binary classifier as <code>positive<\/code> and <code>negative<\/code>, and also generalize the possible output predictions for any binary classifier as <code>positive<\/code> and <code>negative<\/code>, then predictions of <em>any<\/em> binary classifier will fall into one of these 4 groups, according to the actual class of the input and the predicted class of the output:<\/p>\n<ul>\n<li><strong><a href=\"https:\/\/www.sharpsightlabs.com\/blog\/true-positive-explained\/\" target=\"_blank\" rel=\"noopener\">True Positive<\/a><\/strong>: Predict <code>positive<\/code> when the actual input is positive<\/li>\n<li><strong><a href=\"https:\/\/www.sharpsightlabs.com\/blog\/true-negative-explained\/\" target=\"_blank\" rel=\"noopener\">True Negative<\/a><\/strong>: Predict <code>negative<\/code> when the actual input is negative<\/li>\n<li><strong><a href=\"https:\/\/www.sharpsightlabs.com\/blog\/false-positive-explained\/\" target=\"_blank\" rel=\"noopener\">False Positive<\/a><\/strong>: Predict <code>positive<\/code> when the actual input is negative<\/li>\n<li><strong><a href=\"https:\/\/www.sharpsightlabs.com\/blog\/false-negative-explained\/\" target=\"_blank\" rel=\"noopener\">False Negative<\/a><\/strong>: Predict <code>negative<\/code> when the actual input is positive<\/li>\n<\/ul>\n<p>And note that each of these four prediction types has a name, True Positive, True Negative, False Positive, and False Negative.<\/p>\n<p>Why does this matter?<\/p>\n<p>Because almost all classification metrics and classification evaluation tools are built on True Positives, True Negatives, False Positives, and False Negatives.<\/p>\n<p>Accuracy, precision, recall, confusion matrices &#8230;<\/p>\n<p>They all depend on the numbers of these prediction types.<\/p>\n<p>And for the purposes of this blog post, so does F1 score.<\/p>\n<p><a id = \"f1-basics\"><\/a><\/p>\n<h2>F1 Score Basics<\/h2>\n<p>Ok.<\/p>\n<p>We&#8217;re finally ready to talk about F1 score.<\/p>\n<p>F1 score is a classification metric that enables us to evaluate the performance of a classifier.<\/p>\n<p>This measure, which is widely used to evaluate of classification models, balances between precision and recall, two other classification metrics.<\/p>\n<h3>Quick Review of Precision and Recall<\/h3>\n<p>To understand F1, you need to understand precision and recall.<\/p>\n<p><a href=\"https:\/\/www.sharpsightlabs.com\/blog\/classifier-precision-explained\/\" target=\"_blank\" rel=\"noopener\">Precision<\/a> measures the proportion of positive predictions (i.e., model output predictions) that were actually positive.<\/p>\n<p class=\"ql-center-displayed-equation\" style=\"line-height: 46px;\"><span class=\"ql-right-eqno\"> (1) <\/span><span class=\"ql-left-eqno\"> &nbsp; <\/span><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.sharpsightlabs.com\/wp-content\/ql-cache\/quicklatex.com-044caa3289c7a9b705bb7a0698ce0340_l3.png\" height=\"46\" width=\"89\" class=\"ql-img-displayed-equation quicklatex-auto-format\" alt=\"&#92;&#98;&#101;&#103;&#105;&#110;&#123;&#101;&#113;&#117;&#97;&#116;&#105;&#111;&#110;&#42;&#125; &#32;&#32;&#32;&#92;&#102;&#114;&#97;&#99;&#123;&#92;&#116;&#101;&#120;&#116;&#123;&#84;&#80;&#125;&#125;&#123;&#92;&#116;&#101;&#120;&#116;&#123;&#84;&#80;&#32;&#43;&#32;&#70;&#80;&#125;&#125; &#92;&#101;&#110;&#100;&#123;&#101;&#113;&#117;&#97;&#116;&#105;&#111;&#110;&#42;&#125;\" title=\"Rendered by QuickLaTeX.com\"\/><\/p>\n<p>You can think of precision as the accuracy of the positive predictions.<\/p>\n<p>On the other hand, <a href=\"https:\/\/www.sharpsightlabs.com\/blog\/classifier-recall-explained\/\" target=\"_blank\" rel=\"noopener\">recall<\/a> (AKA, sensitivity) is the proportion of positive examples that were correctly classified as positive.<\/p>\n<p class=\"ql-center-displayed-equation\" style=\"line-height: 46px;\"><span class=\"ql-right-eqno\"> (2) <\/span><span class=\"ql-left-eqno\"> &nbsp; <\/span><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.sharpsightlabs.com\/wp-content\/ql-cache\/quicklatex.com-e69223257e8b997e82533f3237335bbb_l3.png\" height=\"46\" width=\"90\" class=\"ql-img-displayed-equation quicklatex-auto-format\" alt=\"&#92;&#98;&#101;&#103;&#105;&#110;&#123;&#101;&#113;&#117;&#97;&#116;&#105;&#111;&#110;&#42;&#125; &#32;&#32;&#32;&#92;&#102;&#114;&#97;&#99;&#123;&#92;&#116;&#101;&#120;&#116;&#123;&#84;&#80;&#125;&#125;&#123;&#92;&#116;&#101;&#120;&#116;&#123;&#84;&#80;&#32;&#43;&#32;&#70;&#78;&#125;&#125; &#92;&#101;&#110;&#100;&#123;&#101;&#113;&#117;&#97;&#116;&#105;&#111;&#110;&#42;&#125;\" title=\"Rendered by QuickLaTeX.com\"\/><\/p>\n<p>(Do you see why we needed to understand True Positives, False Positives, and False Negatives?)<\/p>\n<h4>The Problems of Precision or Recall<\/h4>\n<p>Although precision and recall can both be informative for evaluating classifiers, they both offer a one-sided view of a model&#8217;s performance.  This frequently leads to misleading conclusions if considered in isolation.<\/p>\n<p>Precision focuses on the correctness of the positive predictions made by the classifier. High precision indicates that when the model predicts a positive value, it&#8217;s likely correct. However, a model can achieve high precision simply by being overly conservative in making positive predictions.  Said differently, if a classifier makes fewer positive predictions, only when it&#8217;s extremely confident that the positive prediction is correct, it can achieve high precision.  But doing so might miss many actual positive cases. This can be particularly problematic in situations where failing to detect positive examples has very high cost, like in diagnosing a deadly disease.<\/p>\n<p>Recall, on the other hand, assesses the model&#8217;s ability to properly detect all of the actual positive examples. A high recall means that the model is good at capturing positive cases.  However, high recall <em>fails<\/em> to consider the <em>cost of false positives<\/em>. In some situations, like spam detection, excessive false positives (i.e., labeling a normal email that you want to get as &#8220;spam&#8221;) can be a big inconvenience.  <\/p>\n<p>Again: optimizing a classifier strictly for high precision or high recall can cause problems.<\/p>\n<h3>F1 Score: A Balance Between Precision and Recall<\/h3>\n<p>Enter F1 Score.<\/p>\n<p>F1 score <em>balances<\/em> between precision and recall.<\/p>\n<p>It does this because it&#8217;s computed as the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Harmonic_mean\" target=\"_blank\" rel=\"noopener\">harmonic mean<\/a> of precision and recall, where the harmonic mean is:<\/p>\n<p class=\"ql-center-displayed-equation\" style=\"line-height: 81px;\"><span class=\"ql-right-eqno\"> (3) <\/span><span class=\"ql-left-eqno\"> &nbsp; <\/span><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.sharpsightlabs.com\/wp-content\/ql-cache\/quicklatex.com-397ed8c97837ac02869ebcd643de9d68_l3.png\" height=\"81\" width=\"245\" class=\"ql-img-displayed-equation quicklatex-auto-format\" alt=\"&#92;&#98;&#101;&#103;&#105;&#110;&#123;&#101;&#113;&#117;&#97;&#116;&#105;&#111;&#110;&#42;&#125; &#32;&#32;&#32;&#32;&#72;&#40;&#120;&#95;&#123;&#49;&#125;&#44;&#120;&#95;&#123;&#50;&#125;&#44;&#92;&#108;&#100;&#111;&#116;&#115;&#32;&#44;&#120;&#95;&#123;&#110;&#125;&#41;&#32;&#61;&#32;&#92;&#102;&#114;&#97;&#99;&#32;&#123;&#110;&#125;&#123;&#92;&#100;&#105;&#115;&#112;&#108;&#97;&#121;&#115;&#116;&#121;&#108;&#101;&#32;&#92;&#115;&#117;&#109;&#32;&#95;&#123;&#105;&#61;&#49;&#125;&#94;&#123;&#110;&#125;&#123;&#92;&#102;&#114;&#97;&#99;&#32;&#123;&#49;&#125;&#123;&#120;&#95;&#123;&#105;&#125;&#125;&#125;&#125; &#92;&#101;&#110;&#100;&#123;&#101;&#113;&#117;&#97;&#116;&#105;&#111;&#110;&#42;&#125;\" title=\"Rendered by QuickLaTeX.com\"\/><\/p>\n<p>Therefore, F1 score can be computed as:<\/p>\n<p class=\"ql-center-displayed-equation\" style=\"line-height: 55px;\"><span class=\"ql-right-eqno\"> (4) <\/span><span class=\"ql-left-eqno\"> &nbsp; <\/span><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.sharpsightlabs.com\/wp-content\/ql-cache\/quicklatex.com-4e41c9be724efdd4f4be76139da65c37_l3.png\" height=\"55\" width=\"468\" class=\"ql-img-displayed-equation quicklatex-auto-format\" alt=\"&#92;&#98;&#101;&#103;&#105;&#110;&#123;&#101;&#113;&#117;&#97;&#116;&#105;&#111;&#110;&#42;&#125; &#32;&#32;&#32;&#92;&#116;&#101;&#120;&#116;&#123;&#70;&#49;&#32;&#83;&#99;&#111;&#114;&#101;&#125;&#32;&#61;&#32;&#72;&#40;&#112;&#114;&#101;&#99;&#105;&#115;&#105;&#111;&#110;&#44;&#32;&#114;&#101;&#99;&#97;&#108;&#108;&#41;&#32;&#61;&#32;&#92;&#102;&#114;&#97;&#99;&#123;&#50;&#125;&#123;&#92;&#102;&#114;&#97;&#99;&#123;&#49;&#125;&#123;&#112;&#114;&#101;&#99;&#105;&#115;&#105;&#111;&#110;&#125;&#32;&#43;&#32;&#92;&#102;&#114;&#97;&#99;&#123;&#49;&#125;&#123;&#114;&#101;&#99;&#97;&#108;&#108;&#125;&#125; &#92;&#101;&#110;&#100;&#123;&#101;&#113;&#117;&#97;&#116;&#105;&#111;&#110;&#42;&#125;\" title=\"Rendered by QuickLaTeX.com\"\/><\/p>\n<p>Which simplifies to:<\/p>\n<p class=\"ql-center-displayed-equation\" style=\"line-height: 49px;\"><span class=\"ql-right-eqno\"> (5) <\/span><span class=\"ql-left-eqno\"> &nbsp; <\/span><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.sharpsightlabs.com\/wp-content\/ql-cache\/quicklatex.com-5d7af204b3b169eb9dbdcfa6786d2204_l3.png\" height=\"49\" width=\"320\" class=\"ql-img-displayed-equation quicklatex-auto-format\" alt=\"&#92;&#98;&#101;&#103;&#105;&#110;&#123;&#101;&#113;&#117;&#97;&#116;&#105;&#111;&#110;&#42;&#125; &#32;&#32;&#32;&#92;&#116;&#101;&#120;&#116;&#123;&#70;&#49;&#32;&#83;&#99;&#111;&#114;&#101;&#125;&#32;&#61;&#32;&#92;&#102;&#114;&#97;&#99;&#123;&#50;&#32;&#92;&#99;&#100;&#111;&#116;&#32;&#40;&#112;&#114;&#101;&#99;&#105;&#115;&#105;&#111;&#110;&#32;&#92;&#116;&#105;&#109;&#101;&#115;&#32;&#114;&#101;&#99;&#97;&#108;&#108;&#41;&#125;&#123;&#112;&#114;&#101;&#99;&#105;&#115;&#105;&#111;&#110;&#32;&#43;&#32;&#114;&#101;&#99;&#97;&#108;&#108;&#125; &#92;&#101;&#110;&#100;&#123;&#101;&#113;&#117;&#97;&#116;&#105;&#111;&#110;&#42;&#125;\" title=\"Rendered by QuickLaTeX.com\"\/><\/p>\n<p>And by plugging in the quantities for True Positive, False Positive, and False Negative (and with a bit of math), we can compute F1 score as follows:<\/p>\n<p class=\"ql-center-displayed-equation\" style=\"line-height: 46px;\"><span class=\"ql-right-eqno\"> (6) <\/span><span class=\"ql-left-eqno\"> &nbsp; <\/span><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.sharpsightlabs.com\/wp-content\/ql-cache\/quicklatex.com-1a223e90a2498f146cc1576e86eebca2_l3.png\" height=\"46\" width=\"274\" class=\"ql-img-displayed-equation quicklatex-auto-format\" alt=\"&#92;&#98;&#101;&#103;&#105;&#110;&#123;&#101;&#113;&#117;&#97;&#116;&#105;&#111;&#110;&#42;&#125; &#32;&#32;&#32;&#92;&#116;&#101;&#120;&#116;&#123;&#70;&#49;&#32;&#83;&#99;&#111;&#114;&#101;&#125;&#32;&#61;&#32;&#92;&#102;&#114;&#97;&#99;&#123;&#50;&#84;&#80;&#125;&#123;&#50;&#84;&#80;&#32;&#43;&#32;&#70;&#80;&#32;&#43;&#32;&#70;&#78;&#125; &#92;&#101;&#110;&#100;&#123;&#101;&#113;&#117;&#97;&#116;&#105;&#111;&#110;&#42;&#125;\" title=\"Rendered by QuickLaTeX.com\"\/><\/p>\n<p>(Again: do you see why it&#8217;s important to understand True Positives, False Positives, and False Negatives?)<\/p>\n<h4>F1 provides a metric that balances between minimizing FN and FP<\/h4>\n<p>Ultimately, F1 score harmonizes between both precision and recall.  It provides a single metric that measures the model&#8217;s overall efficiency in correctly identifying positives, while also minimizing both false positives <code>and<\/code> negatives. <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/f1-score_simple-explanation.png\" alt=\"An image that shows F1 score as the harmonic mean of precision and recall.\" width=\"785\" class=\"aligncenter size-full wp-image-7820\" srcset=\"https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/f1-score_simple-explanation.png 1570w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/f1-score_simple-explanation-600x465.png 600w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/f1-score_simple-explanation-1024x794.png 1024w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/f1-score_simple-explanation-768x596.png 768w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/f1-score_simple-explanation-1536x1192.png 1536w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/f1-score_simple-explanation-1080x838.png 1080w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/f1-score_simple-explanation-610x473.png 610w\" sizes=\"(max-width: 1570px) 100vw, 1570px\" \/><\/p>\n<p>By using the harmonic mean, F1 score ensures that a model performs well in both precision and recall to perform well on this metric. <\/p>\n<p>This balance is critical in many practical applications where both False Positives <em>and<\/em> False Negatives have substantial costs. <\/p>\n<p>Therefore, F1 score offers a more comprehensive metric to evaluate a classifier&#8217;s performance, which makes it a preferred metric for many machine learning and AI applications.<\/p>\n<h3>An Example of F1 Score<\/h3>\n<p>To illustrate how we F1 is calculated, let&#8217;s look at a simple example.<\/p>\n<p>Let&#8217;s assume that we&#8217;re using the Cat Detector model that I described above.<\/p>\n<p>We&#8217;ll assume that the model has already been trained, and we&#8217;re evaluating how the model works on a new dataset.<\/p>\n<p>To do this, we&#8217;ll use 100 photos as input examples.  Among these 100 examples are 50 pictures of dogs (non-cats) and 50 pictures of cats.<\/p>\n<p>We&#8217;ll feed the pictures into the Cat Detector and let the system predict whether or not the image is a cat or non-cat.<\/p>\n<p>After doing this, the Cat Detector produces these output predictions (shown here as <a href=\"https:\/\/www.sharpsightlabs.com\/blog\/confusion-matrix-explained\/\" target=\"_blank\" rel=\"noopener\">a confusion matrix<\/a>):<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/classification-accuracy_confusion-matrix-example.png\" alt=\"An image of a confusion matrix with TP, TN, FP and FN, which we can use to calculate classifier recall.\" width=\"693\"  class=\"aligncenter size-full wp-image-7751\" srcset=\"https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/classification-accuracy_confusion-matrix-example.png 1386w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/classification-accuracy_confusion-matrix-example-600x366.png 600w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/classification-accuracy_confusion-matrix-example-1024x625.png 1024w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/classification-accuracy_confusion-matrix-example-768x469.png 768w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/classification-accuracy_confusion-matrix-example-1080x659.png 1080w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/11\/classification-accuracy_confusion-matrix-example-610x372.png 610w\" sizes=\"(max-width: 1386px) 100vw, 1386px\" \/><\/p>\n<p>So, we have the following predictions:<\/p>\n<ul>\n<li>40 True Positives (pictures of cats predicted as cat)<\/li>\n<li>15 False Positives (pictures of non-cats predicted as cats)<\/li>\n<li>35 True Negatives (pictures of non-cats predicted as not cat)<\/li>\n<li>10 False Negatives (pictures of cats predicted as not cat)<\/li>\n<\/ul>\n<p>Using these numbers, and in  particular, using the number of True Positives, True Negatives, and False Negatives, we can calculate F1.<\/p>\n<h4>Calculating F1 Score with TP, TN, FP, and FN<\/h4>\n<p>So, let&#8217;s calculate the F1 Score of our classifier using the quantities shown above, and using equation 6 seen earlier.<\/p>\n<p class=\"ql-center-displayed-equation\" style=\"line-height: 46px;\"><span class=\"ql-right-eqno\"> (7) <\/span><span class=\"ql-left-eqno\"> &nbsp; <\/span><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.sharpsightlabs.com\/wp-content\/ql-cache\/quicklatex.com-1a223e90a2498f146cc1576e86eebca2_l3.png\" height=\"46\" width=\"274\" class=\"ql-img-displayed-equation quicklatex-auto-format\" alt=\"&#92;&#98;&#101;&#103;&#105;&#110;&#123;&#101;&#113;&#117;&#97;&#116;&#105;&#111;&#110;&#42;&#125; &#32;&#32;&#32;&#92;&#116;&#101;&#120;&#116;&#123;&#70;&#49;&#32;&#83;&#99;&#111;&#114;&#101;&#125;&#32;&#61;&#32;&#92;&#102;&#114;&#97;&#99;&#123;&#50;&#84;&#80;&#125;&#123;&#50;&#84;&#80;&#32;&#43;&#32;&#70;&#80;&#32;&#43;&#32;&#70;&#78;&#125; &#92;&#101;&#110;&#100;&#123;&#101;&#113;&#117;&#97;&#116;&#105;&#111;&#110;&#42;&#125;\" title=\"Rendered by QuickLaTeX.com\"\/><\/p>\n<p>Plugging in the numbers, we get:<\/p>\n<p class=\"ql-center-displayed-equation\" style=\"line-height: 45px;\"><span class=\"ql-right-eqno\"> (8) <\/span><span class=\"ql-left-eqno\"> &nbsp; <\/span><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.sharpsightlabs.com\/wp-content\/ql-cache\/quicklatex.com-93e9161f8b364e952787ce8ad3201284_l3.png\" height=\"45\" width=\"257\" class=\"ql-img-displayed-equation quicklatex-auto-format\" alt=\"&#92;&#98;&#101;&#103;&#105;&#110;&#123;&#101;&#113;&#117;&#97;&#116;&#105;&#111;&#110;&#42;&#125; &#32;&#32;&#32;&#92;&#116;&#101;&#120;&#116;&#123;&#70;&#49;&#32;&#83;&#99;&#111;&#114;&#101;&#125;&#32;&#61;&#32;&#92;&#102;&#114;&#97;&#99;&#123;&#50;&#42;&#52;&#48;&#125;&#123;&#50;&#42;&#52;&#48;&#32;&#43;&#32;&#49;&#53;&#32;&#43;&#32;&#49;&#48;&#125; &#92;&#101;&#110;&#100;&#123;&#101;&#113;&#117;&#97;&#116;&#105;&#111;&#110;&#42;&#125;\" title=\"Rendered by QuickLaTeX.com\"\/><\/p>\n<p>Which gives us:<\/p>\n<p class=\"ql-center-displayed-equation\" style=\"line-height: 43px;\"><span class=\"ql-right-eqno\"> (9) <\/span><span class=\"ql-left-eqno\"> &nbsp; <\/span><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.sharpsightlabs.com\/wp-content\/ql-cache\/quicklatex.com-bc61382f68908fb759a3b210b5551d82_l3.png\" height=\"43\" width=\"89\" class=\"ql-img-displayed-equation quicklatex-auto-format\" alt=\"&#92;&#98;&#101;&#103;&#105;&#110;&#123;&#101;&#113;&#117;&#97;&#116;&#105;&#111;&#110;&#42;&#125; &#32;&#32;&#32;&#92;&#102;&#114;&#97;&#99;&#123;&#56;&#48;&#125;&#123;&#49;&#48;&#53;&#125; &#61;&#32;&#46;&#55;&#54; &#92;&#101;&#110;&#100;&#123;&#101;&#113;&#117;&#97;&#116;&#105;&#111;&#110;&#42;&#125;\" title=\"Rendered by QuickLaTeX.com\"\/><\/p>\n<p>So the F1 Score of our system is .76.<\/p>\n<h2>Wrapping Up: Why F1 Score is a Useful Metric<\/h2>\n<p>F1 score is often superior to precision recall because it takes a balanced approach in evaluating classifier performance.<\/p>\n<p>While precision measure the accuracy of the positive predictions, precision doesn&#8217;t directly account for the number of actual positive examples that the model fails to predict.<\/p>\n<p>Recall, on the other hand, assess the ability to correctly identify all of the positive examples, but it overlooks the proportion of positive predictions that were actually correct, which can increase the number of False Positives.<\/p>\n<p>F1 score, being the harmonic mean of both precision and recall, mitigates these limitations by accounting for both precision and recall at the same time.  And by doing this, F1 score provides a more wholistic assessment of the performance of a classifier.<\/p>\n<h3>Further Reading<\/h3>\n<p>If you want to learn more about classification evaluation, you should read our posts about:<\/p>\n<ul>\n<li><a href=\"https:\/\/www.sharpsightlabs.com\/blog\/classifier-precision-explained\/\" target=\"_blank\" rel=\"noopener\">Precision<\/a><\/li>\n<li><a href=\"https:\/\/www.sharpsightlabs.com\/blog\/classifier-recall-explained\/\" target=\"_blank\" rel=\"noopener\">Recall<\/a><\/li>\n<li><a href=\"https:\/\/www.sharpsightlabs.com\/blog\/roc-curve-explained\/\" target=\"_blank\" rel=\"noopener\">ROC Curves<\/a><\/li>\n<li><a href=\"https:\/\/www.sharpsightlabs.com\/blog\/confusion-matrix-explained\/\" target=\"_blank\" rel=\"noopener\">Confusion Matrices<\/a><\/li>\n<\/ul>\n<p>Additionally, I&#8217;m going to write more about F1 score in the future, to cover topics like how to improve F1 score, when to use F1 score (and when not to), and more.<\/p>\n<h3>Leave Your Questions and Comments Below<\/h3>\n<p>Do you have other questions about F1 score?<\/p>\n<p>Are you still confused about something, or want to learn something else about F1 that I didn&#8217;t cover?<\/p>\n<p>I want to hear from you.<\/p>\n<p>Leave your questions and comments in the comments section at the bottom of the page.<\/p>\n<h3>Sign up for our email list<\/h3>\n<p>If you want to learn more about machine learning and AI, then sign up for our email list.<\/p>\n<p>Every week, we publish free long-form tutorials about a variety of machine learning, AI, and data science topics, including:<\/p>\n<ul>\n<li>Scikit Learn<\/li>\n<li>Numpy<\/li>\n<li>Pandas<\/li>\n<li>Machine Learning<\/li>\n<li>Deep Learning<\/li>\n<li>&#8230; and more<\/li>\n<\/ul>\n<p>If you sign up for our email list, then we&#8217;ll deliver those free tutorials to you, direct to your inbox.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>If you want to master modern machine learning and AI, one of the major sub-areas that you need to master is classification. Classification is one of the most important types of task in machine learning and AI. But mastering classification, in part, means mastering how to evaluate classification systems. Which in turn, means understanding the &#8230; <a title=\"F1 Score, Explained\" class=\"read-more\" href=\"https:\/\/www.sharpsightlabs.com\/blog\/f1-score-explained\/\" aria-label=\"Read more about F1 Score, Explained\">Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":7819,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","om_disable_all_campaigns":false,"footnotes":""},"categories":[86,19],"tags":[],"publishpress_future_action":{"enabled":false,"date":"2024-05-15 04:17:56","action":"change-status","newStatus":"draft","terms":[],"taxonomy":"category"},"_links":{"self":[{"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/posts\/7398"}],"collection":[{"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/comments?post=7398"}],"version-history":[{"count":0,"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/posts\/7398\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/media\/7819"}],"wp:attachment":[{"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/media?parent=7398"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/categories?post=7398"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/tags?post=7398"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}