{"id":3521,"date":"2018-05-21T21:37:15","date_gmt":"2018-05-22T02:37:15","guid":{"rendered":"https:\/\/www.sharpsightlabs.com\/?p=3521"},"modified":"2021-07-24T19:17:29","modified_gmt":"2021-07-25T00:17:29","slug":"highlight-data-in-ggplot2","status":"publish","type":"post","link":"https:\/\/www.sharpsightlabs.com\/blog\/highlight-data-in-ggplot2\/","title":{"rendered":"How to highlight data in ggplot2"},"content":{"rendered":"

One of the most important skills you’ll need as a data scientist (and a junior<\/em> data scientist in particular) is telling stories with data<\/a>.<\/p>\n

Companies, hiring managers, and recruiters not only need you to be able to find insights in data; they also need you to be able to communicate<\/em> with data<\/a>.<\/p>\n

In practice, communicating or storytelling with data actually means using charts and graphs. You need to be able to select the right chart to communicate a particular message. But you’ll also need to modify charts and graphs to communicate clearly and precisely. You want people to look at your chart and immediately “get it.” <\/p>\n

How to highlight data in ggplot2<\/h2>\n

One technique you can use to communicate more clearly with your visualizations is highlighting data. You can highlight particular elements of interest in a chart or graph.<\/p>\n

There are a few ways to do this, but I want to show you how to do this in ggplot2<\/code>.<\/p>\n

At a high level, here’s how we’ll do it:<\/p>\n

    \n
  1. Create an “indicator” variable that identifies the items of interest<\/li>\n
  2. Use color to highlight those specific items of interest, based on the “indicator” variable <\/li>\n<\/ol>\n

    So to highlight data in a ggplot2<\/code> plot, you need two core tools: dplyr::mutate()<\/code> and ggplot2::scale_fill_manual()<\/code> (or something similar).<\/p>\n

    I’ll show you how to use these in your code in a moment, but first, I’ll briefly explain these tools and how they fit into our highlighting strategy.<\/p>\n

    Use dplyr::mutate to create an indicator “flag”<\/h4>\n

    To highlight data in a ggplot visualization, the first thing you need to do is create a new indicator variable. Essentially, you’re going to use dplyr::mutate()<\/code> to create a TRUE<\/code>\/FALSE<\/code> indicator variable based on some condition or conditions.<\/p>\n

    For example, if you’re working with sales data and you want to highlight all of the observations where price is greater than $25, you would use mutate()<\/code> to create a flag for observations where price > 25<\/code>.<\/p>\n

    If this doesn’t make sense, don’t worry … I’ll show you a concrete example in a minute.<\/p>\n

    Use scale_fill_manual or scale_color_manual to set the highlight color<\/h4>\n

    Next, you’ll need to use ggplot2::scale_fill_manual()<\/code> or ggplot2::scale_color_manual()<\/code> to modify the color of the data based on your new indicator variable. <\/p>\n

    So for example, if you wanted to highlight data where price is greater than $25, you would use ggplot2::scale_color_manual()<\/code> to set the highlight color for the different TRUE<\/code> or FALSE<\/code> values of the new indicator variable that you just created.<\/p>\n

    Doesn’t make sense yet?<\/p>\n

    Fair enough. Let’s take a look at some examples.<\/p>\n

    Example: how to highlight data in a ggplot bar chart<\/h2>\n

    Let’s work through an example of how to highlight data in ggplot2<\/code>. <\/p>\n

    Here, we’ll highlight a specific bar in a bar chart.<\/p>\n

    First things first: we’ll load the packages and inspect the data.<\/p>\n

    \r\n#==============\r\n# LOAD PACKAGES\r\n#==============\r\nlibrary(ggplot2)\r\nlibrary(tidyverse)\r\n\r\n\r\n#--------\r\n# INSPECT\r\n#--------\r\nmidwest %>% glimpse()\r\nmidwest %>% count(state)\r\n\r\n<\/pre>\n
    This dataset has demographic data for the states and counties in five midwestern states: Illinois, Indiana, Michigan, Ohio, and Wisconsin.<\/p>\n
    Create a simple bar chart<\/h4>\n
    Before we create a full highlighted bar chart, we’re going to do something very simple.  We’ll plot a bar chart of the number of counties in each state.  <\/p>\n
    To accomplish this, we’re going to use the ggplot()<\/code> function with geom_bar()<\/code><\/a> and the correct variable mappings:<\/p>\n
    \r\n#=================\r\n# SIMPLE BAR CHART\r\n#=================\r\nggplot(data = midwest, aes(x = state)) +\r\n  geom_bar()\r\n<\/pre>\n
    \"A<\/p>\n
    This is very simple.  It’s a basic use of ggplot2<\/code>. (If you don’t know how this works, go back and practice it until you have it memorized!)<\/p>\n
    Create a ggplot bar chart using the ‘pipe’ operator<\/h4>\n
    Ok.  Now, I want to create the same bar chart, but we’re going to make one minor syntactical change (you’ll understand why in a moment).  <\/p>\n
    Instead of using the data =<\/code> parameter to specify the dataset inside of the ggplot()<\/code> function, we’re going to do something different.  We’re going to pull the dataset outside of the ggplot()<\/code> function, and we will use the pipe operator, %>%<\/code>, to “pipe” the dataset into ggplot()<\/code>.<\/p>\n
    \r\n#===========================================\r\n# SIMPLE BAR CHART (USING THE PIPE OPERATOR)\r\n#===========================================\r\nmidwest %>% \r\n  ggplot(aes(x = state)) +\r\n    geom_bar()\r\n<\/pre>\n
    \"A<\/p>\n
    In terms of output, this is effectively identical to the first version of the code.  The resulting plot is exactly the same, but syntactically it’s a slightly different way to do it.<\/p>\n
    But if the output is exactly the same, why change it?  The reason is that in the next step, we’re going to insert a new line of code between the dataset (midwest<\/code>) and the ggplot()<\/code> function.<\/p>\n
    Create an “indicator variable” and create a highlighted bar chart<\/h4>\n
    In this step, we’re going to create an indicator variable using using dplyr::mutate()<\/code>.  <\/p>\n
    Essentially, instead of piping the data directly into the ggplot()<\/code> function, we are piping the dataset into mutate<\/code> where we are going to modify the dataset.  The mutate()<\/code> function will add a new variable<\/a> to the dataset, which is then piped into ggplot2<\/code> for plotting.<\/p>\n
    \r\n#===============================\r\n# BAR CHART WITH HIGHLIGHTED BAR\r\n#===============================\r\nmidwest %>% \r\n  mutate(highlight_flag = ifelse(state == 'IL', T, F)) %>% \r\n  ggplot(aes(x = state)) +\r\n    geom_bar(aes(fill = highlight_flag)) \r\n<\/pre>\n
    \"A<\/p>\n
    This isn’t too complicated, but let’s unpack it.<\/p>\n
    The critical piece of code is the new line that uses mutate()<\/code>:<\/p>\n
      mutate(highlight_flag = ifelse(state == 'IL', T, F))<\/code> <\/p>\n
    What exactly does this do?<\/p>\n
    The mutate()<\/code> function indicates that we’re going to add a new variable to the dataset, midwest<\/code>.  The name of the new variable is highlight_flag<\/code>, and the value of highlight_flag<\/code> depends on the value of the state<\/code> variable: if state<\/code> is ‘IL<\/code>‘ then highlight_flag<\/code> is T<\/code> (i.e., True). Otherwise highlight_flag<\/code> is F<\/code> (i.e., False).<\/p>\n
    This new variable \u2013 highlight_flag<\/code> \u2013 is an indicator variable.  If state is Illinois, then it’s true, otherwise it’s false.<\/p>\n
    We will be able to use this variable to change the color of our bars.  If highlight_flag<\/code> is T<\/code>, the bar will be one color, otherwise, it will be another color.<\/p>\n
    The way that we do this is by mapping highlight_flag<\/code> to the fill<\/code> aesthetic.  It happens in the following line of code: <\/p>\n
    geom_bar(aes(fill = highlight_flag))<\/code>.<\/p>\n
    Remember: the fill<\/code> aesthetic is a ggplot2<\/code> parameter that controls the “fill color” of bars in a bar chart.  By mapping highlight_flag<\/code> to the fill<\/code> aesthetic, we are telling ggplot2<\/code> that the bar colors should correspond to the values of the new highlight_flag<\/code> variable.<\/p>\n
    Do you understand the strategy now?<\/p>\n
    We’re using mutate()<\/code> to create a new variable.  The new variable (which has values T<\/code> or F<\/code>), is being mapped to the fill<\/code> aesthetic, which controls the color of the bars in the bar chart.<\/p>\n
    Change the color of the highlighted bar<\/h4>\n
    There’s only one more step to complete the effect.<\/p>\n
    Notice the in the new highlighted bar chart that the colors of the bars are pink and light blue.<\/p>\n
    No offense if you like pink and light blue, but they are … how can I put this … not to my taste.<\/p>\n
    Pink and light blue are the default colors when you map a categorical variable to the fill<\/code> aesthetic in ggplot2<\/code>.  I didn’t choose those colors.  In fact, I’d like the highlighted bar to be colored red.<\/p>\n
    So how do we choose the colors of the bars after we’ve mapped a variable to the fill<\/code> aesthetic?<\/p>\n
    There are a few ways to change the color of the geoms in ggplot2<\/code>, but in this case, we will do it with scale_fill_manual()<\/code>.  We will use scale_fill_manual()<\/code> to set the color that corresponds to different values of the variable we mapped to the fill<\/code> aesthetic.  Essentially, we can use scale_fill_manual()<\/code> to override the defaults and set the fill colors of the bars \u2013 both the highlight color and the color of the other bars.<\/p>\n
    \r\n#===============================\r\n# BAR CHART WITH HIGHLIGHTED BAR\r\n#===============================\r\nmidwest %>% \r\n  mutate(highlight_flag = ifelse(state == 'IL', T, F)) %>% \r\n  ggplot(aes(x = state)) +\r\n    geom_bar(aes(fill = highlight_flag)) +\r\n    scale_fill_manual(values = c('#595959', 'red'))\r\n<\/pre>\n
    \"How<\/p>\n
    Essentially, we’re using scale_fill_manual()<\/code> to set the bar color to ‘red’ if highlight_flag<\/code> is T<\/code> (i.e., true), and setting the color to #595959<\/code> if highlight_flag<\/code> is F<\/code>.<\/p>\n
    How to highlight a bar in a barchart: a finalized, ‘polished’ example<\/h4>\n
    Ok, we’ve created a bar chart with a single highlighted bar.<\/p>\n
    As I commonly do though, I want to show you a “polished” version of the chart.  The bar chart we just created is OK for purposes of analysis, but it lacks “polish.”  The previous bar chart might not be appropriate for a presentation to clients or a management team.<\/p>\n
    That being the case, here’s a version with several modifications.  I’ll add a title; format the title<\/a>; add axis titles; flip the coordinates; and format the overall text in the plot.<\/p>\n
    \r\n#===============================\r\n# BAR CHART WITH HIGHLIGHTED BAR\r\n#===============================\r\nmidwest %>% \r\n  group_by(state) %>% \r\n  summarise(county_count = n()) %>% \r\n  mutate(highlight_flag = ifelse(state == 'IL', T, F)) %>% \r\n  ggplot(aes(x = fct_reorder(state, county_count), y = county_count)) +\r\n    geom_bar(aes(fill = highlight_flag), stat = 'identity') +\r\n    scale_fill_manual(values = c('#595959', 'red')) +\r\n    coord_flip() +\r\n    labs(x = 'State'\r\n         ,y = 'Number of counties'\r\n         ,title = str_c(\"Illinois has the largest number of counties\"\r\n                        , \"\\nof any state in the midwest\")\r\n    ) +\r\n    theme(text = element_text(family = 'Helvetica Neue', color = '#444444')\r\n          ,plot.title = element_text(size = 18, face = 'bold')\r\n          ,legend.position = 'none'\r\n          ,axis.title = element_text(face = 'bold')\r\n          ,axis.title.y = element_text(angle = 0, vjust = .5)\r\n    )\r\n\r\n<\/pre>\n
    \"Finished<\/p>\n
    A few comments:<\/p>\n
    Notice that I grouped the data by state using dplyr::group_by()<\/code>.  I decided to do this because I wanted to reorder the bars in the bar chart by the number of counties (remember: the length of the bar corresponds to the number of counties).  To reorder the bars in this way, I needed to create a new variable called county_count<\/code> that I could explicitly reference in forcats::fct_reorder()<\/code> to reorder the bars.  This is a minor technical detail, but I wanted to call it out so you don’t get confused.<\/p>\n
    Setting aside the technical details, pay attention to how the chart communicates.<\/p>\n
    The title “tells a story.” I’ve used the title to call out a detail in the chart that I want to highlight.  Moreover, the “story” from the headline works in concert with the highlight color of the highlighted bar.  <\/p>\n
    This is how you communicate with data<\/em>.  <\/p>\n
    To communicate with data, you need to use titles, text copy, annotations, and color to communicate to your audience.  You want the elements of the chart to work together to tell your audience something important or interesting that’s in the chart.<\/p>\n
    Example: how to highlight data in a ggplot scatterplot<\/h2>\n
    Let me show you another example of to highlight data in ggplot2<\/code>.  I’ll show you how to highlight points in a scatterplot.<\/p>\n
    First, let’s build a basic scatterplot<\/a> to understand what’s in these variables and how they are related.<\/p>\n
    \r\n#=============================================\r\n# SIMPLE SCATTERPLOT (USING THE PIPE OPERATOR)\r\n#=============================================\r\n\r\nmidwest %>% \r\n  ggplot(aes(x = percollege, y = percadultpoverty)) +\r\n    geom_point()\r\n<\/pre>\n
    \"A<\/p>\n
    There seems to be a modest negative relationship between the variable percollege<\/code> and percadultpoverty<\/code>.  The relationship is not quite linear, and it’s slight, but it is immediately obvious when we plot the data.<\/p>\n
    Having said that, there are few counties (i.e. points) in the upper right hand corner that stand out.  You can see that there are a few counties where the percent of people with a college degree is greater than 30% and the percent of adults in poverty is greater than 20%. <\/p>\n
    Wait … when I describe it to you in words, it’s not immediately clear exactly which points I’m talking about?<\/p>\n
    Fair enough, I can fix that …<\/p>\n
    \r\n#====================================\r\n# SCATTERPLOT WITH HIGHLIGHTED POINTS\r\n#====================================\r\n\r\nmidwest %>% \r\n  mutate(highlight_flag = ifelse(percollege > 30 & percadultpoverty > 15, T, F)) %>% \r\n  ggplot(aes(x = percollege, y = percadultpoverty)) +\r\n    geom_point(aes(color = highlight_flag)) +\r\n    scale_color_manual(values = c('#595959', 'red'))\r\n\r\n<\/pre>\n
    \"\"<\/p>\n
    Do you see what I did there?<\/p>\n
    Something jumped out at me in the plot.  I tried to communicate what I was seeing verbally, but it was a little challenging.  In fact, as I tried to explain what I was seeing to you, it was easier and more effective to just highlight the f*^ing points<\/em>.<\/p>\n
    This is visual communication.<\/p>\n
    Highlighting plot elements is a tool for “telling stories with data”<\/h3>\n
    To “tell stories with data” you need to be able to use the tools of data science<\/a> \u2013 like data visualization and highlighted elements in visualizations \u2013 to communicate with people.  You want people to look at your visualization and just “get the message.”  <\/p>\n
    The good news is that humans are “wired” for visual information.  Evolutionary, the visual system is much older than the areas for language processing.  The human visual system can rapidly interpret information that is presented visually.  <\/p>\n
    As the saying goes, “a picture is worth a thousand words.”  It’s true.  Images are<\/em> more effective than words, if you know how to properly use color, shape, size to communicate to you audience.  It’s even more effective if you use color, shape, and size in combination with text so that they support each other to communicate a specific message.<\/p>\n
    Ultimately, as a data scientist, part of your job is presenting information in a way that takes advantage visual nature of human cognition.  You need to use visual techniques so that your audience just “get’s it.”  One of those techniques is highlighting data in visualizations.<\/p>\n
    How to use highlighted plots in the data analysis process<\/h2>\n
    Now that we’ve talked about how to use this technique and why it’s valuable, let’s quickly talk about when you might use it.<\/p>\n
    Highlighting elements of a plot is a technique that you can use at several phases of the data science workflow, but there are two phases that are particularly important:<\/p>\n
      \n
    • Exploratory data analysis<\/li>\n
    • Analyses and reports<\/li>\n<\/ul>\n
      You might consider these to be one and the same, but that’s not the case.  Performing exploratory data analysis is not quite the same thing as producing a finalized analysis.<\/p>\n
      Let’s briefly talk about these two phases, and how you can use highlighting in them.<\/p>\n
      How to use highlighted plots in exploratory data analysis<\/h3>\n
      One thing you need to master in order to be a great data scientist is exploratory data analysis.  <\/p>\n
      Much like the name implies, exploratory data analysis involves (wait for it …) exploring<\/em> data.  Whether your final deliverable is a report to business partners, or a machine learning model that makes predictions, you will first need to explore your data to find out what’s in it.  <\/p>\n
      In the case of creating a report, you will use EDA to explore your data in an attempt to “find insights.”   When you use EDA to find insights, your first task is to understand the data for yourself personally<\/em>.  You<\/em><\/strong> need to see the interesting elements.   You<\/em><\/strong> need to see the anomolies.  You need to see<\/em><\/u> them.<\/p>\n
      When performing exploratory data analysis, highlighting specific parts of a chart enables you to literally see relevant information in a chart.  <\/p>\n
      I can’t stress this enough.  As an analyst, you need to see<\/em> insights in data.  One of the tools for helping you see the data in new ways is highlighting.<\/p>\n
      How to use highlighted plots in reports and presentations<\/h3>\n
      Some of your deliverables as a data scientist will be reports, analyses, and presentations.  In many of these presentations, you will need to communicate with your data.  As noted earlier, your business partners will want you to “tell a story” with your data.  <\/p>\n
      This actually works hand-in-hand with the exploratory data analysis process.  In the first step, in EDA, you will identify the story and see important insights for yourself<\/em>.  Your first task \u2013 as noted earlier \u2013 will be to see personally.<\/p>\n
      But once you<\/em> see the insights \u2013 once you<\/em> see the story \u2013 you need to be able to share the insights with others in a way that they can easily understand.<\/p>\n
      As I noted previously, the human brain is wired for visual information.  When you create reports and analyses, you can use highlighting to emphasize relevant points or details in the data.  Use highlighting to communicate the insights that you’ve found and seen for yourself.<\/p>\n
      Master the basics, then master details techniques like highlighting data<\/h2>\n
      Highlighting data in ggplot2<\/code> plots can be surprisingly useful.  It’s a technique that you can use in EDA, reporting, analysis, and “data storytelling.”<\/p>\n
      But it is a bit of an intermediate technique.  As you saw in this tutorial, to execute this technique, you need to combine several different tools from ggplot<\/code> and dplyr<\/code> to make this work.<\/p>\n
      That being the case, before you start practicing intermediate techniques like this, make sure that you have a solid foundation in ggplot2<\/code> and dplyr<\/code>.<\/p>\n
      Here at Sharp Sight, we have a firm belief that data science students should master the fundamentals first.  <\/p>\n
      We believe that becoming a great data scientist starts with mastering the fundamentals.  That means mastering basic techniques like the bar chart, line chart, and histogram … even if they are a little un-sexy.<\/p>\n
      Once you master those simple techniques, you can move on to intermediate techniques like highlighting plot elements, just like I showed you in this blog post.<\/p>\n
      My recommendation to you then, is start with simple techniques like ggplot’s geom_bar()<\/code> and geom_line()<\/code>.  Master basic techniques from dplyr<\/code> like mutate()<\/code> and the other major dplyr<\/code> verbs.  Once you’ve mastered those basic techniques, you can expand your toolkit to tools like scale_fill_manual()<\/code> and scale_color_manual()<\/code>, which will enable you to customize your plots and do things like highlight plot elements.  <\/p>\n
      Start by mastering the fundamentals, then steadily expand your toolkit. <\/p>\n
      If you can do that, you’ll be on the path to becoming a top-performing data scientist. <\/p>\n
      \n\t\t\t\t