{"id":4890,"date":"2023-11-29T16:00:22","date_gmt":"2023-11-29T22:00:22","guid":{"rendered":"https:\/\/www.sharpsightlabs.com\/?p=4890"},"modified":"2023-11-29T16:56:37","modified_gmt":"2023-11-29T22:56:37","slug":"pandas-melt","status":"publish","type":"post","link":"https:\/\/www.sharpsightlabs.com\/blog\/pandas-melt\/","title":{"rendered":"Pandas Melt, Explained"},"content":{"rendered":"<p>In this blog post, I&#8217;ll show you how to use the Pandas melt method. <\/p>\n<p>I explain what melt does, how the syntax works, and I&#8217;ll show you a simple example.<\/p>\n<p>If you need something specific, you can click on any of the following links:<\/p>\n<p><strong>Table of Contents:<\/strong><\/p>\n<ul>\n<li><a href=\"#intro\">Introduction<\/a><\/li>\n<li><a href=\"#syntax\">Syntax<\/a><\/li>\n<li><a href=\"#examples\">Examples<\/a><\/li>\n<li><a href=\"#FAQ\">Frequently Asked Questions<\/a><\/li>\n<\/ul>\n<p>Having said that, this Pandas method is a little difficult to understand, and I recommend that you read the whole post and watch the videos.  That will be the best way to really understand what&#8217;s going on.<\/p>\n<p><a id = \"intro\"><\/a><\/p>\n<h2>A Quick Introduction to Pandas Melt<\/h2>\n<p>We use the Pandas to reshape <a href=\"https:\/\/www.sharpsightlabs.com\/blog\/pandas-dataframe\/\" target=\"_blank\" rel=\"noopener\">Pandas dataframes<\/a> from wide format to long format.<\/p>\n<p>This is a type of <a href=\"https:\/\/en.wikipedia.org\/wiki\/Transpose\" target=\"_blank\" rel=\"noopener\">data transposition<\/a>.<\/p>\n<p>And frankly, it&#8217;s a little difficult to explain in words.<\/p>\n<p>You kind of need to <em>see<\/em> it.<\/p>\n<p>Which is why I put together this video, to show you exactly what the Pandas melt method does, and how it works.<\/p>\n<p align=\"center\"><iframe loading=\"lazy\" width=\"560\" height=\"315\" src=\"https:\/\/www.youtube.com\/embed\/mCPgkl5ULZo?si=tYAKwBJYfoJOW0FY\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" allowfullscreen><\/iframe><\/p>\n<p>As you can see in the video, we use Pandas melt to reorganize the data somewhat.<\/p>\n<p>Melt transforms the <em>names<\/em> of multiple columns into categorical values of a new categorical variable.<\/p>\n<p>It also transforms the numeric values within those transformed columns into a new &#8220;value&#8221; variable, that consolidates the numeric values of the original columns into one new column.<\/p>\n<p>It&#8217;s hard to describe in words, so if you&#8217;re confused, you need to watch the video carefully to actually <em>see<\/em> what&#8217;s happening.<\/p>\n<p>Now &#8230; assuming you&#8217;ve watched the video &#8230;<\/p>\n<p>Does it make better sense now?<\/p>\n<p>Good.<\/p>\n<p>(If not, leave me a question in the comments.)<\/p>\n<p>Let&#8217;s take a look at the syntax.<\/p>\n<p><a id = \"syntax\"><\/a><\/p>\n<h2>The Syntax of Pandas Melt<\/h2>\n<p>The syntax of Pandas melt is a little complicated, because how the function works is a little complicated.<\/p>\n<p>Having said that, I&#8217;m going to simplify it as much as possible.<\/p>\n<h4>A quick note<\/h4>\n<p>One quick note: everything I write here assumes that you&#8217;ve imported Pandas and that you have a dataframe that you can use.<\/p>\n<h3>melt syntax<\/h3>\n<p>Let&#8217;s assume that you have a dataframe called <code>myDataFrame<\/code>.<\/p>\n<p>To use the melt method, you need to type the name of the dataframe, and then type <code>.melt()<\/code>.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2024\/12\/pandas-melt-syntax.png\" alt=\"An image that explains the syntax of the Pandas Melt method.\" width=\"800\" class=\"aligncenter size-full wp-image-7815\" srcset=\"https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2024\/12\/pandas-melt-syntax.png 1600w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2024\/12\/pandas-melt-syntax-600x261.png 600w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2024\/12\/pandas-melt-syntax-1024x445.png 1024w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2024\/12\/pandas-melt-syntax-768x334.png 768w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2024\/12\/pandas-melt-syntax-1536x668.png 1536w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2024\/12\/pandas-melt-syntax-1080x470.png 1080w, https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2024\/12\/pandas-melt-syntax-610x265.png 610w\" sizes=\"(max-width: 1600px) 100vw, 1600px\" \/><\/p>\n<p>Inside the parenthesis, there are a few critical parameters that you need to use, the most important being <code>id_vars<\/code> and <code>value_vars<\/code>.<\/p>\n<p>Let&#8217;s take a look at the parameters.<\/p>\n<p><a id = \"parameters\"><\/a><\/p>\n<h3>The Parameters of Pandas Melt<\/h3>\n<p>The parameters of melt are really what makes it work, so you need to understand them.<\/p>\n<p>The 4 that you should know are:<\/p>\n<ul>\n<li><code>id_vars<\/code><\/li>\n<li><code>value_vars<\/code><\/li>\n<li><code>var_name<\/code><\/li>\n<li><code>value_name<\/code><\/li>\n<\/ul>\n<p>The two most important are <code>id_vars<\/code> and <code>value_vars<\/code>, but <code>var_name<\/code> and <code>value_name<\/code> are also important.<\/p>\n<p>Let&#8217;s take a look at each of these, one at a time.<\/p>\n<h6><code>id_vars<\/code> (required)<\/h6>\n<p>The <code>id_vars<\/code> parameter specifies the column that contains the values that will <em>identify<\/em> the rows in the new column.<\/p>\n<p>It&#8217;s essentially the column that contains the row identifiers for the output.<\/p>\n<p>The argument to this parameter is the column or group of columns that contain the row identifiers of the new output.<\/p>\n<h6><code>value_vars<\/code><\/h6>\n<p>The <code>value_vars<\/code> parameter specifies the columns in the input dataframe that contain the values that will populate the new &#8220;value&#8221; column in the output DataFrame.<\/p>\n<p>Remember: when we melt our data, there&#8217;s a set of existing columns that contain values, and we want to consolidate the values of those multiple columns into a single new &#8220;value&#8221; column.<\/p>\n<p>And the <em>names<\/em> of those consolidated columns will become categorical values in a new &#8220;variable&#8221; column.<\/p>\n<p>The <code>value_vars<\/code> parameter specifies which existing columns should be transformed into the new output columns.<\/p>\n<p>The argument to this parameter is the names of the columns that contain the values that you want to consolidate.<\/p>\n<h6><code>var_name<\/code><\/h6>\n<p>The <code>var_name<\/code> parameter allows you to specify the name of the new &#8220;categorical&#8221; variable that&#8217;s being created from the the names of the &#8220;value_vars&#8221; columns.<\/p>\n<p>The argument to this parameter is just a text name, formatted as a string.<\/p>\n<h6><code>value_name<\/code><\/h6>\n<p>The <code>value_name<\/code> parameter allows you to specify the name of the new &#8220;values&#8221; variable that&#8217;s being created from the the values of the &#8220;value_vars&#8221; columns.<\/p>\n<p>The argument to this parameter is also just a text name, formatted as a string.<\/p>\n<p><a id = \"examples\"><\/a><\/p>\n<h2>Examples of How to Use Melt<\/h2>\n<p>Here, I&#8217;ll show you a couple of quick examples of how to use Pandas melt.<\/p>\n<p><strong>Examples:<\/strong><\/p>\n<ul>\n<li><a href=\"#example-1\">Simple Melt Example<\/a><\/li>\n<li><a href=\"#example-2\">Add names to the new variables<\/a><\/li>\n<\/ul>\n<h4>Run this code first<\/h4>\n<p>Run this code first.<\/p>\n<p>It will import Pandas (which you need to already have installed on your machine).<\/p>\n<p>It will also import the data that we&#8217;re going to use.<\/p>\n<pre>\r\n# IMPORT PANDAS\r\nimport pandas as pd\r\n\r\n# GET DATA\r\nrevenue_wide = pd.read_csv('https:\/\/learn.sharpsightlabs.com\/datasets\/pdm\/revenue_wide.csv')\r\n<\/pre>\n<p><a id = \"example-1\"><\/a><\/p>\n<h3>EXAMPLE 1: Simple Melt Example<\/h3>\n<p>Let&#8217;s take a look at a simple example.<\/p>\n<p>Here, we&#8217;re going to transpose <code>revenue_wide<\/code> from wide format to long format.<\/p>\n<pre>\r\nrevenue_wide.melt(id_vars = ['region']\r\n                  ,value_vars = ['Q1','Q2']\r\n                  )\r\n<\/pre>\n<p>OUT:<\/p>\n<pre>\r\n          region variable  value\r\n0         Europe       Q1  65000\r\n1  North America       Q1  60000\r\n2         Europe       Q2  62000\r\n3  North America       Q2  63000\r\n<\/pre>\n<h5>Explanation<\/h5>\n<p>Take a look at the output.  <\/p>\n<p>The &#8220;id var&#8221; that we used was region.  The values of that variable are the row identifiers for the input, and they become part of the unique row identifier in the output (in combination with the new &#8220;<code>variable<\/code>&#8221; column).<\/p>\n<p>We used the code <code>value_vars = ['Q1','Q2']<\/code> to specify the values we want to transpose were in the columns <code>Q1<\/code> and <code>Q2<\/code>.<\/p>\n<p>Notice that in the output, <code>Q1<\/code> and <code>Q2<\/code> have become categorical values under the new <code>variable<\/code> column.  And the values that had previously been under the <code>Q1<\/code> and <code>Q2<\/code> columns of the input, are now values that have been consolidated under the new &#8220;<code>value<\/code>&#8221; column of the output.<\/p>\n<p>The output contains the same data as the input, it&#8217;s just structured differently, where the names of the old columns <code>Q1<\/code> and <code>Q2<\/code> have become values of a new categorical variable.  And the values of the old columns <code>Q1<\/code> and <code>Q2<\/code> have become values of a new &#8220;<code>value<\/code>&#8221; column.<\/p>\n<p><a id = \"example-2\"><\/a><\/p>\n<h3>EXAMPLE 2: Add names to the new variables<\/h3>\n<p>Next, we&#8217;re going to add names to the newly created variables.<\/p>\n<p>Let&#8217;s run the code, and then I&#8217;ll explain:<\/p>\n<pre>\r\nrevenue_wide.melt(id_vars = ['region']\r\n                  ,value_vars = ['Q1','Q2']\r\n                  ,value_name = 'revenue'\r\n                  ,var_name = 'quarter'\r\n                  )\r\n<\/pre>\n<p>OUT:<\/p>\n<pre>\r\n          region quarter  revenue\r\n0         Europe      Q1    65000\r\n1  North America      Q1    60000\r\n2         Europe      Q2    62000\r\n3  North America      Q2    63000\r\n<\/pre>\n<h5>Explanation<\/h5>\n<p>Notice that the structure of the output is the same as the output from <a href=\"#example-1\">example 1<\/a>.<\/p>\n<p>The only difference from example 1 is that we&#8217;ve <em>named the new variables<\/em>.<\/p>\n<p>We named the new categorical column as <code>quarter<\/code> with the code <code>var_name = 'quarter'<\/code>.<\/p>\n<p>And we named the new numerical &#8220;value&#8221; column as <code>revenue<\/code> with the code <code>value_name = 'revenue'<\/code>.<\/p>\n<p>That&#8217;s it.  If you look closely and follow both examples (examples 1 and 2), you should be able to see what&#8217;s going on here.<\/p>\n<h5>Leave your other questions in the comments below<\/h5>\n<p>All of that said, I do realize that Pandas melt is a difficult tool to understand.<\/p>\n<p>So, is there something you&#8217;re still confused about?<\/p>\n<p>Do you have questions?<\/p>\n<p>I want to hear from you.<\/p>\n<p>Leave your questions and comments in the comments section at the bottom of the page.<\/p>\n<h2>For More Python Data Science Tutorials, Sign Up for our Email List<\/h2>\n<p>If you want more FREE tutorials about Pandas, and Python data science, then sign up for our email list<\/p>\n<p>We publish new free tutorials every week on things like:<\/p>\n<ul>\n<li>NumPy<\/li>\n<li>Pandas<\/li>\n<li>Base Python<\/li>\n<li>Scikit learn<\/li>\n<li>Machine learning<\/li>\n<li>Deep learning<\/li>\n<li>\u2026 and more.<\/li>\n<\/ul>\n<p>When you sign up for our email list, we&#8217;ll send our free tutorials directly to your inbox.<\/p>\n<div class=\"et_bloom_inline_form et_bloom_optin et_bloom_make_form_visible et_bloom_optin_7\" style=\"display: none;\">\n\t\t\t\t<style type=\"text\/css\">.et_bloom .et_bloom_optin_7 .et_bloom_form_content { background-color: #999999 !important; } .et_bloom .et_bloom_optin_7 .et_bloom_form_container .et_bloom_form_header { background-color: #2a2b2d !important; } .et_bloom .et_bloom_optin_7 .carrot_edge .et_bloom_form_content:before { border-top-color: #2a2b2d !important; } .et_bloom .et_bloom_optin_7 .carrot_edge.et_bloom_form_right .et_bloom_form_content:before, .et_bloom .et_bloom_optin_7 .carrot_edge.et_bloom_form_left .et_bloom_form_content:before { border-top-color: transparent !important; border-left-color: #2a2b2d !important; }\n\t\t\t\t\t\t@media only screen and ( max-width: 767px ) {.et_bloom .et_bloom_optin_7 .carrot_edge.et_bloom_form_right .et_bloom_form_content:before { border-top-color: #2a2b2d !important; border-left-color: transparent !important; }.et_bloom .et_bloom_optin_7 .carrot_edge.et_bloom_form_left .et_bloom_form_content:after { border-bottom-color: #2a2b2d !important; border-left-color: transparent !important; }\n\t\t\t\t\t\t}.et_bloom .et_bloom_optin_7 .et_bloom_form_content button { background-color: #4598ab !important; } .et_bloom .et_bloom_optin_7 .et_bloom_form_content .et_bloom_fields i { color: #4598ab !important; } .et_bloom .et_bloom_optin_7 .et_bloom_form_content .et_bloom_custom_field_radio i:before { background: #4598ab !important; } .et_bloom .et_bloom_optin_7 .et_bloom_form_content button { background-color: #4598ab !important; } .et_bloom .et_bloom_optin_7 .et_bloom_form_container h2, .et_bloom .et_bloom_optin_7 .et_bloom_form_container h2 span, .et_bloom .et_bloom_optin_7 .et_bloom_form_container h2 strong { font-family: \"Open Sans\", Helvetica, Arial, Lucida, sans-serif; }.et_bloom .et_bloom_optin_7 .et_bloom_form_container p, .et_bloom .et_bloom_optin_7 .et_bloom_form_container p span, .et_bloom .et_bloom_optin_7 .et_bloom_form_container p strong, .et_bloom .et_bloom_optin_7 .et_bloom_form_container form input, .et_bloom .et_bloom_optin_7 .et_bloom_form_container form button span { font-family: \"Open Sans\", Helvetica, Arial, Lucida, sans-serif; } <\/style>\n\t\t\t\t<div class=\"et_bloom_form_container  with_edge carrot_edge et_bloom_form_text_dark et_bloom_form_bottom et_bloom_inline_1_field\">\n\t\t\t\t\t\n\t\t\t<div class=\"et_bloom_form_container_wrapper clearfix\">\n\t\t\t\t<div class=\"et_bloom_header_outer\">\n\t\t\t\t\t<div class=\"et_bloom_form_header et_bloom_header_text_light\">\n\t\t\t\t\t\t\n\t\t\t\t\t\t<div class=\"et_bloom_form_text\">\n\t\t\t\t\t\t<h2>Sign up for FREE data science tutorials<\/h2><p>If you want to master data science fast, sign up for our email list.<\/p>\r\n<p>When you sign up, you'll receive FREE weekly tutorials on how to do data science in R and Python.\u00a0<\/p>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t\t\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"et_bloom_form_content et_bloom_1_field et_bloom_bottom_inline\">\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t<form method=\"post\" class=\"clearfix\">\n\t\t\t\t\t\t<div class=\"et_bloom_fields\">\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t<p class=\"et_bloom_popup_input et_bloom_subscribe_email\">\n\t\t\t\t\t\t\t\t<input placeholder=\"Enter your best email address\">\n\t\t\t\t\t\t\t<\/p>\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t<button data-optin_id=\"optin_7\" data-service=\"aweber\" data-list_id=\"3573443\" data-page_id=\"4890\" data-account=\"josh@sharpsightlabscom\" data-ip_address=\"true\" class=\"et_bloom_submit_subscription\">\n\t\t\t\t\t\t\t\t<span class=\"et_bloom_subscribe_loader\"><\/span>\n\t\t\t\t\t\t\t\t<span class=\"et_bloom_button_text et_bloom_button_text_color_light\">Give me free tutorials!<\/span>\n\t\t\t\t\t\t\t<\/button>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/form>\n\t\t\t\t\t<div class=\"et_bloom_success_container\">\n\t\t\t\t\t\t<span class=\"et_bloom_success_checkmark\"><\/span>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t<h2 class=\"et_bloom_success_message\">Check your email inbox to confirm your subscription ...<\/h2>\n\t\t\t\t\t\n\t\t\t\t<\/div>\n\t\t\t<\/div>\n\t\t\t<span class=\"et_bloom_close_button\"><\/span>\n\t\t\t\t<\/div>\n\t\t\t<\/div>\n","protected":false},"excerpt":{"rendered":"<p>In this blog post, I&#8217;ll show you how to use the Pandas melt method. I explain what melt does, how the syntax works, and I&#8217;ll show you a simple example. If you need something specific, you can click on any of the following links: Table of Contents: Introduction Syntax Examples Frequently Asked Questions Having said &#8230; <a title=\"Pandas Melt, Explained\" class=\"read-more\" href=\"https:\/\/www.sharpsightlabs.com\/blog\/pandas-melt\/\" aria-label=\"Read more about Pandas Melt, Explained\">Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","om_disable_all_campaigns":false,"footnotes":""},"categories":[68],"tags":[],"publishpress_future_action":{"enabled":false,"date":"2024-05-23 20:16:33","action":"change-status","newStatus":"draft","terms":[],"taxonomy":"category"},"_links":{"self":[{"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/posts\/4890"}],"collection":[{"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/comments?post=4890"}],"version-history":[{"count":0,"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/posts\/4890\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/media?parent=4890"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/categories?post=4890"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.sharpsightlabs.com\/wp-json\/wp\/v2\/tags?post=4890"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}