I am trying to change the shape of pandas data by turning one of the columns in the data into rows (by rotation or inaction).
I'm new to this, so I'll probably miss something obvious. I searched many times, but could not successfully apply any solutions that I came across.
df
Location Month Metric Value
0 Texas January Temperature 10
1 New York January Temperature 20
2 California January Temperature 30
3 Alaska January Temperature 40
4 Texas January Color Red
5 New York January Color Blue
6 California January Color Green
7 Alaska January Color Yellow
8 Texas February Temperature 15
9 New York February Temperature 25
10 California February Temperature 35
11 Alaska February Temperature NaN
12 Texas February Color NaN
13 New York February Color Purple
14 California February Color Orange
15 Alaska February Color Brown
I am trying to "expand" Metric values into columns. The final goal is the result:
Location Month Temperature Color
Texas January 10 Red
New York January 20 Blue
California January 30 Green
Alaska January 40 Yellow
Texas February 15
New York February 25 Purple
California February 35 Orange
Alaska February Brown
I tried using pivot, pivot_table, as well as stack methods, but I'm sure something is missing. Many of the complications seem to be due to the fact that I mix strings with numbers and also have some missing values in the data.
This is the closest that I managed to get so far, but I do not want extra rows for the month column, which leads to more empty values:
df.set_index(['Location','Month','Metric'], append=True, inplace=True)
df.unstack()
Value
Metric Color Temperature
Location Month
0 Texas January None 10
1 New York January None 20
2 California January None 30
3 Alaska January None 40
4 Texas January Red None
5 New York January Blue None
6 California January Green None
7 Alaska January Yellow None
. , , , .