How to shift a column in a Pandas DataFrame

Question

I would like to DataFrame column in the Pandas DataFrame , but I could not find a way to do this from the documentation without overwriting the entire DF. Does anyone know how to do this? DataFrame:

 ## x1 x2 ##0 206 214 ##1 226 234 ##2 245 253 ##3 265 272 ##4 283 291 

Required Conclusion:

 ## x1 x2 ##0 206 nan ##1 226 214 ##2 245 234 ##3 265 253 ##4 283 272 ##5 nan 291 

Answer 1

 In [18]: a Out[18]: x1 x2 0 0 5 1 1 6 2 2 7 3 3 8 4 4 9 In [19]: a.x2 = a.x2.shift(1) In [20]: a Out[20]: x1 x2 0 0 NaN 1 1 5 2 2 6 3 3 7 4 4 8 

Answer 2

If you do not want to lose the columns, you will be taken back to the end of your data frame, just add the required number first:

  offset = 5 DF = DF.append([np.nan for x in range(offset)]) DF = DF.shift(periods=offset) DF = DF.reset_index() #Only works if sequential index 

Answer 3

I believe imports

 import pandas as pd import numpy as np 

First add a new line with NaN, NaN,... to the end of the DataFrame ( df ).

 s1 = df.iloc[0] # copy 1st row to a new Series s1 s1[:] = np.NaN # set all values to NaN df2 = df.append(s1, ignore_index=True) # add s1 to the end of df 

He will create a new DF df2. There may be a more elegant way, but it works.

Now you can move it:

 df2.x2 = df2.x2.shift(1) # shift what you want 

Answer 4

Let's define a data frame from your example

 >>> df = pd.DataFrame([[206, 214], [226, 234], [245, 253], [265, 272], [283, 291]], columns=[1, 2]) >>> df 1 2 0 206 214 1 226 234 2 245 253 3 265 272 4 283 291 

Then you can manipulate the index of the second column with

 >>> df[2].index = df[2].index+1 

and finally re-merge the individual columns

 >>> pd.concat([df[1], df[2]], axis=1) 1 2 0 206.0 NaN 1 226.0 214.0 2 245.0 234.0 3 265.0 253.0 4 283.0 272.0 5 NaN 291.0 

Perhaps not fast, but just for reading. Consider setting variables for column names and the actual offset required.

Edit: As a rule, a shift is possible using df[2].shift(1) as it has already been published, however, this will lead to a cutoff of the transfer.

Answer 5

You need to use df.shift here

df.shift (i) shifts the entire data frame by i units down.

So for i = 1

Input data:

  x1 x2 0 206 214 1 226 234 2 245 253 3 265 272 4 283 291 

Exit:

  x1 x2 0 Nan Nan 1 206 214 2 226 234 3 245 253 4 265 272 

So run this script to get the expected result

 import pandas as pd df = pd.DataFrame({'x1': ['206', '226', '245',' 265', '283'], 'x2': ['214', '234', '253', '272', '291']}) print(df) df['x2'] = df['x2'].shift(1) print(df) 

Answer 6

Trying to answer a personal problem similar to yours, I found in Pandas Doc that, in my opinion, would answer this question:

DataFrame.shift ( period = 1, freq = None, axis = 0) Shift the index by the desired number of periods with an optional time frequency

Notes

If freq is specified, the index values ​​are shifted, but the data is not aligned. That is, use freq if you want to expand the index on shift and keep the original data.

Hope to help future issues in this matter.

Answer 7

Here is how I do it:

 df_ext = pd.DataFrame(index=pd.date_range(df.index[-1], periods=8, closed='right')) df2 = pd.concat([df, df_ext], axis=0, sort=True) df2["forecast"] = df2["some column"].shift(7) 

Basically, I generate an empty data frame with the desired index, and then just merge them together. But I really would like to see this as a standard feature in pandas, so I suggested an improvement to pandas.