Calculate the minimum execution distance of all lines

Question

I am using pandas in python and want to do the following: I want to introduce a new column A in my framework. To calculate it, I want to consider all the rows that have in column B the same value as my "current element" (I think the part in which it is now) in column B, and then take the minimum value of column C minus the value of the current element C of all these elements - and exclude the difference 0, namely self-reference.

As an example:

   B  C    A
0  0  1.2  1.7  (calculation: possible rows are 1 and 2 (all have B = 0), the differences are 2.9 - 1.2 and 3.0 - 1.2 => min = 1.7
1  0  2.9  -1.7 (min difference is 1.2 - 2.9)
2  0  3.0  -1.8
3  1  4.1  1.4
4  1  5.5  -1.4

Thank!

Answer 1

Edit to process the new dataset:

df.groupby('B')['C'].transform(lambda x: np.where(x.idxmin() == x.index,
                                                  x.nsmallest(2).iloc[1]-x,
                                                  (x[x.idxmin()] - x)))

Conclusion:

   B    C    A  A_new
0  0  1.2  1.7    1.7
1  0  2.9 -1.7   -1.7
2  0  3.0 -1.8   -1.8
3  1  4.1  1.4    1.4
4  1  5.5 -1.0   -1.4

IIUC, I think you want this, but I'm not sure of 1 in column A. This is the first row in each group. I replace 0 with 1.

df['A_new'] = df.groupby('B')['C'].transform(lambda x: (x[x.idxmin()] - x).replace(0,1))

Conclusion:

   B  C  A  A_new
0  0  1  1      1
1  0  2 -1     -1
2  0  3 -2     -2
3  1  4  1      1
4  1  5 -1     -1

Timings:

:

%timeit df.apply(lambda x: df[(df.B == x.B) & (~df.C.eq(x.C))].min().C - x.C, axis=1)

100 , 3: 9,78

:

%timeit df.groupby('B')['C'].transform(lambda x: np.where(x.idxmin() == x.index,1,(x[x.idxmin()] - x)))

100 , 3: 3,58

Answer 2

, ...

df['new'] = df.B.map(df.groupby('B').C.apply(list))

df.apply(lambda x :min(list(map(lambda y: y - x['C'],list(set(x['new'])-set([x['C']]))))),axis=1)


Out[1013]: 
0    1
1   -1
2   -2
3    1
4   -1
dtype: int64

:

df['NewA']=df.apply(lambda x :min(list(map(lambda y: y - x['C'],list(set(x['new'])-set([x['C']]))))),axis=1)
df
Out[1015]: 
   B  C  A        new  NewA
0  0  1  1  [1, 2, 3]     1
1  0  2 -1  [1, 2, 3]    -1
2  0  3 -2  [1, 2, 3]    -2
3  1  4  1     [4, 5]     1
4  1  5 -1     [4, 5]    -1

---

numpy

A = df.C.values[:, None] - df.C.values.T
np.fill_diagonal(A, 9999999)
G=df.groupby('B')
np.concatenate([np.min(A[y.min():y.max()+1,y.min():y.max()+1],0) for _, y in G.groups.items()])

%timeit df.apply(lambda x: df[(df.B == x.B) & (~df.C.eq(x.C))].min().C - x.C, axis=1)
100 loops, best of 3: 4.14 ms per loop
%timeit df.groupby('B')['C'].transform(lambda x: np.where(x.idxmin() == x.index,1,(x[x.idxmin()] - x)))
100 loops, best of 3: 1.67 ms per loop

def fff(x):
    A = df.C.values[:, None] - df.C.values.T
    np.fill_diagonal(A, 9999999)
    G=df.groupby('B')
    np.concatenate([np.min(A[y.min():y.max()+1,y.min():y.max()+1],0) for _, y in G.groups.items()])
%timeit fff(1)
1000 loops, best of 3: 758 µs per loop

Answer 3

Min conversion and subtraction C

df['new'] = (df.groupby('B')['C'].transform('min')-df['C']).replace(0,1)

   B  C  A  new
0  0  1  1    1
1  0  2 -1   -1
2  0  3 -2   -2
3  1  4  1    1
4  1  5 -1   -1

Edit based on updated data frame:

g = df.groupby('B')
diff = g['C'].transform('min') - df['C']
df['new'] = diff.where(diff!=0,np.nan)
df['new'] = df['new'].fillna(df['new'].abs().groupby(df['B']).transform('min'))

  B    C    A  new
0  0  1.2  1.7  1.7
1  0  2.9 -1.7 -1.7
2  0  3.0 -1.8 -1.8
3  1  4.1  1.4  1.4
4  1  5.5 -1.4 -1.4

Answer 4

Thanks to everyone for the helpful answers. My preferred solution now is the following:

res = df.sort_values(['B', 'C'])
res.loc[res.B.eq(res.shift(1).B), 'A'] = res.C - res.shift(1).C
df = pd.merge(df, res, on=['B', 'C'])

What do you think? So, I get smaller / larger values ​​and can combine them.