I will use the same @Matt data file xf
xf = pd.DataFrame(np.random.rand(5, 5))
However, I will indicate that if the diagonal is zero, the use np.diag(np.diag(xf)) != 0will be violated.
A way to ensure that you are masking the diagonal is to evaluate if the row indices are not equal for the column indices.
Option 1
numpy.indices
, numpy np.indices.
, .
rows, cols = np.indices((5, 5))
print(rows)
[[0 0 0 0 0]
[1 1 1 1 1]
[2 2 2 2 2]
[3 3 3 3 3]
[4 4 4 4 4]]
print(cols)
[[0 1 2 3 4]
[0 1 2 3 4]
[0 1 2 3 4]
[0 1 2 3 4]
[0 1 2 3 4]]
... .
print((cols == rows).astype(int))
[[1 0 0 0 0]
[0 1 0 0 0]
[0 0 1 0 0]
[0 0 0 1 0]
[0 0 0 0 1]]
, ,
xf.mask(np.equal(*np.indices(xf.shape)))
0 1 2 3 4
0 NaN 0.605436 0.573386 0.978588 0.160986
1 0.295911 NaN 0.509203 0.692233 0.717464
2 0.275767 0.966976 NaN 0.883339 0.143704
3 0.628941 0.668836 0.468928 NaN 0.309901
4 0.286933 0.523243 0.693754 0.253426 NaN
pd.DataFrame(
np.where(np.equal(*np.indices(xf.shape)), np.nan, xf.values),
xf.index, xf.columns
)
2
numpy.arange slice
v = xf.values.copy()
i = j = np.arange(np.min(v.shape))
v[i, j] = np.nan
pd.DataFrame(v, xf.index, xf.columns)
0 1 2 3 4
0 NaN 0.605436 0.573386 0.978588 0.160986
1 0.295911 NaN 0.509203 0.692233 0.717464
2 0.275767 0.966976 NaN 0.883339 0.143704
3 0.628941 0.668836 0.468928 NaN 0.309901
4 0.286933 0.523243 0.693754 0.253426 NaN
%%timeit
v = xf.values.copy()
i = j = np.arange(np.min(v.shape))
v[i, j] = np.nan
pd.DataFrame(v, xf.index, xf.columns)
%timeit pd.DataFrame(np.where(np.eye(np.min(xf.shape)), np.nan, xf.values), xf.index, xf.columns)
%timeit pd.DataFrame(np.where(np.equal(*np.indices(xf.shape)), np.nan, xf.values), xf.index, xf.columns)
%timeit xf.mask(np.equal(*np.indices(xf.shape)))
%timeit xf.mask(np.diag(np.diag(xf.values)) != 0)
%timeit xf.mask(np.eye(np.min(xf.shape), dtype=bool)
10000 loops, best of 3: 74.5 µs per loop
10000 loops, best of 3: 85.7 µs per loop
10000 loops, best of 3: 77 µs per loop
1000 loops, best of 3: 519 µs per loop
1000 loops, best of 3: 517 µs per loop
1000 loops, best of 3: 528 µs per loop