How to make numpy array FFT work?

Question

I am reading a specific column of a csv file as a numpy array. When I try to fft this array, I get a NaN array. How do I get fft to work? Here is what I still have:

#!/usr/bin/env python from __future__ import division import numpy as np from numpy import fft import matplotlib.pyplot as plt fileName = '/Users/Name/Documents/file.csv' #read csv file df = np.genfromtxt(fileName, dtype = float, delimiter = ',', names = True) X = df['X'] #get X from file rate = 1000. #rate of data collection in points per second Hx = abs(fft.fft(X)) freqX = fft.fftfreq(len(Hx), 1/rate) plt.plot(freqX,Hx) #plot freqX vs Hx 

Answer 1

Presumably there are some missing values ​​in your csv file. By default, np.genfromtxt will replace the missing values ​​with NaN .

If the array has NaN or Inf , fft will be NaN or Inf s.

For instance:

 import numpy as np x = [0.1, 0.2, np.nan, 0.4, 0.5] print np.fft.fft(x) 

And we get:

 array([ nan +0.j, nan+nanj, nan+nanj, nan+nanj, nan+nanj]) 

However, since FFT works at regular intervals of values, removing non-finite values ​​from an array is a bit more complicated than just dropping them.

pandas has several specialized operations for this, if you are open to using it (e.g. fillna ). However, this is not too difficult to do with pure numpy.

First, I'm going to assume that you are working with a continuous series of data, because you take FFT values. In this case, we want to interpolate the NaN values ​​based on the values ​​around them. Linear interpolation ( np.interp ) may not be ideal in all situations, but it is not a bad default choice:

For instance:

 import numpy as np x = np.array([0.1, 0.2, np.nan, 0.4, 0.5]) xi = np.arange(len(x)) mask = np.isfinite(x) xfiltered = np.interp(xi, xi[mask], x[mask]) 

And we get:

 In [18]: xfiltered Out[18]: array([ 0.1, 0.2, 0.3, 0.4, 0.5]) 

Then we can calculate the FFT:

 In [19]: np.fft.fft(xfiltered) Out[19]: array([ 1.50+0.j , -0.25+0.34409548j, -0.25+0.08122992j, -0.25-0.08122992j, -0.25-0.34409548j]) 

... and get a valid result.