NumPy PolyFit and PolyVal in several sizes?

Question

Suppose that an n-dimensional array of observations, which is transformed as a 2d-array, with each row being one set of observations. Using this shape-changing approach, np.polyfit can calculate second-order matching coefficients for the entire ndarray (vectorized):

 fit = np.polynomial.polynomialpolyfit(X, Y, 2) 

where Y is the shape (304000, 21), and X is the vector. The result is an array of coefficients (304000.3) suitable.

Using an iterator, you can call np.polyval(fit, X) for each row. This is ineffective when a vector approach may exist. Can the fit result be applied to the entire array of observations without iteration? If so, how?

This is in accordance with this SO question.

Answer 1

np.polynomial.polynomial.polyval accepts arrays of multidimensional coefficients:

 >>> x = np.random.rand(100) >>> y = np.random.rand(100, 25) >>> fit = np.polynomial.polynomial.polyfit(x, y, 2) >>> fit.shape # 25 columns of 3 polynomial coefficients (3L, 25L) >>> xx = np.random.rand(50) >>> interpol = np.polynomial.polynomial.polyval(xx, fit) >>> interpol.shape # 25 rows, each with 50 evaluations of the polynomial (25L, 50L) 

And of course:

 >>> np.all([np.allclose(np.polynomial.polynomial.polyval(xx, fit[:, j]), ... interpol[j]) for j in range(25)]) True 

Answer 2

np.polynomial.polynomial.polyval is a great (and convenient) approach to efficiently evaluating polynomial fittings.

However, if the “fastest” is what you are looking for, simply building polynomial inputs and using the multiplication functions on the elementary matrix matrix leads to slightly faster (about 4 times faster) computational speeds.

Customization

Using the same setting as above, we will create 25 different fittings.

 >>> num_samples = 100000 >>> num_lines = 100 >>> x = np.random.randint(0,100,num_samples) >>> y = np.random.randint(0,100,(num_samples, num_lines)) >>> fit = np.polyfit(x,y,deg=2) >>> xx = np.random.randint(0,100,num_samples*10) 

Numpy polyval Function

 res1 = np.polynomial.polynomial.polyval(xx, fit) 

Averaging the main matrices

 inputs = np.array([np.power(xx,d) for d in range(len(fit))]) res2 = fit.T.dot(inputs) 

Terms of performance of functions

Using the same options above ...

 %timeit _ = np.polynomial.polynomial.polyval(xx, fit) 1 loop, best of 3: 247 ms per loop %timeit inputs = np.array([np.power(xx, d) for d in range(len(fit))]);_ = fit.T.dot(inputs) 10 loops, best of 3: 72.8 ms per loop 

Beat the dead horse ...

GPA Efficiency ~ 3.61x faster. Speed ​​fluctuations probably come from random computer processes in the background.