How to use scycit-learn Gaussian Process for 2D inputs, 1D output regression?

Question

Before publishing, I did a lot of searching and found this question , which may be exactly my problem. However, I tried what was offered in response, but unfortunately this could not be fixed, and I could not add a comment to request additional explanations, since I am a new member here.

Anyway, I want to use Gaussian Processes with scikit-learn in Python for a simple but real case to get started (using the examples given in the scikit-learn documentation). I have a two-dimensional input set (8 pairs of 2 parameters) called X. I have 8 corresponding outputs compiled in a 1D y array.

# Inputs: 8 points X = np.array([[p1, q1],[p1, q2],[p1, q3],[p1, q4],[p2, q1],[p2, q2],[p2, q3],[p2, q4]]) # Observations: 8 couples y = np.array([r1,r2,r3,r4,r5,r6,r7,r8]) 

I defined the input test space x :

 # Input space x1 = np.linspace(x1min, x1max) x2 = np.linspace(x2min, x2max) x = (np.array([x1, x2])).T 

Then I create an instance of the GP model, match it with the training data (X, y) and make a 1D prediction y_pred on my input space x :

 kernel = C(1.0, (1e-3, 1e3)) * RBF([5,5], (1e-2, 1e2)) gp = GaussianProcessRegressor(kernel=kernel, n_restarts_optimizer=15) gp.fit(X, y) y_pred, MSE = gp.predict(x, return_std=True) 

And then I make a 3D plot:

 fig = pl.figure() ax = fig.add_subplot(111, projection='3d') Xp, Yp = np.meshgrid(x1, x2) Zp = np.reshape(y_pred,50) surf = ax.plot_surface(Xp, Yp, Zp, rstride=1, cstride=1, cmap=cm.jet, linewidth=0, antialiased=False) pl.show() 

This is what I get:

RBF [5.5]

When I change the kernel parameters, I get something like this, similar to what I got the above poster:

RBF [10.10]

These graphs do not even correspond to the observation from the training starting points (a lower answer is obtained for [65.1,37] and the highest for [92.3, 54]).

I am new to GPs in 2D (also recently started Python), so I think something is missing here ... Any answer would be helpful and would be greatly appreciated, thanks!

Answer 1

I am also quite new using the scikit-learn gaussian process. But after some effort, I was able to successfully implement the 3-dimensional regression of the gaussian process. There are many examples of 1st regression, but nothing of higher input dimensions.

Perhaps you can show the values ​​you are using.

I found that sometimes the format you send the input to may cause some problems. Try formatting input X as:

 X = np.array([param1, param2]).T 

and format the output as:

 gp.fit(X, y.reshape(-1,1)) 

In addition, as I understand it, the implementation assumes an average function m = 0. If the result you are trying to recover is an average value that is significantly different from 0, you should normalize it (this will probably solve your problem). Standardizing the parameter space will also help.