Is there a special type of multivariate regression for multiparametric predictions?

Question

I am trying to use multidimensional regression to play basketball. Specifically, I need to predict the strength of pitch, yaw and gun based on X, Y and distance from the target. I thought of using multivariate regression with multiplex variables for each of the output parameters. Is there a better way to do this?

Also, should you use the solution directly for best fit or use gradient descent?

Answer 1

Multivariate regression is equivalent to performing an inversion of the covariance of a set of input variables. Since there are many solutions to invert the matrix (if the dimension is not very high. Thousand should be fine), you should go directly for the best fit instead of gradient descent.

n is the number of samples, m is the number of input variables, and k is the number of output variables.

X be the input data (n,m) Y be the target data (n,k) A be the coefficients you want to estimate (m,k) XA = Y X'XA=X'Y A = inverse(X'X)X'Y 

X' is the transposition of X.

As you can see, once you find the inverse of X'X , you can calculate the coefficients for any number of output variables with just a few matrix multiplications.

Use any simple math tools to solve this problem (MATLAB / R / Python ..).

Answer 2

ElKamina's answer is correct, but one thing to note is that it is identical to performing k independent ordinary least-square regressions. That is, in the same way as making a separate linear regression from X to pitch, from X to yaw and from X to strength. This means that you are not using correlations between output variables. This may be good for your application, but one alternative that takes advantage of correlation in output reduces regression regression ( the matlab implementation is here ) or somewhat interconnected, you can explicitly non-correlate y by projecting it onto its main components (see PCA , also called PCA bleaching in this case, since you are not reducing the dimension).

I highly recommend Chapter 6 of Izenman's “Modern Multivariate Statistical Methods: Regression, Classification, and Multivalued Learning” textbook for a fairly high level overview of these methods. If you are at a university, it may be available online through your library.

If these alternatives do not work well, there are many complex non-linear regression methods that have several output versions (although most software packages do not have multidimensional modifications), such as vector support regression, Gaussian process regression, tree regression solution, or even neural networks.