Like: the decisive foundation is the quadratic least square

Question

I have two independent variables, GSH and Gls . Using these two variables, I am trying to predict the result, prob . Using a view function:

 prob=a*Gls^2+b*GSH^2+c*Gls+d*GSH+e // (where a,b,c,d,e are coefficients) 

Sample data:

 Gls( 2.3 2.3 2.5 2.5 2.5 2.5 2.7 2.7 2.7 2.7 2.7 2.9 2.9 2.9 2.9 2.9 3.1 3.1 3.1 3.1 3.1 3.1 3.3 3.3 3.3 3.3 3.3 3.3 3.5 3.5 3.5 3.5 3.5) GSH( 0.475 0.525 0.425 0.475 0.525 0.575 0.425 0.475 0.525 0.575 0.625 0.425 0.475 0.525 0.575 0.625 0.375 0.425 0.475 0.525 0.575 0.625 0.375 0.425 0.475 0.525 0.575 0.625 0.425 0.475 0.525 0.575 0.625) prob( 0.263636 0.324159 0.319328 0.291295 0.286086 0.253994 0.233766 0.284644 0.273818 0.263743 0.175182 0.243986 0.284848 0.28066 0.247863 0.183468 0.181818 0.237288 0.269266 0.2555 0.240924 0.206081 0.209677 0.216949 0.263261 0.25966 0.23588 0.203252 0.239316 0.209184 0.234818 0.242424 0.192118) 

I would like to find the best coefficient values ​​to minimize the sum of the least squares.

I read the lots on the foundation solver, but I was not able to decide how to fix this problem in the C # Solver Foundation. All suggestions for using the code are welcome.

thanks

Answer 1

I think for this you do not need a basis for a solution. There is no need for numerical optimization, since a solution (a vector of polynomial coefficients that minimizes the sum of squares of vertical distances between the observed answers in the data set and the predicted answers) exists in a closed form.

See wikipedia for more details.

Answer 2

The following solution is very straightforward, just trying to find a local minimum using the algorithm you described. Using it, I get the following values

a = 0.02527237, b = 0.04768372, c = -0.001549721, d = 0.01382828, e = 0.002026558

with a total square of 0.2139592.

  static void Main(string[] args) { var a = FindLocalMinimum(x => SumSq(x, 0, 0, 0, 0)); var b = FindLocalMinimum(x => SumSq(a, x, 0, 0, 0)); var c = FindLocalMinimum(x => SumSq(a, b, x, 0, 0)); var d = FindLocalMinimum(x => SumSq(a, b, c, x, 0)); var e = FindLocalMinimum(x => SumSq(a, b, c, d, x)); } private static float SumSq(float a, float b, float c, float d, float e) { var gls = new[] { 2.3, 2.3, 2.5, 2.5, 2.5, 2.5, 2.7, 2.7, 2.7, 2.7, 2.7, 2.9, 2.9, 2.9, 2.9, 2.9, 3.1, 3.1, 3.1 , 3.1, 3.1, 3.1, 3.3, 3.3, 3.3, 3.3, 3.3, 3.3, 3.5, 3.5, 3.5, 3.5, 3.5 }; var gsh = new[] { 0.475, 0.525, 0.425, 0.475, 0.525, 0.575, 0.425, 0.475, 0.525, 0.575, 0.625, 0.425, 0.475, 0.525, 0.575, 0.625, 0.375, 0.425, 0.475, 0.525, 0.575, 0.625, 0.375, 0.425, 0.475, 0.525, 0.575, 0.625, 0.425, 0.475, 0.525, 0.575, 0.625 }; var prob = new[] { 0.263636, 0.324159, 0.319328, 0.291295, 0.286086, 0.253994, 0.233766, 0.284644, 0.273818, 0.263743, 0.175182, 0.243986, 0.284848, 0.28066, 0.247863, 0.183468, 0.181818, 0.237288, 0.269266, 0.2555, 0.240924, 0.206081, 0.209677, 0.216949, 0.263261, 0.25966, 0.23588, 0.203252, 0.239316, 0.209184, 0.234818, 0.242424, 0.192118 }; var res = 0.0; for (var i = 0; i < prob.Length; i++) { var p = a*Math.Pow(gls[i], 2) + a*Math.Pow(gsh[i], 2) + c*gls[i] + d*gsh[i] + e; res += Math.Pow(p - prob[i], 2); } return (float)res; } private static float FindLocalMinimum(Func<float, float> f) { float bestV = float.MaxValue; float bestX = 0; float x = 0; float lastV = bestV; float diff = 1000.0f; while (Math.Abs(diff) > 0.0001f) { float v = f(x); if (v < bestV) { bestV = v; bestX = x; } else if (v > lastV) { diff *= -0.5f; } lastV = v; x += diff; } return bestX; } 

Answer 3

You can use the solver foundation. Your regression is already non-linear and is actually a generalized linear regression . In R, you can use a package, such as glm , to perform regression.

In C #, I'm not sure if any open source exists. But in any case, you can solve the optimization yourself, and MSF has a non-linear solver in it! So just write two functions:

• objective function and

• its gradient

As a quick example, you can see my article:

Logistic regression in F # using the Microsoft Solver Foundation

But you don’t need to know about logistic regression, in the article I also included a simpler example showing how to optimize the Rosenbrock function with 2 variables.

MSF also has a built-in language for defining C # using an implicit translation language function. [You can find an example in the MSF docs.]