How big should epsilon be when checking if a dot product is close to 0?

Question

How big should the check be if point-to-point is close to 0?

I am working on a raytracing project and I need to check if the dot product is 0. But that probably will never happen, so I want to count it 0 if its value is in a small area [-eps, + eps], but I not sure how big eps should be?

thanks

Answer 1

Since you describe this as part of a ray tracing project, your accuracy is most likely determined by the "world coordinates" of the scene or, possibly, even the coordinates of the screen into which they are translated. Any of them will give an acceptable goal for an absolute error in your calculation.

One could step back from this to the accuracy needed for the intermediate calculations that you do, for example, to form an internal product that is theoretically considered "zero." For example, you can find the shortest path (reflected light) between two smooth bodies, and the disappearance of the internal product (perpendicularity) gives the location of the point.

In this case, the scalar product may be quadratic in the unknowns (point location) that you are looking for. It is possible that unknowns form a “double root” (zero of multiplicity 2), which makes the location of this root additional, sensitive to the calculation of the internal product, equal to zero.

For such cases, you would like to get about twice as many “zero” digits in the internal product as the location accuracy is needed. In fact, the internal product changes very slowly with the location in the vicinity of the double root.

But your application may not be as sensitive; An analysis of the necessary algorithm is necessary to give you a good answer. As a rule, I make an internal product with double precision in order to get answers that can be reliable with respect to single precision, but it can be too expensive if ray tracing should be performed in real time.

Answer 2

There is no final answer. I use two approaches.

If you are only concerned about the floating-point error, you can use a rather small value comparable to the smallest floating-point number that the compiler can handle. In c / C ++, you can use the definitions given in float.h , like DBL_MIN , to check for these numbers. I would use a small number, for example 10. * DBL_MIN as the value for eps .

If the problem is not a rounding math error, I use a value that is small (say 1%) compared to the modulus of the smallest vector.