When are denormals really useful?

Question

Whenever I search for the term "denormal numbers" or "denormal", I only find ways to detect them and round them to zero. Apparently, they do not like anyone, because dealing with them carries a fine for execution.

And yet they are implemented everywhere. What for? If that were accurate, I would say that you need a larger float or reorder operations to avoid very small intermediate values. It's hard for me to believe that this little extra precision is really worth the precious clock cycles.

Are there any good reasons why denormal numbers will still be used? And if there is no substantial reason to have denormal numbers, why even use them? Only for compliance with IEEE754?)

Answer 1

In short, since gradual overflow preserves some useful mathematical identities (e.g. xy == 0 means x == y). Some explanations of why gradual overflow might be useful:

http://grouper.ieee.org/groups/754/faq.html#underflow

http://www.cs.berkeley.edu/~wkahan/ARITH_17U.pdf

And yes, in some cases they are underutilized due to poor application development, and the correct action is to fix the application. In other cases, applications that work correctly with a gradual overflow fail during a sharp overflow.

Moreover,

• In many cases, a slow but correct one is considered a better default than a fast but dangerous one.

• Since gradual underperformance is the default, Google discovers that people complain about it and want to disable it. If OTOH's abrupt downstream was defaulted, would you probably see more people complaining about mysterious numerical problems instead? Numerical programming is quite complicated as it is!

• Modern hw reduced the penalty for working with subnormal numbers. See http://www.agner.org/optimize/blog/read.php?i=142&v=t

Answer 2

Denormals are extremely useful; There are a number of useful error estimates for floating point calculations that are no longer true if you remove the denormals (most importantly, xy == 0 if and only if x == y ).

It is also important to remember that (a) denormals do not impose a fine on all equipment; There are systems that can process denormals at a speed (or very close to it), and (b) denormals only slow down your calculations if you really run into them. If you don’t use them, you don’t pay for them (and if you end up using them, then your result would most likely be wrong without them - if you just want to get the wrong answer as quickly as possible, you can replace all your calculation on return 0 ).