Normalization of a list of very small double numbers (probabilities)

Question

I am writing an algorithm where, given the model, I calculate the probabilities for the list of data sets, and then I have to normalize (to the likelihood) each of the likelihoods. Thus, it is possible that something like [0.00043, 0.00004, 0.00321] may be like [0.2, 0.03, 0.77]. My problem is that the probability of registration with which I work is rather small (for example, in the log space the values ​​are -269647.432, -231444.981, etc.). In my C ++ code, when I try to add two of them (taking their exponent), I get a response from "Inf". I tried adding them to log-space (summing / subtracting the log) , but came across the same problem again.

Can anyone share their expert opinion on this?

thanks

Answer 1

Assuming that the probability was calculated correctly, you can divide each of them by the highest probability. This can be done in the form of a logarithm by subtracting the largest logarithmic likelihood from each log-likelihood.

Then you can convert from logarithmic space. The largest will be 1.0, because its normalized log is 0. The smaller ones will be between 0 and 1.0 and presented as part of the largest.

Answer 2

This is a standard procedure. Numerically stable Matlab code:

LL = [ . . . ]; % vector of log-likelihoods M = max(LL); LL = LL - M; L = exp(LL); L = L ./ sum(L);