The log-sum-exp trick

desp

The log-sum-exp trick

The log-sum-exp trick improves stability in exponentiation and logarithmic calculations by operating in the logarithmic domain, avoiding numerical overflow or underflow.

import numpy as np

The log1p function is a useful tool for calculating the logarithm of 1 + p, particularly when p is a small value. It helps avoid numerical instability issues that can arise when directly computing np.log(1 + p) when p is close to zero. By using log1p, you can achieve a more accurate and stable calculation of the logarithm.

x = np.array([1, 2, 3, 4, 5])
p = 1e-10
result = np.log(1 + p)  # Direct computation, can be unstable for small p
result_log1p = np.log1p(p)  # More stable computation using log1p
print(result)
print(result_log1p)

1.000000082690371e-10
9.999999999500001e-11

When dealing with small values like 1e-5, using the expression exp(p) - 1 can result in a loss of precision. In such cases, the function expm1(p) is a better alternative as it provides a more accurate result that preserves the precision of the small input value p.

result1 = np.exp(p) - 1
result2 = np.expm1(p)

print("Using exp(p) - 1:")
print(result1)

print("Using expm1(p):")
print(result2)

Using exp(p) - 1:
1.000000082740371e-10
Using expm1(p):
1.00000000005e-10

Using np.log1p to compute log(1 + p) provides a numerically stable and accurate result of 1e-10, even for small values of p.

x = 0
y = 10

result = np.multiply(x, np.log(y))

print(result)

0.0

To avoid numerical instability when computing x log(1 + p) with small values of x and p, it is advisable to use np.log1p instead of directly calculating np.log(1 + p). This ensures better accuracy and stability by preserving the precision of the small input p.

from scipy.special import xlog1py
from scipy.special import xlogy


x = 1
p = 1e-10

result = xlogy(x, 1 + p)
result1 = xlog1py(x, p)

print(result)
print(result1)

1.000000082690371e-10
9.999999999500001e-11