Rectified Gaussian distribution

Not to be confused with truncated Gaussian distribution.

In probability theory, the rectified Gaussian distribution is a modification of the Gaussian distribution when its negative elements are reset to 0 (analogous to an electronic rectifier). It is essentially a mixture of a discrete distribution (constant 0) and a continuous distribution (a truncated Gaussian distribution with interval $(0,\infty)$).

Density function

The probability density function of a rectified Gaussian distribution, for which random variables X having this distribution are displayed as $X \sim \mathcal{N}^{\textrm{R}}(\mu,\sigma^2)$, is given by

$f(x;\mu,\sigma^2) =\Phi(-\frac{\mu}{\sigma})\delta(x)+ \frac{1}{\sqrt{2\pi\sigma^2}}\; e^{ -\frac{(x-\mu)^2}{2\sigma^2}}\textrm{U}(x).$

A comparison of Gaussian distribution, rectified Gaussian distribution, and truncated Gaussian distribution.

Here, $\Phi(x)$ is the cumulative distribution function (cdf) of the standard normal distribution:

$\Phi(x) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^x e^{-t^2/2} \, dt \quad x\in\mathbb{R},$

$\delta(x)$ is the Dirac delta function

$\delta(x) = \begin{cases} +\infty, & x = 0 \\ 0, & x \ne 0 \end{cases}$

and, $\textrm{U}(x)$ is the unit step function:

$\textrm{U}(x)=\begin{cases} 0, & x \leq 0, \\ 1, & x > 0. \end{cases}$

Alternative form

Often, a simpler alternative form is to consider a case, where,

$s\sim\mathcal{N}(\mu,\sigma^2),x=\textrm{max}(0,s),$

then,

$x\sim\mathcal{N}^{\textrm{R}}(\mu,\sigma^2)$

Application

A rectified Gaussian distribution is semi-conjugate to the Gaussian likelihood, and it has been recently applied to factor analysis, or particularly, (non-negative) rectified factor analysis. Harva proposed a variational learning algorithm for the rectified factor model, where the factors follow a mixture of rectified Gaussian; and later Meng proposed an infinite rectified factor model coupled with its Gibbs sampling solution, where the factors follow a Dirichlet process mixture of rectified Gaussian distribution, and applied it in computational biology for reconstruction of gene regulatory network.