# Arranging a Two-Parameter Gamma Distribution into Exponential Family Form

## 2020/12/23

Tags: statistics math

Note: MathJax 3, Hugo, and Netlify just aren’t playing nice for some reason, so pardon any $$\LaTeX$$ spillage.

A probability distribution function (PDF) is part of the exponential family if it can be arranged into the form

$$f(x|\theta) = h(x)c(\theta)\exp\left(\sum_{i=1}^kw_i(\theta)t_i(x)\right)$$

where $$\theta$$ is the vector of parameters. The shape-scale parameterization of the Gamma distribution PDF takes the form

$$f_X(x) = \frac{1}{\Gamma(k)\beta(k)}x^{k-1}e^{-\frac{x}{\beta}}$$

We use $$\beta$$ to notate the rate parameter in the shape-rate parameterization of the Gamma PDF, while $$k$$ is the symbol for the scale parameter.

Can we arrange that PDF into an exponential family form? Spoiler: Yes. Here, we demonstrate that a Gamma PDF given two unknown parameters, $$\beta$$ and $$k$$, belongs to the exponential family.

We start by re-arranging the Gamma distribution:

\begin{align} f(x|k,\beta) &= \frac{1}{\Gamma(k)\beta(k)}x^{k-1}e^{-\frac{x}{\beta}}\\\ &= \frac{1}{\Gamma(k)\beta(k)}e^{(k-1)\ln(x)}e^{-\frac{x}{\beta}}\\\ &= \frac{1}{\Gamma(k)\beta(k)}e^{(k-1)\ln(x) - \frac{x}{\beta}} \end{align}

The log identity $$x^b = e^{b\ln(x)}$$ is a very useful logarithmic identity to remember when trying to arrange PDFs into exponential family form.

Subsequently, let $$h(x) = I_{x>0}(x)$$ (If you see that $$h(x) = 1$$, that is a cue to use an indicator function that ranges through the support of $$x$$ when shaping functions into exponential family form.) Hence, we assign pieces of the re-arranged Gamma PDF to their corresponding exponential family sections:

\begin{align} h(x) &= I_{x>0}(x)\\\ c(k,\beta) &= \frac{1}{\Gamma(k)\beta(k)}\\\ w_1(k,\beta) &= k - 1\\\ w_2(k,\beta) &= -\frac{1}{\beta}\\\ t_1(x) &= \ln(x)\\\ t_2(x) &= x \end{align}

Hence, the Gamma distribution given unknown parameters $$\beta$$ and $$k$$ constituting a two-dimensional parameter vector $$\theta$$ can be shown to be part of the exponential family. A similar process will apply for showing that a Gamma PDF with one unknown parameter, $$\beta$$ or $$k$$ is also part of the exponential family.

>> Home