Note: MathJax 3, Hugo, and Netlify just aren’t playing nice for some reason, so pardon any \(\LaTeX\) spillage.
A probability distribution function (PDF) is part of the exponential family if it can be arranged into the form
$$ f(x|\theta) = h(x)c(\theta)\exp\left(\sum_{i=1}^kw_i(\theta)t_i(x)\right) $$
where \(\theta\) is the vector of parameters. The shape-scale parameterization of the Gamma distribution PDF takes the form
$$ f_X(x) = \frac{1}{\Gamma(k)\beta(k)}x^{k-1}e^{-\frac{x}{\beta}} $$
We use \(\beta\) to notate the rate parameter in the shape-rate parameterization of the Gamma PDF, while \(k\) is the symbol for the scale parameter.
Can we arrange that PDF into an exponential family form? Spoiler: Yes. Here, we demonstrate that a Gamma PDF given two unknown parameters, \(\beta\) and \(k\), belongs to the exponential family.
We start by re-arranging the Gamma distribution:
$$ \begin{align} f(x|k,\beta) &= \frac{1}{\Gamma(k)\beta(k)}x^{k-1}e^{-\frac{x}{\beta}}\\\ &= \frac{1}{\Gamma(k)\beta(k)}e^{(k-1)\ln(x)}e^{-\frac{x}{\beta}}\\\ &= \frac{1}{\Gamma(k)\beta(k)}e^{(k-1)\ln(x) - \frac{x}{\beta}} \end{align} $$
The log identity \(x^b = e^{b\ln(x)}\) is a very useful logarithmic identity to remember when trying to arrange PDFs into exponential family form.
Subsequently, let \(h(x) = I_{x>0}(x)\) (If you see that \(h(x) = 1\), that is a cue to use an indicator function that ranges through the support of \(x\) when shaping functions into exponential family form.) Hence, we assign pieces of the re-arranged Gamma PDF to their corresponding exponential family sections:
$$ \begin{align} h(x) &= I_{x>0}(x)\\\ c(k,\beta) &= \frac{1}{\Gamma(k)\beta(k)}\\\ w_1(k,\beta) &= k - 1\\\ w_2(k,\beta) &= -\frac{1}{\beta}\\\ t_1(x) &= \ln(x)\\\ t_2(x) &= x \end{align} $$
Hence, the Gamma distribution given unknown parameters \(\beta\) and \(k\) constituting a two-dimensional parameter vector \(\theta\) can be shown to be part of the exponential family. A similar process will apply for showing that a Gamma PDF with one unknown parameter, \(\beta\) or \(k\) is also part of the exponential family.
>> Home