Gamma distribution
The gamma distribution is a common choice for modeling continuous quantities that are non-negative: dollar amounts, waiting times, etc. It is governed by two positive parameters: the shape \(\alpha>0\) and the rate \(\beta>0\). Don’t worry about interpreting what those words mean. They’re just labels.
Basic properties
The gamma distribution gets its name because it is related to the so-called gamma function, which appears in the normalizing constant:
\[ \Gamma(\alpha) = \int_0^\infty x^{\alpha-1}e^{-x}\,\text{d}x. \]
We have a lil’ math explainer where you can read more about it.
| Notation | \(X\sim\text{Gamma}(\alpha,\,\beta)\) |
| Range | \([0,\,\infty)\) |
| Parameter space | \(\alpha,\,\beta>0\) |
| \(f(x)=\begin{cases}\frac{\beta^\alpha}{\Gamma(\alpha)}x^{\alpha-1}e^{-\beta x} &x\geq 0\\0 & x<0.\end{cases}\) | |
| MGF | \(M(t)=\left(\frac{\beta}{\beta-t}\right)^\alpha\), \(t<\beta\) |
| Expectation | \(\alpha/\beta\) |
| Variance | \(\alpha/\beta^2\) |
Special cases
Particular special cases of the gamma distribution get special nicknames:
- If \(X\sim\text{Gamma}(1,\,\beta)\), then \(X\) has the exponential distribution, denoted \(X\sim\text{Exp}(\beta)\) This special case is nice because we get closed-form expressions for the CDF and the quantile function:
\[ \begin{aligned} f(x)&=\beta e^{-\beta x}\\ F(x)&=1-e^{-\beta x}\\ F^{-1}(p)&=-\frac{\ln(1-p)}{\beta}. \end{aligned} \]
If \(X\sim\text{Gamma}(\nu/2,\, 1/2)\), then \(X\) has the chi-squared distribution, denoted \(X\sim\chi^2_\nu\). In this context we call the parameter \(\nu>0\) the degrees of freedom of the chi-squared distribution.
If \(X\sim\text{Gamma}(k,\,\beta)\) for some integer \(k\), then \(X\) has the Erlang distribution, denoted \(X\sim\text{Erlang}(k,\, \beta)\)
R commands
Here is the documentation for the suite of commands that let you work with the gamma distribution in R:
Play around!
#| '!! shinylive warning !!': |
#| shinylive does not work in self-contained HTML documents.
#| Please set `embed-resources: false` in your metadata.
#| standalone: true
#| viewerHeight: 700
library(shiny)
ui <- fluidPage(
titlePanel("Gamma distribution CDF and PDF"),
sidebarLayout(
sidebarPanel(
sliderInput("alpha", "Shape (α):",
min = 0.5, max = 5, value = 2, step = 0.1),
sliderInput("beta", "Rate (β):",
min = 0.2, max = 2, value = 0.5, step = 0.05),
hr(),
verbatimTextOutput("moments")
),
mainPanel(
plotOutput("gammaPlot", height = "600px")
)
)
)
server <- function(input, output) {
output$moments <- renderText({
alpha <- input$alpha
beta <- input$beta
mu <- alpha / beta
sd <- sqrt(alpha) / beta
paste0("E(X) = ", round(mu, 3),
"\nsd(X) = ", round(sd, 3))
})
output$gammaPlot <- renderPlot({
alpha <- input$alpha
beta <- input$beta
# Fixed plotting range
x_min <- -2
x_max <- 25
x <- seq(x_min, x_max, length.out = 2000)
# PDF and CDF
pdf_vals <- ifelse(x >= 0, dgamma(x, shape = alpha, rate = beta), 0)
cdf_vals <- ifelse(x >= 0, pgamma(x, shape = alpha, rate = beta), 0)
mu <- alpha / beta
# Fixed y-limits
pdf_ylim <- c(0, 0.8)
cdf_ylim <- c(0, 1)
par(mfrow = c(2, 1), mar = c(4, 4, 2, 1))
# --- CDF ---
plot(x, cdf_vals, type = "l", lwd = 2, col = "blue",
xlim = c(x_min, x_max), ylim = cdf_ylim,
main = "Cumulative Distribution Function (CDF)",
xlab = "", ylab = "F(x)")
abline(h = c(0, 1), col = "gray80", lty = 2)
#abline(v = mu, col = "gray60", lty = 3)
# --- PDF ---
plot(x, pdf_vals, type = "l", lwd = 2, col = "darkred",
xlim = c(x_min, x_max), ylim = pdf_ylim,
main = "Probability Density Function (PDF)",
xlab = "x", ylab = "f(x)")
#abline(v = mu, col = "gray60", lty = 3)
# Label E(X) with mtext along x-axis
if (mu >= x_min && mu <= x_max) {
mtext("E(X)",
side = 1, line = 2.2, at = mu, cex = 1.0)
}
})
}
shinyApp(ui = ui, server = server)
Derivations
Let \(X\sim\text{Exp}(\beta)\), and recall that this means \(X\sim\text{Gamma}(\alpha=1,\, \beta)\). So the pdf is \(f_X(x)=\beta\exp(-\beta x)\) for \(x>0\), and the cdf is
\[ \begin{aligned} F_X(x) &= \int_0^x\beta e^{-\beta t}\,\text{d} t = [ -e^{\beta t} ]_0^x = 1-e^{-\beta x},\quad x>0. \end{aligned} \]
For some \(y\in(0,\, 1)\), we see that
\[ \begin{align*} y&=1-e^{-\beta x}\\ e^{-y}&=1-y\\ -\beta x&=\ln(1-y)\\ x&=\frac{-\ln(1-y)}{\beta}, \end{align*} \]
and so the quantile function is
\[ F_X^{-1}(y)=-\frac{\ln(1-y)}{\beta}, \]
and the median of the exponential distribution is
\[ F_X^{-1}(0.5)=-\frac{\ln(1-0.5)}{\beta}=-\frac{\ln(0.5)}{\beta}=\frac{\ln 2}{\beta}. \]
This is classic “massage and squint.” Let \(t\in(-\infty,\,\beta)\). Then
\[ \begin{align*} M_X(t)&=E\left(e^{tX}\right)\\ &=\int_0^\infty e^{tx}\frac{\beta^\alpha}{\Gamma(\alpha)}x^{\alpha-1}e^{-\beta x}\,\text{d} x && \text{LOTUS}\\ &=\frac{\beta^\alpha}{\Gamma(\alpha)}\int_0^\infty x^{\alpha-1}e^{tx}e^{-\beta x}\,\text{d} x && \text{pull out constant}\\ &=\frac{\beta^\alpha}{\Gamma(\alpha)}\int_0^\infty x^{\alpha-1}e^{tx-\beta x}\,\text{d} x&& \text{combine base-e terms}\\ ` &=\frac{\beta^\alpha}{\Gamma(\alpha)}\int_0^\infty x^{\alpha-1}e^{(t-\beta) x}\,\text{d} x&& \text{factor out }x\\ ` &=\frac{\beta^\alpha}{\Gamma(\alpha)}\int_0^\infty x^{\alpha-1}e^{-(\beta-t) x}\,\text{d} x&& \text{factor out -1}\\ &=\frac{\beta^\alpha}{\Gamma(\alpha)}\int_0^\infty \underbrace{x^{\alpha-1}e^{-(\beta-t)x}}_{\text{kernel of Gamma}(\alpha,\,\beta-t)}\,\text{d} x && \text{recognize}\\ &=\frac{\beta^\alpha}{{\Gamma(\alpha)}}\frac{{\Gamma(\alpha)}}{(\beta-t)^\alpha}\\ &=\left(\frac{\beta}{\beta-t}\right)^\alpha. \end{align*} \]
In order to invoke the gamma density kernel, we require that \(t<\beta\) so that the rate parameter of the implied gamma is positive.