Chapter 5 Binomial & Poisson Distributions

The Poisson distribution is suitable to model outcomes that represent numbers of events or occurrences. When the recorded data is in the form \(\lbrace (y_i,x_i, n_i)\rbrace_{i=1,\cdots,N}\) where the outcome \(y_i\) is the number of successes amongst \(n_i\) trials, the binomial distribution seems the most suitable distribution to adapt to different exposures \(n_i\) because this information appears explicitly in the mathematical definition of the Binomial distribution. The Poisson distribution can also be adapted to deal with various exposures. The relation between Binomial and Poisson distributions is then investigated when \(n\rightarrow +\infty\).

5.1 Offset and Exposure

Let considers the Poisson distribution for modelling a count \(y\) (out of \(n\) trial) such that:

• \(y\sim\mathrm{Poisson}(\lambda)\) so the distribution for \(y\) given parameter \(\lambda\) is: \[p_{y|\lambda}(y|\lambda)= \frac{\lambda^y\ \exp(-\lambda)}{y!}\] In this case, \(\mathbb{E}[y]=\lambda\)

• Remember that when considering the Binomial distribution \(y\sim\mathrm{Bin}(n,\theta)\) then \(\mathbb{E}[y]=n\theta\) where \(\theta\) is a proportion between 0 and 1.

We have the following correspondence with the proportion \(\theta\): \[\mathbb{E}[y]=\lambda=n\theta\] or \(\theta=\frac{\lambda}{n}\) and the link function \(g\) is applied to the proportion as follow: \[g(\theta)=g\left(\frac{\lambda}{n}\right)=x^{T}\beta\] The canonical link function for the Poisson distribution is the log function so: \[\log\left(\frac{\lambda}{n}\right)=x^{T}\beta \quad \text{or} \quad \log(\lambda)=\log(n)+x^{T}\beta\]

Definition 5.1 (Poisson Regression: offset and exposure) When using the Poisson distribution, the population \(n\) is called the exposure (e.g. for the Beetle case study, \(n\) beetles are exposed to a toxic substance) while \(\log (n)\) is called the offset. The log is used as a link function such that \[\log\left(\mathbb{E}[y]\right)=\log(\lambda)=\log(n)+x^{T}\beta\] or \[\theta=\frac{\lambda}{n}= \ \exp(x^{T}\beta)\] Offsets are used to correct for the group size or differing time periods of observation when using the Poisson distribution.

5.2 Relation between Poisson and Binomial distributions when \(n\rightarrow +\infty\)

Show that \[\lim_{n\rightarrow\infty}\ \underbrace{\frac{n!}{(n-y)! y!}\ \theta^{y} (1-\theta)^{n-y}}_{\text{Binomial}(n,\theta)}= \underbrace{\frac{\lambda^y\ \exp(-\lambda)}{y!}}_{\text{Poisson}(\lambda)}\] with \(\lambda=n\theta\).

Changing \(\theta=\frac{\lambda}{n}\) in the binomial distribution: \[\frac{n!}{(n-y)! y!} \left(\frac{\lambda}{n}\right)^{y} \left(1- \frac{\lambda}{n}\right)^{n-y} =\underbrace{\frac{n!}{(n-y)! \ n^y}}_{A_n}\ \frac{\lambda^y}{y!}\ \underbrace{\left(1- \frac{\lambda}{n}\right)^{n}}_{B_n} \ \left(1- \frac{\lambda}{n}\right)^{-y}\] We see that \(\lim_{n\rightarrow\infty}\ \left(1- \frac{\lambda}{n}\right)^{-y} =1\), hence \[\lim_{n\rightarrow\infty}\ \frac{n!}{(n-y)! y!} \left(\frac{\lambda}{n}\right)^{y} \left(1- \frac{\lambda}{n}\right)^{n-y} = \frac{\lambda^y}{y!} \ \lim_{n\rightarrow\infty}\ A_n \ B_n\]

• \(\lim_{n\rightarrow\infty}\ A_n\)? \[A_n=\frac{n!}{(n-y)! n^y}=\frac{n\ (n-1)\ \cdots (n-y+1)}{n\times n\times \cdots \times n}=1 \ \left(1-\frac{1}{n}\right)\ \left(1-\frac{2}{n}\right) \cdots \ \left(1-\frac{y-1}{n}\right)\] so \[\lim_{n\rightarrow\infty}\ A_n =1\]

• \(\lim_{n\rightarrow\infty}\ \ B_n\)? we know \((1+s)^n=\sum_{k=0}^n \left( \begin{array}{c} n\\ k\\ \end{array} \right) \ s^k\) hence \[\left(1-\frac{\lambda}{n}\right)^n = \sum_{k=0}^n\ \underbrace{\frac{n!}{(n-k)!\ n^k}}_{A_n}\frac{(-\lambda)^k}{k!}\] The Taylor expansion of \(\exp(-\lambda)\) is: \[\exp(-\lambda)= \sum_{k=0}^{\infty} \frac{(-\lambda)^k}{k!}\] So \[\lim_{n\rightarrow\infty}\ \ B_n = \exp(-\lambda)\]

Hence with \(\lambda=n\theta\) \[\lim_{n\rightarrow\infty}\ \underbrace{\frac{n!}{(n-y)! y!}\ \theta^{y} (1-\theta)^{n-y}}_{\text{Binomial}(n,\theta)}= \underbrace{\frac{\lambda^y\ \exp(-\lambda)}{y!}}_{\text{Poisson}(\lambda)}\]