Distributions of Random Variables
Normal distribution
Usually also called Gaussian Distribution
Unimodal and symmetric, bell shaped curve
Many variables are nearly normal, but none are exactly normal
Denoted as \(N(\mu,\sigma^2)\) or \(N(\mu, \sigma)\) \(\rightarrow\) Normal with mean \(\mu\) and variance \(\sigma^2\)
Normal distributions with different parameters
\(\mu\): mean, \(\sigma\): standard deviation \[N(\mu = 0, \sigma = 1) \hspace{1.4cm} N(\mu = 19, \sigma = 4)\] ![image]()
![image]()
Normal Distribution
Very common distribution for modeling phenomena like heights, test scores, errors, etc.
Standard Normal Distribution
A random variable having a normal distribution with a mean of 0 and a standard deviation of 1 is said to have a standard normal probability distribution.
The letter Z is used to designate the standard normal random variable.
Converting to the Standard Normal Distribution \[Z = \frac{X - \mu}{ \sigma}\]
Probability density function: \[f(z) = \frac{1}{\sqrt{2\pi}} e^{-\frac{x^2}{2}}\]
Mean: \[E(Z) = 0\]
Variance: \[\text{Var}(Z) = 1\]
Example: Pep Zone
Pep Zone sells auto parts and supplies including a popular multi-grade motor oil. When the stock of this oil drops to 80 litres, a replenishment order is placed.
The store manager is concerned that sales are being lost due to stockouts while waiting for an order.
It has been determined that demand during replenishment lead-time is normally distributed with a mean of 60 litres and a standard deviation of 24 litres.
The manager would like to know the probability of a stockout, P(X > 80).
Example: Pep Zone (using tables)
Solving for the Stockout Probability
Example: Pep Zone (using tables)
Solving for the Stockout Probability
Step 1: Convert x to the standard normal distribution. \[z = \frac{X - \mu}{\sigma}= \frac{80 - 60}{24} = 0.83\]
Step 2: Find the area under the standard normal curve to the left of z = 0.83. \[\mathbb{P}(Z \leq 0.83) = 0.7967\]
Step 3: Compute the area under the standard normal curve to the right of Z = 0.83. \[\mathbb{P}(Z > 0.83) = 1- \mathbb{P}(Z \leq 0.83) = 1 - 0.7967 = 0.2033\]
Example: Quality control
At Heinz ketchup factory the amounts which go into bottles of ketchup are supposed to be normally distributed with mean 36 oz. and standard deviation 0.11 oz. Once every 30 minutes a bottle is selected from the production line, and its contents are noted precisely. If the amount of ketchup in the bottle is below 35.8 oz. or above 36.2 oz., then the bottle fails the quality control inspection. What percent of bottles have less than 35.8 ounces of ketchup? Let \(X\) = amount of ketchup in a bottle: \(X \sim N(\mu = 36, \sigma = 0.11)\)
\[Z = \frac{35.8 - 36}{0.11} \approx -1.82\]
Finding the exact probability - using Python
> from scipy.stats import norm
> norm.cdf(-1.82,loc=0, scale=1)
0.03437950244588998
\(\:\)
OR
\(\:\)
> norm.cdf(35.8, loc = 36, scale = 0.11)
0.03451817399720564
Practice
What percent of bottles pass the quality control inspection?
1.82%
3.44%
6.88%
93.12%
96.56%
\[\begin{aligned}
Z_{35.8} &=& \frac{35.8 - 36}{0.11} = -1.82 \\
Z_{36.2} &=& \frac{36.2 - 36}{0.11} = 1.82 \\
P(35.8 < X < 36.2) &=& P(-1.82 < Z < 1.82) = 0.9656 - 0.0344 = 0.9312
\end{aligned}\]
Bernoulli distribution: Milgram experiment
Stanley Milgram, a Yale University psychologist, conducted a series of experiments on obedience to authority starting in 1963.
Experimenter (E) orders the teacher (T), the subject of the experiment, to give severe electric shocks to a learner (L) each time the learner answers a question incorrectly.
The learner is actually an actor, and the electric shocks are not real, but a prerecorded sound is played each time the teacher administers an electric shock.
These experiments measured the willingness of study participants to obey an authority figure who instructed them to perform acts that conflicted with their personal conscience.
Milgram found that about 65% of people would obey authority and give such shocks.
Over the years, additional research suggested this number is approximately consistent across communities and time.
Bernouilli random variables
Each person in Milgram’s experiment can be thought of as a trial.
A person is labelled a success if she refuses to administer a severe shock, and failure if she administers such shock.
Since only 35% of people refused to administer a shock, probability of success is \(p = 0.35\).
When an individual trial has only two possible outcomes, it is called a Bernoulli random variable.
Geometric distribution
Dr. Smith wants to repeat Milgram’s experiments but she only wants to sample people until she finds someone who will not inflict a severe shock. What is the probability that she stops after the first person?
\[P(1^{st}~person~refuses) = 0.35\]
... the third person? \[P(1^{st}~and~2^{nd}~shock,~3^{rd}~refuses) = \begin{array}{c}\underline{S}\\0.65\end{array} \times \begin{array}{c}\underline{S}\\0.65\end{array} \times \begin{array}{c}\underline{R}\\0.35\end{array} = 0.65^2 \times 0.35 \approx 0.15\]
... the tenth person? \[P(9~shock,~10^{th}~refuses) = \underbrace{\begin{array}{c}\underline{S}\\0.65\end{array} \times \cdots \times \begin{array}{c}\underline{S}\\0.65\end{array}}_{9~of~these} \times \begin{array}{c}\underline{R}\\0.35\end{array} = 0.65^9 \times 0.35 \approx 0.0072\]
Geometric distribution (cont.)
Geometric distribution describes the waiting time until a success for independent and identically distributed (iid) Bernouilli random variables.
\(\:\)
\(\:\)
Geometric probabilitiesIf \(p\) represents probability of success, \((1-p)\) represents probability of failure, and \(n\) represents number of independent trials \[\mathbb{P}(success~on~the~n^{th}~trial) = (1-p)^{n-1} p\]
Expected value
How many people is Dr. Smith expected to test before finding the first one that refuses to administer the shock?
The expected value, or the mean, of a geometric distribution is defined as \(\frac{1}{p}\). \[\mu = \frac{1}{p} = \frac{1}{0.35} \approx 2.86\]
She is expected to test 2.86 people before finding the first one that refuses to administer the shock.
But how can she test a non-whole number of people?
Expected value and its variability
Mean and standard deviation of geometric distribution \[\mu = \frac{1}{p} \qquad \qquad \sigma = \sqrt{\frac{1-p}{p^2}}\]
Going back to Dr. Smith’s experiment:
\[\sigma = \sqrt{\frac{1-p}{p^2}} = \sqrt{\frac{1-0.35}{0.35^2}} \approx 2.3\]
Dr. Smith is expected to test 2.86 people before finding the first one that refuses to administer the shock, give or take 2.3 people.
These values only make sense in the context of repeating the experiment many many times.
Binomial distribution
Suppose we randomly select 4 individuals to participate in this experiment. What is the probability that exactly 1 of them will refuse to administer the shock? Let’s call these people Allen (A), Bob (B), Cindy (C), and Douglas (D). Each one of the four scenarios below will satisfy the condition of “exactly 1 of them refuses to administer the shock":
Scenario 1: \(\begin{array}{c}\underline{0.35}\\\text{(A) refuse}\end{array} \times \begin{array}{c}\underline{0.65}\\\text{(B) shock}\end{array} \times \begin{array}{c}\underline{0.65}\\\text{(C) shock}\end{array} \times \begin{array}{c}\underline{0.65}\\\text{(D) shock}\end{array} = 0.0961\)
Scenario 2: \(\begin{array}{c}\underline{0.65}\\\text{(A) shock}\end{array} \times \begin{array}{c}\underline{0.35}\\\text{(B) refuse}\end{array}\times \begin{array}{c}\underline{0.65}\\\text{(C) shock}\end{array} \times \begin{array}{c}\underline{0.65}\\\text{(D) shock}\end{array} = 0.0961\)
Scenario 3: \(\begin{array}{c}\underline{0.65}\\\text{(A) shock}\end{array} \times \begin{array}{c}\underline{0.65}\\\text{(B) shock}\end{array} \times \begin{array}{c}\underline{0.35}\\\text{(C) refuse}\end{array}\times \begin{array}{c}\underline{0.65}\\\text{(D) shock}\end{array} = 0.0961\)
Scenario 4: \(\begin{array}{c}\underline{0.65}\\\text{(A) shock}\end{array} \times \begin{array}{c}\underline{0.65}\\\text{(B) shock}\end{array} \times \begin{array}{c}\underline{0.65}\\\text{(C) shock}\end{array} \times \begin{array}{c}\underline{0.35}\\\text{(D) refuse}\end{array} = 0.0961\)
The probability of exactly one 1 of 4 people refusing to administer the shock is the sum of all of these probabilities. \[0.0961+ 0.0961 + 0.0961 + 0.0961 = 4 \times 0.0961 = 0.3844\]
Binomial distribution
The question from the prior slide asked for the probability of given number of successes, \(k\), in a given number of trials, \(n\), (\(k = 1\) success in \(n = 4\) trials), and we calculated this probability as \[\#~of~scenarios \times \mathbb{P}(single~scenario)\]
\(\#~of~scenarios\): counting rules, combinations, \[{n \choose k} = \frac{n!}{k! (n - k)!}\]
\(\mathbb{P}(single~scenario) = p^k~(1-p)^{(n-k)}\), probability of success to the power of number of successes, probability of failure to the power of number of failures
\(\:\)
\(\:\)
The Binomial distribution describes the probability of having exactly \(k\) successes in \(n\) independent Bernoulli trials with probability of success \(p\).
Binomial distribution (cont.)
Binomial probabilities If \(p\) represents probability of success, \((1-p)\) represents probability of failure, \(n\) represents number of independent trials, and \(k\) represents number of successes \[P(k~successes~in~n~trials) = {n \choose k}~p^k~(1-p)^{(n-k)}\]
\(\:\)
\(\:\)
There are several characteristics of binomial distribution,
The experiment consists of a sequence of \(n\) identical trials.
Two outcomes, success, and failure, are possible on each trial.
The probability of a success does not change from trial to trial.
The trials are independent.
Binomial distribution (cont.)
We let \(X\) denote the number of successes occurring in the \(n\) trials.
\(X \sim Bin(n, p)\) or \(B(n, p)\)
Probability mass function: \[P(X=k) = \binom{n}{k} p^k (1-p)^{n-k}\]
First and second moment:
Mean: \[\mathbb{E}(X) = np\]
Variance: \[\text{Var}(X) = \mathbb{E}[X -\mathbb{E}(X)]^2 = np(1-p)\]
Standard deviation: \[\sigma(X) = \sqrt{\text{Var}}(X) = \sqrt{np(1-p)}\]
Distributions of number of successes
Hollow histograms of samples from the binomial model where \(p = 0.10\) and \(n = 10\), \(30\), \(100\), and \(300\). What happens as \(n\) increases?
How large is large enough?
The sample size is considered large enough if the expected number of successes and failures are both at least 10 (in some cases, 5 is also fine). \[np \ge 10 \qquad \text{ and } \qquad n(1-p) \ge 10\]
Normal approximation to the binomial
When the sample size is large enough, the binomial distribution with parameters \(n\) and \(p\) can be approximated by the normal model with parameters \(\mu = np\) and \(\sigma = \sqrt{np(1-p)}\).
In the case of a scenario \(n = 245\) and \(p = 0.25\). \[\mu = 245 \times 0.25 = 61.25 \qquad \sigma = \sqrt{245 \times 0.25 \times 0.75} = 6.78\]
\(Bin(n = 245, p = 0.25) \approx N(\mu = 61.25, \sigma = 6.78)\).
Compare the exact probabilities - using Python
> from scipy.stats import norm
> from scipy.stats import binom
\(\:\)
>>> binom.cdf(70, 245, 0.25)
0.912329297526947
>>> norm.cdf(70, 61.25, 6.78)
0.9015719349740392
The normal approximation breaks down on small intervals
The normal approximation to the binomial distribution tends to perform poorly when estimat- ing the probability of a small range of counts, even when the conditions are met.
This approximation for intervals of values is usually improved if cutoff values are extended by 0.5 in both directions.
The tip to add extra area when applying the normal approximation is most often useful when examining a range of observations. While it is possible to also apply this correction when computing a tail area, the benefit of the modification usually disappears since the total interval is typically quite wide.
Poisson Distribution
The Poisson distribution is a discrete probability distribution that expresses the probability of a given number of events occurring within a fixed interval of time or space.
The rate for a Poisson distribution is the average rate at which events happen and is typically denoted by \(\lambda\).
Using the rate, we can describe the probability of observing exactly \(k\) rare events in a single unit of time.
Probability mass function: \[\mathbb{P}(X=k) = \frac{\lambda^k e^{-\lambda}}{k!}\]
Mean: \[E(X) = \lambda\]
Variance: \[\text{Var}(X) = \lambda\]
Used for modelling rare events like number of phone calls, web hits, or accidents per hour.
Poisson Distribution: How to understand PMF
Here’s the formula again: \[\mathbb{P}(X=k) = \frac{\lambda^k e^{-\lambda}}{k!}\]
\(\lambda\): This is the average rate at which events happen. For example, if on average 3 customers arrive at a bank every hour, \(\lambda\) would be 3.
\(\lambda^k\): This term accounts for those occasions when the event happens exactly \(k\) times instead of the average \(\lambda\) times.
It basically models the likelihood of that specific scenario—when exactly \(k\) events happen, even though the average rate is \(\lambda\).
For example, if 3 customers usually arrive every hour (\(\lambda=3\)), \(\lambda^k\) would model scenarios like what happens if exactly 2 customers arrive, or exactly 4 customers arrive, and so on.
\(e^{-\lambda}\):It’s a kind of "balancing factor" to make the probabilities add up to 1.
\(k!\): This is the product of all positive integers up to \(k\). It adjusts for the number of ways \(k\) events could happen. It makes sure we’re not overcounting different arrangements of the same outcome.
Poisson Distribution vs Binomial Distribution
Credit Card Fraud: Imagine a large bank that has issued 1 million credit cards. The chance of any individual card being used fraudulently is quite low—let’s say 0.1% over a given month. That’s a classic Binomial setting: each card either experiences fraudulent activity (success) or it doesn’t (failure).
Binomial: consider each card as a Bernoulli trial with two possible outcomes: fraud (with a probability of 0.1%) or no fraud (with a probability of 99.9%).
Poisson:
The average rate of fraud can be calculated as: \[\lambda =n \times p=1,000,000 \times 0.001=1000\] So, on average, the bank would expect to see 1000 instances of fraudulent activity per month across all cards.
Poisson Distribution vs Binomial Distribution
Why Poisson?
Computational Simplicity: Binomial calculations with such a large \(n\) would be computationally expensive, whereas Poisson only involves dealing with \(\lambda\), which is far simpler.
Good for Rare Events: The Poisson distribution is well-suited for modelling rare events and can offer insights into how the bank should set up its fraud detection algorithms, what kind of resources it would need to handle fraud cases, etc.
Flexible Timeframes: The Poisson model allows the bank to easily adjust the timeframe. Instead of a month, they could look at fraudulent rates over a day, a week, or any other time period simply by adjusting \(\lambda\).
Resource Allocation: Knowing the average rate of frauds allows the bank to optimally allocate resources for fraud detection and customer service. For example, if they expect around 1000 frauds, they can gauge how many staff they’ll need to handle this volume of cases.
The Poisson distribution is a limiting case of a Binomial distribution!
Poisson Distribution vs Normal Distribution
![]()
https://www.geeksforgeeks.org/poisson-distribution/
When \(\lambda\) is large, a Poisson distribution with parameter \(\lambda\) can be approximated by a Normal distribution with mean \(\mu = \lambda\) and variance \(\sigma^2 = \lambda\).
\[Poisson(\lambda) \sim N(\mu=\lambda, \sigma^2 = \lambda)\]
Log-normal Distribution
The log-normal distribution results from taking the exponential of a normally distributed random variable.
A random variable \(X\) is said to follow a log-normal distribution if \(\ln(X)\) follows a normal distribution.
Probability density function: \[f(x) = \frac{1}{x\sigma\sqrt{2\pi}} e^{-\frac{(\ln x - \mu)^2}{2\sigma^2}}, x > 0\]
Mean: \[E(X) = e^{\mu + \frac{\sigma^2}{2}}\]
Variance: \[\text{Var}(X) = (e^{\sigma^2} - 1)e^{2\mu + \sigma^2}\]
Used to model positive skewed distributions, for example income, size of cities, etc.
Log-normal Distribution: Characteristics
Non-Negativity: A key feature is that a Log-Normally distributed variable takes only positive values, making it useful for modelling quantities like prices, incomes, and other financial metrics that cannot be negative.
Skewness: Unlike the Normal distribution, the Log-Normal distribution is not symmetric. It is skewed to the right, meaning it has a longer right tail.
Multiplicative Effects: \[\ln(X_1 \times X_2) = \ln(X_1) + \ln(X_2)\] The Log-Normal distribution is often used in contexts where the quantity of interest can be viewed as the result of multiplicative effects of several independent variables. This contrasts with the additive effects often assumed in Normal distributions.
Log-normal Distribution: Applications
The Log-Normal distribution is often used when your data are the result of compound, independent processes, especially if those processes multiply your data value.
For example, the return on a stock is often modelled as a percentage change, which is a multiplicative factor.
Over time, these factors multiply together to give you the stock’s price, making a Log-Normal distribution a natural model.
Further reasons:
Positivity: Stock prices are always positive.
Compounding Returns: Stock returns often compound over time, meaning that returns are multiplicative.
Volatility: Stock prices are influenced by a myriad of factors, many of which can be thought of as multiplicative. This can include earnings, interest rates, and broader economic indicators, which all contribute to the stock’s volatility—a key parameter in the Log-Normal distribution.
Black-Scholes-Merton model
= 
- 
