Moment Generating Functions: 13 Important Facts

Moment generating function    

Moment generating function is very important function which generates the moments of random variable which involve mean, standard deviation and variance etc., so with the help of moment generating function only, we can find basic moments as well as higher moments, In this article we will see moment generating functions for the different discrete and continuous random variables. since the Moment generating function(MGF) is defined with the help of mathematical expectation denoted by M(t) as

gif

and using the definition of expectation for the discrete and continuous random variable this function will be

gif.latex?M%28t%29%3D%5Cleft%5C%7B%5Cbegin%7Barray%7D%7Bll%7D%20%5Csum %7Bx%7D%20e%5E%7Bt%20x%7D%20p%28x%29%20%26%20%5Ctext%20%7B%20if%20%7D%20X%20%5Ctext%20%7B%20is%20discrete%20with%20mass%20function%20%7D%20p%28x%29%20%5C%5C%20%5Cint %7B

which by substituting the value of t as zero generates respective moments. These moments we have to collect by differentiating this moment generating function for example for first moment or mean we can obtain by differentiating once as

gif

This gives the hint that differentiation is interchangeable under the expectation and we can write it as

gif

and

gif

if t=0 the above moments will be

gif

and

gif

In general we can say that

gif

hence

gif

Moment generating function of Binomial distribution||Binomial distribution moment generating function||MGF of Binomial distribution||Mean and Variance of Binomial distribution using moment generating function

The Moment generating function for the random variable X which is Binomially distribution will follow the probability function of binomial distribution with the parameters n and p as

gif

which is the result by binomial theorem, now differentiating and putting the value of t=0

gif

which is the mean or first moment of binomial distribution similarly the second moment will be

gif

so the variance of the binomial distribution will be

gif

which is the standard mean and variance of Binomial distribution, similarly the higher moments also we can find using this moment generating function.

Moment generating function of Poisson distribution||Poisson distribution moment generating function||MGF of Poisson distribution||Mean and Variance of Poisson distribution using moment generating function

 If we have the random variable X which is Poisson distributed with parameter Lambda then the moment generating function for this distribution will be

gif

now differentiating this will give

gif

this gives

gif

which gives the mean and variance for the Poisson distribution same which is true

Moment generating function of Exponential distribution||Exponential distribution moment generating function||MGF of Exponential distribution||Mean and Variance of Exponential distribution using moment generating function

                The Moment generating function for the exponential random variable X by following the definition is

gif

here the value of t is less than the parameter lambda, now differentiating this will give

gif

which provides the moments

gif

clearly

gif

Which are the mean and variance of exponential distribution.

Moment generating function of Normal distribution||Normal distribution moment generating function||MGF of Normal distribution||Mean and Variance of Normal distribution using moment generating function

  The Moment generating function for the continuous distributions also same as the discrete one so the moment generating function for the normal distribution with standard probability density function will be

this integration we can solve by adjustment as

%202%7D%20%5Cend%7Barray%7D

since the value of integration is 1. Thus the moment generating function for the standard normal variate will be

%202%7D

from this we can find for any general normal random variable the moment generating function by using the relation

gif

thus

%202%7D%20%5C%5C%20%26amp%3B%3De%5E%7B%5Cleft%5C%7B%5Cfrac%7B%5Csigma%5E%7B2%7D%20t%5E%7B2%7D%7D%7B2%7D+%5Cmu%20t%5Cright%5C%7D%7D%20%5Cend%7Baligned%7D

so differetiation gives us

gif

thus

gif

so the variance will be

gif

Moment generating function of Sum of random variables

The Moment generating function of sum of random variables gives important property that it equals the product of moment generating function of respective independent random variables that is for independent random variables X and Y then the moment generating function for the sum of random variable X+Y is

Moment generating function
MGF OF SUM

here moment generating functions of each X and Y are independent by the property of mathematical expectation. In the succession we will find the sum of moment generating functions of different distributions.

Sum of Binomial random variables

If the random variables X and Y are distributed by Binomial distribution with the parameters (n,p) and (m,p) respectively then moment generating function of their sum X+Y will be

gif

where the parameters for the sum is (n+m,p).

Sum of Poisson random variables

The distribution for the sum of independent random variables X and Y with respective means which are distributed by Poisson distribution we can find as

gif

Where

gif

is the mean of Poisson random variable X+Y.

Sum of Normal random variables

     Consider the independent normal random variables X and Y with the parameters

gif

then for the sum of random variables X+Y with parameters

gif

so the moment generating function will be

gif

which is moment generating function with additive mean and variance.

Sum of random number of random variables

To find the moment generating function of the sum of random number of random variables let us assume the random variable

gif

where the random variables X1,X2, … are sequence of random variables of any type, which are independent and identically distributed then the moment generating function will be

gif
gif

Which gives the moment generating function of Y on differentiation as

gif

hence

gif

in the similar way the differentiation two times will give

gif

which give

gif

thus the variance will be

gif

Example of Chi-square random variable

Calculate the moment generating function of the Chi-squared random variable with n-degree of freedom.

Solution: consider the Chi-squared random variable with the n-degree of freedom for

gif

the sequence of standard normal variables then the moment generating function will be

gif

so it gives

%202%7D%20%5Cend%7Baligned%7D

the normal density with mean 0 and variance σ2 integrates to 1

%202%7D

which is the required moment generating function of n degree of freedom.

Example of Uniform random variable

Find the moment generating function of random variable X which is binomially distributed with parameters n and p given the conditional random variable Y=p on the interval (0,1)

Solution: To find the moment generating function of random variable X given Y

gif

using the binomial distribution, sin Y is the Uniform random variable on the interval (0,1)

gif.latex?%5Cbegin%7Barray%7D%7Bl%7D%20E%5Cleft%5Be%5E%7Bt%20X%7D%5Cright%5D%3D%5Cint %7B0%7D%5E%7B1%7D%5Cleft%28p%20e%5E%7Bt%7D+1 p%5Cright%29%5E%7Bn%7D%20d%20p%20%5C%5C%3D%5Cfrac%7B1%7D%7Be%5E%7Bt%7D 1%7D%20%5Cint %7B1%7D%5E%7Be%5E%7Bt%7D%7D%20y%5E%7Bn%7D%20d%20y%5C%5C%20%3D%5Cfrac%7B1%7D%7Bn+1%7D%20%5Cfrac%7Be%5E%7Bt%28n+1%29%7D 1%7D%7Be%5E%7Bt%7D

Joint moment generating function

The Joint moment generating function for the n number of random variables X1,X2,…,Xn

gif

where t1,t2,……tn are the real numbers, from the joint moment generating function we can find the individual moment generating function as

gif

Theorem: The random variables X1,X2,…,Xn are independent if and only if the joint mement generating function

gif

Proof: Let us assume that the given random variables X1,X2,…,Xn are independent then

gif

Now assume that the joint moment generating function satisfies the equation

gif
  • to prove the random variables X1,X2,…,Xn are independent we have the result that the joint moment generating function uniquely gives the joint distribution(this is another important result which requires proof) so we must have joint distribution which shows the random variables are independent, hence the necessary and sufficient condition proved.

Example of Joint Moment generating function

1.Calculate the joint moment generating function of the random variable X+Y and X-Y

Solution : Since the sum of random variables X+Y and subtraction of random variables X-Y are independent as for the independent random variables X and Y so the joint moment generating function for these will be

%202%7D%20%5C%5C%20%26amp%3B%3De%5E%7B2%20%5Cmu%20t+%5Csigma%5E%7B2%7D%20t%5E%7B2%7D%7D%20e%5E%7B%5Csigma%5E%7B2%7D%20s%5E%7B2%7D%7D%20%5Cend%7Baligned%7D

as this moment generating function determine the joint distribution so from this we can have X+Y and X-Y are independent random variables.

2. Consider for the experiment the number of events counted and uncounted distributed by poisson distribution with probability p and the mean λ, show that the number of counted and uncounted events are independent with respective means λp and λ(1-p).

Solution: We will consider X as the number of events and Xc the number of counted events so the number of uncounted events is X-Xc,the joint moment genrating function will generate moment

gif

and by the moment generating function of binomial distribution

gif

and taking expectation off these will give

gif

Conclusion:

By using the standard definition of moment generating function the moments for the different distributions like binomial, poisson, normal etc were discussed and the sum of these random variables either the discrete or continuous the moment generating function for those and joint moment generating function were obtained with suitable examples , if you require further reading go through below books.

For more articles on Mathematics, please see our Mathematics page.

A first course in probability by Sheldon Ross

Schaum’s Outlines of Probability and Statistics

An introduction to probability and statistics by ROHATGI and SALEH