Linear Combinations of Random Variables

The joint distribution of a particular pair of linear combinations of random variables which are independent of each other is a bivariate normal distribution. It forms the basis for all calculations involving arbitrary means and variances relating to the more general bivariate normal distribution. The property of rotational symmetry implies that the joint distribution of any two linear combinations of the two variables is bivariate normal.

The same concept can also be extended to linear combinations of any number of independent normal variables. By applying scaling the condition the two variables need not be standard in nature.

If there are two linear combinations of independent normal variables V and W then their joint distribution is bivariate normal. The paramaters of the bivariate distribution can then be easily calculated.

The following result is implied from the above definitions –

Two linear combinations of independent normal variables are independent only when the two sets of data are not correlated with each other.

What can be said about linear combinations of random variables can also be said about functions of random varaibles that are formed by a linear relationship. A function of random variables is itself a random variable and it can either be built from linear or non-linear relationships between variables.

Given that Y is a linear combination of variables X_1, _X_2,…,_Xp and constants c_1, _c_2, …, _cp

Y\= c_1_X_1 + _c_2_X_2 + … + _cpXp

Multi-Variate Normal Distribution

The bivariate case which applies to linear combinations of two sets of independent random variables can also be applied in the case of linear combinations of several sets of independent random variables and is called a multivariate normal distribution.

The independence property holds for the several sets of variables only if the covariance between each and every one of the sets of data is zero i.e. the changes in value between any two variables are not linked to each other in any way. However, it must be noted that the reverse is not necessarily true i.e. if two or more sets of data are uncorrelated it does not necessarily mean they are independent.

The mutlivariate normal disribution has some very important properties.

If X is distributed multivariate normally:

  • Linear combinations of X are normally distributed
  • All subsets of X are multivariate normally distributed
  • The zero covariance between pairs of variables indicated the variables are independent
  • Conditional distributions of X are also mutlivariate normal

Dependent Normal Variables

In the case the normal variables have a relationship of dependence among them i.e. there is covariance   among the sets of data in other words they display correlation trends amongst each other then any possible linear combinations between these sets of data will never be normally distributed.

This issue is addressed by the concept of copulas which try to measure the degree of dependence between two sets of variables.