Definition

The most basic form of continuous probability distribution function is called the uniform distribution. It is a rectangular distribution with constant probability and implies the fact that each range of values that has the same length on the distributions support has equal probability of occurrence.  This belongs to the category of maximum entropy probability distributions.

The probability density function for the variable x given that a ≤ x ≤ b is given by:

<img class="aligncenter size-full wp-image-10240" alt="ud1" src="https://financetrain.com/wp-content/uploads/ud1.gif" width="285" height="60" />

Characteristics

The following are the key characteristics of the uniform distribution:
<ul>
	<li>The density function integrates to unity</li>
	<li>Each of the inputs that go in to form the function have equal weighting</li>
	<li>Mean of the unifrom function is given by:</li>
</ul>
<img class="aligncenter size-full wp-image-10244" alt="ud2" src="https://financetrain.com/wp-content/uploads/ud2.gif" width="125" height="64" />
<ul>
	<li>The variance is given by the equation:</li>
</ul>
<img class="aligncenter size-full wp-image-10243" alt="ud3" src="https://financetrain.com/wp-content/uploads/ud3.gif" width="154" height="65" />
The plot of the uniform function is as below:

<img class="aligncenter size-full wp-image-10242" alt="ud4" src="https://financetrain.com/wp-content/uploads/ud4.gif" width="607" height="237" />

The location of the interval has little influence in deciding if the uniformly distributed variable falls within the fixed length. Two factors that influence this the most are the interval size and the fact that the interval falls within the distributions support.
<ul>
	<li>Standard uniform distribution: If a =0  and b=1 then the resulting function is called a standard unifrom distribution. This has very important practical applications.</li>
	<li>There are variables in physical, management and biological sciences that have the properties of a uniform distribution and hence it finds application is these fields.</li>
</ul>
Example

Let's say the amount of gasoline sold every day at a service station is uniformly distributed. The minimum sold is 1000 gallons and maximum sold is 3000 gallons.

We can calculate the probability that the service station will sell atleast 2,000 gallons using the uniform distribution properties.

<img class="aligncenter size-full wp-image-10241" alt="ud5" src="https://financetrain.com/wp-content/uploads/ud5.gif" width="424" height="187" />

P(X ≥ 2,000) = (3,000 – 2,000) x  (1/2000)   = .50

Applications
<ul>
	<li>The most common use of the uniform distribution is as a starting point for the process of random number generation. A classic example of this would be in programming languages.</li>
	<li>It is most commonly used for sampling arbitrary distributions. The most general method used is the inverse transform sampling method which uses the cumulative distribution function of the target random variable. There is very wide application in the theoretical space in this case. Inverting the CDF is a very important requirement in this process and when the CDF is not known in closed form alternative methods have been invented of which one is called rejection sampling. A prime example of this is the use of the normal distribution method to get over the inefficiences of the inverse transform technique.</li>
	<li>When analog signals are converted to digital format errors happen due to rounding or truncation which are called quantization errors. In cases where the error does not have significant correlation with the signal this is uniformly distributed.</li>
</ul>

Properties of Uniform Distribution

Definition

The Bernoulli distribution is a discrete probability distribution which consists of Bernoulli trials.

Each Bernoulli trial has the following characteristics:
<ul>
	<li>There are only two outcomes a 1 or 0, i.e., success or failure each time.</li>
	<li>If the probability of success is p then the probability of failure is  1-p and this remains the same across each successive trial.</li>
	<li>The probabilites are not affected by the outcomes of other trials which means the trials are independent.</li>
</ul>
The probability mass function commonly abbreviated as by PMF is given by:

<img class="aligncenter size-full wp-image-10251" alt="bd1" src="https://financetrain.com/wp-content/uploads/bd1.gif" width="283" height="93" />The graph of the Bernoulli probabilities is as depicted below:

<img class="aligncenter size-full wp-image-10250" alt="bd2" src="https://financetrain.com/wp-content/uploads/bd2.gif" width="380" height="348" />

An example that best illustrates the Bernoulli distribution is a single toss of a coin. Every successive toss is independent of the previous tosses when it comes to determining the outcome.

In case there are more than one trial or in case of many trials the Bernoulli distribution extends to the Binomial distribution. This is then called a Binomial experiment and gives rise to a binomial random variable.

The key characteristics of a binomial experiment are as follows:
<ul>
	<li>A number of Bernoulli trials are to be performed under one experiment and these are pre-determined.</li>
	<li>Each trial has an experiment whereby there are two labelled outcomes success or failure,</li>
	<li>The probability of success in each trial is the same.</li>
	<li>The trials are independent of each other which mean that one trial’s outcome is not affected by the outcome of any other trial.</li>
</ul>
History and relevance 
<ul>
	<li>Swiss scientist Jacob Bernoulli is accredited with the invention of this distribution and he also came up with the idea of the Binomial distribution.</li>
	<li>It is used in situations where a random variable is associated with two outcomes.</li>
</ul>
Properties

The expected value of the random variable is given by E(x) = p and can be derived as follows

E(x) = 0*(1-p) + 1 * p = p

The variance of the Bernoulli variable is given by p*(1-p) and is given as follows:

Var(X) = p - p2 = p*(1-p)

Applications
<ul>
	<li>There are real life situations which involve noting if a specific event occurs or not which is recorded as a success or a failure. The Bernoulli distribution finds application in this situation.</li>
	<li>Some examples that best explain such scenarios are the probability of getting a head in a single coin flip, probability of having a boy child or the probability of getting a hike in the salary package.</li>
</ul>

Properties of Bernoulli Distribution

<h4>Definition</h4>
It is one of the most important continuous probability distributions which finds wide applications in real life by describing variables that display randomness. The distribution is characterized by bell curve which is more weight in the center and tapers off on either side which means it has tails on either side. It takes only two moments i.e. the mean µ and the variance σ2 to describe this function and is therefore a parametric function.

The mean gives more information about the location and the variance gives an idea of how dispersed the values are.

The density function is given as follows:
<img class="aligncenter size-full wp-image-10260" alt="nd1" src="https://financetrain.com/wp-content/uploads/nd1.gif" width="249" height="79" />

<h4>Standard Normal Distribution</h4>
If the mean of the normal distribution is 0 and the variance is set to 1 then we get the standard normal density distribution function which is symmetrical around the mean. In other words the mean and mode of this function are the same as the median. The skewness of this distribution is zero and the kurtosis has a value of 3. The variable which represents this distribution is called the standard normal variable ε.

The standard normal distribution is depicted by the graph below:

<img class="aligncenter size-full wp-image-10259" alt="nd2" src="https://financetrain.com/wp-content/uploads/nd2.gif" width="521" height="287" />

The area under this graph for the range of values of ε from -2 to +2 represents 95% of the distribution. For the value range from -1 to +1 the area represents 68% of the distribution. A table that gives the values of the standard normal variable against the percentage of the distribution is called the standard normal table and the percentage number is called the confidence level in finance.

Most financial variables display this characteristic and the distribution plays a key role in this field. One of the most famous applications is in the Black Scholes pricing model where the cumulative standard normal density function is used to derive the prices of options.

The normal variable is derived from the standard normal by the following definition:
<img class="aligncenter size-full wp-image-10258" alt="nd3" src="https://financetrain.com/wp-content/uploads/nd3.gif" width="120" height="42" />
And this has the desired moments.
<h4>Value at Risk (VAR)</h4>
The maximum possible loss of a portfolio given the exposure to movement in market factors is given by a multiplication of the standard deviation and the confidence level which is the standard normal deviate factor.

This is given by the following equation:
 <img class="aligncenter size-full wp-image-10257" alt="nd4" src="https://financetrain.com/wp-content/uploads/nd4.gif" width="389" height="47" />
Q(X, c) represents the quantile at a specified confidence level c.
<h4>Scalability</h4>
An important property of the normal distribution is that a linear combination of jointly normally distributed random variables is normally distributed in other words it shows stability under addition. This is of extreme relevance because it is important to know the portfolio mean and variance to reconstruct the distribution.

Properties of Normal Distribution

Definitions

Parametric Distribution: A parametric distribution is used in statistics when an assumption is made of the way the underlying data is distributed. An example would be when a variable is assumed to be normally distributed. All subsequent analysis will then rely on this assumption. The parameters associated with this assumption like mean and standard deviation also contribute to the analysis.

Non-Parametric Distribution: This class of distributions is used in cases where assumptions about the pattern or form of the underlying probability distribution from which the data are drawn are not needed. Typically these are used in cases where an attribute of the population needs description; its relationship with another attribute needs to be determined and/or differences on that attribute across population, time or related constructs needs to be derived without an underlying population being distributed in a certain form or requiring interval level measurement. An example of this type of test is the Wilcoxon rank-sum test.

This applies to situations where very weak assumptions have been made about the actual form of the distribution of observations.

Applications

In the field of statistics sampling is most commonly used as it is almost impossible to include each and every member of the population under consideration. A population typically is composed of a very large number of observations.

There are real-world scenarios where the sample set is very small in size for example less than 30. This as well the fact that it is diifficult to determine a pattern associated with the distribution and interval scale measurement has not been made justifies the use of non-parametric distributions for analysis purposes. This is combined with a very desirable characteristic that very few assumptions about the pattern of behaviour are made in this method.

Parameteric methods should only be used when the assumtions about the distribution of the underlying are well met and any violation justifies the use of non-parameteric methods. Whenever there is convergence between the results of parametric and non-parametric analysis the former can be used. In practice most research questions are bivariate and if the bivariate results of both types of tests converge then it would be good to use parametric techniques.

Comparison of Results 

The results of the non-parametric tests are more difficult to interpret than that of the parametric tests.

Whenever the underlying assumption about the distribution of the population holds parametric tests necessarily produce more accurate results than non-paramteric methods.

An Example

Patients in a hospital have been classified based on their gender and their duration of stay needs to be compared. The distribution for females is strongly skewed while that of males is not. The median of both shows strong convergence whereas the means show strong inequality. The parametric test is less suitable in this case as the assumption of normality is not reasonable and a non-parametric approach is more advisable.

Paramteric vs Non-Parametric Distributions

Definition

If the logarithm to the power of the variable x is normally distributed then the variable itself is said to be lognormally distributed. In other words if ln(x) is normally distributed then the variable x is supposed to have a log-normal distribution.

The probability density function for this variable is as follows:

<img class="aligncenter size-full wp-image-10275" alt="ln1" src="https://financetrain.com/wp-content/uploads/ln1.gif" width="375" height="74" />

In this equation µ and σ are the mean and the standard deviation of the variable’s natural logarithm.

A typical log-normal function looks as depicted in the graph below:

<img class="aligncenter size-full wp-image-10274" alt="ln2" src="https://financetrain.com/wp-content/uploads/ln2.gif" width="598" height="204" />

The plot of the log-normal distribution for various values of the standard deviation is as below:

<img class="aligncenter size-full wp-image-10273" alt="ln3" src="https://financetrain.com/wp-content/uploads/ln3.gif" width="574" height="423" />

Occurrence

In financial markets the returns on asset prices are assumed to be normally distributed. If the return is denoted by the following equation:

r = (P1 – P0) / P0

Where P0 and P1 are the prices at time 0 and 1 respectively, then in theory it is possible that P1 might turn out to be negative as r could end up below -1. In order to account for such situations it would be safe to use the log-normal distribution.

If a random variable X is defined as P1/P0 and the logartihm of this variable ln(X) is normally distributed, since X can never be negative it means that P1 can never be negative.

However in the real world it is rare that stock prices are negative as businesses do not have situations where they have large liabilities or outgoing cashflows. Also in practice when the changes in price are small and the time period is not that large the possibility of having a negative price is very small.

Applications

This pattern finds wide application in finance and one of the most famous applications is in the <a title="Black-Scholes-Merton (BSM) Option Pricing Model" href="https://financetrain.com/black-scholes-merton-bsm-option-pricing-model/">Black-Scholes option pricing model</a> commonly used to value options. This model assumes that the returns of commonly used financial asset values like stock prices or foreign exchange rates or price indices or stock market indices are log-normally distributed. Which means the log of the returns should be normally distributed.

However there have been several situations where this assumption fails as in case of very sudden changes in market factors like in the case of stock market crashes or situations involving economic collapses like the Asian Financial Crisis which saw liquidity drying up in the market rapidly causing dramatic changes in the shape of the yield curve with short term rates suddenly sky rocketing. In this case the distributions develop very fat tails and models other than Black-Scholes like log-levy distributions are used to factor in this effect.

Properties of Log-Normal Distribution

<h2>Definition</h2>
I.I.D’s or independent and identically distributed variables are commonly used in probability theory and statistics and typically refer to the sequence of random variables. If the sequence of random variables has similar probability distributions but they are independent of each other then the variables are called independent and identically distributed variables.

This is a pre-reqeusitie for many key theorems like the Central Limit theorem which form the basis of concepts like the normal distribution and many other statistical theories. It must be noted that this assumption does not always hold true in the real world i.e. in practice. This is however the default model for random variables.
<h2>Characteristics</h2>
Sum of I.I.D’s

The sum of independent and identically distributed functions has a moment generating function and it has a continuous probability density function.

<img class="aligncenter size-full wp-image-10305" alt="iid1" src="https://financetrain.com/wp-content/uploads/iid11.gif" width="517" height="500" />

It can be shown that the characteristic function is absolutely integrable and the i.i.d follows a continuous and bounded uniform continuous density function given by:

 <img class="aligncenter size-full wp-image-10303" alt="iid2" src="https://financetrain.com/wp-content/uploads/iid2.gif" width="271" height="98" />

Expected Value and Variance of average of I.I.D’s

As above let’s assume that there are n independent and identically distributed random variables and we take the average of them.

The resulting variable is given by the following equation:

<img class="aligncenter size-full wp-image-10302" alt="iid3" src="https://financetrain.com/wp-content/uploads/iid3.gif" width="549" height="360" />

<img class="aligncenter size-full wp-image-10301" alt="iid4" src="https://financetrain.com/wp-content/uploads/iid4.gif" width="651" height="303" />

Financial Volatility

Financial series data are adequately expressed by Gaussian distributions. In order to calculate the volatility of a series of financial data like the Brazilian real / US$ exchange rates they are expressed as a series of reduced independent and identically distributed variables which form a best fit for the real world data.

Exponential law when applied to this set of reduced variables helps explain their volatilities. It is to be noted that this is based on the assumption that the stochastic process always exhibits a characteristic period.

Applications

Some of the most common uses of I.I.D’s are illustrated in their use in the following situations:
<ul>
	<li>A series of consequent fair or unfair tosses of a coin</li>
	<li>A series of consequent fair or unfair rolls of dice</li>
	<li>A series of results of fair or unfair roulette wheel spins</li>
</ul>
Some of the other most common applications are in signal or image processing and in testing the hyotheses of the means of random variables which assumes the central limit theorem.

Independent and Identically Distributed Variables

The joint distribution of a particular pair of linear combinations of random variables which are independent of each other is a bivariate normal distribution. It forms the basis for all calculations involving arbitrary means and variances relating to the more general bivariate normal distribution. The property of rotational symmetry implies that the joint distribution of any two linear combinations of the two variables is bivariate normal.

The same concept can also be extended to linear combinations of any number of independent normal variables. By applying scaling the condition the two variables need not be standard in nature.

If there are two linear combinations of independent normal variables V and W then their joint distribution is bivariate normal. The paramaters of the bivariate distribution can then be easily calculated.

The following result is implied from the above definitions –

Two linear combinations of independent normal variables are independent only when the two sets of data are not correlated with each other.

What can be said about linear combinations of random variables can also be said about functions of random varaibles that are formed by a linear relationship. A function of random variables is itself a random variable and it can either be built from linear or non-linear relationships between variables.

Given that Y is a linear combination of variables X1, X2,…,Xp and constants c1, c2, …, cp

Y= c1X1 + c2X2 + … + cpXp
<h2>Multi-Variate Normal Distribution</h2>
The bivariate case which applies to linear combinations of two sets of independent random variables can also be applied in the case of linear combinations of several sets of independent random variables and is called a multivariate normal distribution.

The independence property holds for the several sets of variables only if the covariance between each and every one of the sets of data is zero i.e. the changes in value between any two variables are not linked to each other in any way. However, it must be noted that the reverse is not necessarily true i.e. if two or more sets of data are uncorrelated it does not necessarily mean they are independent.

The mutlivariate normal disribution has some very important properties.

If X is distributed multivariate normally:
<ul>
	<li>Linear combinations of X are normally distributed</li>
	<li>All subsets of X are multivariate normally distributed</li>
	<li>The zero covariance between pairs of variables indicated the variables are independent</li>
	<li>Conditional distributions of X are also mutlivariate normal</li>
</ul>
<h2>Dependent Normal Variables</h2>
In the case the normal variables have a relationship of dependence among them i.e. there is covariance   among the sets of data in other words they display correlation trends amongst each other then any possible linear combinations between these sets of data will never be normally distributed.

This issue is addressed by the concept of copulas which try to measure the degree of dependence between two sets of variables.

Linear Combinations of Random Variables

Probability Distributions

This frontmatter description will appear below the title

COURSE

Probability Distributions

Lessons

R Programming Bundle: 25% OFF