In this course, you will learn about hypothesis testing, used for validating dataset hypotheses, along with techniques to test a hypothesis.&nbsp;

Hypothesis Testing

Many a times, we want to test the validity of a statement. For example, is the mean return from this mutual fund more to the mean return from the benchmark? While answering such a question, our interest is not to find the actual mean returns of the mutual fund, but to test whether the statement holds true or not.

A hypothesis is a statement about one or more populations like the statement above.

A hypothesis testing is a standard procedure to test the hypothesis. There could be two possible results: 1) The hypothesis is correct and hence should be accepted. 2) The hypothesis is incorrect and should be rejected.

Steps in Hypothesis Testing

The hypothesis testing process consists of the following steps:
<ol>
 	<li>Stating the hypotheses.</li>
 	<li>Identifying the appropriate test statistic and its probability distribution.</li>
 	<li>Specifying the significance level.</li>
 	<li>Stating the decision rule.</li>
 	<li>Collecting the data and calculating the test statistic.</li>
 	<li>Making the statistical decision.</li>
</ol>
Null and Alternative Hypothesis

When we form a hypothesis to be tested, the hypothesis is called a null hypothesis. The null hypothesis is written as H0. A null hypothesis will be a simple statement about the population parameter. For example, the hypothesis that the mean returns of a mutual fund will be greater than or equal to 8% will be states as H0: m &gt;= 8%.

The alternative hypothesis is the hypothesis accepted when the null hypothesis is rejected. It is represented as HA. In our example, the alternative hypothesis will be HA: m &lt; 8%.

In general, a hypothesis can be formed as follows:
<ol start="1">
 	<li>H0: θ = θ0 versus HA: θ ≠ θ0</li>
 	<li>H0: θ ≤ θ0 versus HA: θ &gt; θ0</li>
 	<li>H0: θ ≥ θ0 versus HA: θ &lt; θ0</li>
</ol>
Where θ represents a population parameter and θ0 is a value assumed in the hypothesis.

One-tailed and Two-tailed Tests

A hypothesis test can be a one-tailed or a two-tailed test. A one-tailed test means that the hypothesis is one-sided such as the second and third formulation above. The second formulation tests whether the population parameter is greater than a certain value (one-sided). The third formulation tests whether the population parameter is greater than a certain value (again one-sided). The first formulation is two-sided; hence the hypothesis test will be twp-tailed. It tests for deviation of value on both sides of θ0.

Example

We will use this example to illustrate the concepts related to hypothesis testing in the following pages.

A portfolio manager believes that the mean returns from a mutual fund are zero. He collects the returns data over the past 100 days and calculate the sample mean returns and the sample standard deviation as follow:

x-bar = 0.2%

s = 0.50%

Based on what we have learned till now, we can state the null and alternative hypothesis as follows:

H0: m = 0 versus HA: m ≠ 0

This is a two-tailed test. We will reject the null hypothesis if the mean is not equal to 0.

What is Hypothesis Testing

<h3>Test Statistic</h3>
A test statistic is a quantity, calculated based on a sample, whose value is the basis for deciding whether or not to reject the null hypothesis.

<img class="aligncenter size-full wp-image-19924" src="https://financetrain.com/wp-content/uploads/ts1.gif" alt="ts1" width="564" height="60">

In our example, the sample statistic is the mean. Therefore, the test statistic will be:

<img class="aligncenter size-full wp-image-19923" src="https://financetrain.com/wp-content/uploads/ts2.gif" alt="ts2" width="556" height="75">
<h3>Type I and Type II Error</h3>
When drawing an inference (from a sample statistic, about a population parameter), there can be two types of errors: Type I and Type II.

Type I error, also known as error of the first kind, occurs when the null hypothesis is true, but is rejected.

Type II error, also known as the error of the second kind, occurs when the null hypothesis is false, but is accepted as true.
<h3>Significance Level</h3>
The level of significance of a test is the probability of a Type I error that we accept in conducting a hypothesis test. It is denoted by α. For example, a significance level of 5% means that there is a 5% probability that a true null hypothesis will be rejected.

It is essential to specify a significance level in a hypothesis test. We then identify the critical value (such as z-statistic) and compare it with our test statistic to accept or reject a hypothesis.

Let’s continue with our example. If the level of significance is 5%, then using the z-test the critical value will be +/-1.96 for a two tailed test.

<img class="aligncenter size-full wp-image-19922" src="https://financetrain.com/wp-content/uploads/ts3.gif" alt="ts3" width="563" height="236">
<h3>Observing z-values</h3>
Two-tailed Test

For a two-tailed test, at a 5% level of significance, we have 2.5% on either tail.

z-value for the left tail will be observed as follows: In the z-table, look for the value of z where F(Z) is 0.0250, i.e., z = -1.96.

z-value for the right tail will be observed as follows: In the z-table, look for the value of z where F(Z) is 0.9750, i.e., z = 1.96

One-tailed Test

For a one-tailed test, at a 5% level of significance, we have 5% in the tail.

z-value for the left tail will be observed as follows: In the z-table, look for the value of z where F(Z) is 0.05, i.e., z = -1.645.

z-value for the right tail will be observed as follows: In the z-table, look for the value of z where F(Z) is 0.95, i.e., z = 1.645.

Test Statistic, Type I and type II Errors, and Significance Level

A decision rule is the rule based on which the null hypothesis is rejected or not rejected.
We first state the hypothesis. Then we determine if it is a one-tailed or a two tailed test. We then specify a significance level, and calculate the test statistic. Now we calculate the critical value. If the test statistic follows a normal distribution, we determine critical value from the standard normal distribution, i.e., the z-statistic. Using the test statistic and the critical value, the decision rule is formulated.
For a 5% level of significance, the decision rules look as follows:
1. H0: θ = θ0 versus Ha: θ ≠ θ0
Reject the null hypothesis if test-statistic &gt; 1.96 or if test-statistic &lt; -1.96.
2.  H0: θ ≤ θ0 versus Ha: θ &gt; θ0
Reject the null hypothesis if test-statistic &gt; 1.645
3. H0: θ ≥ θ0 versus Ha: θ &lt; θ0
Reject the null hypothesis if test-statistic &lt; -1.645
In our example, the decision rule will be as follows:
Reject the null hypothesis if test-statistic &gt; 1.96 or if test-statistic &lt; -1.96.
<img class="aligncenter size-full wp-image-19928" src="https://financetrain.com/wp-content/uploads/dr11.gif" alt="dr1" width="605" height="210" />
Our value of test-statistic was 4, which is greater than 1.96. Therefore, null hypothesis should be rejected. This was a two-tailed test.
The following chart shows the rejection point at 5% significance level for a one-sided test using z-test.
<img class="aligncenter size-full wp-image-19927" src="https://financetrain.com/wp-content/uploads/dr21.gif" alt="dr2" width="497" height="255" />
<h3>Power of Test</h3>
The power of test is the probability of correctly rejecting the null (rejecting the null when it is false).
The power of test = 1 – P(Type II Error)
The following table illustrates the correct decision, Type I error and Type II error.
<table>
<tbody>
<tr>
<td width="205"> </td>
<td width="205">H0 is true</td>
<td width="205">H0 is false</td>
</tr>
<tr>
<td width="205">H0 is rejected</td>
<td width="205">Type I ErrorSignificance level, a. Probability of Type I error.</td>
<td width="205">Correct</td>
</tr>
<tr>
<td width="205">H0 is not rejected</td>
<td width="205">Correct</td>
<td width="205">Type II ErrorPower of Test. 1 – P(Type II Error)</td>
</tr>
</tbody>
</table>
It is difficult to control for the probability of making a Type II error. Statisticians avoid the risk of making a Type II error by using “do not reject H0” and not “accept H0.


<h3>Statistical Result Vs Economically Meaningful Result</h3>



When we use a hypothesis test to reject a null hypothesis, we have results that are statistically significant. However, this does not necessarily mean that the results are meaningful economically.



For example, our hypothesis may statistically prove that a certain strategy produces returns consistently above the benchmark. However, it does not mean that when we implement that strategy, we will get economically meaningful returns above the benchmark. While implementing we will have to consider many other factors such as taxes, and transaction costs.

Decision Rule in Hypothesis Testing

The p-value is the probability, computed using the test statistic, that measures the support (or lack of support) provided by the sample for the null hypothesis.



Once we have the test statistic, we look into the z-table to calculate a value greater than or less than that. This will depend on whether it’s a two-tailed or a one-tailed test.



<h3>Lower-tailed Test</h3>



In a left-tailed hypothesis test, the decision rule will be as follows:



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/p-value-0.png" alt="" class="wp-image-28064" width="447" height="52"/></figure></div>



Assume that the level of significance is 10% (a = 0.10)



Under critical value approach, the decision rule will be as follows:



Reject the null hypothesis if test-statistic &lt; -1.28



If the test statistic is -1.46, we will reject the null hypothesis.



We can calculate the p-value as follows:&nbsp; We will look for the value of F(Z) where Z=-1.46. This gives us p-value = 0.072.



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://financetrain.com/wp-content/uploads/pvalue-1.png" alt="" class="wp-image-28065"/></figure></div>



Under p-value approach, the decision rule will be as follows:



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/p-value-2.png" alt="" class="wp-image-28066" width="552" height="63"/></figure></div>



Since p-value (0.072) is less than a (0.10), we reject the null hypothesis. This is illustrated below:



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/p-value-3.png" alt="" class="wp-image-28067" width="550" height="402"/></figure></div>



<h3>Upper-tailed Test</h3>



In a right-tailed hypothesis test, the decision rule will be as follows:



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/ut-1.png" alt="" class="wp-image-28068" width="486" height="59"/></figure></div>



Assume that the level of significance is 4% (a = 0.04)



Under critical value approach, the decision rule will be as follows:



Reject the null hypothesis if test-statistic &gt; 1.75



1.75 is the value of z in z-table where F (Z) is (1 – 0.04) = 0.96



If the test statistic is 2.29, we will reject the null hypothesis.



We can calculate the p-value as follows:&nbsp; We will look for the value of F(Z) where Z=2.29. This gives us p-value = 0.011. Note: At z=2.29, P(Z)=0.989. p-value = 1 – 0.989 = 0.011.



Under p-value approach, the decision rule will be as follows:



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/ut-2.png" alt="" class="wp-image-28069" width="493" height="61"/></figure></div>



Since p-value (0.011) is less than a (0.04), we reject the null hypothesis. This is illustrated below:



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/ut-3.png" alt="" class="wp-image-28070" width="494" height="379"/></figure></div>



<h3>Two-tailed Test</h3>



In a two-tailed hypothesis test, the decision rule will be as follows:



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/tt-1.png" alt="" class="wp-image-28071" width="464" height="53"/></figure></div>



Assume that the level of significance is 3% (a = 0.03)



Under critical value approach, the decision rule will be as follows:



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/tt-2.png" alt="" class="wp-image-28072" width="435" height="52"/></figure></div>



Reject the null hypothesis if test-statistic &lt;= -2.17 or test-statistic &gt;=2.17



2.17 is the value of z in z-table where F (Z) is (1 – 0.015) = 0.985



If the test statistic is -2.74 or 2.74, we will reject the null hypothesis.



We can calculate the p-value as follows:&nbsp; We will look for the value of F(Z) where Z = 2.74. At z=2.74, P(Z)=0.9690. Probability of getting a value greater than 2.74 = 1-0.9690 = 0.0031. Since it’s a two tailed test, p-value = 2*0.0031 = 0.0062.



Under p-value approach, the decision rule will be as follows:



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/tt-3.png" alt="" class="wp-image-28073" width="421" height="51"/></figure></div>



Since p-value (0.0062) is less than a (0.03), we reject the null hypothesis. This is illustrated below:



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/tt-4.png" alt="" class="wp-image-28074" width="538" height="311"/></figure></div>

p-Value in Hypothesis Testing

You can identify the appropriate test statistic for your data as follows:



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://financetrain.com/wp-content/uploads/test-statistic-1-1024x712.png" alt="" class="wp-image-28085"/></figure></div>

Selecting the Appropriate Test Statistic

t-statistic is the computed value of the test statistic based on the Student’s t distribution.



t-statistic with n-1 degrees of freedom is calculated as follows:



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/t-statistic.png" alt="" class="wp-image-28089" width="267" height="87"/></figure></div>



In the hypothesis test, we compare the t-statistic with the critical t-value at a given level of significance and appropriate degrees of freedom.

Hypothesis Testing with t-statistic

z-statistic is the computed value of the test statistic based on the standard normal distribution.



z-statistic with is calculated as follows:



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/zt-1.png" alt="" class="wp-image-28092" width="355" height="113"/></figure></div>



In the hypothesis test, we compare the z-statistic with the critical z-value at a given level of significance and appropriate degrees of freedom.



Note that when the sample size is large, the population standard deviation (s) can be replaced with sample standard deviation (s).



<h3>Example</h3>



Widget Inc. is a manufacturer of widgets. The mean thickness of the widget produced by them is 5 inches. However sometimes their machine will defect and the thickness will vary. The quality control team takes a sample of widgets every day to check the quality. If the quality is not acceptable the machine will be sent for maintenance. On one such day the quality control person takes a sample of 25 widgets. The mean thickness of the sample is 5.2 inches. The population standard deviation is known to be 0.1 inches. At a level of significance of 5% identify whether the machine needs to be sent for maintenance.



Step 1: State the hypothesis



The null hypothesis will be that the machine is fine and alternative hypothesis will be that the machine is not fine and should be sent for maintenance.



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/zt-2.png" alt="" class="wp-image-28093" width="378" height="57"/></figure></div>



This is a two-tailed test and null hypothesis will be rejected for value above or below 5.



Step 2: Identify the appropriate test statistic



The distribution is normal, sample size is small, and the population standard deviation is known. Therefore, we will use z-statistic.



Step 3: Specify the significance level



The level of significance is 5%. There is a 5% probability of rejecting a true null hypothesis (Type I error)



Step 4: State the decision rule



Since it’s a two tailed test, both the values on the left and on the right are rejection regions. The level of significance is 5%, so area of rejection in each tail is 2.5%. We will look for critical value of z(0.025) which is 1.96.



The decision rule will be as follows:



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/zt-3.png" alt="" class="wp-image-28094" width="444" height="106"/></figure></div>



Step 5: Collect the data and calculate the test statistic



The test statistic will be calculated as follows:



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/zt-4.png" alt="" class="wp-image-28095" width="397" height="93"/></figure></div>



Step 6: Make the statistical decision



The calculated test statistic (10) is greater than critical value (1.96). Therefore, we reject the null hypothesis. The machine should be sent for maintenance.

Hypothesis Testing with z-statistic

Sometimes we want to test if the mean values differ between two populations. We can assume that the two populations are normally distributed and that the samples are drawn independently.



We can combine observations from both samples to get a pooled estimate of the unknown population variance.



The hypothesis can be formed as follows:



<ol type="1"><li>H0: µ1 - µ2 = 0 versus HA: µ1 - µ2 ≠ 0</li><li>H0: µ1 - µ2 ≤ 0 versus HA: µ1 - µ2 &gt; 0</li><li>H0: µ1 - µ2 ≥ 0 versus HA: µ1 - µ2 &lt; 0</li></ol>



<h3>Case 1: Normally distributed populations, population variances unknown, but assumed to be equal</h3>



<figure class="wp-block-image size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/tests-1.png" alt="" class="wp-image-28098" width="410" height="450"/></figure>



<h3>Case 2: Normally distributed populations, population variances unequal and unknown</h3>



<figure class="wp-block-image size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/tests-2.png" alt="" class="wp-image-28099" width="307" height="385"/></figure>

Tests Concerning Differences in Means

In the above cases, we assumed that the samples are independent. However, sometimes the samples may not be independent.



If the samples are not independent, a test of mean difference is done using paired observations. Such a test is called paired comparison test.



<ol type="1"><li>H0: µd = µd0 versus HA: µd ≠ µd0</li><li>H0: µd ≤ µd0 versus HA: µd &gt; µd0</li><li>H0: µd ≥ µd0 versus HA: µd &lt; µd0</li></ol>



In the first hypothesis we want to test if the mean of the differences in pairs is zero.



µd is the mean of the population of paired differences.



µd0 is the hypothesized value of mean of paired differences. This is usually assumed to be zero.



<h3>Calculating t-statistic</h3>



To calculate the t-statistic, we first need to find the sample mean difference:



<figure class="wp-block-image size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/sample-mean-d-1024x439.png" alt="" class="wp-image-28102" width="428" height="183"/></figure>



The sample variance is:



<figure class="wp-block-image size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/mean-variance.png" alt="" class="wp-image-28103" width="176" height="117"/></figure>



The standard deviation of the mean is:



<figure class="wp-block-image size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/standard-dev.png" alt="" class="wp-image-28104" width="123" height="79"/></figure>



 The test statistic, with n – 1 df is:



<figure class="wp-block-image size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/test-stat-1.png" alt="" class="wp-image-28105" width="164" height="108"/></figure>

Paired Comparision Tests - Mean Differences When Populations are Not Independent

Till now the tests we saw were related to population mean. We can also conduct hypothesis tests concerning the variance of the population.



There are two types of tests:



<ul><li>Tests concerning the value of a single population variance</li><li>Tests concerning the differences between two population variances</li></ul>



The hypothesis will be formed as follows:



<figure class="wp-block-image size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/hypothesis.png" alt="" class="wp-image-28109" width="326" height="120"/></figure>



Where &nbsp;is the population variance and &nbsp;is the hypothesized value of the variance.

Hypothesis Tests Concerning Variances

The tests concerning the value of a population variance of a normally distributed population are called Chi-square tests.



For this test, we calculate a Chi-square distributed test statistic. A chi-square distribution is an asymmetrical distribution. Its shape approaches a normal distribution as the degrees of freedom increases. A chi-square distribution has a lower bound of 0. So, it does not have negative values.



The Chi-square test statistic is calculated as follows:



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/chi-1-1.png" alt="" class="wp-image-28113" width="322" height="86"/></figure></div>



Where the sample variance is:



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/chi-2.png" alt="" class="wp-image-28114" width="323" height="105"/></figure></div>



<h3>Example:</h3>



A mutual fund claims that the standard deviation of their quarterly returns is 5% based on the historical data.



An investor wants to verify this claim. He collects the sample returns for the past 16 quarters. The standard deviation based on this sample returns data is 6%. At a 5% level of significance can the investor say that the actual standard deviation of claimed returns is indeed 5%.



Step 1: State the hypothesis



Variance of monthly returns = 0.05^2 = 0.0025



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/chi-3.png" alt="" class="wp-image-28115" width="410" height="50"/></figure></div>



This is a two-tailed test.



Step 2: Identify the appropriate test statistic



The appropriate test statistic for a value of population variance is Chi-square.



Step 3: Specify the significance level



The level of significance is 5%. There is a 5% probability of rejecting a true null hypothesis (Type I error). This is a probability of 2.5% in each tail.



Step 4: State the decision rule



Since it’s a two tailed test, both the values on the left and on the right are rejection regions. The level of significance is 5%, so area of rejection in each tail is 2.5%.



We will use the chi-square table to look for the critical values. For 15 degrees of freedom (16 – 1) and probabilities 0.9750 and 0.025, the critical values are 6.262 and 27.488



The decision rule will be as follows:



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/chi-4.png" alt="" class="wp-image-28116" width="422" height="42"/></figure></div>



The Chi-square distribution for this two-tailed test is illustrated below:



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/chi-5.png" alt="" class="wp-image-28117" width="427" height="282"/></figure></div>



Step 5: Collect the data and calculate the test statistic



The test statistic will be calculated as follows:



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/chi-6-1.png" alt="" class="wp-image-28119" width="382" height="146"/></figure></div>



Step 6: Make the statistical decision



The calculated test statistic (21.6) is between 6.262 and 27.488 we cannot reject the null hypothesis. We will conclude that the statistical mean is not different from the population mean at 5% level of significance.

Chi-square Test – Test for value of a single population variance

The tests concerning the differences between two population variances are called F-test. We are testing the equality of two variances.



For this test, we calculate a F-distributed test statistic that follows the F-distribution.



Here also we assume that the samples are normally distributed and are independent.



The hypothesis is formed as follows:



<figure class="wp-block-image size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/ft-1.png" alt="" class="wp-image-28122" width="345" height="131"/></figure>



Assume we have two samples with n1 and n2 observations. The test statistic for F-test is calculated as follows:



<figure class="wp-block-image size-large is-resized"><img src="https://financetrain.com/wp-content/uploads/ft-2-1024x164.png" alt="" class="wp-image-28123" width="455" height="73"/></figure>



We use the degrees of freedom to identify the critical value from the F-table.



The F distribution is an asymmetric distribution that has a minimum value of 0, but no maximum value. The curve reaches a peak not far to the right of 0, and then gradually approaches the horizontal axis the larger the F value is. The F distribution approaches, but never quite touches the horizontal axis.



All the other steps in conducting the hypothesis test are the same, except the calculation of test statistic.

F-test - Test for the Differences Between Two Population Variances

In parametric tests we make assumptions about the distribution of the population and each parametric test is specific to a population parameter such as mean or variance, for example, a z-test.



A nonparametric test is not concerned with a parameter or makes minimal assumptions about the population being sampled.



A nonparametric test is primarily used in three situations:



<ul><li>when data does not meet distributional assumptions</li><li>&nbsp;when data are given in ranks</li><li>when the hypothesis we are addressing does not concern a parameter</li></ul>

Non-parametric Tests

This quiz is a part of the CFA Level 1 reading 'Hypothesis Testingn'. 

This frontmatter description will appear below the title

Hypothesis Testing

Course Resources

Course Quizzes

Lessons

Free Guides - Getting Started with R and Python

Take the Next Step in Your Data Career