In the real world, it is often not possible to collect data about an entire population to conduct a statistical analysis. For example, to make conclusions about the saving habits of families in a city, it would be impractical to collect and analyse savings data about each and every family in the city. Instead, a small group of families, called a sample, can be used to represent the entire population. We can then use this sample to perform our statistical analysis and estimate parameters for the population. These parameters will however only be estimates of the actual parameters of the population.



Learn about how to obtain sample data and use the sample to estimate parameters of the population.

Sampling and Estimation

<h3>Simple Random Sampling</h3>
Simple random sampling is a type of sampling method, in which each element of the population has an equal chance of being selected in the sample.

A simple random sample can be selected as follows: List all the items in the population say from 1 to N, where N is the total number of items in the population. We have given an identification number (1 to N) to each item in the population. Using a random number algorithm (a computer based random number generator or some other method of generating random numbers) select n units from the list of N units one at a time without replacing the items.

This method of sampling is useful where the population is small. A problem with this method is that the sample may not be a good representative of the population as it may not evenly capture all dimensions of the population.
<h3>Sampling Distribution</h3>
A sampling distribution is a probability distribution of the sample statistic. An example of a sample statistic is the mean of sample data.

A sampling distribution of the mean is the probability distribution of sample mean obtained by drawing all possible samples of the same size from the same population.

Let’s say we are looking at a population of 500 stocks. We can draw random samples of 50 stocks from the population and calculate their mean returns. The ‘sample mean returns’ is a sample statistic. We can draw many such samples of 50 stocks and calculate sample mean returns for each random sample. All the sample mean returns are an estimate of the population mean. These ‘sample mean returns’ can be plotted as a sampling distribution of the mean.

Since the process is based on random sampling, the sampling distribution will resemble a normal distribution, even if the population is not normally distributed.

Just like any other distribution, a sampling distribution can be described by its mean and standard deviation.

Mean (μ): The mean of the sampling distribution equals the mean of the population.

Standard Deviation (σ): This measures how spread out the distribution is and represents approximately the amount by which the sample means deviate from the population mean. Note that the larger the sample size, the smaller will be the standard deviation. The standard deviation of the sampling distribution is called the standard error.

Simple Random Sampling and Sampling Distribution

Sampling error is the difference between the sample statistics (such as sample mean) and the corresponding population parameter (such as population mean).

Sampling error occurs due to the random selection of the sample and can be reduced by increasing the size of the sample, or by ensuring that the sample more closely represents the population.

Sampling Error

Stratified random sampling is a sampling method that goes one step further than <a title="Simple Random Sampling and Sampling Distribution" href="https://financetrain.com/simple-random-sampling-sampling-distribution/">simple random sampling</a>. It can be used in situations where the population can be separated naturally into sub-groups or strata. For example, our population of 500 stocks can be separated into sub-groups, with each group representing an industry such as Energy, Automobiles, FMCG, Retail, Infrastructure, etc. Then from each stratum, we can take a random sample. The sample size taken from each stratum will depend on the size of the stratum itself with respect to the population.

Bond fund managers make use of stratified random sampling in bond indexing strategies. In a <a title="Classification of Bond Indexing Strategies" href="https://financetrain.com/classification-of-bond-indexing-strategies/">bond indexing strategy</a>, the investor constructs a portfolio that mimics the performance of a bond index such as Barclays Capital Aggregate Bond Index.

However, a bond index usually contains thousands of bonds which makes it difficult for the investor to implement a <a title="Pure Bond Indexing Strategy" href="https://financetrain.com/pure-bond-indexing-strategy/">pure indexing strategy</a>. Even if a manager wants to do so, it will be very expensive due to the high transaction costs. In such a situation, the investor can use stratified random sampling to construct a portfolio that has the same characteristics as the index. For a bond index, the various bonds can be grouped based on certain risk factors such as duration, credit quality, coupon rate, maturity etc. Then based on the weight of each group in the population, a sample can be drawn from each group to create a sample that replicates the index in terms of these risk factors.

Let’s say there are 1000 bonds in an index. We can classify these bonds based on their <a title="Bond Duration and Convexity Simplified – Part 1 of 2" href="https://financetrain.com/bond-duration-and-convexity-simplified-part-1-of-2/">duration</a> and the coupon rate. For duration we have two intervals: below 5 years and above 5 years. For coupon we have two intervals: below 8% and above 8%. The following table shows this classification:
<table>
<tbody>
<tr>
<td width="168">&nbsp;</td>
<td width="142">Coupon below 8%</td>
<td width="151">Coupon above 8%</td>
</tr>
<tr>
<td width="168">Duration below 5 years</td>
<td width="142">250</td>
<td width="151">300</td>
</tr>
<tr>
<td width="168">Duration above 5 years</td>
<td width="142">200</td>
<td width="151">250</td>
</tr>
</tbody>
</table>
As you can see, there are 4 stratums. Suppose we take a sample of 100 bonds. The number of bonds selected from each cell will be as follows:
<ol>
 	<li>D&lt;5 years, C&lt;8% = 250/1000*100 = 25</li>
 	<li>D&lt;5 years, C&gt;8% = 300/1000*100 = 30</li>
 	<li>D&gt;5 years, C&lt;8% = 200/1000*100 = 20</li>
 	<li>D&gt;5 years, C&gt;8% = 250/1000*100 = 25</li>
</ol>
Stratified random sampling ensures that each type of bond is represented in the sample in the right proportion.

Stratified Random Sampling

In investment analysis, we observe two types of data, namely, time-series data and cross-sectional data.

Time-series data refers to observations made over a period of time at regular intervals. For example, when we take daily closing prices of a stock for 1 year, it is time-series data. The time unit of observation could be anything such as day, week, month, or year.

Note that time-series data contains observations on a single phenomenon (prices of one stock) over multiple periods of time.

Cross-sectional data on the other hand, contains observations on multiple phenomena observed at a single point of time. For example, closing price of 100 stocks at the end of a year.

The following tables illustrate the difference.

Time Series Data
<table>
<tbody>
<tr>
<td width="102">Day</td>
<td width="123">Closing Stock Price of Stock A</td>
</tr>
<tr>
<td width="102">1</td>
<td width="123">.</td>
</tr>
<tr>
<td width="102">2</td>
<td width="123">.</td>
</tr>
<tr>
<td width="102">3</td>
<td width="123">.</td>
</tr>
<tr>
<td width="102">4</td>
<td width="123">.</td>
</tr>
<tr>
<td width="102">5</td>
<td width="123">.</td>
</tr>
</tbody>
</table>
Cross Sectional Data
<table>
<tbody>
<tr>
<td width="103">Closing Stock Price</td>
<td width="103">Stock A</td>
<td width="103">Stock B</td>
<td width="103">Stock C</td>
<td width="103">Stock D</td>
<td width="103">Stock E</td>
</tr>
<tr>
<td width="103">Dec 31, 2013</td>
<td width="103">.</td>
<td width="103">.</td>
<td width="103">.</td>
<td width="103">.</td>
<td width="103">.</td>
</tr>
</tbody>
</table>
As you can see, both time-series data and cross-sectioned data are one-dimensional.

We can combine time-series and cross-sectional data to form two-dimensional data sets.

Panel Data

Observations on multiple phenomena over multiple time periods are called panel data. The following table shows closing price of 5 stocks for years. This is an example of panel data. Note that it contains multi-period data (5 years) of a single characteristic (closing price) of multiple entities (5 different stocks).
<table>
<tbody>
<tr>
<td width="103">Closing Stock Price</td>
<td width="103">Stock A</td>
<td width="103">Stock B</td>
<td width="103">Stock C</td>
<td width="103">Stock D</td>
<td width="103">Stock E</td>
</tr>
<tr>
<td width="103">Year 1</td>
<td width="103">.</td>
<td width="103">.</td>
<td width="103">.</td>
<td width="103">.</td>
<td width="103">.</td>
</tr>
<tr>
<td width="103">Year 2</td>
<td width="103">.</td>
<td width="103">.</td>
<td width="103">.</td>
<td width="103">.</td>
<td width="103">.</td>
</tr>
<tr>
<td width="103">Year 3</td>
<td width="103">.</td>
<td width="103">.</td>
<td width="103">.</td>
<td width="103">.</td>
<td width="103">.</td>
</tr>
<tr>
<td width="103">Year 4</td>
<td width="103">.</td>
<td width="103">.</td>
<td width="103">.</td>
<td width="103">.</td>
<td width="103">.</td>
</tr>
<tr>
<td width="103">Year 5</td>
<td width="103">.</td>
<td width="103">.</td>
<td width="103">.</td>
<td width="103">.</td>
<td width="103">.</td>
</tr>
</tbody>
</table>
Longitudinal 

We can also have a data set that contains multi-period data (say 5 years) of multiple characteristics (say closing price, <a title="Price to Earnings (P/E) Ratio (Leading P/E and Trailing P/E)" href="https://financetrain.com/price-to-earnings-pe-ratio-leading-pe-and-trailing-pe/">P/E ratio</a>, and Market cap) of a single entity (Say a single stock – Stock A).
<table>
<tbody>
<tr>
<td width="103">Stock A</td>
<td width="103">Closing Price</td>
<td width="103">P/E Ratio</td>
<td width="103">Market Cap</td>
</tr>
<tr>
<td width="103">Year 1</td>
<td width="103">.</td>
<td width="103">.</td>
<td width="103">.</td>
</tr>
<tr>
<td width="103">Year 2</td>
<td width="103">.</td>
<td width="103">.</td>
<td width="103">.</td>
</tr>
<tr>
<td width="103">Year 3</td>
<td width="103">.</td>
<td width="103">.</td>
<td width="103">.</td>
</tr>
<tr>
<td width="103">Year 4</td>
<td width="103">.</td>
<td width="103">.</td>
<td width="103">.</td>
</tr>
<tr>
<td width="103">Year 5</td>
<td width="103">.</td>
<td width="103">.</td>
<td width="103">.</td>
</tr>
</tbody>
</table>
&nbsp;

Time Series and Cross Sectional Data

The <a title="Central Limit Theorem" href="https://financetrain.com/central-limit-theorem/">Central Limit Theorem</a> is a fundamental theorem of probability and describes the characteristics of the population of the means. According the Central Limit Theorem, for <a title="Simple Random Sampling and Sampling Distribution" href="https://financetrain.com/simple-random-sampling-sampling-distribution/">simple random samples</a> from any population with finite mean and variance, as n becomes increasingly large, the sampling distribution of the sample means is approximately normally distributed. Here n represents the size of each sample.

The sample mean will be normally distributed regardless of the population distribution, i.e., regardless of the distribution of parent.

The Central Limit Theorem tells us what happens when we have the sum of a large number of independent random variables each of which contributes a small amount to the total.

The central limit theorem has the following characteristics:
<ul>
 	<li>The mean of the sample means is equal to the mean of the population (μ) from which the samples were drawn.</li>
 	<li>The standard deviation of the distribution of sample means is s divided by the square root of n (σ/Sqrt(n)). This is also called the standard error of the sample mean.</li>
 	<li>For the sampling distribution to approximate a normal distribution, the sample size must be large (n &gt;=30).</li>
</ul>

Central Limit Theorem

The standard error of the sample <a title="Calculating Arithmetic Mean" href="https://financetrain.com/calculating-arithmetic-mean/">mean</a> is calculated using the following formula.

<img class="aligncenter size-full wp-image-19637" src="https://financetrain.com/wp-content/uploads/se1.gif" alt="se1" width="522" height="55">

Note that the larger the sample size, the smaller will be the standard deviation.

Let’s say the monthly average savings of a family in a city are $500. Based on a sample size of 50 families, the standard error will be:
Standard Error = 500/(Sqrt(50) = $70.71
This means that if we take all possible samples of size 50 each, calculate the sample means of each sample and create a sampling distribution, then this sampling distribution will have a mean of $500 and a standard error of $70.71.

In reality, since we don’t have the population standard deviation, we use the <a title="Variance and Standard Deviation" href="https://financetrain.com/variance-standard-deviation/">standard deviation</a> of the sample mean to calculate the standard error.

Standard Error of the Sample Mean

In statistics, statistical inference refers to drawing conclusion based on the data. Statistical inferences are drawn in two broad ways, namely, hypothesis testing, and parameter estimation.

In hypothesis testing, we make a hypothesis and then we determine whether the sample data supports the hypothesis or does not support it. The hypothesis could be something like - Population mean is equal to 10. Then based on our sample data, we either accept or reject the hypothesis.

In contrast with hypothesis testing, under parameter estimation we try to estimate the population parameter by making use of the information available in the sample.

There are two types of parameter estimators: point estimates and confidence interval estimates.

Parameter Estimation

A point estimate is a single statistic value that is the “best guess” for the parameter value (such as population mean). The point estimates are calculated using formulas such as the formula to calculate the sample mean or standard deviation. These formulas are called estimators and the values calculated using these estimates are called estimates. To give an example, the sample mean calculated from a sample is a point estimate of the population.

Properties of a Good Estimator

There are three main properties associated with a good estimator. They are:

Unbiasedness: Unbiasedness here means that the expected value of the estimate should equal the parameter the estimate is estimating. In other words, the sampling distribution of the estimator equals the parameter value. For example, the mean of its sampling distribution equals the mean of the population (E(x̄) = μ).

Efficiency: An unbiased estimator is most efficient among all the unbiased estimators, when it has the smallest standard error.&nbsp; No other unbiased estimator has a sampling distribution with lower variance.

For example, both mean and median are estimators to find the centre value of a set of data. When the data comes from distributions with thick tails, the sample median is more efficient. When the data come from distributions with a thin tail, like the normal distribution, the sample mean is more efficient.

Consistency: Consistency means that the estimator converges in probability with the estimated parameter. In other words, as the sample size increases, the probability of the estimates being close to the value of population parameter increases. We can also say that as n reaches infinity, the variance of the estimator approaches 0.

Point Estimates

Along with point estimate we may also want to find a range of values within which our <a title="Parameter Estimation" href="https://financetrain.com/parameter-estimation/">population parameter</a> lies with a certain confidence level (1 – α). This is called confidence interval estimate. The α is known as the significance level and the probability (1-α) is known as the degree of confidence or confidence level. An interval estimate provides more information about a population characteristic than does a <a title="Point Estimates" href="https://financetrain.com/point-estimates/">point estimate</a>.

For example, we can say that the population mean is between 8 and 10 with a 95% confidence. 95% is the degree of confidence and 5% is the level of significance.

<img class="aligncenter wp-image-19672" src="https://financetrain.com/wp-content/uploads/ci11.png" alt="ci1" width="468" height="157">

Confidence intervals are derived from point estimates using the following general formula:

<img class="aligncenter size-full wp-image-19671" src="https://financetrain.com/wp-content/uploads/ci2.gif" alt="ci2" width="595" height="176">

We will learn about how to calculate the confidence interval for a population mean, given a <a title="Normal Distribution" href="https://financetrain.com/normal-distribution/">normal distribution</a>, under three situations:
<ul>
 	<li>Population variance is known</li>
 	<li>Population variance is not known</li>
 	<li>Population variance is not known and sample size is large</li>
</ul>

Confidence Interval Estimates

We have the following assumptions:
<ul>
 	<li>Population variance σ2 is known</li>
 	<li>Population is normally distributed</li>
</ul>
Under these assumptions, the confidence interval estimate will be given as follows:

<img class="aligncenter size-full wp-image-19679" src="https://financetrain.com/wp-content/uploads/cip1.gif" alt="cip1" width="584" height="248">

Example

Let’s take an example to compute this.

We take a sample of 16 stocks from a large population with a mean return of 5.2%. We know that the population standard deviation is 1.5%.

Calculate the 95% confidence interval for the population mean.

For 95% confidence interval, zα/2 = 1.96

The confidence interval will be:

<img class="aligncenter size-full wp-image-19678" src="https://financetrain.com/wp-content/uploads/cip2.gif" alt="cip2" width="497" height="137">

We are 95% confidence that the true mean is between 4.465% an­­d 5.935%.

z is obtained from the standard normal distribution table as shown below. F(Z) value is 0.025 at z = -1.96 and F(Z) value is 0.9750 at z = 1.96.

<img class="aligncenter size-full wp-image-19677" src="https://financetrain.com/wp-content/uploads/cip3.gif" alt="cip3" width="489" height="237">

The most commonly used confidence intervals are 90%, 95%, 99% and 99.9%. The z values are given below.
<table>
<tbody>
<tr>
<td width="177">Confidence Level</td>
<td width="198">Za/2 Value</td>
</tr>
<tr>
<td width="177">90%</td>
<td width="198">1.645</td>
</tr>
<tr>
<td width="177">95%</td>
<td width="198">1.96</td>
</tr>
<tr>
<td width="177">99%</td>
<td width="198">2.58</td>
</tr>
<tr>
<td width="177">99.9%</td>
<td width="198">3.27</td>
</tr>
</tbody>
</table>

Confidence Interval for a Population mean, with a known Population Variance

If the population variance is not known, then we do the following change to the above confidence interval formula:
<ol>
 	<li>Substitute the population variance (s) with the sample variance (s)</li>
 	<li>Us t-distribution instead of normal distribution (explained in the following pages)</li>
</ol>
We use t-distribution because the use of sample variance introduces extra uncertainty as s varies from sample to sample.

<img class="aligncenter size-full wp-image-19883" src="https://financetrain.com/wp-content/uploads/ci11.gif" alt="ci1" width="601" height="129">

Example

We take a sample of 16 stocks from a large population with a mean return of 5.2% and a standard deviation of 1.2%. The population standard deviation is not known.

Calculate the 95% confidence interval for the population mean.

The confidence interval will be:

<img class="aligncenter size-full wp-image-19882" src="https://financetrain.com/wp-content/uploads/ci21.gif" alt="ci2" width="550" height="149">

The value of t is observed from the t-table.

Confidence Interval for a Population mean, with an Unknown Population Variance

When the distribution is normal, we use the z-statistic when the population variance is known and we use t-statistic when the <a title="Confidence Interval for a Population mean, with an Unknown Population Variance" href="https://financetrain.com/confidence-interval-population-mean-unknown-population-variance/">population variance is unknown</a>.

However, when the distribution is not normal, we cannot create a <a title="Confidence Interval Estimates" href="https://financetrain.com/confidence-interval-estimates/">confidence interval</a> if the sample size n&lt;30.

If sample size &gt;30 and the distribution is non-normal then:
<ul>
 	<li>If population variance is known, we use z-statistic</li>
 	<li>If population variance is unknown, we use t-statistic. Even z-statistic is acceptable, but t-statistic is more common.</li>
</ul>
&nbsp;

Confidence Interval for a Population Mean, when the Distribution is Non-normal

Student’s t distribution, or simply called t-distribution, is a form of continuous probability distributions which is formed when we are trying to estimate the mean of a population that is normally distributed, but we have a small sample size and we don’t know the population standard deviation. When we say that the sample size is small, we mean a sample size of less than 30.

Consider a random sample of n observations with mean &nbsp;and standard deviation σ, from a normally distributed population with mean μ. The t-statistic is calculated as follows:

<img class="aligncenter size-full wp-image-19890" src="https://financetrain.com/wp-content/uploads/t-dist0.gif" alt="t-dist0" width="504" height="75">

The variable t follows a Student’s t distribution with (n-1) degrees of freedom. Degrees of freedom refer to the number of observations that are free to vary after sample mean has been calculated.

The t-distribution is symmetric and bell-shaped just like a normal distribution. However, it has heavier tails, which means more observations are far from the mean.

The t-distribution has a degree of freedom equal to n-1. The larger the degree of freedom, the closer is the t-distribution to normal distribution (z-statistic).

The following chart depicts t-distribution with varying degrees of freedom (df).

<img class="aligncenter size-full wp-image-19891" src="https://financetrain.com/wp-content/uploads/t-dist1.gif" alt="t-dist1" width="500" height="285">

Student’s t Distribution

<a title="Student’s t Distribution" href="https://financetrain.com/students-t-distribution/">Student’s t distribution</a> table has the following structure:

<img class="aligncenter size-full wp-image-19897" src="https://financetrain.com/wp-content/uploads/t-table1.gif" alt="t-table1" width="420" height="248">

The row represents the upper tail area, while the column represents the degrees of freedom. The body contains the t values. Note that for one-tail distribution the values are for a and for two-tailed distribution values are for a/2.

Let’s say n = 3, the df= 3-1 = 2. If significance level a is 0.10 then a/2 = 0.05. From the table we can observe that t-value = 2.920.

<img class="aligncenter size-full wp-image-19896" src="https://financetrain.com/wp-content/uploads/t-table2.gif" alt="t-table2" width="347" height="189">

How to Read Student’s t Table

As we have seen in this chapter, all our estimates are based on the sample selected from the population. It is therefore critical that we choose the samples correctly so that our results are not biased. That said, there are many issues that could come in that would make our samples biased and lower the quality of parameter estimates. Let’s look at some of these:

Appropriate Sample Size

We learned that a larger sample size reduces sampling error. However, it has two problems:
<ul>
 	<li>When we increase the sample size, we are at the risk of choosing data that doesn’t represent the population correctly. This is especially the case with any time series data (such as returns from a mutual fund), because the population parameters may change over time.</li>
 	<li>A large sample size also increases our cost of sampling.</li>
</ul>
Data-mining Bias

Data mining involves mining or searching through historical data in an effort to find statistically significant patterns or trading strategies that work. One such pattern is the turn of the day effect – returns tended to be negative and lower on Monday as compared to the averages of the four other days of the week. Traders can make use of this information to outperform the market.

Data mining bias refers to errors that may occur from overly relying on the data mining practices. While some patterns observed from data mining may be useful but not all. Even the ones that were valid, they may not be valid in today’s market scenario. A trader is advised to investigate the data mining practices for their validity before forming their trading strategy. For example, the trader may want to conduct an out-of-sample test to find out whether the model works even in the periods that were not a part of the period considered for data mining.

Sample Selection Bias

This happens when you are not able to take a sample of certain part of the population because of non-availability of data. As a result the sample data selected is not completely random.

Survivorship Bias

Survivorship bias is a form of sample selection bias where certain financial databases tend to exclude information from the database. For example, where unsuccessful funds are removed from the index, the past index values are adjusted to remove the data of the dropped fund. Since a fund is more likely to be dropped from an index because of poor performance, such actions create bias in the index.

Look-ahead Bias

Look-ahead bias occurs when the researcher uses information that was not available on the test date while calculating an estimate. The researcher just assumes the information. In investment analysis, analysts may be using information that doesn’t exist at the time they are making investment decisions.

Time-period Bias 

Time-period bias occurs when the test is based on a time period that may make the results time-period specific.

Biases in Sampling

This quiz is a part of the CFA Level 1 reading 'Sampling and Estimation'. 

This frontmatter description will appear below the title

COURSE

Sampling and Estimation

Course Resources

Course Quizzes

Lessons

R Programming Bundle: 25% OFF

Course Resources

Course Quizzes