Loan Data - Training and Test Data Sets

For building the model, we will divide our data into two different data sets, namely training and testing datasets. The model will be built using the training set and then we will test it on the testing set to evaluate how our model is performing.

There are many ways in which we can split the data. If we had multi-year data, we could have used data for some years as training data and other years as testing data. Our data is for the same period (2016 Q1). We will use a simple approach to randomly divide the dataset into training and test set.

We can use the "sample" command to randomly select certain index number and then use the selected index numbers to divide the dataset into training and testing dataset. Below is the code for doing this. In the code below we use 30% of the data for testing and rest of the 70% for training.

1# Sample Indexes
2> indexes = sample(1:nrow(loandata), size=0.3*nrow(loandata))
3>
4# Split data
5> data_test = loandata[indexes,]
6> dim(data_test)
7[1] 17960   145
8 
9> data_train = loandata[-indexes,]
10> dim(data_train)
11[1] 41909   145
12>
13

We can now remove the original loandata dataset from R to free up memory.

While building the model, we will emphasize that we avoid picking loans that can default as we don’t want to spoil our ROI. At the same time, we don’t want to pick up just a number of loans as we want to make a sizeable investment.

In the next few lessons we will focus on cleaning the dataset and then training the model.